InSelectFrombyJun Wei NgWriting unit tests for Airflow custom operators and hooksHow to mock Airflow connections and variables, DAG run context, and Airflow hook initialisation during unit testingMar 8, 2022371Mar 8, 2022371
Jun Wei NgWriting unit tests for an Airflow DAGUnit tests are the fundamental of the test pyramid. They should be fast to execute and cover as much of the code as logically sensible. The…Mar 7, 2022312Mar 7, 2022312
Jun Wei NgCreate a dynamic Airflow DAG using a YAML file (or any other flat file)We can control a DAG’s workflow using a flat file that contains the dynamic variable. Sounds easy, but it does have its downside tooMar 14, 2022443Mar 14, 2022443
Jun Wei NgUsing nested operators to achieve a dynamic Airflow DAGNesting operators might do the trick, but at what costs?Mar 14, 202264Mar 14, 202264
Gary A. StaffordDevOps for DataOps: Building a CI/CD Pipeline for Apache Airflow DAGsBuild an effective CI/CD pipeline to test and deploy your Apache Airflow DAGs to Amazon MWAA using GitHub ActionsDec 13, 2021166Dec 13, 2021166
InIT’s TinkoffbyMaxim BelousovTesting of Apache Airflow’s DAGs with docker compose and pytestOur team is developing a data catalog Data Detective. We collect data from various sources in miscellaneous formats and transform data to a…Dec 15, 202114Dec 15, 202114
InSelectFrombyJun Wei NgAirflow Integration Testing using Docker ComposeHow we can do integration testing for our Airflow DAGs with the help of Docker ComposeMar 2, 202291Mar 2, 202291
InTDS ArchivebyMadison SchottTired of Airflow? Try This.A new tool that does what Airflow does but betterOct 5, 20218566Oct 5, 20218566
InNinja Van TechbyTeddy HartantoA Complete Guide to Setting up a Local Development Environment for Airflow (with Docker and…Collaborate on Airflow workflows with ease using this complete setup that includes a docker-compose, PyCharm, and DAG validation tests.May 23, 20207775May 23, 20207775
