InSelectFrombyJun Wei NgWriting unit tests for Airflow custom operators and hooksHow to mock Airflow connections and variables, DAG run context, and Airflow hook initialisation during unit testingMar 8, 20221Mar 8, 20221
Jun Wei NgWriting unit tests for an Airflow DAGUnit tests are the fundamental of the test pyramid. They should be fast to execute and cover as much of the code as logically sensible. The…Mar 7, 20222Mar 7, 20222
Jun Wei NgCreate a dynamic Airflow DAG using a YAML file (or any other flat file)We can control a DAG’s workflow using a flat file that contains the dynamic variable. Sounds easy, but it does have its downside tooMar 14, 20223Mar 14, 20223
Jun Wei NgUsing nested operators to achieve a dynamic Airflow DAGNesting operators might do the trick, but at what costs?Mar 14, 2022Mar 14, 2022
Gary A. StaffordDevOps for DataOps: Building a CI/CD Pipeline for Apache Airflow DAGsBuild an effective CI/CD pipeline to test and deploy your Apache Airflow DAGs to Amazon MWAA using GitHub ActionsDec 13, 2021Dec 13, 2021
InIT’s TinkoffbyMaxim BelousovTesting of Apache Airflow’s DAGs with docker compose and pytestOur team is developing a data catalog Data Detective. We collect data from various sources in miscellaneous formats and transform data to a…Dec 15, 2021Dec 15, 2021
InSelectFrombyJun Wei NgAirflow Integration Testing using Docker ComposeHow we can do integration testing for our Airflow DAGs with the help of Docker ComposeMar 2, 20221Mar 2, 20221
InTDS ArchivebyMadison SchottTired of Airflow? Try This.A new tool that does what Airflow does but betterOct 5, 20216Oct 5, 20216
InNinja Van TechbyTeddy HartantoA Complete Guide to Setting up a Local Development Environment for Airflow (with Docker and…Collaborate on Airflow workflows with ease using this complete setup that includes a docker-compose, PyCharm, and DAG validation tests.May 23, 20205May 23, 20205
