📄️ Add Documentation
How to add Markdown documentation directly to Airflow DAGs in Datacoves so operators can read context in the Airflow UI and Grid view.
📄️ External Python Scripts
Use DatacovesBashOperator to run external Python scripts from Airflow DAGs in Datacoves, with custom packages pre-installed in the worker environment.
📄️ Dynamic Schedule
Use environment variables or Airflow Variables to dynamically configure DAG schedule intervals in Datacoves for flexible pipeline orchestration.
📄️ Generate from YAML
Use dbt-coves to auto-generate Airflow Python DAG files from simple YAML definitions, reducing boilerplate and standardizing pipeline creation.
📄️ Get Git Branch Name
Retrieve the current git branch name from within an Airflow DAG task in Datacoves for branch-specific logic or conditional pipeline behavior.
📄️ S3 to Snowflake
Use Airflow in Datacoves to load files from Amazon S3 into Snowflake using the S3ToSnowflakeOperator, with staging area and copy options.
📄️ Run ADF Pipeline
Use Azure Data Factory operators in Airflow DAGs within Datacoves to trigger and monitor ADF pipelines as part of your data orchestration workflow.
📄️ Run Airbyte Syncs
Orchestrate Airbyte data syncs from Airflow DAGs in Datacoves using the Airbyte provider, with examples for triggering connections and monitoring status.
📄️ Run dbt
Use the Datacoves dbt decorator or operator to run dbt commands from Airflow tasks, with full examples for dbt run, dbt test, and dbt build.
📄️ Retry dbt Tasks
Configure automatic retries for failed dbt tasks in Airflow DAGs in Datacoves, including retry count, delay settings, and email-on-retry behavior.
📄️ Run Databricks Notebook
Trigger and monitor Databricks Notebook runs from Airflow DAGs in Datacoves using the Databricks provider operator with cluster configuration.
📄️ Run Fivetran Syncs
Orchestrate Fivetran connector syncs from Airflow DAGs in Datacoves using the Fivetran provider, with connection ID configuration and monitoring.
📄️ Test DAGs
How to run automated Airflow DAG tests in your CI/CD pipeline using pytest and the Datacoves CLI to catch import errors before deploying to production.
📄️ Variables & Connections
How to access and use Airflow Variables and Connections in Datacoves DAGs, with best practices for avoiding performance issues during DAG parsing.