★★★★★
"The structure, the practice exams, the instructor — all top tier. Passed first try."
AS
Aarti SharmaSenior Project Manager · TCS
— The journey
Course Outline
What the programme covers, module by module.
Module 1: Introduction to Apache Airflow
- Data orchestration concepts
- Apache Airflow overview
- Workflow automation
- Airflow use cases
- ETL and ELT orchestration
- Airflow ecosystem
Module 2: Airflow Architecture
- Scheduler
- Webserver
- Metadata database
- Executor
- Workers
- Airflow components
Module 3: Airflow Environment Setup
- Installation concepts
- Project structure
- Configuration files
- Environment variables
- Airflow CLI
- Development environment
Module 4: Understanding DAGs
- DAG fundamentals
- DAG structure
- DAG definitions
- Task relationships
- Workflow design
- DAG lifecycle
Module 5: Building Your First DAG
- DAG creation
- Task definition
- Dependencies
- Scheduling
- Start dates
- DAG execution
Module 6: Airflow Operators
- Operator concepts
- PythonOperator
- BashOperator
- EmptyOperator
- SQL operators
- Custom operators
Module 7: Task Dependencies
- Upstream tasks
- Downstream tasks
- Dependency patterns
- Branching
- Trigger rules
- Workflow sequencing
Module 8: Scheduling & Timetables
- Scheduling concepts
- Cron expressions
- Schedule intervals
- Data intervals
- Catchup
- Backfilling
Module 9: Airflow TaskFlow API
- TaskFlow concepts
- Task decorators
- Python functions
- Task dependencies
- Data passing
- Workflow development
Module 10: XComs & Data Sharing
- XCom concepts
- Pushing values
- Pulling values
- Task communication
- Data exchange patterns
- XCom best practices
Module 11: Sensors
- Sensor concepts
- File sensors
- SQL sensors
- External task sensors
- Waiting strategies
- Sensor optimization
Module 12: Connections, Variables & Configuration
- Airflow connections
- Variables
- Connection management
- Runtime configuration
- Environment configuration
- Secure configuration practices
Module 13: Dynamic Workflows
- Dynamic task mapping
- Parameterized DAGs
- Reusable workflows
- Dynamic dependencies
- Scalable task generation
- Workflow flexibility
Module 14: Error Handling & Retries
- Task failures
- Retry configuration
- Failure callbacks
- Timeouts
- Trigger rules
- Recovery strategies
Module 15: Data Pipeline Orchestration
- ETL pipelines
- ELT workflows
- Data ingestion
- Transformation workflows
- Pipeline dependencies
- Data validation
Module 16: Database & Cloud Integration
- SQL databases
- Data warehouses
- Object storage
- Cloud services
- Provider packages
- Integration patterns
Module 17: Monitoring & Logging
- Task logs
- DAG monitoring
- Workflow status
- Performance monitoring
- Alerting concepts
- Troubleshooting
Module 18: Testing Airflow Workflows
- DAG validation
- Task testing
- Unit testing concepts
- Integration testing
- Debugging
- Pipeline quality checks
Module 19: Airflow Security & Access
- Authentication concepts
- Role-based access
- Secrets management
- Connection security
- Environment security
- Operational best practices
Module 20: Airflow Deployment & Scaling
- Deployment architecture
- Executor selection
- Worker scaling
- Containerized deployment
- Kubernetes concepts
- Production considerations
Module 21: Airflow Optimization & Best Practices
- DAG optimization
- Scheduler performance
- Resource management
- Idempotent tasks
- Maintainable workflows
- Production best practices
Module 22: Practical Data Orchestration Project
- Pipeline architecture
- DAG development
- Task dependencies
- Data integration
- Monitoring and retries
- Production workflow design
— 01.2 · Is it right for you?
Who it's for & what's included
Pick a delivery method to see exactly who it suits and everything you receive.
Who it's for
Classroom
Best for learners who want face-to-face tuition and to network with peers in person.
What's included
Everything you get
- ✓ Live instructor on-site
- ✓ Printed workbook & materials
- ✓ Group exercises & case studies
Who it's for
Online Instructor-Led
Best for learners who want a live instructor and a fixed schedule, without the travel.
What's included
Everything you get
- ✓ Live instructor via video call
- ✓ Digital workbook & resources
- ✓ Session recordings
Who it's for
Self-Paced
Best for self-motivated learners who need maximum flexibility around work and life.
What's included
Everything you get
- ✓ On-demand video lessons
- ✓ Interactive quizzes
- ✓ 24/7 access on any device