Get Latest De

Email:info@onlinetrainings.in

The Ultimate Hands On Course To Master Apache Airflow

Course

THE ULTIMATE HANDS ON COURSE TO MASTER APACHE AIRFLOW

Category

Apache Airflow IT Training

Eligibility

Graduates and Technology Aspirants

Mode

Both Classroom and Online Classes

Batches

Week Days and Week Ends

Duration :

60 Days

Apache Airflow What will you learn?

•Understand the concepts in Apache Airflow
•Understand how to navigate and use Apache Airflow.
•A Beginner’s Guide to Apache Airflow Coding from scratch
•How to perform read and write operations in Apache Airflow.
•You will know how to configure a Apache Airflow jobs.
•Learn Apache Airflow at your own pace with quality learning videos.
•An easy way to learn one of the widely used Apache Airflow
•Understand Apache Airflow and how to use it to write styles programmatically in Apache Airflow.
•Have an understand of Apache Airflow and how to apply it in your own programs

the ultimate hands on course to master apache airflow Course Highlights

•24 × 7 = 365 days supportive faculty
•Basic Training starting with fundamentals
•Job Placement Assistance with Good Companies
•Online Training with 100% placement assistance
•We provide Classroom and Online training in Metro Cities
• Finessing your tech skills and help break into the IT field
•We provide one to one mentorship for the students and Working Professionals
• Our dedicated HR department will help you search jobs as per your module & skill set, thus, drastically reducing the job search time

Who are eligible for Apache Airflow

•Application Server, Problem Mgmt, SAP Technical/Functional, BO Developer, Automotive Developer, Protocols, Embedded C, AutoSar, Window Applications
•Front End, Javascript, Computer Graphics, Html, Css, Problem Solving, CSS, Web Technologies, Design, Software Development, Full Stack Developer
•Java/J2EE, .Net C#, Networking, Oracle DBA, Embedded Developers, HTML5, Android Framework, Android Developers, MSTR Developer, Cognos, SAN, Windows Admin
•OBIEE, Oracle Fusion Middleware, Oracle Database, Oracle apps DBA, Oracle core DBA, Apex Developer, Java/J2EE developer, Data Architect, Orcale Fusion
•Web Application Developers, Java Developers, DBA LEAD, DBA Manager, Asset Control developer, embedded software engineer, oracle applications technical

THE ULTIMATE HANDS ON COURSE TO MASTER APACHE AIRFLOW Syllabus

Introduction
•Important Prerequisites
•Course Objectives
•Who I am?
•Development Environment
•The basics of Apache Airflow
•Why Airflow?
•What is Airflow?
•How Airflow works?
•[Practice] Installing Airflow
•[Practice] Quick tour of Airflow UI
•[Practice] Quick tour of Airflow CLI
•[Practice] Controlling your DAGs with the CLI
•The Forex Data Pipeline
•Docker reminder
•Troubleshoot Docker performances on MacOS
•Project: The Forex Data Pipeline
•What is a DAG?
•[Practice] Defining your first DAG
•What is an Operator?
•[Practice] Checking if the API is available – HttpSensor
•[Practice] Checking if the currency file is available – FileSensor
•[Practice] Downloading the forex rates from the API – PythonOperator
•[Practice] Saving the forex rates in the HDFS – BashOperator
•[Practice] Creating the Hive table forex_rates – HiveOperator
•[Practice] Processing the forex rates with Spark – SparkSubmitOperator
•[Practice] Sending an email notification – EmailOperator
•[Practice] Sending a Slack notification – SlackAPIPostOperator
•Operator Relationships and Bitshift Composition
•[Practice] Adding dependencies between tasks
•[Practice] The Forex Data Pipeline in action!
•Mastering your DAGs
•Start_date and schedule_interval parameters demystified
•[Practice] Manipulating the start_date with schedule_interval
•Backfill and Catchup
•[Practice] Catching up non triggered DAGRuns
•Dealing with timezones in Airflow
•[Practice] Making your DAGs timezone aware
•How to make your tasks dependent
•[Practice] Creating task dependencies between DagRuns
•How to structure your DAG folder
•[Practice] Organizing your DAGs folder
•[Practice] How the Web Server works
•How to deal with failures in your DAGs
•[Practice] Retry and Alerting
•How to test your DAGs
•[Practice] Unit testing your DAGs
•Distributing Apache Airflow
•Sequential Executor with SQLite
•Local Executor with PostgreSQL
•[Practice] Executing tasks in parallel with the Local Executor
•[Practice] Ad Hoc Queries with the metadata database
•Scale out Apache Airflow with Celery Executors and Redis
•[Practice] Set up the Airflow cluster with Celery Executors and Docker
•[Practice] Distributing your tasks with the Celery Executor
•[Practice] Sending tasks to a specific worker with Queues
•[Practice] Pools and priority_weights: Limiting parallelism – prioritizing tasks
•Kubernetes Reminder
•Scaling Airflow with Kubernetes Executors
•[Practice] Set up a 3 nodes Kubernetes Cluster with Vagrant and Rancher
•[Practice] Installing Airflow with Rancher and the Kubernetes Executor
•[Practice] Running your DAGs with the Kubernetes Executor
•Improving your DAGs with advanced concepts
•Minimising Repetitive Patterns With SubDAGs
•[Practice] Grouping your tasks with SubDAGs and Deadlocks
•Making different paths in your DAGs with Branching
•[Practice] Make Your First Conditional Task Using Branching
•Trigger rules for your tasks
•[Practice] Changing how your tasks are triggered
•Avoid hard coding values with Variables, Macros and Templates
•[Practice] Templating your tasks
•How to share data between your tasks with XCOMs
•[Practice] Sharing (big?) data with XCOMs
•TriggerDagRunOperator or when your DAG controls another DAG
•[Practice] Trigger a DAG from another DAG
•Dependencies between your DAGs with the ExternalTaskSensor
•[Practice] Make your DAGs dependent with the ExternalTaskSensor
•Deploying Airflow on AWS EKS with Kubernetes Executors and Rancher
•Quick overview of AWS EKS
•[Practice] Set up an EC2 instance for Rancher
•[Practice] Create an IAM User with permissions
•[Practice] Create an ECR repository
•[Practice] Create an EKS cluster with Rancher
•How to access your applications from the outside
•[Practice] Deploy Nginx Ingress with Catalogs (Helm)
•[Practice] Deploy and run Airflow with the Kubernetes Executor on EKS
•[Practice] Cleaning your AWS services
•Monitoring Apache Airflow
•How the logging system works in Airflow
•[Practice] Setting up custom logging
•[Practice] Storing your logs in AWS S3
•Elasticsearch Reminder
•[Practice] Configuring Airflow with Elasticsearch
•[Practice] Monitoring your DAGs with Elasticsearch
•Introduction to metrics
•[Practice] Monitoring Airflow with TIG stack
•[Practice] Triggering alerts for Airflow with Grafana
•Airflow maintenance DAGs
•Security in Apache Airflow
•[Practice] Encrypting sensitive data with Fernet
•[Practice] Rotating the Fernet Key
•[Practice] Hiding variables
•[Practice] Password authentication and filter by owner
•[Practice] RBAC UI
•BONUS – APPENDIX
•How to define variables through environment variables
•[Practice] Adding new worker nodes with the Celery Executor