Hello Airflow community! Operating distributed data platforms at scale introduces complex architectural challenges: multi-cluster isolation, real-time cross-region monitoring, GPU workload scheduling, and microservice-grade CI/CD for thousands of DAGs.
The *Airflow Summit Online Reconnect <https://airflowsummit.org/onlinereconnect/?utm_source=organic&utm_medium=mail&utm_campaign=asor26&utm_content=mailinglistdevs> (November 4β5)* features deep technical talks built for platform engineers, infrastructure architects, and core developers pushing Airflow to its limits. Highlighted technical sessions for developers include: - *Adaptive DAGs (Satej Sahu)*: Implementing dynamic query plan rewriting and runtime task resizing to prevent OOM errors during massive volume spikes. - *Building Data Products on Airflow 3 (Bhavani Ravi)*: How to leverage Airflow 3's plugin ecosystem (FastAPI, React, DAGFactory, and built-in connections) to eliminate review bottlenecks and enable self-service schema mapping for end users. - *Cross-Deployment Monitoring via Kafka (Christos Bisias)*: Using the new Apache Kafka Provider plugin for zero-polling, event-driven state change streaming across isolated regional instances. - *Operating GPU Workloads at Scale (Aleksandr Shirokov & Roman Khomenko)*: Advanced multi-cluster orchestration patterns for heavy ML workloads on Kubernetes. - *Multi-Tenant Governance (Serjesh Sharma & John MacMillan)*: Architectural patterns for strict data privacy and tenant isolation in enterprise SaaS environments. - *The State of Airflow (Vikram Koka)*: Explore what orchestration needs to become as production workloads get more intelligent and dynamic in the Airflow 3 era. If you are architecting the next evolution of your organization's data platform, join us to review production code patterns, benchmark results, and architectural blueprints. π Format: 100% Virtual & Free π Register for the Developer Track <https://airflowsummit.org/onlinereconnect/?utm_source=organic&utm_medium=mail&utm_campaign=asor26&utm_content=mailinglistdevs> *P.S.* Based in or visiting Mexico City? *Join us for the upcoming local Apache Airflow CDMX meetup!* Itβs a great opportunity to connect, share knowledge, and network with fellow data engineers and practitioners in the area. Oct 29 Β· 6:00 PM to 8:00 PM Check out the details and RSVP here <https://www.meetup.com/mexico-apache-airflow-meetup/events/316923031/?eventOrigin=group_upcoming_events> . Best regards, Airflow Summit team.
