October Monthly Digest View this email in your browser <https://mailchi.mp/79d21270332b/apache-airflow-newsletter-october-9295402?e=ece21dcc4d>
Another month passed, and Airflow continues to grow and improve! We had two core releases 2.4.1 and 2.4.2 covering a wide swath of bug fixes and some code cleanup. Going into this winter, events are speeding up as well, with 3 conferences, a meetup, and more. Find the details in the “Upcoming Events” section below. Also coming up, we’ll be looking to give this newsletter a facelift, with a new name and some cleaner design. Look for a vote on the dev list soon! Thanks for reading! See you next month On October 24, we released Apache Airflow 2.4.2! 🎉 📦 PyPI: https://pypi.org/project/apache-airflow/2.4.2/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b5c5191150&e=ece21dcc4d> 📚 Docs: https://airflow.apache.org/docs/apache-airflow/2.4.2 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7db20ba21a&e=ece21dcc4d> 🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/2.4.2/release_notes.html <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b92f356a01&e=ece21dcc4d> 🪶 Sources: https://airflow.apache.org/docs/apache-airflow/2.4.2/installation/installing-from-sources.html <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=415957ed4a&e=ece21dcc4d> On October 14, we also released Apache Airflow Helm chart 1.7.0. 📦 ArtifactHub: https://artifacthub.io/packages/helm/apache-airflow/airflow <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7063ca7a45&e=ece21dcc4d> 📚 Docs: https://airflow.apache.org/docs/helm-chart/1.7.0/ <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2d3a60150b&e=ece21dcc4d> 🛠️ Release Notes: https://airflow.apache.org/docs/helm-chart/1.7.0/release_notes.html <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=232a69c866&e=ece21dcc4d> 🪶 Sources: https://airflow.apache.org/docs/helm-chart/1.7.0/installing-helm-chart-from-sources.html <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=06165130ac&e=ece21dcc4d> Become the last #ApacheAirflow <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=324acfa286&e=ece21dcc4d> speaker of 2022! Register here to present your awesome Airflow use case or demo: http://bit.ly/3lhp1aR <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=338db7c499&e=ece21dcc4d> & win some amazing swag! Here are some upcoming Airflow-related events: - November 1: Batch Inference with Airflow and SageMaker webinar <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=853430f2f0&e=ece21dcc4d>, Ganapathi Krishnamoorthi - Senior ML Solutions Architect at Amazon and Faisal Hoda, Enterprise Architect at Astronomer - November 2, 9, and 16 or December 1: Data Engineering From Notebook to Production <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0471c3fa87&e=ece21dcc4d> course - November 7: London Apache Airflow Meetup <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=fd7242d514&e=ece21dcc4d> - November 10: Trino Summit <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cc9e5eb1ce&e=ece21dcc4d> including Using Trino with Apache Airflow for (almost) all your data problems with Philippe Gagnon - November 14-15, 16. International Conference on Semantic Technologies for Big Data Integration and Analysis <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5a772bf2c1&e=ece21dcc4d>, Paris event - November 22-23, Sustainability Reporting & Communications Europe 2022 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=848b749a8a&e=ece21dcc4d>, London conference - December 6, Astro Days <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=8b2bbf18c9&e=ece21dcc4d>, Chicago event If you missed any of these, you can catch a recap, watch on-demand, or dig into the slides: - October 3-6: ApacheCon North America <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=3e1ea63e12&e=ece21dcc4d>, New Orleans, LA - available slides <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4b260ece02&e=ece21dcc4d> - October 4: What’s New in Airflow 2.4 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=818705f825&e=ece21dcc4d> with Kenten Danas and Viraj Parekh - October 11: Data Driven Scheduling <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2dcb065c70&e=ece21dcc4d> with Kenten Danas and Viraj Parekhmediu - October 19: Data Lake Community Meetup- Apache Iceberg: An architectural look under the covers <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=05d3e407f7&e=ece21dcc4d> with Dipankar Mazumdar This month we had a tie between two wholly diserving PRs! *@kazanzhy <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c809eedf01&e=ece21dcc4d>* #25717 Add SQLExecuteQueryOperator <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e3ba829e2e&e=ece21dcc4d> Adds a catch-all operator to execute SQL queries against diverse databases *@ephraimbuddy <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=96aedd80e1&e=ece21dcc4d>* #27297 Fix IntegrityError during webserver startup <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=aba735b819&e=ece21dcc4d> A fix to preload the webserver app before forking workers, fixing the workers throwing IntegrityError As Always, tag a PR with #protm to nominate it - How to keep Data Quality in check with Airflow <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=677be4c1be&e=ece21dcc4d>, Tamara Fingerlin, Astronomer - Writing a good Airflow DAG - Part 1 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=3a2870a2b8&e=ece21dcc4d> and Part 2 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=d077441a1b&e=ece21dcc4d>, Najma Bader, Medium - Web scraping automation using Airflow <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=2aa4fa6437&e=ece21dcc4d>, Muhammad Fadlil Ismail, Medium - Introducing Airflow SLA Miss Report <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=007d8c81a5&e=ece21dcc4d>, Nikhil Manjunatha, Clairvoyant Blog - Tracking the War in Ukraine: A Data Engineering Project <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=db313747eb&e=ece21dcc4d>, Nate DiRenzo, Python in Plain English - Airflow is on the Cloud | ELT Pipeline Orchestration With Airflow & AWS <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=0635893849&e=ece21dcc4d>, Kaan Boke, Towards AI - Orchestration, or how to become a data-driven company <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4df10f1d16&e=ece21dcc4d>, Steven Hillion, Astronomer - Using Google Trends with Airflow <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=5444710fa5&e=ece21dcc4d>, Davide Gazzè, Medium - AWS Managed Workflows for Apache Airflow vs. Glue <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=902adcab66&e=ece21dcc4d>, Minseok Song, Towards Data Science - Using XCOMs with Airflow - scenario based examples with code <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=19e8f519f6&e=ece21dcc4d>, Suman Kumar Gangopadhyay, Medium - Micropipelines: A Microservice Approach for DAG Authoring in Apache Airflow <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7730fe29ef&e=ece21dcc4d>, Vikram Koka, Astronomer - Votes: - Release Apache Airflow Providers prepared on October 4 [approved] - Release Apache Airflow Helm Chart 1.7.0 based on 1.7.0rc1 [approved] - Release Airflow 2.4.2 from 2.4.2rc1 [approved] - Discussed: - Creating system to handle webserver_config.py in non-webserver pods - Proposed: - Clarifications of triage team role including strengthening importance of active triaging - Split tracking for DAG SLAs off from tracking of queue to ensure timely SLA pushes - Announced: - Apache Airflow 2.4.2 Released - Apache Airflow Helm Chart version 1.7.0 Released - Breeze rewrite to Python completed! - Apache Airflow 2.4.1 Released - Airflow 2.4.3 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=8bf5eeb1b4&e=ece21dcc4d> - Focused on Bugfixes - Airflow 2.5.0 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=b61e1a6fc2&e=ece21dcc4d> - Focused on new features - Airflow Helm Chart 1.8.0 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e3b88fd3f5&e=ece21dcc4d> - For more info about upcoming releases and in-development projects, visit the current projects’ issue (#10176 <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=361e04fd3b&e=ece21dcc4d> ). Please subscribe to this newsletter to ensure you always receive the latest issue: http://eepurl.com/hXUA3r <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=455ce4a1fb&e=ece21dcc4d> . Know of an upcoming event or publication the community should know about? Let us know at [email protected] or [email protected].
