> View this email in your browser > <https://mailchi.mp/c750d1fc299c/apache-airflow-newsletter-august-9279594?e=a2e8531eaf> > > > This month in the Airflow community, we: > > released Airflow Client 2.3.0 > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=22d09d7702&e=a2e8531eaf> > and Airflow 2.3.4 > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ded85b2e1b&e=a2e8531eaf> > held events on using plugins for Webserver > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4c01142009&e=a2e8531eaf> > and building pipelines with dbt and Great Expectations > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1b94cf2f12&e=a2e8531eaf> > discussed consolidating DAG scheduling params and other topics on the dev > list > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c2b2315030&e=a2e8531eaf> > and much more! > > Please subscribe to this newsletter to ensure you always receive the latest > issue: http://eepurl.com/hXUA3r > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7a50486f2a&e=a2e8531eaf>. > > > Airflow 2.4.0 > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=29c2fc2ac7&e=a2e8531eaf>, > expected in mid-September, will significantly upgrade Airflow’s data > handling capabilities thanks to AIP-48 > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=8dca048e93&e=a2e8531eaf>: > data-aware scheduling. > > From the proposal: > > "Airflow currently depends on XComs to handle data, but several common use > cases would benefit from a more "data aware" approach. For example, in many > organizations, different teams need to access and modify the same datasets. > Currently, a data dependency can only be expressed as a task dependency > within a single DAG, so organizations end up writing composite DAGs with many > tasks. Even so, underlying data dependencies remain invisible. > > AIP-48 will: > > enable a data dependency mechanism which can be used across DAGs > > enable the triggering of DAGs based on these dataset updates > > enable explicit, visible relationships between DAGs based on data > dependencies." > > > On August 23rd, we published Airflow 2.3.4. You can find the package and > documentation using these links: > PyPI > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6b4ee2800e&e=a2e8531eaf> > | Docs > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=539e5dde63&e=a2e8531eaf> > | Notes > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=c8323fcee2&e=a2e8531eaf> > | Constraints > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=610860c224&e=a2e8531eaf> > Docker image: docker pull apache/airflow:2.3.4 > On August 3rd, we released Airflow Client 2.3.0. You can find it here: > PyPI > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=30878aa4c4&e=a2e8531eaf> > | Docs > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=85569d3c08&e=a2e8531eaf> > > > This month we had a tie! > > @rossturk > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a64b7ae4e9&e=a2e8531eaf> > > #25888 Prefer the local Quick Start in docs > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6afbf67621&e=a2e8531eaf> > Makes the Docker Compose Quick Start procedure into a How To, and puts the > standalone procedure in its place > > @pierrejeambrun > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6459dbed71&e=a2e8531eaf> > > #25610 Grid logs for mapped instances > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=7ec024e8d6&e=a2e8531eaf> > Adds support for displaying mapped instance task logs in the Grid details > side panel > To nominate a PR > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=81d56e4b76&e=a2e8531eaf> > for a future newsletter, flag it by adding #protm to a comment. > > Become the last #ApacheAirflow > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=15053a04dd&e=a2e8531eaf> > speaker of 2022! Sign up to host a meetup at http://bit.ly/3lhp1aR > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=4229815273&e=a2e8531eaf> > & win some amazing swag! > > September 27: Astronomer's Astro Days: NYC > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=bd7e87cc48&e=a2e8531eaf> > > If you missed any of these, you can catch a recap, watch on-demand, or dig > into the slides: > > August 9: "Airflow 101: Essential Tips for Beginners" > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=cdf4b37a60&e=a2e8531eaf> > with Tamara Fingerlin & Kenten Danas > > August 10: “Build a Robust Data Pipeline with Airflow, dbt, and Great > Expectations – with Interactivity” > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=afe825ec84&e=a2e8531eaf> > with Sam Bail > > August 16: "Live with Astronomer: The SQL Table Check Operator" > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=ff7ef9955e&e=a2e8531eaf> > with Kenten Danas & Viraj Parekh > > August 17: “Using Plugins for Webserver” > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=a07eca048c&e=a2e8531eaf> > with Faizan Qazi > > August 23: "The Astro Python SDK" > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=6a569d5131&e=a2e8531eaf> > with Daniel Imberman & Kenten Danas > > > Pierre Jeambrun @pierrejeambrun > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1b057ec1fe&e=a2e8531eaf> > Where are you based? > I live in Paris. > > What is your current role (& industry if not tech)? > I am a senior full stack developer. In January 2022 I started working > freelance, since then I have been working for Engie doing back-end > development. > > Why did you start contributing to Airflow? > After a few years in the industry using tons of amazing open source tools, I > felt like it was time for me to contribute to the community. I was looking > for a very cool project, not too 'low level' and most importantly with great > people to work with. Apache Airflow checked all the boxes. > > What do you use it for? > I have used Airflow in a couple of side projects at small scale, but never in > a production/commercial environment. Currently the only DAGs I write are for > Airflow development purposes. > > When was your first PR, and what was it? > My first PR was in December 2021, the goal was to rewrite the Opsgenie hook > to use the official Python SDK. > > What was your latest PR? > Lately I have been working on the Grid page of the UI, mostly around logs. My > last contribution adds support for displaying logs of mapped task instances. > > What do you like or enjoy about working on OSS projects? > For me this is a unique opportunity to work with talented people from all > over the world, to build an amazing product that can be used by everyone. > > > “Everything You Should Know About Airflow 2.3.0’s New Grid View” > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=1f2fd9872c&e=a2e8531eaf>, > Brent Bovenzi, Astronomer Blog > > “We’re All Using Airflow Wrong and How to Fix It” > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=15df105edb&e=a2e8531eaf>, > Jessica Laughlin, Medium > > “Astro Is Now Available on All Major Cloud Providers” > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=054005d742&e=a2e8531eaf>, > Ryan Fox, Astronomer Blog > > “Reimagining Airflow for Data Engineers and Data Scientists with the Astro > Python SDK” > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=e4dafdbdf2&e=a2e8531eaf>, > Kenten Danas & Mike Shwe, Astronomer Blog > > Votes > > AIP-44 - Airflow internal API [approved] > > Airflow Providers release [approved] > > Release Airflow 2.3.4 from 2.3.4rc1 [approved] > > Lazy consensus > > Consolidate DAG scheduling params > > Discussed > > Move “contrib” and all old classes to a separate package > > AIP naming correction > > AIP-44 Airflow database API > > “Use existing venv support” for PythonVirtualenvOperator as counterpart to > AIP-46 > > Vending AWS system test results back to the community > > User Kinceo1 posting spam review approvals > > Should value passed via XComArg be templated? > > Proposed > > A fix for the SLA alerts > > Creating DAGs through the REST API > > Airflow pluggable scheduler > > Announced > > Airflow Python client version 2.3.0 released > > Airflow Providers released > > Apache Airflow 2.3.4 released > > > Airflow Helm Chart 1.7.0 > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=dd6094e979&e=a2e8531eaf> > The next release for the Airflow Helm Chart, fixing bugs and adding more > features. > > For more info about upcoming releases and in-development projects, visit the > current projects’ issue (#10176 > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=98b7fd9355&e=a2e8531eaf>). > > Please subscribe to this newsletter to ensure you always receive the latest > issue: http://eepurl.com/hXUA3r > <https://apache.us14.list-manage.com/track/click?u=fe7ef7a8dbb32933f30a10466&id=afc83edf5b&e=a2e8531eaf>. > > Is there an upcoming event or publication the community should know about? > Let us know at [email protected] > <mailto:[email protected]> or [email protected] > <mailto:[email protected]>. > Want to change how you receive these emails? > You can update your preferences > <https://apache.us14.list-manage.com/profile?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=a2e8531eaf&c=8dffcacaf4> > or unsubscribe from this list > <https://apache.us14.list-manage.com/unsubscribe?u=fe7ef7a8dbb32933f30a10466&id=65cb5665fa&e=a2e8531eaf&c=8dffcacaf4>. >
