Hey everyone, I updated our meeting notes document in the Airflow wiki to capture the notes from our dev call last week. The link for those notes is here <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=308153072#Airflow3Devcall:MeetingNotes-Summary.20>
As always, it was wonderful to see the progress and everyone working together towards delivering this release. To everyone who attended the meeting, please check the summary and add anything that I may have missed. For those who could not join, please let us know if you disagree with anything discussed and agreed upon in the meeting. Also, please do ask questions if something is unclear. As discussed in the meeting, we are switching to a weekly cadence. Our next meeting is on the 27th of March at the same time. Please let me know if you would like to add anything to the agenda <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=308153072#Airflow3Devcall:MeetingNotes-ProposedAgenda.5> . Best regards, Vikram -- Below is the summary from the call - Development Updates and Presentations - AIP-81 CLI Authentication Design Update (Bugra Ozturk, Jed Cunningham) - Bugra and Jed explained how AIP-81 would create a completely separate remote CLI package, which will be "airflow-ctl". - For local execution, all of the CLI commands (which are DB backed) will just work. - All of the merged code for remote CLI execution will be moved into airflow-ctl as part of beta4 and will also simplify packaging and release timelines. - AIP-72 Task Execution Interface aka Task SDK Update (Ash Berlin-Taylor, Kaxil Naik, Amogh Desai) - Kaxil gave an update on the AIP-72 detailing that JWT Integration, Custom XCom backend, and Secrets backend work is all completed. - He added that the Triggerer backend now works without going through the public API, to avoid all the authentication passing complexity. - Kaxil said that the Task Notifiers is the one open item, which TP is currently working on to finish. - Amogh added that the Secrets backend also enables a separate secrets backend for Workers as compared to the Secrets backend configured for the Scheduler. - Ash added that each Task now has a JWT token that is associated with it, bound to a Task Try ID. - Ash also added that the Execution API versioning was now merged for the Task Execution Interface and implemented Calendar versioning, based on Cadwyn. - Ash added that this was only for the Task Execution Interface, not for the public API, nor for UI internal API. - Shahar added that he had heard that some AWS provider system tests were breaking right and that he would be investigating. - Test plan update (Rahul Vats) - Rahul shared the updated Airflow 3 Test plan <https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+3.0+Test+Plan> and walked through a quick progress update by AIP. - Rahul asked everyone on the team to put on their testing hat, treat beta4 as being the internal testing vehicle, and to test it out, so that we can find more bugs before release. - Rahul also added that AIP-81 needed to be tested for CLI regressions. - Discussion Topics: - Packaging Update (Jarek Potiuk) - Jarek highlighted the big PR which introduces the "airflow-core" package within the main airflow repo. - He highlighted how there is no longer an "airflow" folder in the main repo. - Instead, there is "airflow-core" which has all the key items inside it including the pyproject home. - The second thing we have is an airflow meta package which has all the configuration for builds and additional elements for packaging and distribution. - Jarek said that this was still work in progress, with a few things to be fixed, but was mostly done. - Jarek also spoke about how UV works with the new packaging structure and stressed how this simplifies Airflow development. - Cleanup of Airflow 3 open items in Github (Elad Kalif) - Elad asked all the AIP owners to review the list of Github issues originally targeted for Airflow 3 and still open - Specifically, these are airflow3.0:candidate <https://github.com/apache/airflow/issues?q=is%3Aissue%20state%3Aopen%20label%3Aairflow3.0%3Acandidate> and airflow3.0:breaking <https://github.com/apache/airflow/issues?q=is%3Aissue%20state%3Aopen%20label%3Aairflow3.0%3Abreaking> - Elad made the point that these should either be closed as having been done or moved to a subsequent release tag as part of the Airflow 3 RCs. - Action items - What's critical to finish up - Jens brought this up in the context of the UI, if there are open critical items to be completed before 3.0 release - Brent took the action item to highlight this async and shout out if anything needed help - Change dev calls to be weekly until release - Elad suggested that we make the dev calls weekly, since we are very close to release. - Vikram to update the calendar invite to be weekly and to send out a reminder if a meeting is required that week. Vikram Koka Chief Strategy Officer Email: vik...@astronomer.io <https://www.astronomer.io/>
