Hi everyone, *IMPORTANT! We are seeing the first week-over-week downward trend in PRs opened, largely due to these improvements.*
I have re-created the adoption PR following further testing and fixes: [1]. I am looking for reviews and a quick merge to switch to the "steward" versions of our skills, which are now faster and self-improving with every triage session (and reusable for other projects). Once merged, skills will be removed from the main Airflow repo and replaced with a purely optional /setup-steward skill. This process is now worktree-friendly and includes an automated self-update feature. We currently have 187 cases ready for review. I highly recommend using the /maintainer-review skill to manage this volume. My goal is to turn "quick merges" into a regular agent-assisted task to further reduce the backlog. We are also working with Andre and Yeonguk to optimize workflows and improve triage accuracy in Steward. Switching to Steward now from the "embedded" skills is therefore quite important as the two systems keep on diverging more and more. I'm developing agentic improvements to replace the manual triage tasks I currently perform. This allows the triager to focus on reviewing and approving (or correcting) the proposals generated by the agent. You can view the current triage and review stats here: [2]. I plan to discuss further enhancements, such as better PR classification and UX design for the maintainer experience (I am talking to some UX designers I met at FOSDEM).,. I will show more at the next dev call. In the meantime, please share any feedback or wishes here or in the #auto-triage-feedback Slack channel. Any wild ideas, comments, or wishes you have as maintainers or contributors in this area are absolutely welcome! [1] Airflow Adopts Steward PR: https://github.com/apache/airflow/pull/66677 [2] Triage status: https://htmlpreview.github.io/?https://gist.githubusercontent.com/potiuk/466fb7efcf8ea331a65545f2ab2139a7/raw/stats-dashboard.html Best, Jarek On Sun, May 3, 2026 at 10:27 PM Jarek Potiuk <[email protected]> wrote: > https://github.com/apache/airflow/pull/66319 -> has exactly this: -7500 > lines of skills +1400 lines of "adoption" bootstrap that has been added > automatically at adoption time. > > On Sun, May 3, 2026 at 7:37 PM Jarek Potiuk <[email protected]> wrote: > >> Ok. Adoption should be super simple now: >> >> 1) Project does not need to have **anything** >> 2) Tell your agent: "Adopt apache/airflow-steward in my repo". >> 3) The agent does everything. You answer questions during configuration. >> At the end you have a "setup-steward" skill in your repo. Commit the PR >> 4) Those who wish to use steward in their copy should ask their agent to >> "set up steward". That's it. All skills chosen for a steward at adoption >> time will be installed >> 5) At any point, you can run `setup-steward upgrade` to upgrade to the >> latest steward, or `setup-steward verify` to verify if it is set up >> correctly >> 6) We also have basic scaffolding to modify steward workflows with local >> "variants" -> "setup-steward override SKILL" (which will be >> intention-driven). >> >> https://github.com/apache/airflow-steward#how-adoption-works >> >> Working on the next step -wiring the skills and PR to add it to airflow - >> using the approach above - I will test it this way :) >> >> On Sun, May 3, 2026 at 11:47 AM Jarek Potiuk <[email protected]> wrote: >> >>> Actually... After your message, Shahar, I woke up today with a much >>> better idea :D. >>> >>> I figured out that with agentic CLIs - we don't have to do subrepo or >>> submodule..... We can have literally 1 SKILL to copy to "airiflow'` SKILL >>> directory: `Install and manage "Apache Steward" :). No subrepot, No >>> submodule, simply instructions on how to locally copy the "steward" from >>> either released SVN or Github repo (tag or branch), adding .gitignore. >>> >>> This will be even better, and likely much more acceptable for projects >>> that do not want to use submodules. When you want to modify "steward" >>> behaviour, we will have protection so that you cannot modify the gitignored >>> steward. Instead you will add local "modification" of the flow >>> locally (which will not be .gitignored), and when you would like to >>> contribute it back you will have to ask agent to turn your local >>> modification into a proper PR to the repo. It's really magical with agents >>> when you do not have to limit yourself to "common tools" (like submodule) - >>> but you can simply describe your intentions and let the agent execute those >>> :) >>> >>> I will close the PR and will follow that path :) >>> >>> On Sat, May 2, 2026 at 11:02 PM Shahar Epstein <[email protected]> >>> wrote: >>> >>>> Yeah well - I forgot that there's the option of not installing the >>>> submodule if you don't use it, and then there's no problem for most >>>> cases >>>> (we could indeed handle the rest, with some AI skill and guidance in the >>>> docs). >>>> In any case it should be better than subrepo, according to your >>>> description. >>>> Let's give it a shot then this way. >>>> >>>> >>>> Shahar >>>> >>>> On Sat, May 2, 2026, 22:43 Jarek Potiuk <[email protected]> wrote: >>>> >>>> > Subrepo has another disadvantage - namely that any "sync" of the >>>> subrepo is >>>> > a full "snapshot" of all changed there - and the files end up as >>>> regular >>>> > files in the target repo. Those files are physically committed files >>>> in >>>> > Airflow repo, and subrepo is not "official" git feature - it's an >>>> external >>>> > package https://github.com/ingydotnet/git-subrepo. I think subrepo >>>> is grep >>>> > if you want to "vendor-in" part of an external "dependent" >>>> repository—one >>>> > that you plan to migrate to the next version, where a submodule is >>>> much >>>> > better for "WIP," especially if none of your code actually depends on >>>> > whether the submodule is checked out or not. >>>> > >>>> > Our only dependency on "steward" now is that symlinks to "skills" >>>> will be >>>> > missing if the submodule is not updated. >>>> > >>>> > So ... skills in Claude won't work in this case—that's the "default" >>>> stage >>>> > for people who do not use skills. No big deal (because they don't use >>>> them) >>>> > >>>> > On the other hand if some wants to start using skills this is a >>>> question of >>>> > two commands (explained in setup): >>>> > >>>> > >>>> > git submodule update --recursive >>>> > >>>> > Followed by (in Agent) >>>> > >>>> > /setup-apache-steward >>>> > >>>> > And the latter command will add a "post-merge" hook that removes the >>>> need >>>> > of manually updating the submodule.Once setup is complete, it should >>>> be >>>> > totally transparent. >>>> > >>>> > Of course - this is theory, and with a little practice (it seems to >>>> work), >>>> > there will be some edge cases. But I would rather learn about those >>>> edge >>>> > cases and fix them :) >>>> > >>>> > J >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > On Sat, May 2, 2026 at 9:25 PM Shahar Epstein <[email protected]> >>>> wrote: >>>> > >>>> > > Thanks Jarek for working on it! >>>> > > Confirming a smooth experience with "Steward" - some setup >>>> > > shenanigans, but we figured it out eventually :) >>>> > > >>>> > > Overall the PR looks solid. However, I'm concerned regarding the >>>> usage >>>> > > of submodules in Airflow's main repository. Maintaining a submodule >>>> > > over time, as well as removing a submodule from the repo, are not >>>> > > trivial for the average contributor and people could easily get >>>> > > troubled with it. AI can surely solve everything related with a >>>> single >>>> > > prompt - but still, I have to be the devil's advocate here. >>>> > > Is it possible to make the transition to subrepo/automation before >>>> > merging? >>>> > > >>>> > > >>>> > > Shahar >>>> > > >>>> > > On Sat, May 2, 2026 at 8:39 PM Jarek Potiuk <[email protected]> >>>> wrote: >>>> > > > >>>> > > > Hi everyone, >>>> > > > >>>> > > > I am moving Airflow's "maintainer skill" rules to the shared >>>> "steward" >>>> > > > project, which will serve as the base for the future ASF-wide >>>> > initiative. >>>> > > > For those curious, I have already ported the skills for security >>>> > > workflows, >>>> > > > PR triage, and review to the steward repository: >>>> > > > >>>> https://github.com/apache/airflow-steward/tree/main/.claude/skills. >>>> > > > >>>> > > > I propose wiring these skills into Airflow via a Git submodule. >>>> While >>>> > we >>>> > > > have had past difficulties with submodules, this approach is >>>> justified >>>> > > here: >>>> > > > >>>> > > > >>>> > > > - Users who do not utilize agentic skills will be unaffected. >>>> > > > - We developed a skill to automate submodule updates, >>>> addressing >>>> > > > previous pain points. >>>> > > > - This removes all skill-related code and almost all >>>> documentation >>>> > > > (except bootstrapping) from the main Airflow repository. Once >>>> we >>>> > > switch, >>>> > > > and make sure all "airflow-specific" things are in Airflow, >>>> further >>>> > > > "steward" work should happen outside of Airflow repo - no more >>>> 1000+ >>>> > > > English markdown PRs in Airflow :). >>>> > > > >>>> > > > >>>> > > > We have a similar setup for our security project, "airflow-s." I >>>> am >>>> > > > applying feedback from Shahar; we previously collaborated on some >>>> > > > security-management related work using it, and I recall it being a >>>> > smooth >>>> > > > experience (Shahar, please confirm). >>>> > > > >>>> > > > I would appreciate (positive :)) reviews on this prerequisite PR: >>>> > > > https://github.com/apache/airflow/pull/66283. Once merged, I will >>>> > > follow up >>>> > > > by replacing the current skills with symbolic links to the >>>> submodule. >>>> > We >>>> > > > can transition to a subrepo or automated extraction once the >>>> "steward >>>> > (to >>>> > > > be renamed)" project stabilizes. >>>> > > > >>>> > > > Best, >>>> > > > Jarek >>>> > > > >>>> > > > On Wed, Apr 29, 2026 at 11:37 AM Jarek Potiuk <[email protected]> >>>> > wrote: >>>> > > > >>>> > > > > Hi everyone, >>>> > > > > >>>> > > > > I would like to provide an update regarding the recent >>>> > > "airflow-steward" >>>> > > > > and "airflow-s" messages you may have seen. I mistakenly >>>> configured a >>>> > > new >>>> > > > > repository to send notifications to the dev@ list, >>>> unintentionally >>>> > > > > sharing some early-stage details a bit sooner than I thought. >>>> > > > > >>>> > > > > I have been working at an accelerated pace (pretty much 150% of >>>> my >>>> > time >>>> > > > > over the last one-two months) due to an emerging security >>>> "crisis" in >>>> > > the >>>> > > > > industry. New AI models, such as Anthropic’s Mythos (which >>>> resulted >>>> > in >>>> > > the >>>> > > > > recently announced cross-industry Project Glasswing [1]), have >>>> become >>>> > > > > exceptionally fast at identifying and chaining security >>>> > > vulnerabilities. >>>> > > > > This has forced a shift in how we handle security, moving from >>>> > reactive >>>> > > > > responses to proactive scanning. We expect an exponential >>>> growth in >>>> > > > > high-quality security reports, and we are preparing to catch >>>> these >>>> > > issues >>>> > > > > during the development process—before code is even released. >>>> > > > > >>>> > > > > Last month, the ASF announced the Responsible AI Initiative [2], >>>> > > supported >>>> > > > > by Anthropic and Alpha-Omega. I am currently leading a proposal >>>> to >>>> > > > > transition our recent work into a new "Apache Steward" PMC, >>>> which is >>>> > > > > expected to be discussed at the next ASF board meeting on May >>>> 20th. >>>> > > > > >>>> > > > > Because of the urgency, I have had to make some rapid decisions >>>> and >>>> > > move >>>> > > > > faster than our usual PMC pace. My goal is to ensure Apache >>>> Steward >>>> > > helps >>>> > > > > the open-source community navigate this transition >>>> successfully. For >>>> > > > > Airflow (and other projects), this starts with self-cleaning of >>>> > > security >>>> > > > > issues first. It very likely means a renewed focus on the >>>> "Airflow >>>> > > Beach >>>> > > > > Cleaning" [3] initiative to prune historical baggage and manage >>>> > > > > dependencies more effectively using new agentic tooling (it had >>>> > > previously >>>> > > > > hit some scaling roadblocks - that are now unblocked with >>>> Agentic >>>> > AI). >>>> > > > > >>>> > > > > We now have Airflow Steward [4] as the first stage of the >>>> "extracting >>>> > > > > tooling" - this will likely become "Apache Steward" - if all >>>> stars >>>> > > align. >>>> > > > > >>>> > > > > I apologize for the hectic communication. I will continue to >>>> keep >>>> > > everyone >>>> > > > > updated as things progress. If you have any specific questions, >>>> > please >>>> > > feel >>>> > > > > free to ask here. We can keep that thread to keep everyone >>>> updated. >>>> > > > > >>>> > > > > Best regards, >>>> > > > > >>>> > > > > Jarek Potiuk >>>> > > > > >>>> > > > > [1] Project Glasswing https://www.anthropic.com/glasswing >>>> > > > > [2] Responsible AI Initiative of ASF - >>>> > > > > >>>> > > >>>> > >>>> https://news.apache.org/foundation/entry/the-apache-software-foundation-launches-10m-responsible-ai-initiative-with-initial-1-75m-donation >>>> > > > > [3] Airflow Beach Cleaning Keynote from Airflow Summit 2024 - >>>> > > > > https://www.youtube.com/watch?v=f6gfoVJXWEE >>>> > > > > [4] Airflow Steward - https://github.com/apache/airflow-steward >>>> > > > > >>>> > > > > >>>> > > >>>> > > >>>> --------------------------------------------------------------------- >>>> > > To unsubscribe, e-mail: [email protected] >>>> > > For additional commands, e-mail: [email protected] >>>> > > >>>> > > >>>> > >>>> >>>
