Thanks Jarek for working on it! Confirming a smooth experience with "Steward" - some setup shenanigans, but we figured it out eventually :)
Overall the PR looks solid. However, I'm concerned regarding the usage of submodules in Airflow's main repository. Maintaining a submodule over time, as well as removing a submodule from the repo, are not trivial for the average contributor and people could easily get troubled with it. AI can surely solve everything related with a single prompt - but still, I have to be the devil's advocate here. Is it possible to make the transition to subrepo/automation before merging? Shahar On Sat, May 2, 2026 at 8:39 PM Jarek Potiuk <[email protected]> wrote: > > Hi everyone, > > I am moving Airflow's "maintainer skill" rules to the shared "steward" > project, which will serve as the base for the future ASF-wide initiative. > For those curious, I have already ported the skills for security workflows, > PR triage, and review to the steward repository: > https://github.com/apache/airflow-steward/tree/main/.claude/skills. > > I propose wiring these skills into Airflow via a Git submodule. While we > have had past difficulties with submodules, this approach is justified here: > > > - Users who do not utilize agentic skills will be unaffected. > - We developed a skill to automate submodule updates, addressing > previous pain points. > - This removes all skill-related code and almost all documentation > (except bootstrapping) from the main Airflow repository. Once we switch, > and make sure all "airflow-specific" things are in Airflow, further > "steward" work should happen outside of Airflow repo - no more 1000+ > English markdown PRs in Airflow :). > > > We have a similar setup for our security project, "airflow-s." I am > applying feedback from Shahar; we previously collaborated on some > security-management related work using it, and I recall it being a smooth > experience (Shahar, please confirm). > > I would appreciate (positive :)) reviews on this prerequisite PR: > https://github.com/apache/airflow/pull/66283. Once merged, I will follow up > by replacing the current skills with symbolic links to the submodule. We > can transition to a subrepo or automated extraction once the "steward (to > be renamed)" project stabilizes. > > Best, > Jarek > > On Wed, Apr 29, 2026 at 11:37 AM Jarek Potiuk <[email protected]> wrote: > > > Hi everyone, > > > > I would like to provide an update regarding the recent "airflow-steward" > > and "airflow-s" messages you may have seen. I mistakenly configured a new > > repository to send notifications to the dev@ list, unintentionally > > sharing some early-stage details a bit sooner than I thought. > > > > I have been working at an accelerated pace (pretty much 150% of my time > > over the last one-two months) due to an emerging security "crisis" in the > > industry. New AI models, such as Anthropic’s Mythos (which resulted in the > > recently announced cross-industry Project Glasswing [1]), have become > > exceptionally fast at identifying and chaining security vulnerabilities. > > This has forced a shift in how we handle security, moving from reactive > > responses to proactive scanning. We expect an exponential growth in > > high-quality security reports, and we are preparing to catch these issues > > during the development process—before code is even released. > > > > Last month, the ASF announced the Responsible AI Initiative [2], supported > > by Anthropic and Alpha-Omega. I am currently leading a proposal to > > transition our recent work into a new "Apache Steward" PMC, which is > > expected to be discussed at the next ASF board meeting on May 20th. > > > > Because of the urgency, I have had to make some rapid decisions and move > > faster than our usual PMC pace. My goal is to ensure Apache Steward helps > > the open-source community navigate this transition successfully. For > > Airflow (and other projects), this starts with self-cleaning of security > > issues first. It very likely means a renewed focus on the "Airflow Beach > > Cleaning" [3] initiative to prune historical baggage and manage > > dependencies more effectively using new agentic tooling (it had previously > > hit some scaling roadblocks - that are now unblocked with Agentic AI). > > > > We now have Airflow Steward [4] as the first stage of the "extracting > > tooling" - this will likely become "Apache Steward" - if all stars align. > > > > I apologize for the hectic communication. I will continue to keep everyone > > updated as things progress. If you have any specific questions, please feel > > free to ask here. We can keep that thread to keep everyone updated. > > > > Best regards, > > > > Jarek Potiuk > > > > [1] Project Glasswing https://www.anthropic.com/glasswing > > [2] Responsible AI Initiative of ASF - > > https://news.apache.org/foundation/entry/the-apache-software-foundation-launches-10m-responsible-ai-initiative-with-initial-1-75m-donation > > [3] Airflow Beach Cleaning Keynote from Airflow Summit 2024 - > > https://www.youtube.com/watch?v=f6gfoVJXWEE > > [4] Airflow Steward - https://github.com/apache/airflow-steward > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
