What I'm interested in is an outside contributors PR being reviewed by a bot, how this process is triggered (e.g. automatically, by the maintainer, by the reviewer), does the bot correctly pick up on AI slop issues, how does the contributor react to this, does this help or get in the way of maintainers moderating slop issues, etc.?
Damian -----Original Message----- From: Jarek Potiuk <[email protected]> Sent: Friday, February 27, 2026 11:49 AM To: [email protected] Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs automatically Here it is Damian: https://github.com/apache/airflow/pull/62575 This is somewhat of an inception PR. It updates the agent instructions to create a PR as we want. I used Claude Code to generate the PR and then I asked Claude Code to create the PR—it followed the instructions it had just created. After this PR: * The PRs will be created via the web interface - which will give contributors a chance to review the PR before they create it (adds a bit of **good** friction there) * It will use our template and we will prefill the "Generated-by" information so the user does not have to do it manually. Which is actually pretty cool - because that will also likely increase the likelihood we will get the actual information about the agent used (we basically instruct the agent to tell who they are). J. On Thu, Feb 26, 2026 at 5:47 PM Damian Shaw <[email protected]> wrote: > Would love to see an example of it working with a real PR, i.e., > submitted by an outside contributor not a test PR, so see how the > interaction ends up going, will consider this for other open-source projects > I maintain. > > Damian > > -----Original Message----- > From: Amogh Desai <[email protected]> > Sent: Thursday, February 26, 2026 3:46 AM > To: [email protected] > Subject: Re: New: AGENTS.md overhaul + Copilot catches AI-slop PRs > automatically > > This is amazing, Kaxil! Thank you! > > Thanks & Regards, > Amogh Desai > > > On Wed, Feb 25, 2026 at 9:38 PM Vincent Beck <[email protected]> wrote: > > > Very much needed! Thank you Kaxil! > > > > On 2026/02/25 08:16:47 Jarek Potiuk wrote: > > > Very cool :). > > > > > > On Wed, Feb 25, 2026 at 4:14 AM Kaxil Naik <[email protected]> > wrote: > > > > > > > Hey all, > > > > > > > > tl;dr: AI coding tools now get proper Airflow guardrails via > > AGENTS.md, and > > > > Copilot code review will automatically flag common mistakes in > > > > PRs > > > > -- including AI-slop patterns as long as we add Copilot as reviewer. > > > > > > > > We've been getting more AI-generated PRs lately, and reviewers > > > > keep catching the same things: N+1 queries, architecture > > > > boundary > > violations, > > > > assert in production code, fabricated diffs, etc. I wanted to > > > > automate that. > > > > > > > > I've been using AGENTS.md (CLAUDE.md , Cursor rules, skills and > > > > such) > > with > > > > Cursor, Claude Code, and Copilot and various other AI harnesses > > > > since > > last > > > > year and have iterated on it many times. Three PRs landed today: > > > > > > > > - https://github.com/apache/airflow/pull/62440 -- AGENTS.md > > overhaul. > > > > Replaced the sparse doc-index with concrete guidance: > > > > environment > > setup, > > > > architecture boundaries, coding/testing standards, commit > > conventions. > > > > Added nested AGENTS.md for Execution API (Cadwyn versioning) > > > > and providers. > > > > - https://github.com/apache/airflow/pull/62442 -- Copilot > > > > code > > review > > > > instructions. Added .github/instructions/ > > code-review.instructions.md > > > > scoped to review only. Covers architecture violations, N+1 > > > > queries, run_id > > > > scoping, unbounded caches, and AI-slop signals (fabricated > > > > diffs, narrating > > > > comments, over-engineered solutions, etc.). > > > > - https://github.com/apache/airflow/pull/62443 -- Symlink > > CLAUDE.md to > > > > AGENTS.md so Claude Code reads the same file. > > > > > > > > To validate the Copilot instructions, I opened a test PR ( > > > > https://github.com/apache/airflow/pull/62447) with 8 > > > > deliberately > > planted > > > > violations. Copilot caught all 8: > > > > > > > > 1. N+1 query (loop calling get_task_instances() per dag run) > > > > 2. time.time() instead of time.monotonic() > > > > 3. assert in production code > > > > 4. Unbounded @lru_cache (no maxsize) > > > > 5. run_id query without dag_id > > > > 6. Narrating comments > > > > 7. Import inside function body > > > > 8. No tests for new behavior > > > > > > > > This won't replace human review, but should save time on > > > > mechanical catches. If you have patterns you'd like Copilot to > > > > flag, PRs to .github/instructions/code-review.instructions.md > > > > are > welcome. > > > > > > > > Regards, > > > > Kaxil > > > > > > > > > > > -------------------------------------------------------------------- > > - To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > ________________________________ > Strike Technologies, LLC (“Strike”) is part of the GTS family of > companies. Strike is a technology solutions provider, and is not a > broker or dealer and does not transact any securities related business > directly whatsoever. This communication is the property of Strike and > its affiliates, and does not constitute an offer to sell or the > solicitation of an offer to buy any security in any jurisdiction. It > is intended only for the person to whom it is addressed and may > contain information that is privileged, confidential, or otherwise protected > from disclosure. > Distribution or copying of this communication, or the information > contained herein, by anyone other than the intended recipient is > prohibited. If you have received this communication in error, please > immediately notify Strike at [email protected], and delete and > destroy any copies hereof. > ________________________________ > > CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any > attachments are intended solely for the addressee. This transmission > is covered by the Electronic Communications Privacy Act, 18 U.S.C > ''2510-2521. The information contained in this transmission is > confidential in nature and protected from further use or disclosure > under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be > subject to attorney-client or other legal privilege. Your use or > disclosure of this information for any purpose other than that > intended by its transmittal is strictly prohibited, and may subject > you to fines and/or penalties under federal and state law. If you are > not the intended recipient of this transmission, please DESTROY ALL > COPIES RECEIVED and confirm destruction to the sender via return transmittal. > ________________________________ Strike Technologies, LLC (“Strike”) is part of the GTS family of companies. Strike is a technology solutions provider, and is not a broker or dealer and does not transact any securities related business directly whatsoever. This communication is the property of Strike and its affiliates, and does not constitute an offer to sell or the solicitation of an offer to buy any security in any jurisdiction. It is intended only for the person to whom it is addressed and may contain information that is privileged, confidential, or otherwise protected from disclosure. Distribution or copying of this communication, or the information contained herein, by anyone other than the intended recipient is prohibited. If you have received this communication in error, please immediately notify Strike at [email protected], and delete and destroy any copies hereof. ________________________________ CONFIDENTIALITY / PRIVILEGE NOTICE: This transmission and any attachments are intended solely for the addressee. This transmission is covered by the Electronic Communications Privacy Act, 18 U.S.C ''2510-2521. The information contained in this transmission is confidential in nature and protected from further use or disclosure under U.S. Pub. L. 106-102, 113 U.S. Stat. 1338 (1999), and may be subject to attorney-client or other legal privilege. Your use or disclosure of this information for any purpose other than that intended by its transmittal is strictly prohibited, and may subject you to fines and/or penalties under federal and state law. If you are not the intended recipient of this transmission, please DESTROY ALL COPIES RECEIVED and confirm destruction to the sender via return transmittal.
