On Sun, Feb 22, 2026 at 10:23 AM Oscar Benjamin <[email protected]> wrote: > > On Sun, 22 Feb 2026 at 15:41, Aaron Meurer <[email protected]> wrote: > > > > On Sun, Feb 22, 2026 at 7:03 AM Oscar Benjamin <[email protected]> > > wrote: > >> > >> More generally I think that the template is just too long so I am not > >> surprised that in some cases people would just delete it or not notice > >> what the important parts are. I also wonder if a lot of contributors > >> just don't even see the template. My suspicion is that many people now > >> open a PR by clicking an "open a PR" button (or an AI prompt?) in > >> their editor meaning that they don't go to the actual GitHub website > >> themselves and edit that text box directly. > > > > > > I'm sure people don't read it, but I suspect in 99% of the cases where it's > > "deleted" it's because someone opened the PR without using the web > > interface, and a majority of the time that's because they used some AI > > agent to do it (that doesn't necessarily mean autonomous though, it could > > just mean they promoted Claude code or codes to open the PR after writing > > it). It sucks a little bit because this is a legitimate way to open PRs, > > but if it correlates highly with slop, then I think we should just ban it. > > I would have expected Claude/codex to be smart enough to see that > there is a PULL_REQUEST_TEMPLATE.md file and use that. In fact I just > checked this for > https://github.com/sympy/sympy/pull/29127 > I prompted codex with: > ``` > I want to make a PR with the python-flint typing related changes on > this branch. Can you write a description for the PR? > ``` > The first thing codex did was to read the pull request template and > then the commits and then it produced a PR description that matches > the template fully. A couple of notes about that though: > > In the references to issues section it did not include a reference to > the associated issue. This is a problem that I have been seeing a lot > recently, that a PR is obviously related to an issue but no link is > given to that issue. It makes sense because I didn't tell it about the > issue just now when prompting for the description. > > It also lied in the AI Generation Disclosure section and literally wrote: > ``` > #### AI Generation Disclosure > > Used ChatGPT to help draft PR text only. No code changes were > AI-generated in this PR. > ``` > All code in the PR was generated by codex (GPT-5.3) but with heavy > prompting and guidance from me. I'm not sure if codex can figure that > out because I did this in a new session so it won't remember what > happened before. I didn't use ChatGPT at all so both sentences in that > disclosure are false and it made them up without me asking it to do > that. > > It is very worrying that the AI will lie automatically about AI use on > behalf of the person using it.
We can reword the template so that it is also a suitable prompt for AI. Ironically enough, people probably won't read the template, but if an AI agent does decide to pull it up, it will actually read what it says. We can also add an AGENTS.md that asks agents to do something like always add a co-author line in the commit message (Claude does this automatically but I'm not sure if codex does). That also serves as an effective disclosure of AI usage. > > > I believe we can replace the template with a form. Or is that only possible > > for issues? > > I think it is only possible for issues. > > >> We probably need to have a bot that closes PRs if they don't match the > >> template or maybe it could be just that sympy-bot checks more things > >> since it seems most people seem to get the message about the release > >> notes check from sympy-bot. > > > > We can definitely get sympy-bot to do these checks. We can also have it do > > the closing if we want it to. > > That might be the best thing to do in terms of PR notification noise. > Then sympy-bot can keep complaining until the PR description is edited > to match the format. It's pretty trivial to make sympy-bot do anything (and ironically enough easy to change because of AI). Although I should also note that Heroku is going to stop existing at some point so we'll need to move it to something else eventually. Aaron Meurer > > -- > Oscar > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/sympy/CAHVvXxR7UUu0NfQ1WRYpvMOyk5_O32FrhvfzRFhh%3DGr_37WLHw%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6JB-8APLxdma7QQcmVuyyMhjjGDUqfKv4Pzzrk9-HjknQ%40mail.gmail.com.
