On Sun, Feb 22, 2026 at 7:07 PM Oscar Benjamin via NumPy-Discussion < [email protected]> wrote:
> On Sat, 21 Feb 2026 at 13:38, Oscar Benjamin <[email protected]> > wrote: > > > > Another example is that I recently made this PR using codex (GPT-5.3) > myself: > > > > https://github.com/sympy/sympy/pull/29127 > > I just tried asking codex to write me a description for that PR. > > The prompt (in a fresh session) was > ``` > I want to make a PR with the python-flint typing related changes on > this branch. Can you write a description for the PR? > ``` > Then codex went and looked at the PULL_REQUESTS_TEMPLATE.md, looked at > the commits, and then produced a PR description matching that > template. It filled out the AI disclosure part of the PR template for > me > ``` > #### AI Generation Disclosure > > Used ChatGPT to help draft PR text only. No code changes were > AI-generated in this PR. > ``` > Both of those sentences are false and it just lied automatically on my > behalf without me asking it to do that and without asking for any > clarification about what to put there. > Based on what you wrote, that seems like user error to me. The commits on the branch you made the PR from do not include the `Generated-by` or `Co-authored-by` attribution to indicate that those commits were generated by an LLM in part or in full. So if you ask Codex in a fresh session, where it doesn't have context about the previous work, to look at that branch / those commits, how is it supposed to know that the commit authorship on those commits is in fact incorrect? It's indeed possible that there is a model that deliberately and systematically lies in order to increase the chances of it being accepted, but it's much more likely that the PR message draft you ask for is actually correct based on the commit history. I just tried based on a branch with 4 commits that were all mine, and one with `Co-authored-by`. First I just asked it to draft a PR message in the session where it had the development context for those commits. It did, short and to the point. Then I gave it more context: Prompt: "Additional content: the upstream repo uses the following PR template:" Output ends with: ``` #### AI Generation Disclosure AI tools (Claude via Warp) were used to generate the initial workflow files and iterate on them based on review feedback. All code was reviewed and tested by the author. ``` Note that it includes actual details of the interaction that aren't in the commit messages (Claude via Warp, the nature of the actual interaction). Next I repeated it in a fresh session. It gave me a much more verbose PR message, more generic-AI sounding with "files changed" etc. I then again fed it the whole SciPy PR template. And it produced this: ``` AI Generation Disclosure Parts of this PR (the CI workflow files) were co-authored with Warp AI. All code was reviewed and edited by the author. ``` Note that it still kept "Warp AI", which it got from the Co-authored-by, but dropped the "Claude" part, which isn't in the commit message. As for the "All code was reviewed and edited by the author.": it can't really know if that is 100% accurate, but based on me being the sole author of 3 of the commits and the co-author of the 4th commit, it seems like the most likely interpretation of the commit history. tl;dr seems to work as advertised. And inaccuracies and omissions are still the responsibility of the human in the loop.
_______________________________________________ NumPy-Discussion mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/numpy-discussion.python.org Member address: [email protected]
