gravitystorm left a comment (openstreetmap/openstreetmap-website#6723)
Thanks for the suggestion, but I'm not convinced about merging this, for a
number of reasons. I speak with experience of using LLMs (specifically, Claude)
in multiple projects.
* Each LLM/agent has its own preference for where to put instructions, e.g.
CLAUDE.md or AGENTS.md. I don't think there's really any conventions yet.
* The contents of the agents files are usually just regurgitated versions of
the instructions found elsewhere in the repo, without much original content.
This makes them more of a "generated" or "compiled" source of knowledge, and
such artefacts aren't generally stored under source control.
* If the agents file contained original content (e.g. telling it how to do
something, but we had no other documentation about how to do that) then it
would be a candidate for source control. But I'd be really really curious why
we would be telling the agents about something that we're not telling everyone
else about!
* I don't want to end up with 10 different agents files, one for each LLM/agent.
* However, it's not clear what value there is giving them all a common agents
file, rather than each agent generating their own files themselves, which will
(perhaps) match their specific instructional needs more closely.
* Since everything moves so quickly, it's not even clear how to maintain these
files. Would we have to regenerate them every month or two when there's new
releases? Do we treat them as diffable text, or will the output each
regeneration be completely different making diffs meaningless? Do we edit them
manually when we add a new linter?
It's different for organizations that have many developers, all using the same
version of the same agent, where it saves each developer a few seconds (e.g.
waiting on `/init` run) when working on a common codebase.
I'm not even really convinced that the concept of an AGENTS file has any
longevity, as LLMs develop. Between internal knowledge (e.g. most LLMs will
know how to run test suites) and human-orientated documentation (e.g.
INSTALL.md) then it seems to me like the agents files are just a minor
implementation detail and could in most cases (and very likely so in ours) be
hidden from developers.
But in summary, since this stuff is all so new, I'm happy to be persuaded that
I'm wrong on any of the above!
--
Reply to this email directly or view it on GitHub:
https://github.com/openstreetmap/openstreetmap-website/pull/6723#issuecomment-3767872697
You are receiving this because you are subscribed to this thread.
Message ID:
<openstreetmap/openstreetmap-website/pull/6723/[email protected]>
_______________________________________________
rails-dev mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/rails-dev
