Julien Cabieces via QGIS-Developer <[email protected]> writes:
[I'm in favor of the stricter policy.] >> - LLMs may only be used as an improved auto-completion mechanism, or >> for repeated tasks (mechanical refactoring) that could potentially be >> completed with a deterministic algorithm. > > Shall we not also allow AI when it's used as a chatbot to answer > speific/technical questions. I see this as crossing a line that we don't today need to cross. One can separate "problems from LLMs arising from PRs" into: 1) effects on the project: review burden, low-quality code, code without adequately clear licensing, asking humans to read LLM-generated text 2) more general concerns: the ethics of using tools built with abusive scraping, proprietary/centralization concerns, environmental concerns Of course, different people see all of this differently. (I have run into people who think LLMs are beings worthy of being protected by a Code of Conduct, implying that those who think LLM contributions should be banned are discriminatory people who should be shunned by other humans.) So far QGIS (and gdal as insiration) are addressing category 1 only. This means human engages with an LLM to figure out a problem human comes to understand the code and larger issues human writes a patch and commit message human submits that to qgis is not a problem (not a category 1 problem). > I also seen contributors using it to identify where and how to fix an > issue. I've never tried it myself and don't know if it's something that > actually work. Would it be allowed ? Under the policy text I see, it would not be prohibited, if the human creates the fix after gaining understanding. If the patch comes from the LLM, it's not allowed. My own view on this got stricter after I and several others spent an hour reviewing a vibe-coded patch to an unrelated project. There was a lot of vague tutorial/generalities in the commit message and comments, unrelated code rototilling, and in the end we decided it was wrong. There's a further issue not addressed by the revised policy, which is LLM-generated text being sent to mailing lists, matrix room, etc. I have recently seen an instance (in an unrelated project) where someone asked a question (about a fairly difficult subject), and got a few generally-helpful accurate answers from humans. Then, a very long answer appeared, superficially appearing authoritative, but it got things wrong. Pretty obviously this was LLM-generated text. I see this as being in the same category as PRs and would suggest amending the policy to be - All contributions including code, ticket comments, commit messages, mailing list messages, and chat messages and should be fully understood by the author(s) submitting them to the project. except that I find the gdal text to be not strict enough. While the "mild LLM assistance is not prohibited if you end up with a contribution that is understood, accurate enough, and not overly verbose", my perception is that people who think it's ok to use LLMs think these problems are not serious and they're going to go ahead anyway. A big point is that it's not ok to ask humans to read text generated by an LLM. Perhaps that could be emphasized in the modified policy. Also, while prohibiting LLM usage, I think it's important to avoid prohibiting straight machine translation or grammar editing of text. The point is that if the translated text has the same semantics as the original, that's fine, but if there new semantics, it's LLM-generated text, not a translation. We should not exclude people who are not fluent in English as part of stopping vibe-coded PRs. _______________________________________________ QGIS-Developer mailing list [email protected] List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
