my turn... Various unordered thoughts
- My personal experience as a "practitioner" with LLM has been
essentially limited to chat bots, and experimenting with LLM code
reviews. The today's online chatbots are quite good when you're an
expert in the field where you query them and can have a critical eye to
assess their suggestion. They might be practical you need to do
throw-away coding for things you have no clue about. I guess that's the
end of the positive part of this message :-)
- I tried a couple months ago making a few "open" models work on my
computer, a laptop from 2020 with 32 GB RAM and 4 GB graphics memory.
Just testing chatbot experience, that "worked" in the sense, it had the
performance of a ChatGPT of 3 years ago that is good at making you
laugh, and at a very slow pace.
- In a project like GDAL, bumping something as the version of the C++
standard generally lead to some heated conversation about if it is
really needed, if it is the good time to do so, if we couldn't defer by
a few more years because you know someone still needs to build a modern
GDAL with a 10+ years compiler. Technical conservatism is kind in the
DNA of the project, so adoption of breakthrough technology is not
something in our habits
- We (GDAL) had a burst of low value/low quality vibe coded
contributions towards the beginning of the year, with some strong
correlation with Google Summer of Code preparation (which the project
doesn't participate at. We issued a first version of an AI tool use
policy that was mostly inspired by the LLVM one whose philosophy is
"there must be a human in the loop". That wasn't very effective. We had
to considerably harden it:
https://gdal.org/en/stable/community/ai_tool_policy.html . I strongly
suspect that some PRs we receive don't follow those guidelines, but hard
to prove and humanly messy to deal with.
- We also add a burst of LLM based vulnerability reports, but quite
modest regarding the size of GDAL and its huge vulnerability surface.
Likely reason: GDAL has been enrolled in OSSFuzz since many years and we
already have fixed hundreds of issues. I'm not sure if we're completely
done on the front of LLM based vulnerability reports though.
- The point of failures between LLM based contributions and human based
ones are different. It is very difficult to debunk errors in LLM based
contributions, because you get comments all over the place which tend to
make your critical eye asleep.
- One recurring trait I've observed is overly verbose tests compared to
what a human do. Like 3 times more verbose. It is likely humans don't
write enough tests, but it is not like more tests is always better. They
also add to technical debt. Anectodal evidence of that: many years ago
(well before AI was a thing), one contributor nearly spent one year
refactoring the whole GDAL test suite from its home-made framework to
pytest (I guess some could say: an agent would do that in a few
hours/days nowadays)
- AI bot scrapping has visible effects on our infrastructure.
ReadTheDocs which hosts gdal.org and proj.org documentation was (and
apparently still is) victim of DDOS, and had to turn on CloudFare human
check tests to access pages, for a whole range of IP address, which
includes mine. I assume AI bots must be behind that. End result: some of
our resources, like JSON schemas, that are supposed to be accessible
through a simple curl request are no longer accessible by a portion of
users.
- The amount of contributions by some people who adopt agentic AI is
strongly demultiplied, at least in quantity. On a couple examples
recently seen at looking at their github activity, I've seen 10 or 16
times bumps between 2024 and 2026. I guess that this could seen as a
win, but that can strongly modify project dynamics if some developers
adopt it at scale, and others don't
- I'm *very* concerned by the environment impacts of those technologies,
at least those which are used in practice today and are in the hands of
a few big players. They are clearly not the only responsible of all our
sins in that domain, but it is again one more thing humanity didn't
fundamentally need and adds to its long basket of things that are
heading towards the wrong direction, and are likely to be very hard to
compensate.
- What could be an acceptable AI for me ? Maybe something like:
* that has been trained on material whose licensing allows to do so
and/or material whose authors have explicitly approved used for LLM
training (some explicit tag in /robots or equivalent)
* where scrapping is done at a reasonable rate, not effectively
DDOS'ing websites
* you can make run inference / agents standalone on your own
average computer with normal hardware specs
* whose environmental impact is similar to normal computer uses
Even
--
http://www.spatialys.com
My software is free, but my time generally not.
LLMs contribute to global warming and brain rot
_______________________________________________
Discuss mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/discuss
