Hello Lucas and others,
I've been reading this discussion, but didn't reply so far (aside from
the message I just sent). I'd like to share my opinion as well, so here
it is.
Let's start with the big picture: my opinion is that LLMs are an
interesting technology, but their current effect on society is extremely
problematic. I think regular people (i.e.: everyone except scientists)
should avoid using them, especially for tasks for which other tools are
available, such as translations or writing code. Even if those other
tools are technically inferior.
When I talk about the effects on society, I agree with most points from
Johannes, and I'll add one more below. I also want to thank Lucas for
the respectful way that he handles this. (Side note: I am not offended
by the summary he posted; I think it makes sense in this discussion to
show what LLMs can do, especially because that was also being discussed.)
The "new" point (it was mentioned before, but not discussed much) is
about human brains: the way LLMs are presenting themselves, is that they
will do all your work. The way our brains respond to that is "let me see
what you can do". We closely check their work for a while and once we're
confident that they are indeed good at it, we scale down those reviews:
they are boring and apparently unnecessary. But LLMs can do a lot of
things right, and then mess up in ways that actual intelligence would
not. So this leads to things breaking, unless you are extremely careful
to avoid that, and I'm not even sure if our brains are capable of that.
What makes it worse, is that they claim that they can do the thinking
for you. Brains are always looking for shortcuts to spend less energy,
and thinking takes a lot of energy. So in the same way, unless we are
very careful (and again I'm not sure if our brains can even do that in
the long term), we are going to let them do the things that used to
require our thinking, and we're doing it less and less ourselves.
Apparently science has already shown that LLMs make people dumber; I
think that is this effect. When we are very careful, as Lucas suggests,
I think it may be possible to slow that effect. I'm not so sure if it's
possible to entirely stop it, and it worries me a lot that so many
people are stepping into this "trap".
Another related effect is that it removes creativity. I've heard from
multiple sides that when people started using LLMs, all their works
started looking the same, "soulless". Their games are no longer fun,
their texts are no longer interesting to read. This may not be the most
relevant for packaging (because a boring package is good), but for most
tasks that LLMs seem to be used for, I think creativity is important and
leaving all the "boring" parts to an LLM results in default choices for
all of those, which is very unpleasant to experience as a player/reader/etc.
Getting back to the main discussion: In terms of this GR, I think there
are two parts that are being discussed: a position statement and rules
for our project.
For the position statement, we do not need to worry about enforcing
anything. I would summarize my position, which I hope the project will
agree with, as: The way LLMs are currently organized in the world has
extreme costs that accelerate the destruction of the climate, both
natural and social, that was already ongoing before LLMs appeared. This
is extremely worrying, and I strongly recommend everyone to avoid using
LLMs that are controlled by big players on the market.
On 8/1/26 6:50 PM, Lucas Nussbaum wrote:
> Of course, the landscape of AI corporations is frightening. But
> there's hope to build alternatives, with open-source agents and
> open-weight or maybe even open-source models.
While those may be good in the future, I think that right now we should
have a position on the current state, which is that those corporations
are having a disastrous effect on the climate and they are pushing
fascism. And I don't think "it may get better later, so let's jump on
board" is a good response to the severity of the problems. It wasn't
this bad when the internet started; that didn't literally threaten our
ecosystem, for example. And while they were capitalists and therefore
fascism-related, it was nowhere near the level of people like Elon Musk.
When it comes to rules for ourselves (be it by changing the social
contract, or amending the code of conduct), I think it is good to set an
example that follows the spirit of the above summary. I do not think the
SC is the right document for that, as things may change in the future
and the SC is almost impossible to change. Also I don't think we need to
change it: I believe as it currently reads, it can already be
interpreted to mean this. After all, turning the planet into an
unlivable wasteland is neither good for our users, nor for free
software. So all we need to do is explain that we agree with that
interpretation.
Note that I do not think it makes sense to put sanctions on things we
cannot check. LLMs were initially designed to be turing machines and
that continues to be a large part of their focus. Saying "if you use an
LLM, we will punish you" is sure to lead to a lot of false positives and
false negatives. So I think the best we can do is say "you are not
supposed to do this", and trust people to follow our rules.
Also, while I would love to live in a world again where many people
avoid LLMs, we don't live in that world. So for practical reasons, we
cannot set a rule that upstream work must be made without LLM
assistance. We can still ask for it, but we must accept that not
everyone will follow our advice. I also do not think we need to tell
people that they cannot copy LLM-assisted work from a bug report, for
example. I'd just treat that as upstream work. As a DD however, I do not
think it should be acceptable to use an LLM with an upstream hat on and
then copy that into the package with a maintainer hat on, or to ask
someone else to use the LLM for you. I think it should be easy to
understand what is and what isn't in the spirit of the rules I propose.
And of course we have procedures for handling disagreements if it gets
that far.
I feel most of the discussion (with the exception of the mail I'm
replying to) has been people talking past each other: anti-LLM people
have mostly argued that the climate and fascism are the big problems,
while pro-LLM people have almost exclusively made the point that LLM
output can technically be of high quality. Both of these can be true at
the same time.
But to eleborate a little on that last point: they are only high quality
if they are used properly. Lucas explained how he works with them, and
that does sound like a good workflow (but I still think the it carries
the risk of losing brain power in the long run). However, on the
internet we see a lot of examples of people using them the wrong way.
That leads to them going crazy and breaking their software, among other
things.
> Also,
> both challenge teaching and evaluation practices (like calculators
> before), but that's a pedagogy problem, not a technology problem.
This is correct, but just saying that is not good enough. If we are
deciding that LLMs are acceptable to use (which again, I don't think we
should), then IMO we must set up systems to train people on using them.
Or else we are going to see a lot more broken packages.
Also, it's worth noting that it might be a bit early to weigh costs and
benefits of AI.
When talking about a total ban for all eternity, sure. But that doesn't
mean we can't do anything now. The current situation is extremely
problematic, and we are allowed to have an opinion on it (including
"don't use them"), even if the situation (and then also our opinion) may
change in the future.
So, despite all the costs you listed, I think that the Free Software
community should explore how to build freer, more independent and better
AI, even if not everything will be super-free and perfect from day one
I agree, which is why I made an exception for science. I do not think it
helps towards that goal at all when people are using LLMs in their
current state to search for facts, or to translate some text. Such uses
do still have all the downsides though.
Thanks,
Bas
