Hi Lucas and others,
Lucas Nussbaum <[email protected]> schreef op 3 augustus 2026 14:08:40 CEST:
>Hi Bas,
First of all, it worries me that mails which say 'climate change is the
primary issue, but also X and Y' get responses about X and Y, but not
about climate change.
IMO you (Lucas) make a fairly convincing point that LLMs are more than
'planet-burning slop extractors': if careful, I believe you that it is
possible to create non-slop. However, I have seen no serious attempt (by
you or others) to claim that they are not planet-burners.
I don't understand how anyone can agree that LLMs destroy our ecosystem
and not feel like that is a deal-breaker.
The only response I've seen to this point was that other activities are
also bad (some even worse) for the climate. How is this even an
argument? The planet is burning, we should try to stop that as much and
as fast as we can! If others aren't stopping, isn't that even more
reason to compensate for their irresponsible actions?
And while I'm on that point: All of our users live on this planet, so
protecting that planet is a matter of life and death for them. In other
words, caring about something as big as climate change during our Debian
work is not (as was suggested elsewhere in the discussion) a violation
of the social contract. On the contrary, I would argue that the social
contract requires us to care about climate change and prevent it as much
as we can.
Anyway, while climate change is by far the most important point IMO,
I'll also reply to your answer:
>On 02/08/26 at 09:28 +0200, Bas Wijnen wrote:
>> The "new" point (it was mentioned before, but not discussed much) is
about
>> human brains: the way LLMs are presenting themselves, is that they
will do
>> all your work. The way our brains respond to that is "let me see
what you
>> can do". We closely check their work for a while and once we're
confident
>> that they are indeed good at it, we scale down those reviews: they are
>> boring and apparently unnecessary. But LLMs can do a lot of things
right,
>> and then mess up in ways that actual intelligence would not. So this
leads
>> to things breaking, unless you are extremely careful to avoid that,
and I'm
>> not even sure if our brains are capable of that.
>>
>> What makes it worse, is that they claim that they can do the
thinking for
>> you. Brains are always looking for shortcuts to spend less energy, and
>> thinking takes a lot of energy. So in the same way, unless we are very
>> careful (and again I'm not sure if our brains can even do that in
the long
>> term), we are going to let them do the things that used to require our
>> thinking, and we're doing it less and less ourselves. Apparently
science has
>> already shown that LLMs make people dumber; I think that is this effect.
>> When we are very careful, as Lucas suggests, I think it may be
possible to
>> slow that effect. I'm not so sure if it's possible to entirely stop
it, and
>> it worries me a lot that so many people are stepping into this "trap".
>
>Some comments about this:
>
>I see the brain analogy as mostly PR (originally from Anthropic, I
>think).
I'm not sure what analogy you're talking about; I'm talking about actual
human brains and how they react to using LLMs.
>At some point, some people described LLMs as "super-fast interns", then
>as "assistants". But I find that the analogy that works best is the one
>of the exoskeleton: it's a tool that extends the user's powers, but the
>user still decides what it does (unless they decide not to decide on
>some details, which is essentially what they do when vibe-coding).
I agree with this. My point is that our brains tend to prefer to limit
their own thinking. Because of this, I expect many people to jump
straight to vibe-coding and others to move in that direction over time.
>I'm not sure that LLMs make people dumber. My feeling is that the
>problem is that LLMs make dumb/simple/automatable activities obvious. As
>a consequence, when people try LLMs, first there's excitement at being
>able to automate some of one's activities, but then there's Fear Of
>Becoming Obsolete: if some activities can be performed just as well by
>LLMs, what essence does one's job retain?
I think this fear exists for some people, but I'm not one of them. My
issue is not with automation, I love that part. My issue is that it
doesn't work well. You described before how much effort it takes to get
the LLM to output good code. Non-LLM-automation is not as powerful, but
you also don't need to keep checking its output every time. For LLMs you
do, and most peope are not going to (at some point, even if they did
initially).
>Of course, the fact that more activities can be automated transforms
>jobs. The future of software engineering is probably more about
>engineering than about coding craftsmanship: coding can be largely
>automated, so what remains are questions about general strategy,
>technological choices, specification and consistency with requirements,
>design, architecture, test, verification, codebase consistency, etc.
That's how it should be, but I disagree that LLMs are that road. Again,
you need to keep checking whether LLMs didn't hallucinate. So instead of
spending all your time on the high level tasks, you have to spend it on
reviewing code which is mostly correct (but the mistakes can be in
unexpected places). And if you find a problem, you don't even get the
satisfaction of learning something new (or seeing someone else learn
something new). So it's very boring and deeply unsatisfying.
>There are two consequences to this:
>1/ Some people are attached to doing manually what can now be automated
>(similarly to manually repairing your car even if it's more time/cost
>efficient to have a professional do it for you). I think that's part of
>Johannes' "taking my hobby away" argument. As a volunteer-based
>organization, it of course makes sense for Debian to respect one's
>willingness to perform tasks either the traditional way, or the
>modern/efficient ways.
I would agree with this, if I would not have climate objections and if I
would agree that it would result in high quality work (as a rule, not as
an exception). Neither of those are true, though.
>2/ Mental fatigue: as the "easy" tasks are now automated, what remains
>are mostly mentally-challenging tasks. I don't think that's an issue for
>Debian as most contributors only spend a few hours per day or week on
>Debian stuff, but it is clearly a problem in professional environments.
I think this is a capitalist/exploitation issue. If this is handled
properly, a full time job is fewer hours for that kind of work. (Not
that the CEOs of large corporations are going to agree to that...)
For volunteer work, as we do in Debian, it certainly isn't a problem to
just work fewer hours if we still get the same or more work done.
>> Another related effect is that it removes creativity. I've heard from
>> multiple sides that when people started using LLMs, all their works
started
>> looking the same, "soulless". Their games are no longer fun, their
texts are
>> no longer interesting to read. This may not be the most relevant for
>> packaging (because a boring package is good), but for most tasks
that LLMs
>> seem to be used for, I think creativity is important and leaving all the
>> "boring" parts to an LLM results in default choices for all of
those, which
>> is very unpleasant to experience as a player/reader/etc.
>
>I think it's useful to distinguish between true creativity, and what I
>would call "industrial creativity". Coding is often an "industrial
>creativity" task, where one re-does stuff that has already been done
>elsewhere, with some adaptations (to context, or change of programming
>language, etc.).
My point is that even things that feel industrial still affect the
result in ways that you may not expect. Even if you don't realize it,
you make lots of choices during 'industrial' work.
What makes it worse, is that LLM users typically think the result is
better than it actually is; they believe the bot at least a little when
it praises them: it has been shown that when people use an LLM, they are
more confident (and thus less likely to ask for help), but the quality
of their work is lower.
And some believe it when it tells them they're the new messiah, of course.
Thanks,
Bas
