For code, I have several concerns:
1. The provenance [place of origin, history] is lost when you use AI
to write code for you. This is significant because AI tends to repeat
its training data.
This isn't a made-up theoretical example. For example, the US
Copyright Office notes that “Generative AI models sometimes output
material that replicates or closely resembles copyrighted works” and
“Such outputs likely infringe the reproduction right and, to the
extent they adapt the originals, the right to prepare derivative
works.” (Copyright and Artificial Intelligence Part 3: Generative AI
Training. United States Copyright Office. May 2025 report, p 31.)
Here's where that matters: If you're using AI to write some code for
you, did that chunk of code replicate someone else's code? For a small
contribution (a few lines) that may not be much of a concern; there
are only so many ways to open a file and read it.
But for larger code contributions ("vibe coding") the risk becomes
much greater that AI will inject someone else's code without you being
aware of it. Let's say you're forking a program that is under the MIT
license, and you use AI to "vibe code" a huge chunk of new features.
Without knowing it, AI has generated some code that copies some other
code (that it was trained on) that is under the GNU GPL. You can't
insert GNU GPL code into an MIT-licensed project. (You can insert MIT
code into a GNU GPL project, but not the other way around.) Now you
have a license issue.
In a more extreme example: Let's say that "vibe coding" session copied
some proprietary code (the AI may have been trained on leaked source
code, which is out there). Now you have a different kind of license
issue. No one wants to be the next "SCO vs. Linux" case.
2. At least in the US, the courts and US Copyright Office have held
that the more you use AI to make a thing, the less strong your claim
on copyright.
The US Copyright Office May 2025 report (see above) summarized their
understanding of Gen AI and how it works, including opinions about
whether or not Gen AI output is sufficiently “transformative” to
bestow copyright. This is a long report, but essentially they advise
that AI-generated contributions on their own are not subject to
copyright. And without copyright, you can't actually claim a license,
because all licenses (GNU GPL, MIT, BSD, Apache, ..etc) rely on
copyright.
In one well known (at least, in my circles) someone used AI to
generate a digital "painting," titled "Théâtre d’Opéra Spatial." He
entered it into an art fair, and won, but the US Copyright Office held
that he was ineligible for copyright because only humans can claim
copyright. (Related, there's another case involving a monkey selfie
and PETA tried to say "The monkey has the copyright" and the courts
clearly said "No, only humans can claim copyright.")
The US Copyright Office doesn't say all AI contributions are
ineligible for copyright, but the more you use AI ("vibe coding") the
less eligible you are to claim copyright. Small stuff, you're probably
okay. The more you use AI to write something, not likely.
If you vide coded a compiler by feeding AI the C Programming Language
specification, and directed AI to create test cases based on that spec
and to create code that addressed gaps and issues, I'd say you've used
AI to the point that you can't claim copyright. And that means you
can't say "this is GNU GPL" or "this is MIT" or any other license,
because you need copyright to claim a license.
3. I'll also add that I've experimented with AI for coding, but only
to see what it would do (I don't use AI for coding anything related to
FreeDOS or anything else I would release online.) In one experiment, I
asked AI to write a formatting program like nroff. And it spat out
some very unsafe code.
That one was an interesting case because I recognized the code as
something I wrote in an article. It was a very recognizable chunk of
code, and I think AI put more priority on it because it was more
recent than other nroff-like code it might have been trained on. But
in the article, I told the readers that this wasn't the best way to
write the program, and it would break under certain edge cases
(memory, string bounds, etc) but it was the simplest to explain to
nonprogrammers. And that's okay for the article because I wasn't going
to hit those edge cases. The AI generated code did not come with that
warning.
On Sat, Oct 3, 2026 at 6:24 PM Aitor Santamaría via Freedos-devel
<[email protected]> wrote:
>
> Hello,
>
> This leads me to question what is the main concern about AI-generated code
> that we are posing as an open source community.
>
> It has been pointed out in the thread that AI can make mistakes, introduce
> risk for prompt injection, etc. It is true, and thus, it would require lots
> of human testing and supervision.
> But that is also true of human produced code.
> Everytime there is a bug, it means that the code hasn't been proved
> thoroughly enough by someone.
> There is no cure for this, other than automatic testing (for which AI may
> help), as well as being open source, which implies that anyone can see the
> code has been produced, and point out defectuous (even malicious) software
> (for which, by the way, AI may also help to discover).
>
> Second, I understood from previous threads that the most worrying issue with
> AI-generated code is that, being unclear where the LLM manufacturer has
> trained the engine from, you may be incurring into copyright issues, as
> produced code may have been excerpted from a non-open source code repository.
> It sounds to me like a reasonable argument against AI-generated code, which
> is difficult to control, especially if the LLM used is something like ChatGPT
> or Gemini or Claude, which may have not taken enough care for this issue, or
> worse, may at sometime claim some kind of ownership in the produced code (I
> have not had an in-deep read of the LLM using license).
> We will have this problem until there is a LLM that is good enough to produce
> decent code, and has been trained with such a guarantee (I can imagine some
> open source initiative that does this (I am not a LLM expert, so this may
> already exist).
> I point this out after Gregory's mail, because in this case, having a human
> check the code after producing is not enough.
>
> Incidentally, it has been mentioned that AI is not good for code, but good
> for translations. I have my issues with this argument, because I have always
> considered documentation as software. I don't know why it is "safer" from the
> concerns above about something written in French or Spanish than something
> written in C or Pascal.
>
> Speaking about other user cases, I suppose that it would be safe to use AI to
> assist in testing, as well in reading and analysing code ("tell me if you
> find any serious issue within this code").
>
> To sum up:
> I don't have a clear idea about our FreeDOS community statement about AI used
> for generating code. The second argument (potential copyright issues) looks
> concerning to me, whereas the first argument of code stability/quality does
> not make much sense (in the sense that it introduces any special concern that
> did not exist for human-generated code). Unless I am missing something.
>
> Thanks for your attention and open for comments,
> Aitor
>
_______________________________________________
Freedos-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/freedos-devel