On 2026-06-08 02:56, Akira Urushibata via libreplanet-discuss wrote:
I discuss three issues in the Project Glasswing initial update
of May 22 in this message.
Reference:
https://www.anthropic.com/research/glasswing-initial-update
Project whatever within Antropic context I just guess it is proprietary.
Why not discuss free-software licensed large language model on this
list? That is appropriate.
How about you purchase one of the new Intel GPUs (Intel Arc A-Series
(e.g., A770)) and install some free software model on your computer and
try it out?
アントロピックのコンテキスト内で「whatever」を投影する。私はそれが proprietary
であるとおもいます。このリストでフリーソフトウェアライセンスの大容量言語モデルについて議論するのはなぜですか。それは適切です。
新しいインテルGPU（インテル Arc Aシリーズ (例:
A770)）のいずれかを購入し、コンピュータにフリーソフトウェアモデルをインストールして試してみませんか？
1. Disagreement in severity assessment between Anthropic and maintainer
The chart has boxes on the bottom line which say this:
1,586 findings Reported to maintainers
1,451 findings Acknowledged by maintainer
It is to me more or less irrelevant on mailing list for LibrePlanet as
it is proprietary.
それは私にとって、LibrePlanetのメーリングリストでは proprietaryであるため、多少関連性が薄いです。
The Anthropic team has discovered "3,900 high-or critical-severity
vulnerabilities in open-source code". 1000 projects were scanned.
You may discover it yourself using locally running models just by
putting attention on discovering. Developers discover daily thousands of
bugs.
That is their public relation stunt.
That doesn't mean they are truly supporting free software, they do it to
draw attention on their proprietary language models.
Best to do is not to give them attention and to build ways so that each
user can freely run free software models on their own hardware.
ローカルで実行されているモデルに注意を向けるだけで、あなた自身が発見できるかもしれません。開発者は毎日数千のバグを発見しています。
それは彼らの広報用の演出です。
彼らが真にフリーソフトウェアをサポートしているわけではありません。彼らは自前のプロプライエタリな言語モデルに注意を引くためにそれを行っています。
最も良いのは、彼らに注意を与えず、各ユーザーが自身のハードウェア上で自由にフリーソフトウェアのモデルを実行できるようにする仕組みを整えることです。
3. Quality of code written by "AI"
The security-related findings by Claude Mythos seem to show that
"AI" is getting better at writing code.
On the contrary we have reports like this which claim that code
produced by "AI" often introduces vulnerabilities:
Report finds AI-generated code poses security risks
July 30, 2025
https://www.eenewseurope.com/en/report-finds-ai-generated-code-poses-security-risks/
Instead of generalities, better would be you do it yourself on your own
computer.
Hermes Agent — The Agent That Grows With You | Nous Research:
https://hermes-agent.nousresearch.com/#features
Once you install Hermes, and purchase the Intel Arc cards (maybe 2 of
them?) then put some of free language models, and run your code
generation yourself.
There are many ways how local model with good agent software can
generate good code.
Veracode has unveiled its 2025 GenAI Code Security Report, revealing
critical security flaws in AI-generated code. The study analysed 80
curated coding tasks across more than 100 large language models
(LLMs), revealing that while AI produces functional code, it
introduces security vulnerabilities in 45 per cent of cases.
Veracode’s statistic — “45% of AI-generated code contains security
flaws” — sounds alarming, but it’s based on 80 curated coding tasks
across “more than 100 LLMs.” That’s a tiny, narrow sample relative to
the millions of models (fine-tuned variants, task-specific, open-source)
on Hugging Face. It cannot support a general claim about “AI-generated
code” writ large.
LLMs are not designed for secure-by-default coding. They are designed to
predict tokens by probability, nothing more. If people find useful
applications for them, that's fine — but expecting probabilistic text
generation to resolve actual security reasoning for humans is setting
the bar impossibly high. It's akin to expecting Joseph Dunninger or
Houdini to resolve a cryptographic vulnerability: their craft is
illusion and showmanship, not engineering rigor. The problem isn't that
LLMs fail at security — it's that we keep asking a probabilistic parrot
to be a security auditor.
Finally, it is not AI. Text generation alone doesn't make it
"intelligent" in anyway.
Using the agent software in combination with the LLM I would say gives
feeling of intelligence, yet doesn't reach it.
The research shows that despite advances in AI-generated code and
the ability of LLMs to generate syntactically correct code, security
performance has not kept up, remaining unchanged over time.
Professor, why not run the model yourself?
It is not good duplicating from Internet some statement, and without
personal experience, without specifics, to simply make claims that are
not even meant to exist in the first place = models were trained on
code, but not with purpose to solve security performance.
If you think of generated code and generated text as "draft text or code
proposal" that is better attitude, that way you have got the draft to
edit and verify it as user.
We may rely on generated code in future, yet, today, expectation is too
high. We have to know that any generated output is just a draft, nothing
else.
There is no model or company so far providing the LLM to resolve your
security issues.
Finding security issues is one thing, that may be good domain for the
LLM.
Resolving security issues is quite different one.
Both the MIT and Apache 2.0 licenses contain explicit disclaimers
stating that the software is provided "AS IS" with no warranties of any
kind, including warranties of "FITNESS FOR A PARTICULAR PURPOSE".
All proprietary LLMs also have disclaimers of similar kind.
It is in first place wrong to have expectation that any model or their
authors ever intended the purpose of resolving security issues. No. They
have only trained the model and model can generate probabilistic
answers. There are many ways how to use such generation, yet, without
warranted purpose.
Another
concerning trend is that when presented with a choice between secure
and insecure coding methods, GenAI models opted for the insecure
option 45 per cent of the time.
They are meant to generate text, not to be smart, no matter the
illusion.
The purpose of a university is to teach students, not to guarantee their
success in life. Some students will thrive; many will not. A few may
even face devastating outcomes like unemployment, addiction, or suicide.
But no one blames the university for those failures — because that was
never the university's purpose. Its purpose is to provide knowledge,
tools, and an environment for learning. What students build with that is
their own responsibility.
The same applies to large language models. Their purpose is to generate
probable tokens based on training data, not to guarantee secure,
correct, or fit-for-purpose code. If a developer takes LLM output and
deploys it without review, the failure is not in the model's purpose —
it's in the misuse of the tool. Expecting an LLM to resolve security
issues is like expecting a university diploma to resolve life. Both are
aids, not guarantees.
Finding a large number of vulnerabilities is not the same as a
guarantee that all vulnerabilities will be found. "AI" can write code
and scan code for problems but that gives us no guarantee that it will
be always aware of problems in its output or input.
Correct. And no one promised a guarantee — except the people selling "AI
security solutions."
Some people are interested in this. They feed old versions of a
program with a well-known issue and test whether the LLM can actually
find it. There is no mention of tests of this kind by Anthropic in
the initial report.
Exactly. Testing on old, well-known vulnerabilities is not real research
— it's just checking a box. The absence of such tests from Anthropic in
the initial report is not an oversight. It's because those tests tell
you nothing about a model's actual capability to find novel or subtle
issues. They're for marketing, not science.
--
Jean Louis
_______________________________________________
libreplanet-discuss mailing list
[email protected]
https://lists.libreplanet.org/mailman/listinfo/libreplanet-discuss
