Hi,
thanks for the great web site with all those bugs. I scrolled through
them and of course stopped on the MMapDirectory related ones. The one
with reporting false offsets on random access multi-mmap was personally
known to me, but I never decided to fix it, because the error message is
valid, just a wrong offset reported, which may prevent easy debugging,
but on other hand to have multi-mmap indexes, the index segment must be
larger than 16 GiB and that no longer happens unless you force merge.
Therefor I'd ignore this one.
The shared arena one looks very relevant and I always had the "feeling"
that something is wrong with them. It was also reported to me privately
by Gianluca Graziadei on/after Apache Community over Code already (or at
least what I got from the report is closely related as it causes similar
symptoms). I don't understand the issue fully, so maybe Chris Hegarthy
can help me fixing it.
Uwe
Am 27.10.2026 um 19:42 schrieb Shitij Bhargava:
Hi all,
I work in Amazon advertising where we use Lucene for a lot of our ad
candidate retrieval services. We also develop custom codecs and other
customizations for cases where the stock Lucene options aren't enough.
While developing one of them, I asked an AI agent to find bugs in my
POC implementation, and it ended up finding an unreported bug in
Lucene itself.
I kept going with this approach and it kept finding more unreported
bugs, so I've gone deeper into AI-assisted bug finding with formal
verification and other techniques. As of today the agents have found
592 bugs in Lucene (and counting) that appear to be unreported in
GitHub or the mailing list. It also found (or already knew from its
training data) nearly 200 bugs that have already been reported, which
I'm not counting here. To avoid noise in these findings, the approach
has each potential bug independently and adversarially verified by a
second model (Claude vs GPT) with a minimal unit test repro and making
sure the behavior is not intended from past discussions in mailing
lists, tests, GitHub discussion & commit/PR history.
Many of them seem to be minor or low impact (392 low, 196 medium, 4
high classified by AI), but a few look important. For example:
- After a flush where the only changes are doc-values updates,
openIfChanged returns null and isCurrent() stays true so the update
can be missed: https://github.com/apache/lucene/issues/16900
- HNSW merge-time graph reuse can leave the entry node with no
neighbours, which drops kNN recall to zero in 10.4.0 and later:
https://shbhar.github.io/lucene-ai-bug-hunt/bugs/LFV-005.html
I've posted the full running list with some categorization here:
https://shbhar.github.io/lucene-ai-bug-hunt/. I've excluded a few that
might be security sensitive according to AI, will send them privately
to security@ soon.
I'm working on understanding, triaging and verifying them myself, then
filing issues on GitHub in my own words in line with Lucene's AI
policy (so far I've filed only one). The agents help by preparing
minimal repro tests, diagrams and short explainer videos (not on site
yet) for surrounding context, but understanding and verifying each bug
properly still takes a lot of time, and the community's help would
make a big difference.
Creating this thread to start a discussion on how to validate and make
these findings actually useful, in particular:
- Could folks help in spot-checking some bugs for areas they might be
familiar with?
- Is it beneficial to track this work in an umbrella github issue?
Thanks,
Shitij
--
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
