Hello, On Tue, Jun 02, 2026 at 07:55:58PM +0200, Bruno Haible via Gnulib discussion list wrote: > Anyone out here who is familiar with LLMs (or wants to get familiar with > LLMs): How about using it not for coding, but for checking commits that > went into gnulib master? > > Since 2026-01-01, at least 17 gnulib commits contained regressions, that > had to be fixed subsequently. We often detect regressions by code review > or by a CI run. The problems: > - Not all commits gets reviewed from a different developer than the > committer. (Like many free software projects, Gnulib lacks good > reviewers.) > - The CI runs possibly a week later. (We can't increase the frequency, > because some CI runs fail due to network problems or other noise, > and this noise needs to be filtered out.) > > As a complement to these QA techniques, Paul Eggert suggests to use an > LLM to analyze the commits that have been pushed into gnulib master. > > This should be promising, because I read recently that LLMs outperform > all classical static analysis tools, when it comes to analyzing source code. > > I can't do this myself, because I'm already quite loaded with the existing > QA techniques and with my work on other GNU packages. > > Therefore, if you volunteer, please step up!
I can run the analysis manually as a first try. Since I don't know which commits are these 17 that contained regressions and I don't follow gnulib development that closely, I think I am a good candidate to perform a double blind experiment. Perhaps you could start by providing some commits to analyze (some of them that contain known regressions, some of them that don't - without indicating which one is which) to have a smaller set than e.g. all commits since 2026-01-01? Regards, Pavel
