On Wed, Oct 08, 2025 at 08:35:39AM +0200, Paolo Bonzini wrote:
> Comparing the results from RightTyper and AI, both tools benefit from human
> help: the dead code removal which I mentioned, the small cleanups in patch 1,
> the final manual fixes for the remaining (mostly trivial) errors. But at
> least in this case, AI is a winner:
>
> - it left basically no unannotated code: fixing the above errors was enough
> to pass "mypy --strict", unlike RightTyper which needed a little more work
> due to its profiling-based nature and a few other limitations (see patch 5).
>
> - most importantly, trying various other tools that didn't work, as well as
> figuring out how to use RightTyper, took a couple hours more. Surprising
> as it was, I could not find any static type inferencing tool for Python;
> neither pytype nor pyre worked for me. This is also why I think this
> is not apples to oranges, but a fair comparison between AI-based and
> regular tooling.
FWIW, there was a third possible option here, a sort of hybrid...
prompt> find me tools that can add python type annotatinos to code
prompt> show me how to invoke tool <blah> on the QEMU codebase.
prompt> run the tool <blah> on the QEMU code, run mypy tests and
commit the result
that would probably have saved the hours learning how to use RightTyper
making it more competitive with the pure AI approach.
> After the diffstat, you can find a diff from this series to the version
> based on Claude Code. It's impossible to be 100% objective but,
> besides being functionally equivalent, I don't think either would be
> identifiable as written by an LLM, by a person, by a tool+human combo,
> or even by a good type inferencing tool (if it existed).
AFAICS the diff there is either whitespace changes, or a differing
style for importing a module vs importing a name from a module.
Whitespace we should be enforcing with 'black' formatting rules.
For import style, I'd say that's an irrelevant difference.
IOW, I'd call the two patches identical for any practical purposes.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
