FWIW, I have been experimenting with Opus 5 Ultra in the last few months, and to my surprise and in spite of being very skeptical of LLMs, I’ve got very impressed with what you can achieve with it. So, in as far as I'm concerned, there are no technical reasons to reject whatever was produced with it if you ensure that the output is the same or better as you would have done completely on your own, and can put your name on it… it’s a bit like asking if the code written with PyCharm has any technical disadvantages as compared to what you’d do in vim.
However, it’s going to be nothing like what everybody is raving about, as in “bruh I just vibe-coded a full stack application with this and that by giving a few prompts”. It’s simply a very different type of development, and you’ve got to constantly read what it does and be on top of everything, or it will quickly go sideways. And once you let it slide, it’s extremely difficult to make sure that you can trust the results, even though they will superficially look flawless. What worked for me was to write down what I want to see in terms of code, documentation, and architecture in CLAUDE.md, and iteratively refine it by asking it upon every violation why it did it and what I can do to stop it from doing it. In the end, funnily enough, I’ve formalized my decades of experience in a nice document I can just give any human developer working for me as guidance. Another crucial part was to keep iteratively building the tooling that mechanically prevents it from doing bad stuff. Prompts and guidelines are soft guardrails, and these probabilistic models are terrible at doing deterministic stuff. I’ve now got dozens of different AST walkers with elaborate rule sets which are applied to every change, and Claude has to win a fight against them, before I even look at the results. Finally, I figured out not so long ago that somehow iterative development works badly and the best way to get good results is to first put it in “creative” mode, and then have it “review” its own mess. Mostly I don’t even have to do multiple iterations to get to a fix point. What also helps a lot in terms of reviews is the ponytail rug ladder; this really does suppress its creative diarrhea. In any case, the central question for me is the question of trust, and I’ve come to realize that it’s no different than dealing with human developers who used to work for me. Only the disparity between our productivity used to be so high that I could be constantly reviewing streams of work from 15 people, and now I’m getting mentally exhausted by trying to keep up with one Opus. So my previous approaches didn’t scale, but I wasn’t in a position to notice it, and now I’m constantly balancing on the verge of falling behind or losing momentum - as I refuse to lower the bar. Unsurprisingly, I first went into the obvious directions of just enforcing classic quality standards to degrees never achievable before, and it was quite crazy to be able to experience their limits, as a few bugs crept through even with 100% line and branch coverage, full mutation testing, and fuzzing. Now I’m moving more and more towards formal verification, and it’s fascinating what becomes accessible in practice without being part of the S3 team at Amazon and having Jeff bankrolling your experiments… I only wish I could get paid doing stuff like that. I’ve also got a few interesting learnings in terms of token efficiency, but if you have a Max account, I can only envy you :) - you probably don’t need them. Hope that helps. > On Aug 19, 2026, at 15:47, Ronny Pfannschmidt <[email protected]> > wrote: > > Hi everyone, > > as i currently have access to claude max, > i'm taking the opportunity to try and do something positive, human > communicated/driven for projects i love > > i started playing around with this on > https://github.com/RonnyPfannschmidt/pypy/pull/1 > > where i steer claude to get rpython to import on python 3 without actually > fixing anything about actually running on python3 > > my rough plan for this experiment is to > > 1. import clean, world is broken > 2. run the testrunner on python3.x without setting everything on fire > 3. run some simple tests on python3, passes all tests on python2 > 4. try to reach rpython on python3.6 ops > 5. try to reach rpython on python3.11 ops > > > however i also want to ensure this experiment is something that can land > > so the opinions of the pypy core developers on the approach and llm usage in > general would be helpful > > -- Ronny > > > _______________________________________________ > pypy-dev mailing list -- [email protected] > To unsubscribe send an email to [email protected] > https://mail.python.org/mailman3//lists/pypy-dev.python.org > Member address: [email protected] _______________________________________________ pypy-dev mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3//lists/pypy-dev.python.org Member address: [email protected]
