Ihor Radchenko <[email protected]> writes: > Christian Moe <[email protected]> writes: > >> Jim Porter <[email protected]> writes: >>> One issue here is review burden [...] With LLM-generated code, the >>> patch is often cleaner (at least superficially), which in my >>> experience requires much closer attention from the reviewer. >> >> I lack experience in reviewing code, but this resonates with my >> experience in my field (translation and copy-editing).
> Maybe translations are somewhat more similar to the code. > For translations, LLMs sometimes lack broader (beyond the text given) > contextual understanding and use incorrect words for > translation. However, under the hood, this is often caused by underlying > ambiguity of the text that is taken out of broader (company or field) > context. The LLM is forced to give /some/ output, and it gives it, without > clarifying ambiguities, instead randomly choosing one possible way to > translate. There are ways to fight it - most important is giving the > extended context, but an alternative is also prompting to surface the > ambiguities. My experience with AI translation is much worse than the pretty simple picture you paint here (just give the context or ask for ambiguities). I translated my own article from German to English and then corrected all the errors (took almost as long as doing the translation myself, but spread over several days as I discovered more and more errors). Then I evaluated what kinds of errors were in there. Some were trivial (like missing items in lists), but some caused deep corruption of the core message of paragraphs, down to inverting its meaning. But if I hadn’t known the original text, I would never have guessed that they were mistranslations (but would have thought that the writer isn’t a nice person). The deepest problems I found: https://www.draketo.de/software/ai-translation-evaluated#completely-changed Since seeing that I consider errors in code writteng by LLMs to be just as hard to find as errors in obfuscated C contests. > I think I once had to write several pages of instructions to produce a > useful non-trivial code. Are you then still saving time? Best wishes, Arne -- Unpolitisch sein heißt politisch sein, ohne es zu merken. https://www.draketo.de
signature.asc
Description: PGP signature
