vidakovic commented on PR #6366: URL: https://github.com/apache/fineract/pull/6366#issuecomment-5553354773
@meonkeys the bare minimum I think we should do is actually require the *disclosure of AI tool usage*. We will not do ourselves a favor if we might be later required to sort this out (if/when we receive complaints). Using AI tools to detect if a file was generated by AI tools is very unreliable and gives at best some percentage values that you still have to interpret yourself; I see currently no way to automate this. Disclosure should be anyway easy enough: we have to write these commit messages anyway and we already have all those checklists in the current PR templates, so we could extend the checklist there. Right now I'm only aware of Anthropic watermarking their generated results (no idea how it works nor how it could be used to detect generated code automatically). In the end this is then only one provider anyway, others don't do this, maybe later some standard will emerge eventually. If that happens then we could probably forget the manual disclosure and automate this process. Until then personally I'll mark every PR/commit with a note which provider/harness was used if parts of the code were created with the help of an AI tool. At the moment the only scenario that I could imagine this to happen in my workflow is if one day an AI based security scanner shows up that auto-suggests the fixes. Right now I don't see this happening (I recently tested a provider... not worth the money... at all). Most likely though I would use AI to kickstart documentation, but would still mark it even if I think that I've changed it considerably. And in general I find all of the providers in various degrees useful for discovery/research. So, if we feel that as a community not disclosing generated code puts us strategically in any better position and this conforms with he Apache license and the current responsible AI usage rules then be it. My concerns are still: - people are already using these tools without any broader consent (maybe silent, but no vote) nor discussion (we just started it); I see a massively increased influx of PRs compared to just a couple of months ago - I take issue that we let consequential stuff like this pass as if this was choosing an IDE for development or an operating system; on the other hand we discuss(ed) smaller, discrete, controlled improvements way longer and with more ceremony (good for me if that is not necessary anymore)... maybe I missed the discussion in the flood of messages that we receive from Github - can we agree on one channel where discussions like these are happening? I thought it's the mailing list (I'm as big of a fan of it like the next one)... but right now we have Mailing list (dev, private, security), Matrix, PRs, some are still on Slack, Jira tickets, Confluence... this makes it really easy to miss important discussions; if it's only me then ignore - that we actually might increase technical debt with these tools; does every author really know what they are generating? I'd have less issues with members that have more experience (both with the code base and/or with the tools), but if we allow it to one then we allow it to everyone (can't anyway be enforced then) - review fatigue due to the increased amount of PRs... but also by entering in these review - AI tool makes changes - review again loops - what does this mean for PR approvals if some are extensively using AI tools, but rely on approval of others that don't feel comfortable with them. Does signing off on a PR create transitive responsibility for the reviewer? - more inexperienced community members will be probably pressured to use the help of AI tools to keep up with this arms race; there is still a lot free beer (tokens), but this will change with an Oligopoly of providers bleeding more money in a month than some countries' GDP and therefor teetering on bankruptcy; not everyone can keep up with those budget requirements... not really inclusive - I don't think that watching code being generated like a game being played on Twitch (read: without resistance and challenges) helps with learning how to code... I thought this might have been a secondary goal of projects like these I have more, but for the sake of coming to an end here I'll stop. I have a feeling that the moment for a proper (pros and cons) discussion has already passed (see "inevitability"... what a word); not expecting all those points to be answered, but wanted to post them anyway. I really hope that these tools help everyone create better and more robust code (not necessarily more). I'll try with a more "classic" approach, but glad to change my mind. For now though my personal process won't change. I'm reviewing these tools once in a while and if I see a sensible opportunity I might change my setup (still with disclosure), but only if I feel more comfortable with the consequences (you can't control everything... I know). In all, just my 2 cents. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
