manuzhang commented on issue #26257: URL: https://github.com/apache/datafusion/issues/26257#issuecomment-6170142312
Artificial Analysis just [added a new evaluation](https://artificialanalysis.ai/articles/frontiercode) to test whether AI models can write code maintainers would merge. Some key findings 🤯 > Leading models still fail more than 40% of the time > 9 of the 100 tasks are passed by no model on any attempt, and no task is passed by every model on every attempt -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
