phillipleblanc commented on issue #26257: URL: https://github.com/apache/datafusion/issues/26257#issuecomment-6147976525
I found this post to be aligned with my current thinking on how AI is transforming software engineering: https://thorstenball.com/blog/2026/09/19/what-i-believe-about-the-future-of-software-development/ > TLDR; I worry that the ease of adding features with AI tools is leading DataFusion to become overly complicated and that the additional complexity will eventually slow down / ruin the project I don't think this will happen, and if it does then it can be solved. The frontier models are getting so good that most problems can be solved with more tokens. However... I do think DataFusion falls into this edge case called out by the blog post above: > Performance critical contributions by humans will stay what they are: an edge case Telling Claude "make datafusion faster, no mistakes" won't work - that will still require human judgement and creativity. But all of the grunt work and stuff that would have distracted from focusing on that core can be handled by AI. I do think DataFusion has the benefit of focusing on being a small extensible core rather than trying to solve the world. If anything we should focus more in that direction to enable the "building block economy": https://mitchellh.com/writing/building-block-economy And be ruthless about saying no and telling people to build on top of the building blocks that exist. > I'm much more purposeful about creating building blocks and encouraging applications or forks on top of that. I think this is resulting in a happier community, a larger community, and ultimately better mainline software. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
