alamb commented on issue #11442: URL: https://github.com/apache/datafusion/issues/11442#issuecomment-2227387071
> It would be great to have one or two quarters where we focus on perf. I think we are at a pretty good place in terms of extensibility/customizability (evidenced by rapidly increasing number of projects), but the situation could be much better wrt performance. I agree with this sentiment. Something about performance improvements is I think they take sustained engineering investment and significant existing engine expertise (thus it is hard to have newcomers to the project make singificant performance improvements) I will try and find time from myself and InfluxData these next two quarters to meaninfully invest in improvements in this area However, I can't realistically do that if I am also helping to shepherd other large projects along (I am thinking specifically of https://github.com/apache/datafusion/pull/11160 from @notfilippo) so I need to make some hard choices there > That being said, team Synnada will keep adding baseline mechanisms to upstream DF to enable streaming use cases (when appropriate and not overfitting, obviously) by downstream projects. Thank you. Your help (and everyone else's help) with documentation and reviews I think would also be tremendously beneficial -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
