alamb commented on PR #7942: URL: https://github.com/apache/arrow-datafusion/pull/7942#issuecomment-1868278955
Here is my suggestion in how to proceed with this PR 1. Create some basic end to end planning performance benchmarks (I elaborated on @Dandandan 's idea https://github.com/apache/arrow-datafusion/issues/8638 https://github.com/apache/arrow-datafusion/pull/7942#issuecomment-1864376725) 2. Use that information to guide which part(s) of this PR are the most valuable for increasing performance. @sadboy, do you have any benchmarks you could share that model your existing workload? > +1 to the importance of this -- our workloads involve lots of analysis/transformations on the Datafusion LogicalPlan, so any perf improvements in this department would be extremely beneficial to us. > It would be great if there's some kind of benchmark to demonstrate the concrete effects of this change -- perf-related impacts can often times be counter-intuitive and surprising. 100% agree -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
