alamb commented on issue #13748: URL: https://github.com/apache/datafusion/issues/13748#issuecomment-2542462344
I spent some time reviewing the flamegraph.  * remove_redundant_entries is called / takes up 1/3 of the time * normalized_oeq_class requires 2/3 of the time The high level observation is that continually recomputing normalized equivalence groups requires significaint amounts of time. What I am going to try is to make these implementations much more efficient (avoid allocation and recomputation) and Sketch: 1. try and avoid calling normalized_oeq_class by ensuring that the oeq_class is always normalized 2. I think we could reduce remove_redundant_entries by storing it in an IndexSet rather than a Vec -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
