IvanVergiliev commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-501584982 Thanks for the feedback! Just clarifying before I make any changes. - @cloud-fan sounds good, I can get rid of the benchmark changes. > and we write a simple microbenmark to test OrcFilters Would that be different from the benchmark I added to the PR description per @gatorsmile's suggestion? The one in the PR does have disk access, but I think it's pretty easy to understand, and it does an excellent job at showing that the exponential behaviour goes away. Let me know if you have something else in mind. > Could you make a separate benchmark for both ORC v1 OrcFilters.createFilter and ORC v2 OrcFilters.createFilter. - @dongjoon-hyun it seems this suggestion would no longer be relevant if we get rid of the benchmark changes and only keep the benchmark results in the PR? Let me know if I got this wrong, and what you had in mind if you'd still like me to do this.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
