cloud-fan commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-475701648 I agree with you that building `ExpressionTree` directly is not a good idea, but there is another idea I'd like to try. The problem we have is, the orc filter builder has no way back, so we have to do an extra check before calling the builder methods. The idea I have is, create a tree-like orc filter representations by our own (similar to `ExpressionTree`), and convert the Spark filters to this tree representations. After the tree is built, call the real orc filter builder to create the `SearchArgument`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
