IvanVergiliev commented on issue #24068: [SPARK-27105][SQL] Optimize away exponential complexity in ORC predicate conversion URL: https://github.com/apache/spark/pull/24068#issuecomment-494300029 @cloud-fan I took a stab at a slightly different approach to structuring the code in https://github.com/IvanVergiliev/spark/pull/2/files . The idea is to implement filtering and building in the same match expression, with an enum that tells us whether to perform a filter or a build operation. This has the following benefits: - All the logic for a given predicate is grouped logically in the same place. You don't have to scroll across the whole file to see what the filter action for an `And` is while you're looking at the build action. - You can't really add a new predicate to the set of filtered predicates without also defining a Build action for it - this fails the exhaustiveness check on `ActionType`. The only annoying part is the need for the `Either` to allow the filter and build actions to return different types. I experimented a bit with using type members on `ActionType` so that I can define the correct return type for each action and then use that as the return type for `performAction`, but I don't think this is really expressible in Scala. I don't think this is a big problem. Overall I'm pretty happy with that version of the code. Let me know if that looks reasonable to you, and if so - I can merge that branch into this one and we can then wrap up the review.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
