jcamachor edited a comment on issue #952: HIVE-23006 ProbeDecode compiler support URL: https://github.com/apache/hive/pull/952#issuecomment-608201286 @pgaref , thanks for changes. Patch looks good. About the name, I think it is fine (probe side for the filtering generated statically in case of expressions or dynamically for joins). I think main remaining issue is related to selection in case of multiple MJ operators. I was thinking that for the time being, we could make the policy pluggable via config and have two very simple ones: 1) keep the one with lowest `ndv_JOIN_probe_key_column / ndv_TS_target_column` ratio, or 2) keep those with ratio below a certain threshold specified through a config value. Alternatively, in this patch you could maybe go only with option 1 and tackle other in follow-up. You have the information about the stats in the operators themselves (follow `op.getStatistics()` calls to see how it is retrieved per column basis). Thus, I would make that decision in the TezCompiler (once we gather all the context information) rather than GenTezUtils.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
