[
https://issues.apache.org/jira/browse/HIVE-23882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23882:
------------------------------------------
Description:
In probe we cannot currently support Key expressions (on the big table Side) as
ORC CVs Probe directly the smalltable HT (there is no expr evaluation at that
level).
TezCompiler should take this into account when picking MJs to push probe
details, In this ticket we extend probedecode compiler logic to:
* Validate that MJ is a single Key join, where the bigTable keyCol is only a
ExprNodeColumnDesc
* For the selected MJ use backtracking logic to find the orignal TS key it maps
to (on the same vertex) - this actually revealed several missed or wrong
optimizations (updated q outs)
* Finally, extend the optimization logic to check if there a type Cast between
src and destination (as the types have to match in the probe case) and only use
it on the LLAP mode for now
was:
In probe we cannot currently support Key expressions (on the big table Side) as
ORC CVs Probe directly the smalltable HT (there is no expr evaluation at that
level).
TezCompiler should take this into account when picking MJs to push probe details
> Compiler should skip MJ keyExpr for probe optimization
> ------------------------------------------------------
>
> Key: HIVE-23882
> URL: https://issues.apache.org/jira/browse/HIVE-23882
> Project: Hive
> Issue Type: Sub-task
> Reporter: Panagiotis Garefalakis
> Assignee: Panagiotis Garefalakis
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> In probe we cannot currently support Key expressions (on the big table Side)
> as ORC CVs Probe directly the smalltable HT (there is no expr evaluation at
> that level).
> TezCompiler should take this into account when picking MJs to push probe
> details, In this ticket we extend probedecode compiler logic to:
> * Validate that MJ is a single Key join, where the bigTable keyCol is only a
> ExprNodeColumnDesc
> * For the selected MJ use backtracking logic to find the orignal TS key it
> maps to (on the same vertex) - this actually revealed several missed or wrong
> optimizations (updated q outs)
> * Finally, extend the optimization logic to check if there a type Cast
> between src and destination (as the types have to match in the probe case)
> and only use it on the LLAP mode for now
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
