[
https://issues.apache.org/jira/browse/HIVE-18873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392656#comment-16392656
]
Ankit Singhal commented on HIVE-18873:
--------------------------------------
bq. IIRC, storage handlers use hive.optimize.ppd.storage to push filters down
to storage handlers. Maybe we can restrict this only when
hive.optimize.index.filters is used so that storage handlers don't get impacted
by HIVE-15680.
[~prasanth_j], How about skipping this check when storage handlers implement
HiveStoragePredicateHandler?
AFAICS, Actually, the storage handlers were receiving the right predicate for
respective alias while creating the input splits. So once the split is created,
we don't need to push down filter again during recordReader as InputFormat
should be carrying all required info through their custom InputSplit. (though
it seems not true for HiveHBaseTableInputFormat only).
> Skipping predicate pushdown for MR silently at HiveInputFormat can cause
> storage handlers to produce erroneous result
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-18873
> URL: https://issues.apache.org/jira/browse/HIVE-18873
> Project: Hive
> Issue Type: Bug
> Affects Versions: 3.0.0
> Reporter: Ankit Singhal
> Assignee: Ankit Singhal
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18873.patch
>
>
> {code:java}
> // disable filter pushdown for mapreduce when there are more than one table
> aliases,
> // since we don't clone jobConf per alias
> if (mrwork != null && mrwork.getAliases() != null &&
> mrwork.getAliases().size() > 1 &&
> jobConf.get(ConfVars.HIVE_EXECUTION_ENGINE.varname).equals("mr")) {
> return;
> }
> {code}
> I believe this needs to be handled at OpProcFactory so that hive doesn't
> believe that predicate is handled by storage handler.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
