limit can be optimized if the limit is happening on the reducer
---------------------------------------------------------------
Key: HIVE-1035
URL: https://issues.apache.org/jira/browse/HIVE-1035
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Reporter: Namit Jain
A query like:
select ... from A join B.. limit 10;
where the limit is performed on the reducer can be further optimized.
Currently, all the operators on the reduce side will be done, but the
ExecReducer will un-necessarily deserialize all the rows.
The following optimizations can be done:
1. Do nothing in reduce() in ExecReducer.
2. Modify map-reduce framework so that it does not even invoke the reduce()
method in ExecReducer.
2. may require some work from hadoop - but we should minimally do 1.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
