Split combination may return too many block locations to map/reduce framework
-----------------------------------------------------------------------------
Key: PIG-1648
URL: https://issues.apache.org/jira/browse/PIG-1648
Project: Pig
Issue Type: Bug
Affects Versions: 0.8.0
Reporter: Yan Zhou
Assignee: Yan Zhou
Fix For: 0.8.0
For instance, if a small split has block locations h1, h2 and h3; another small
split has h1, h3, h4. After combination, the composite split contains 4 block
locations. If the number of component splits is big, then the number of block
locations could be big too. In fact, the number of block locations serves as a
hint to M/R as the best hosts this composite split should be run on so the list
should contain a short list, say 5, of the hosts that contain the most data in
this composite split.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
