[
https://issues.apache.org/jira/browse/TEZ-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siddharth Seth updated TEZ-2454:
--------------------------------
Attachment: TEZ-2454.2.txt
Tested this with a 100GB sort - with varying number of reducers to trigger
different merge scenarios.
The patch makes the OrderedGroupedFetcher work similar to the Broadcast Fetcher
- so it should be much simpler to consolidate these two.
Fetcher logic moves from Shuffle to ShuffleScheduler.
[~rajesh.balamohan] - please review. I'll rebase the patch after TEZ-2450 goes
in. Submitting this as is for review and jenkins.
> Change FetcherOrderedGroup to work as Callables instead of blocking threads
> ---------------------------------------------------------------------------
>
> Key: TEZ-2454
> URL: https://issues.apache.org/jira/browse/TEZ-2454
> Project: Apache Tez
> Issue Type: Improvement
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-2454.1.wip.txt, TEZ-2454.2.txt
>
>
> The Fetcher threads for Ordered Input currently run and block till merge
> completes, which makes it difficult to use them via ThreadPools.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
