[
https://issues.apache.org/jira/browse/TEZ-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15434246#comment-15434246
]
Jonathan Eagles commented on TEZ-3361:
--------------------------------------
In this patch the design is to allow for only a single partition range instead
of a list of partitions ranges. This is to better align with usage.
Also, the patch has changed the shuffle header and payload. Since multiple
partitions are fetched the partition headers are placed up front to allow for a
single transfer to command to slurp the partition spills in one go. This may
need to add an additional header specifying the number of partitions returned,
though the client should know since it requested the number.
> Fetch Multiple Partitions from the Shuffle Handler
> --------------------------------------------------
>
> Key: TEZ-3361
> URL: https://issues.apache.org/jira/browse/TEZ-3361
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Jonathan Eagles
> Assignee: Jonathan Eagles
> Attachments: TEZ-3361.1.patch
>
>
> Provide an API that allows for fetching multiple partitions at once from a
> single upstream task. This is to better support auto-reduce parallelism where
> a single downstream task is impersonating several (possibly?) consecutive
> downstream tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
