[
https://issues.apache.org/jira/browse/TAJO-982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jihoon Son updated TAJO-982:
----------------------------
Fix Version/s: (was: 0.11.0)
0.12.0
> Improve Fetcher to get multiple shuffle outputs through a request
> -----------------------------------------------------------------
>
> Key: TAJO-982
> URL: https://issues.apache.org/jira/browse/TAJO-982
> Project: Tajo
> Issue Type: Improvement
> Components: Data Shuffle
> Reporter: Hyunsik Choi
> Assignee: Jihoon Son
> Fix For: 0.12.0
>
>
> Currently, Fetcher only can request at most a fetch for one shuffle output at
> a time. The implementation can cause performance degradation even though
> intermediate data is actually small.
> For example, If an input data set of the first stage is big and the
> intermediate data is very small, QueryMaster will choose a few of nodes for
> next execution block. Since each node keeps limited threads for fetch, it
> will take a long time for the nodes for next stage to fetch all intermediate.
> If Fetcher can get multiple shuffle outputs through a request, it would solve
> the slowness which occurs in some cases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
