Hyunsik Choi created TAJO-982:
---------------------------------
Summary: Improve Fetcher to get multiple shuffle outputs through a
request
Key: TAJO-982
URL: https://issues.apache.org/jira/browse/TAJO-982
Project: Tajo
Issue Type: Bug
Components: data shuffle
Reporter: Hyunsik Choi
Fix For: 0.9.0
Currently, Fetcher only can request at most a fetch for one shuffle output at a
time. The implementation can cause performance degradation even though
intermediate data is actually small.
For example, If an input data set of the first stage is big and the
intermediate data is very small, QueryMaster will choose a few of nodes for
next execution block. Since each node keeps limited threads for fetch, it will
take a long time for the nodes for next stage to fetch all intermediate.
If Fetcher can get multiple shuffle outputs through a request, it would solve
the slowness which occurs in some cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
