[
https://issues.apache.org/jira/browse/TAJO-1996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyunsik Choi updated TAJO-1996:
-------------------------------
Fix Version/s: (was: 0.12.0)
0.13.0
> Too many index lookups during range shuffle
> -------------------------------------------
>
> Key: TAJO-1996
> URL: https://issues.apache.org/jira/browse/TAJO-1996
> Project: Tajo
> Issue Type: Improvement
> Components: Data Shuffle, Pull Server
> Reporter: Jihoon Son
> Priority: Critical
> Fix For: 0.13.0
>
>
> During range shuffle, there are a huge number of index lookups, thereby
> taking at least the half time of the total shuffle time. This is because it
> is difficult to know which data is stored on which machine, so each machine
> needs to ask to all machines including itself.
> Unfortunately, this problem is difficult to resolved using statistics. With
> an assumption of even data distribution, every machine is expected to have
> the data of the whole range.
> So, we need to investigate an alternative solution to reduce the frequent
> index lookups.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
