[
https://issues.apache.org/jira/browse/TAJO-385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13849288#comment-13849288
]
Jinho Kim commented on TAJO-385:
--------------------------------
I've tested insert clause. output size worked well. but we need more test for
intermediate data
before
{noformat}
//snappy compressed text
final state: QUERY_SUCCEEDED, response time: 352.074 sec
//snappy compressed rcfile(text serde)
Progress: 100%, response time: 304.059 sec
{noformat}
After
{noformat}
//snappy compressed text
final state: QUERY_SUCCEEDED, response time: 337.719 sec
//snappy compressed rcfile(text serde)
final state: QUERY_SUCCEEDED, response time: 319.776 sec
{noformat}
> Refactoring TaskScheduler to assign multiple fragments
> ------------------------------------------------------
>
> Key: TAJO-385
> URL: https://issues.apache.org/jira/browse/TAJO-385
> Project: Tajo
> Issue Type: Improvement
> Components: query master
> Affects Versions: 0.8-incubating
> Reporter: Jihoon Son
> Assignee: Jihoon Son
> Attachments: TAJO-385.patch, TAJO-385_2.patch, TAJO-385_3.patch
>
>
> In the current implementation, each task processes only one fragment.
> However, processing multiple fragments in a task will increase the query
> processing performance according to the storage layout and the user queries.
> In this issue, TaskScheduler is refactored to enable assigning multiple
> fragments to each task.
> Followings should be contained.
> * Schedule Fragments instead of QueryUnits in TaskScheduler
> ** The QueryUnit creation is postponed until TaskScheduler receives task
> requests from workers.
> ** When TaskScheduler receives task requests from workers, it dynamically
> creates an QueryUnit and assigns one or more fragments.
> ** The fragment scheduling should take into account the disk load balancing.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
