[
https://issues.apache.org/jira/browse/TAJO-743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13963925#comment-13963925
]
Tajo QA commented on TAJO-743:
------------------------------
{color:red}*-1 overall.*{color} Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12639360/TAJO-743.patch
against master revision 631f3d0.
{color:green}+1 @author.{color} The patch does not contain any @author
tags.
{color:red}-1 tests included.{color} The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac.{color} The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc.{color} The applied patch does not increase the
total number of javadoc warnings.
{color:green}+1 checkstyle.{color} The patch generated 0 code style errors.
{color:red}-1 findbugs.{color} The patch appears to introduce 204 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit.{color} The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests.{color} The patch passed unit tests in
tajo-common tajo-core/tajo-core-backend tajo-storage.
Test results:
https://builds.apache.org/job/PreCommit-TAJO-Build/318//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TAJO-Build/318//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core-backend.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-TAJO-Build/318//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/318//console
This message is automatically generated.
> Change the default resource allocation policy of leaf tasks
> -----------------------------------------------------------
>
> Key: TAJO-743
> URL: https://issues.apache.org/jira/browse/TAJO-743
> Project: Tajo
> Issue Type: Improvement
> Components: resource manager
> Affects Versions: 0.8-incubating, 1.0-incubating
> Reporter: Jinho Kim
> Assignee: Jinho Kim
> Attachments: TAJO-743.patch, TAJO-743_branch-0.8.0.patch
>
>
> Currently, resource allocation is calculated by memory base. If a machine
> have a large memory, in default settings, heavy disk IO per disk is usually
> caused by high task concurrency. However, it is likely to seem to be
> problematic.
> When i tested the leaf task scan by 2(concurrency of SATA disk), the
> performance was better. if you have SAS Storage or SSD, you can increase the
> disk concurrency. This patch changes the default resource allocation policy
> to use disk resource.
> The following configs have been available so far:
> * tajo.worker.resource.disks - available disk resource of each worker
> * tajo.task.disk-slot.default - how many disk resource is consumed per task
> Below config is newly introduced in this patch
> * tajo.worker.resource.dfs-dir-aware - it can be true/false. If it is true,
> each worker uses the number of HDFS datanode's data dirs in the worker as the
> disk resource. So, tajo.worker.resource.disks is ignored.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
