-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20149/
-----------------------------------------------------------
(Updated April 9, 2014, 7:22 a.m.)
Review request for Tajo.
Bugs: TAJO-743
https://issues.apache.org/jira/browse/TAJO-743
Repository: tajo
Description
-------
Currently, resource allocation is calculated by memory base. If a machine have
a large memory, in default settings, heavy disk IO per disk is usually caused
by high task concurrency. However, it is likely to seem to be problematic.
When i tested the leaf task scan by 2(concurrency of SATA disk), the
performance was better. if you have SAS Storage or SSD, you can increase the
disk concurrency. This patch changes the default resource allocation policy to
use disk resource.
The following configs have been available so far:
* tajo.worker.resource.disks - available disk resource of each worker
* tajo.task.disk-slot.default - how many disk resource is consumed per task
Below config is newly introduced in this patch
* tajo.worker.resource.dfs-dir-aware - it can be true/false. If it is true,
each worker uses the number of HDFS datanodes
Diffs
-----
tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
db96eb829a567e34f07c168e5d75e01545a680ef
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/TajoResourceAllocator.java
bcf10ddb4f41bfe68816d5659680a0a00b4f8a77
tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/worker/WorkerHeartbeatService.java
3843b8f303b6138fa979314bc9002755e0f2082d
tajo-storage/src/main/java/org/apache/tajo/storage/v2/DiskUtil.java
bb90c3917bc577f701ec05b44ae54584a1bb0d1a
Diff: https://reviews.apache.org/r/20149/diff/
Testing (updated)
-------
mvn clean install
Thanks,
Jinho Kim
