Implement Worker Pool
---------------------
Key: HAMA-516
URL: https://issues.apache.org/jira/browse/HAMA-516
Project: Hama
Issue Type: Improvement
Affects Versions: 0.4.0
Reporter: Thomas Jungblut
Priority: Minor
Fix For: 0.5.0
To decrease job startup time we should implement worker pools.
Worker pools should start BSPTask JVM's based on the configured task capacity.
This should greatly improve cold-start time for jobs. However, this cost is
quite low compared to the long-running Hama task.
The idea is from
http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a
(slide 4). Google Tenzing uses this, and I read about the gmail priority inbox
jobs which also uses this task reuse.
This will be the start of a number of tasks that will profile and improve
startup time of jobs and cluster. (Umbrella follows).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
