[
https://issues.apache.org/jira/browse/HADOOP-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619707#action_12619707
]
Doug Cutting commented on HADOOP-3675:
--------------------------------------
The motivation for re-using the JVM is presumably performance. Since most
nodes are multicore these days, it would be a shame if tasks of a job were
executed serially on each node, no? So, if you don't intend to use threads,
then I think you need to lift the one-jvm-per-job limit. The tasktracker could
run instead one jvm per task slot, restarting them when tasks arrive from a
different job. Could that work?
> Provide more flexibility in the way tasks are run
> -------------------------------------------------
>
> Key: HADOOP-3675
> URL: https://issues.apache.org/jira/browse/HADOOP-3675
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Reporter: Brice Arnould
> Assignee: Brice Arnould
> Priority: Minor
> Attachments: TaskWrapper_v0.patch, userBasedInsulator.sh
>
>
> *The aim*
> With [HADOOP-3421] speaking about sharing a cluster among more than one
> organization (so potentially with non-cooperative users), and posts on the ML
> speaking about virtualization and the ability to re-use the TaskTracker's VM
> to run new tasks, it could be useful for admins to choose the way TaskRunners
> run their children.
> More specifically, it could be useful to provide a way to imprison a Task in
> its working directory, or in a virtual machine.
> In some cases, reusing the VM might be useful, since it seems that this
> feature is really wanted ([HADOOP-249]).
> *Concretely*
> What I propose is a new class, called called SeperateVMTaskWrapper which
> contains the current logic for running tasks in another JVM. This class
> extends another, called TaskWrapper, which could be inherited to provide new
> ways of running tasks.
> As part of this issue I would also like to provide two other TaskWrappers :
> the first would run the tasks as Thread of the TaskRunner's VM (if it is
> possible without too much changes), the second would use a fixed pool of
> local unix accounts to insulate tasks from each others (so potentially
> non-cooperating users will be hable to share a cluster, as described in
> [HADOOP-3421]).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
