[
https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623619#comment-16623619
]
Zac Zhou commented on YARN-8806:
--------------------------------
[~tangzhankun] Thanks a lot for your suggestion. :)
I thought your idea before. But I'm not sure that azkaban, zeppelin and other
systems would submit a subamrine job in a new process or in their own processes.
If a new process should be created as a recommended method, we should use the
way you suggested ~
> Enable local staging directory and clean it up when submarine job is submitted
> ------------------------------------------------------------------------------
>
> Key: YARN-8806
> URL: https://issues.apache.org/jira/browse/YARN-8806
> Project: Hadoop YARN
> Issue Type: Sub-task
> Environment: In the /tmp dir, there are launch scripts which are not
> cleaned up as follows:
> -rw-r--r-- 1 hadoop netease 1100 Sep 18 10:46
> PRIMARY_WORKER-launch-script8635233314077649086.sh
> -rw-r--r-- 1 hadoop netease 1100 Sep 18 10:46
> WORKER-launch-script129488020578466938.sh
> -rw-r--r-- 1 hadoop netease 1028 Sep 18 10:46
> PS-launch-script471092031021738136.sh
> Reporter: Zac Zhou
> Assignee: Zac Zhou
> Priority: Major
> Attachments: YARN-8806.001.patch, YARN-8806.002.patch,
> YARN-8806.003.patch
>
>
> YarnServiceJobSubmitter.generateCommandLaunchScript creates container launch
> scripts in the local filesystem. Container launch scripts would be uploaded
> to hdfs staging dir, but would not be not deleted after the job is submitted
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
