[jira] [Created] (YARN-8698) Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-21 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8698: -- Summary: Failed to add hadoop dependencies in docker container when submitting a submarine job Key: YARN-8698 URL: https://issues.apache.org/jira/browse/YARN-8698 Project:

[jira] [Updated] (YARN-8698) Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8698: --- Description: When a standalone submarine tf job is submitted, the following error is got :

[jira] [Updated] (YARN-8698) Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8698: --- Description: when a standalone submarine tf job is submitted, the following error is got :

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-22 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589597#comment-16589597 ] Zac Zhou commented on YARN-8698: Thanks a lot, [~leftnoteasy] :) Hi [~tangzhankun], I think this issue is

[jira] [Comment Edited] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-23 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591040#comment-16591040 ] Zac Zhou edited comment on YARN-8698 at 8/24/18 1:47 AM: - Hi [~tangzhankun],

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-23 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591040#comment-16591040 ] Zac Zhou commented on YARN-8698: Hi [~tangzhankun], yeah, I specified "DOCKER_HADOOP_HDFS_HOME" to

[jira] [Comment Edited] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-23 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591040#comment-16591040 ] Zac Zhou edited comment on YARN-8698 at 8/24/18 1:55 AM: - Hi [~tangzhankun],

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591243#comment-16591243 ] Zac Zhou commented on YARN-8698: In my case, launch_container.sh specified HADOOP_COMMON_HOME like this:

[jira] [Comment Edited] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591243#comment-16591243 ] Zac Zhou edited comment on YARN-8698 at 8/24/18 7:27 AM: - In my case, 

[jira] [Created] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-08-28 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8725: -- Summary: Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times Key: YARN-8725 URL:

[jira] [Updated] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-08-28 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8725: --- Description: Submarine jobs upload core-site.xml, hdfs-site.xml, job.info and 

[jira] [Updated] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-08-28 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8725: --- Description: Submarine jobs upload core-site.xml, hdfs-site.xml, job.info and 

[jira] [Commented] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-08-28 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16594904#comment-16594904 ] Zac Zhou commented on YARN-8725: [~leftnoteasy], [~tangzhankun]  Any comments? > Submarine job staging

[jira] [Commented] (YARN-8722) Failed to get native service application status when security is enabled

2018-08-28 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595840#comment-16595840 ] Zac Zhou commented on YARN-8722: [~leftnoteasy] Yeah, without the proxy settings,we can't submit the job

[jira] [Updated] (YARN-8722) Failed to get native service application status when security is enabled

2018-08-27 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8722: --- Attachment: standlone-tf.json stack.txt > Failed to get native service application status

[jira] [Updated] (YARN-8722) Failed to get native service application status when security is enabled

2018-08-27 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8722: --- Environment: The environment context is as follows: 1) Security enabled. kerberos 2) Klist output Ticket

[jira] [Created] (YARN-8722) Failed to get native service application status when security is enabled

2018-08-27 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8722: -- Summary: Failed to get native service application status when security is enabled Key: YARN-8722 URL: https://issues.apache.org/jira/browse/YARN-8722 Project: Hadoop YARN

[jira] [Commented] (YARN-8698) [Submarine] Failed to add hadoop dependencies in docker container when submitting a submarine job

2018-08-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16591468#comment-16591468 ] Zac Zhou commented on YARN-8698: Sure, [~tangzhankun] I've checked, launch_container.sh invokes 

[jira] [Commented] (YARN-8852) Add documentation for submarine installation details

2018-10-08 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641399#comment-16641399 ] Zac Zhou commented on YARN-8852: [~sunilg], thanks a lot for pointing out the mistake~ The typo is fixed.

[jira] [Commented] (YARN-8852) Add documentation for submarine installation details

2018-10-07 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641378#comment-16641378 ] Zac Zhou commented on YARN-8852: [~leftnoteasy] , thanks a lot for your comments.  License headers & 

[jira] [Updated] (YARN-8852) Add documentation for submarine installation details

2018-10-07 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8852: --- Attachment: YARN-8852.002.path > Add documentation for submarine installation details >

[jira] [Updated] (YARN-8852) Add documentation for submarine installation details

2018-10-08 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8852: --- Attachment: (was: YARN-8852.002.path) > Add documentation for submarine installation details >

[jira] [Updated] (YARN-8852) Add documentation for submarine installation details

2018-10-08 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8852: --- Attachment: YARN-8852.002.patch > Add documentation for submarine installation details >

[jira] [Commented] (YARN-8852) Add documentation for submarine installation details

2018-10-08 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641577#comment-16641577 ] Zac Zhou commented on YARN-8852: Fix whitespace errors > Add documentation for submarine installation

[jira] [Updated] (YARN-8852) Add documentation for submarine installation details

2018-10-08 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8852: --- Attachment: YARN-8852.003.patch > Add documentation for submarine installation details >

[jira] [Updated] (YARN-8879) Kerberos principal is needed when submitting a submarine job

2018-10-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8879: --- Attachment: YARN-8879.002.patch > Kerberos principal is needed when submitting a submarine job >

[jira] [Updated] (YARN-8879) Kerberos principal is needed when submitting a submarine job

2018-10-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8879: --- Attachment: (was: YARN-8879.patch) > Kerberos principal is needed when submitting a submarine job >

[jira] [Updated] (YARN-8879) Kerberos principal is needed when submitting a submarine job

2018-10-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8879: --- Attachment: YARN-8879.001.patch > Kerberos principal is needed when submitting a submarine job >

[jira] [Commented] (YARN-8879) Kerberos principal is needed when submitting a submarine job

2018-10-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651307#comment-16651307 ] Zac Zhou commented on YARN-8879: Thanks,  [~sunilg],  There is logic for kerberosPrincipal.getKeytab()

[jira] [Created] (YARN-8879) Kerberos principal is needed when submitting a submarine job

2018-10-16 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8879: -- Summary: Kerberos principal is needed when submitting a submarine job Key: YARN-8879 URL: https://issues.apache.org/jira/browse/YARN-8879 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-8960) Can't get submarine service status using the command of "yarn app -status" under security environment

2018-10-31 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.003.patch > Can't get submarine service status using the command of "yarn app -status"

[jira] [Created] (YARN-8960) Can't get submarine service status using the command of "yarn app -status" under security environment

2018-10-30 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8960: -- Summary: Can't get submarine service status using the command of "yarn app -status" under security environment Key: YARN-8960 URL: https://issues.apache.org/jira/browse/YARN-8960

[jira] [Commented] (YARN-8960) Can't get submarine service status using the command of "yarn app -status" under security environment

2018-10-30 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16668550#comment-16668550 ] Zac Zhou commented on YARN-8960: The root cause is that the submarine service doesn't have keytab and

[jira] [Updated] (YARN-8960) Can't get submarine service status using the command of "yarn app -status" under security environment

2018-10-30 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.002.patch > Can't get submarine service status using the command of "yarn app -status"

[jira] [Updated] (YARN-8960) {Submarine} Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-01 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Summary: {Submarine} Can't get submarine service status using the command of "yarn app -status" under

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-01 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Summary: [Submarine] Can't get submarine service status using the command of "yarn app -status" under

[jira] [Updated] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-10 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9001: --- Attachment: YARN-9001.002.patch > [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs >

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-10 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682419#comment-16682419 ] Zac Zhou commented on YARN-8960: Maybe, we need both of them, To enable "yarn app -status", submarine

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684639#comment-16684639 ] Zac Zhou commented on YARN-9001: Sure, Wangda The following test cases have been executed: # submarine

[jira] [Comment Edited] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684664#comment-16684664 ] Zac Zhou edited comment on YARN-8714 at 11/13/18 3:14 AM: -- Looks great, it would

[jira] [Comment Edited] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684664#comment-16684664 ] Zac Zhou edited comment on YARN-8714 at 11/13/18 3:13 AM: -- Looks great, it would

[jira] [Commented] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684664#comment-16684664 ] Zac Zhou commented on YARN-8714: Looks great, it would be convenient for notebook app, like Zeppline, to

[jira] [Comment Edited] (YARN-8714) [Submarine] Support files/tarballs to be localized for a training job.

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684664#comment-16684664 ] Zac Zhou edited comment on YARN-8714 at 11/13/18 3:14 AM: -- Looks great, it would

[jira] [Updated] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-11 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9001: --- Attachment: YARN-9001.003.patch > [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs >

[jira] [Comment Edited] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684788#comment-16684788 ] Zac Zhou edited comment on YARN-8960 at 11/13/18 6:53 AM: -- As discussion offline,

[jira] [Comment Edited] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684788#comment-16684788 ] Zac Zhou edited comment on YARN-8960 at 11/13/18 6:50 AM: -- As discussion offline,

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684788#comment-16684788 ] Zac Zhou commented on YARN-8960: As discussion offline, we can use the same kerberos keytab parameter for

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-13 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.004.patch > [Submarine] Can't get submarine service status using the command of "yarn

[jira] [Updated] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9001: --- Attachment: YARN-9001.004.patch > [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs >

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-12 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16683661#comment-16683661 ] Zac Zhou commented on YARN-9001: The ut error seems is not related to the patch, resubmit the patch >

[jira] [Updated] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9001: --- Attachment: YARN-9001-branch-3.2.001.patch > [Submarine] Use AppAdminClient instead of ServiceClient to

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-13 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686076#comment-16686076 ] Zac Zhou commented on YARN-9001: Thanks, [~leftnoteasy] I checked the code and found there were no

[jira] [Comment Edited] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687457#comment-16687457 ] Zac Zhou edited comment on YARN-8960 at 11/15/18 4:18 AM: -- Add a parameter, named 

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.006.patch > [Submarine] Can't get submarine service status using the command of "yarn

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.006.patch > [Submarine] Can't get submarine service status using the command of "yarn

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: (was: YARN-8960.006.patch) > [Submarine] Can't get submarine service status using the command

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.007.patch > [Submarine] Can't get submarine service status using the command of "yarn

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16687457#comment-16687457 ] Zac Zhou commented on YARN-8960: Add a parameter, named distribute_keytab, which can be used to specify

[jira] [Updated] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8960: --- Attachment: YARN-8960.005.patch > [Submarine] Can't get submarine service status using the command of "yarn

[jira] [Commented] (YARN-8960) [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment

2018-11-14 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16686692#comment-16686692 ] Zac Zhou commented on YARN-8960:   Thanks,[~leftnoteasy] For comment 1 {quote}1) doLoginIfSecure, could

[jira] [Commented] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-09 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16681388#comment-16681388 ] Zac Zhou commented on YARN-9001: I'll submit a patch shortly > [Submarine] Use AppAdminClient instead of

[jira] [Created] (YARN-9001) [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs

2018-11-09 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-9001: -- Summary: [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs Key: YARN-9001 URL: https://issues.apache.org/jira/browse/YARN-9001 Project: Hadoop YARN

[jira] [Assigned] (YARN-8852) Add documentation for submarine installation details

2018-10-05 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou reassigned YARN-8852: -- Assignee: Zac Zhou > Add documentation for submarine installation details >

[jira] [Created] (YARN-8852) Add documentation for submarine installation details

2018-10-05 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8852: -- Summary: Add documentation for submarine installation details Key: YARN-8852 URL: https://issues.apache.org/jira/browse/YARN-8852 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-8852) Add documentation for submarine installation details

2018-10-05 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8852: --- Attachment: YARN-8852.001.patch > Add documentation for submarine installation details >

[jira] [Updated] (YARN-8852) Add documentation for submarine installation details

2018-10-05 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8852: --- Attachment: (was: YARN-8852.001.patch) > Add documentation for submarine installation details >

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.003.patch > Enable local staging directory and clean it up when submarine job is

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16623619#comment-16623619 ] Zac Zhou commented on YARN-8806: [~tangzhankun] Thanks a lot for your suggestion. :) I thought your idea

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-25 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.004.patch > Enable local staging directory and clean it up when submarine job is

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-25 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.005.patch > Enable local staging directory and clean it up when submarine job is

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-25 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16626868#comment-16626868 ] Zac Zhou commented on YARN-8806: Thanks [~leftnoteasy] for your comments. As deleteOnExit() would fail if

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-25 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.006.patch > Enable local staging directory and clean it up when submarine job is

[jira] [Commented] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-25 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628263#comment-16628263 ] Zac Zhou commented on YARN-8806: [~bsteinbach], your suggestions are awesome. Thanks a lot for your 

[jira] [Created] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-20 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-8806: -- Summary: Enable local staging directory and clean it up when submarine job is submitted Key: YARN-8806 URL: https://issues.apache.org/jira/browse/YARN-8806 Project: Hadoop YARN

[jira] [Comment Edited] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-09-20 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622159#comment-16622159 ] Zac Zhou edited comment on YARN-8725 at 9/20/18 2:51 PM: - [~leftnoteasy],

[jira] [Commented] (YARN-8725) Submarine job staging directory has a lot of useless PRIMARY_WORKER-launch-script-***.sh scripts when submitting a job multiple times

2018-09-20 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622159#comment-16622159 ] Zac Zhou commented on YARN-8725: [~leftnoteasy], [~tangzhankun]. Thanks a lot for your efforts to make

[jira] [Updated] (YARN-8806) Enable local staging directory and clean it up when submarine job is submitted

2018-09-20 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8806: --- Attachment: YARN-8806.002.patch > Enable local staging directory and clean it up when submarine job is

[jira] [Updated] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2018-12-31 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9161: --- Attachment: YARN-9161.001.patch > Absolute resources of capacity scheduler doesn't support GPU and FPGA >

[jira] [Updated] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2018-12-31 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9161: --- Attachment: (was: YARN-9161.001.patch) > Absolute resources of capacity scheduler doesn't support GPU and

[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-02 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732613#comment-16732613 ] Zac Zhou commented on YARN-9161: Thanks a lot, [~leftnoteasy], [~sunilg] {code:java} // W/o unit for

[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744183#comment-16744183 ] Zac Zhou commented on YARN-9161: Add UT for container allocation and fix some code style issues >

[jira] [Updated] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9161: --- Attachment: YARN-9161.005.patch > Absolute resources of capacity scheduler doesn't support GPU and FPGA >

[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-21 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748468#comment-16748468 ] Zac Zhou commented on YARN-9161: I tested the TestCapacitySchedulerSurgicalPreemption locally, it doesn't

[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-22 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749498#comment-16749498 ] Zac Zhou commented on YARN-9161: [~sunilg], [~leftnoteasy], [~tangzhankun]  welcome to add some comments

[jira] [Commented] (YARN-6695) Race condition in RM for publishing container events vs appFinished events causes NPE

2019-01-22 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749497#comment-16749497 ] Zac Zhou commented on YARN-6695: I checked the logic for TimelineServiceV2Publisher. If 

[jira] [Updated] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9161: --- Attachment: YARN-9161.004.patch > Absolute resources of capacity scheduler doesn't support GPU and FPGA >

[jira] [Commented] (YARN-9205) When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION)

2019-01-17 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745759#comment-16745759 ] Zac Zhou commented on YARN-9205: [~tangzhankun] {quote}private static Map

[jira] [Updated] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-17 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9161: --- Attachment: YARN-9161.006.patch > Absolute resources of capacity scheduler doesn't support GPU and FPGA >

[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-17 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745779#comment-16745779 ] Zac Zhou commented on YARN-9161: Update patch to fix failed UT cases > Absolute resources of capacity

[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA

2019-01-22 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16749522#comment-16749522 ] Zac Zhou commented on YARN-9161: [~sunilg] , Thanks a lot for your comments. Checked with [~tangzhankun]

[jira] [Created] (YARN-9144) WebAppProxyServlet can't redirect to ATS V1.5 when a yarn native service app is finished

2018-12-18 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-9144: -- Summary: WebAppProxyServlet can't redirect to ATS V1.5 when a yarn native service app is finished Key: YARN-9144 URL: https://issues.apache.org/jira/browse/YARN-9144 Project:

[jira] [Updated] (YARN-9155) Can't re-run a submarine job, if a previous job with the same service name has finished

2018-12-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9155: --- Summary: Can't re-run a submarine job, if a previous job with the same service name has finished (was: Can't

[jira] [Updated] (YARN-9155) Can't re-run a submarine job, if the previous job with the same service name has finished

2018-12-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-9155: --- Summary: Can't re-run a submarine job, if the previous job with the same service name has finished (was:

[jira] [Commented] (YARN-9155) Can't re-run a submarine job, if the previous job with the same service name has finished

2018-12-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728293#comment-16728293 ] Zac Zhou commented on YARN-9155: Or should we add an after service listeners interface to yarn native

[jira] [Created] (YARN-9155) Can't submit a submarine job, if a previous job with the same service name has finished

2018-12-24 Thread Zac Zhou (JIRA)
Zac Zhou created YARN-9155: -- Summary: Can't submit a submarine job, if a previous job with the same service name has finished Key: YARN-9155 URL: https://issues.apache.org/jira/browse/YARN-9155 Project:

[jira] [Commented] (YARN-9155) Can't submit a submarine job, if a previous job with the same service name has finished

2018-12-24 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728276#comment-16728276 ] Zac Zhou commented on YARN-9155: Could we add a parameter in Service API to indicate that the service

[jira] [Commented] (YARN-9003) Support multi-homed network for docker container

2018-12-26 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728938#comment-16728938 ] Zac Zhou commented on YARN-9003: I'm not sure whether --net can specify multiple networks.  I tried 

[jira] [Comment Edited] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-12-11 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716694#comment-16716694 ] Zac Zhou edited comment on YARN-8489 at 12/11/18 9:47 AM: -- [~suma.shivaprasad] 

[jira] [Commented] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-12-11 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16716694#comment-16716694 ] Zac Zhou commented on YARN-8489: @[~suma.shivaprasad] any Updates? Or would you mind if I take it, as this

[jira] [Updated] (YARN-8489) Need to support "dominant" component concept inside YARN service

2018-12-16 Thread Zac Zhou (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zac Zhou updated YARN-8489: --- Attachment: YARN-8489.003.patch > Need to support "dominant" component concept inside YARN service >

  1   2   >