[jira] [Commented] (YARN-4990) Re-direction of a particular log file within in a container in NM UI does not redirect properly to Log Server ( history ) on container completion

2017-01-05 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801743#comment-15801743 ] Jason Lowe commented on YARN-4990: -- This would be a nice fix to get into 2.8 and seems to be low risk.

[jira] [Commented] (YARN-5258) Document Use of Docker with LinuxContainerExecutor

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801657#comment-15801657 ] Daniel Templeton commented on YARN-5258: [~sidharta-s] or [~vvasudev], any comments? I'd love to

[jira] [Commented] (YARN-5554) MoveApplicationAcrossQueues does not check user permission on the target queue

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801637#comment-15801637 ] Daniel Templeton commented on YARN-5554: In doing a last pass, I have two questions on the test

[jira] [Commented] (YARN-6027) Support fromId for flows/flowrun apps

2017-01-05 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801934#comment-15801934 ] Varun Saxena commented on YARN-6027: By the way in YARN-5585 we have kept fromId and fromIdPrefix as

[jira] [Commented] (YARN-5556) Support for deleting queues without requiring a RM restart

2017-01-05 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802176#comment-15802176 ] Xuan Gong commented on YARN-5556: - [~Naganarasimha] Thanks for the comments [~leftnoteasy] Please comment

[jira] [Updated] (YARN-3550) Improve YARN RM REST API error messages

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3550: - Fix Version/s: (was: 2.7.1) > Improve YARN RM REST API error messages >

[jira] [Updated] (YARN-3614) FileSystemRMStateStore throw exception when failed to remove application, that cause resourcemanager to crash

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3614: - Fix Version/s: (was: 2.7.1) > FileSystemRMStateStore throw exception when failed to remove

[jira] [Updated] (YARN-6057) yarn.scheduler.minimum-allocation-* and yarn.scheduler.maximum-allocation-* descriptions are incorrect about behavior when a request is out of bounds

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton updated YARN-6057: --- Summary: yarn.scheduler.minimum-allocation-* and yarn.scheduler.maximum-allocation-*

[jira] [Updated] (YARN-3955) Support for priority ACLs in CapacityScheduler

2017-01-05 Thread Sunil G (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sunil G updated YARN-3955: -- Attachment: YARN-3955.0008.patch Thanks [~leftnoteasy] for the detailed comments. I have some more doubts here.

[jira] [Commented] (YARN-6040) Remove usage of ResourceRequest from AppSchedulerInfo / SchedulerApplicationAttempt

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802354#comment-15802354 ] Hadoop QA commented on YARN-6040: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6057) yarn.scheduler.minimum-allocation describtion update required

2017-01-05 Thread Bibin A Chundatt (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801897#comment-15801897 ] Bibin A Chundatt commented on YARN-6057: IIUC for minimum allocation the requests gets rounded up

[jira] [Commented] (YARN-5556) Support for deleting queues without requiring a RM restart

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802255#comment-15802255 ] Wangda Tan commented on YARN-5556: -- Just offline discussed with [~xgong]. I think we don't need the

[jira] [Updated] (YARN-5280) Allow YARN containers to run with Java Security Manager

2017-01-05 Thread Greg Phillips (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Phillips updated YARN-5280: Attachment: YARN-5280.006.patch > Allow YARN containers to run with Java Security Manager >

[jira] [Commented] (YARN-5846) Improve the fairscheduler attemptScheduler

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802340#comment-15802340 ] Junping Du commented on YARN-5846: -- Hi, we shouldn't set fix version here unless the commit get checked

[jira] [Comment Edited] (YARN-6057) yarn.scheduler.minimum-allocation describtion update required

2017-01-05 Thread Bibin A Chundatt (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801897#comment-15801897 ] Bibin A Chundatt edited comment on YARN-6057 at 1/5/17 5:15 PM: IIUC for

[jira] [Commented] (YARN-6057) yarn.scheduler.minimum-allocation describtion update required

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801933#comment-15801933 ] Daniel Templeton commented on YARN-6057: In the case of both minimum and maximum, resource requests

[jira] [Assigned] (YARN-6057) yarn.scheduler.minimum-allocation describtion update required

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Templeton reassigned YARN-6057: -- Assignee: Daniel Templeton > yarn.scheduler.minimum-allocation describtion update

[jira] [Commented] (YARN-5959) RM changes to support change of container ExecutionType

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802113#comment-15802113 ] Wangda Tan commented on YARN-5959: -- Committing ... > RM changes to support change of container

[jira] [Commented] (YARN-6041) Opportunistic containers : Combined patch for branch-2

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802169#comment-15802169 ] Wangda Tan commented on YARN-6041: -- [~asuresh], How you plan to merge these changes to branch-2? I think

[jira] [Comment Edited] (YARN-3955) Support for priority ACLs in CapacityScheduler

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802330#comment-15802330 ] Wangda Tan edited comment on YARN-3955 at 1/5/17 7:45 PM: -- 1) bq. There is one

[jira] [Updated] (YARN-6032) SharedCacheManager cleaner task should rm InMemorySCMStore some cachedResources which does not exists in hdfs fs

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-6032: - Fix Version/s: (was: 2.7.1) > SharedCacheManager cleaner task should rm InMemorySCMStore some >

[jira] [Commented] (YARN-3955) Support for priority ACLs in CapacityScheduler

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802330#comment-15802330 ] Wangda Tan commented on YARN-3955: -- 1) bq. There is one issue here. If approvedPriorityACL comes are null,

[jira] [Updated] (YARN-5964) Lower the granularity of locks in FairScheduler

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5964: - Fix Version/s: (was: 2.7.1) > Lower the granularity of locks in FairScheduler >

[jira] [Commented] (YARN-6032) SharedCacheManager cleaner task should rm InMemorySCMStore some cachedResources which does not exists in hdfs fs

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802329#comment-15802329 ] Junping Du commented on YARN-6032: -- Remove fix version as the jira haven't be resolved. >

[jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters

2017-01-05 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801911#comment-15801911 ] Varun Saxena commented on YARN-5585: Thanks [~rohithsharma] for the latest patch. The patch LGTM.

[jira] [Resolved] (YARN-5234) ResourceManager REST API missing descriptions for what's returned when using Fair Scheduler

2017-01-05 Thread Grant Sohn (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Sohn resolved YARN-5234. -- Resolution: Fixed Fix Version/s: 3.0.0-alpha1 Looked at latest docs and this has been addressed.

[jira] [Commented] (YARN-3955) Support for priority ACLs in CapacityScheduler

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802298#comment-15802298 ] Hadoop QA commented on YARN-3955: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-5585) [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters

2017-01-05 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801940#comment-15801940 ] Sangjin Lee commented on YARN-5585: --- +1. Thanks [~rohithsharma]! > [Atsv2] Reader side changes for

[jira] [Created] (YARN-6058) Support for listing all applications i.e /apps

2017-01-05 Thread Rohith Sharma K S (JIRA)
Rohith Sharma K S created YARN-6058: --- Summary: Support for listing all applications i.e /apps Key: YARN-6058 URL: https://issues.apache.org/jira/browse/YARN-6058 Project: Hadoop YARN Issue

[jira] [Updated] (YARN-6040) Remove usage of ResourceRequest from AppSchedulerInfo / SchedulerApplicationAttempt

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-6040: - Attachment: YARN-6040.006.patch Attached 006 patch, rebased to latest trunk. [~asuresh] could you please

[jira] [Commented] (YARN-5959) RM changes to support change of container ExecutionType

2017-01-05 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802237#comment-15802237 ] Hudson commented on YARN-5959: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11075 (See

[jira] [Updated] (YARN-6050) AMs can't be scheduled on racks or nodes

2017-01-05 Thread Robert Kanter (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated YARN-6050: Attachment: YARN-6050.004.patch The 004 patch - Removes the {{getAMContainerResourceRequest}} check

[jira] [Commented] (YARN-5831) Propagate allowPreemptionFrom flag all the way down to the app

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802328#comment-15802328 ] Hadoop QA commented on YARN-5831: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-5955) Use threadpool or multiple thread to recover app

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802336#comment-15802336 ] Junping Du commented on YARN-5955: -- Remove fix version as JIRA hasn't been resolved. > Use threadpool or

[jira] [Created] (YARN-6057) yarn.scheduler.minimum-allocation describtion update required

2017-01-05 Thread Bibin A Chundatt (JIRA)
Bibin A Chundatt created YARN-6057: -- Summary: yarn.scheduler.minimum-allocation describtion update required Key: YARN-6057 URL: https://issues.apache.org/jira/browse/YARN-6057 Project: Hadoop YARN

[jira] [Commented] (YARN-6050) AMs can't be scheduled on racks or nodes

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802134#comment-15802134 ] Wangda Tan commented on YARN-6050: -- [~rkanter], bq. Though I think we should still throw and exception if

[jira] [Commented] (YARN-6041) Opportunistic containers : Combined patch for branch-2

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802212#comment-15802212 ] Wangda Tan commented on YARN-6041: -- [~asuresh], so it will generate 10 commits (plus one for suggestions

[jira] [Updated] (YARN-5936) when cpu strict mode is closed, yarn couldn't assure scheduling fairness between containers

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5936: - Fix Version/s: (was: 2.7.1) > when cpu strict mode is closed, yarn couldn't assure scheduling fairness

[jira] [Updated] (YARN-5846) Improve the fairscheduler attemptScheduler

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5846: - Fix Version/s: (was: 2.7.1) > Improve the fairscheduler attemptScheduler >

[jira] [Updated] (YARN-5955) Use threadpool or multiple thread to recover app

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5955: - Fix Version/s: (was: 2.7.1) > Use threadpool or multiple thread to recover app >

[jira] [Commented] (YARN-6056) Yarn NM using LCE shows a failure when trying to delete a non-existing dir

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15801998#comment-15801998 ] Hadoop QA commented on YARN-6056: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-4990) Re-direction of a particular log file within in a container in NM UI does not redirect properly to Log Server ( history ) on container completion

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802185#comment-15802185 ] Junping Du commented on YARN-4990: -- I am fine with it. > Re-direction of a particular log file within in

[jira] [Commented] (YARN-6041) Opportunistic containers : Combined patch for branch-2

2017-01-05 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802195#comment-15802195 ] Arun Suresh commented on YARN-6041: --- [~leftnoteasy], So what I plan to do was, like you mentioned,

[jira] [Updated] (YARN-3795) ZKRMStateStore crashes due to IOException: Broken pipe

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3795: - Fix Version/s: (was: 2.7.1) > ZKRMStateStore crashes due to IOException: Broken pipe >

[jira] [Created] (YARN-6059) Update paused container state in the state store

2017-01-05 Thread Hitesh Sharma (JIRA)
Hitesh Sharma created YARN-6059: --- Summary: Update paused container state in the state store Key: YARN-6059 URL: https://issues.apache.org/jira/browse/YARN-6059 Project: Hadoop YARN Issue Type:

[jira] [Updated] (YARN-3681) yarn cmd says "could not find main class 'queue'" in windows

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3681: - Fix Version/s: 2.8.0 > yarn cmd says "could not find main class 'queue'" in windows >

[jira] [Updated] (YARN-3725) App submission via REST API is broken in secure mode due to Timeline DT service address is empty

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3725: - Fix Version/s: 2.8.0 > App submission via REST API is broken in secure mode due to Timeline DT > service

[jira] [Updated] (YARN-3733) Fix DominantRC#compare() does not work as expected if cluster resource is empty

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3733: - Fix Version/s: 2.8.0 > Fix DominantRC#compare() does not work as expected if cluster resource is > empty

[jira] [Updated] (YARN-3544) AM logs link missing in the RM UI for a completed app

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3544: - Fix Version/s: 2.8.0 > AM logs link missing in the RM UI for a completed app >

[jira] [Updated] (YARN-4424) Fix deadlock in RMAppImpl

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4424: - Fix Version/s: 2.8.0 > Fix deadlock in RMAppImpl > - > > Key:

[jira] [Updated] (YARN-4610) Reservations continue looking for one app causes other apps to starve

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4610: - Fix Version/s: 2.8.0 > Reservations continue looking for one app causes other apps to starve >

[jira] [Updated] (YARN-3193) When visit standby RM webui, it will redirect to the active RM webui slowly.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3193: - Fix Version/s: (was: 2.7.1) > When visit standby RM webui, it will redirect to the active RM webui

[jira] [Commented] (YARN-6040) Remove usage of ResourceRequest from AppSchedulerInfo / SchedulerApplicationAttempt

2017-01-05 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802425#comment-15802425 ] Arun Suresh commented on YARN-6040: --- Thanks for updating the patch [~leftnoteasy], # Looks like

[jira] [Commented] (YARN-6022) Revert changes of AbstractResourceRequest

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802779#comment-15802779 ] Daniel Templeton commented on YARN-6022: Bah, nevermind. The conflict is trivial. I'll take care

[jira] [Updated] (YARN-5246) NMWebAppFilter web redirects drop query parameters

2017-01-05 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-5246: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks, Varun! I committed this to branch-2.8 as

[jira] [Commented] (YARN-4348) ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding blocking ZK's event thread

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802493#comment-15802493 ] Junping Du commented on YARN-4348: -- Hi [~jianhe] and [~ozawa], Does this fix need to go to

[jira] [Commented] (YARN-4348) ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding blocking ZK's event thread

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802604#comment-15802604 ] Junping Du commented on YARN-4348: -- Got it. Thanks for confirmation here, Jian! >

[jira] [Commented] (YARN-6041) Opportunistic containers : Combined patch for branch-2

2017-01-05 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802732#comment-15802732 ] Wangda Tan commented on YARN-6041: -- +1 to latest patch, [~asuresh] please wait another day before

[jira] [Commented] (YARN-6041) Opportunistic containers : Combined patch for branch-2

2017-01-05 Thread Karthik Kambatla (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802796#comment-15802796 ] Karthik Kambatla commented on YARN-6041: The changes to config names and methods look good to me.

[jira] [Commented] (YARN-4675) Reorganize TimeClientImpl into TimeClientV1Impl and TimeClientV2Impl

2017-01-05 Thread Sangjin Lee (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802756#comment-15802756 ] Sangjin Lee commented on YARN-4675: --- One implication of refactoring the interface is that code that uses

[jira] [Updated] (YARN-3850) NM fails to read files from full disks which can lead to container logs being lost and other issues

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3850: - Fix Version/s: 2.8.0 > NM fails to read files from full disks which can lead to container logs being >

[jira] [Updated] (YARN-3426) Add jdiff support to YARN

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3426: - Fix Version/s: 2.8.0 > Add jdiff support to YARN > - > > Key:

[jira] [Created] (YARN-6060) Linux container executor fails to run container on directories mounted as noexec

2017-01-05 Thread Miklos Szegedi (JIRA)
Miklos Szegedi created YARN-6060: Summary: Linux container executor fails to run container on directories mounted as noexec Key: YARN-6060 URL: https://issues.apache.org/jira/browse/YARN-6060

[jira] [Updated] (YARN-5222) DockerContainerExecutor dosn't set work directory

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5222: - Fix Version/s: (was: 2.7.2) > DockerContainerExecutor dosn't set work directory >

[jira] [Commented] (YARN-4348) ZKRMStateStore.syncInternal shouldn't wait for sync completion for avoiding blocking ZK's event thread

2017-01-05 Thread Jian He (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802518#comment-15802518 ] Jian He commented on YARN-4348: --- No, it doesn't need to. The zkstore implementation has been changed by using

[jira] [Commented] (YARN-5280) Allow YARN containers to run with Java Security Manager

2017-01-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802560#comment-15802560 ] Hadoop QA commented on YARN-5280: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-6022) Revert changes of AbstractResourceRequest

2017-01-05 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802770#comment-15802770 ] Daniel Templeton commented on YARN-6022: Sorry, [~leftnoteasy], there's a conflict now. Mind

[jira] [Updated] (YARN-3764) CapacityScheduler should forbid moving LeafQueue from one parent to another

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3764: - Fix Version/s: 2.8.0 > CapacityScheduler should forbid moving LeafQueue from one parent to another >

[jira] [Commented] (YARN-5222) DockerContainerExecutor dosn't set work directory

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802485#comment-15802485 ] Junping Du commented on YARN-5222: -- Also, please don't set fix version field as we don't have any patch

[jira] [Updated] (YARN-5309) Fix SSLFactory truststore reloader thread leak in TimelineClientImpl

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5309: - Fix Version/s: 2.8.0 > Fix SSLFactory truststore reloader thread leak in TimelineClientImpl >

[jira] [Updated] (YARN-3701) Isolating the error of generating a single app report when getting all apps from generic history service

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3701: - Fix Version/s: 2.8.0 > Isolating the error of generating a single app report when getting all apps > from

[jira] [Updated] (YARN-4990) Re-direction of a particular log file within in a container in NM UI does not redirect properly to Log Server ( history ) on container completion

2017-01-05 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated YARN-4990: - Fix Version/s: (was: 2.9.0) 2.8.0 Thanks, Xuan! I committed this to branch-2.8 as

[jira] [Updated] (YARN-3522) DistributedShell uses the wrong user to put timeline data

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3522: - Fix Version/s: 2.8.0 > DistributedShell uses the wrong user to put timeline data >

[jira] [Updated] (YARN-3485) FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3485: - Fix Version/s: 2.8.0 > FairScheduler headroom calculation doesn't consider maxResources for Fifo and >

[jira] [Updated] (YARN-3385) Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3385: - Fix Version/s: 2.8.0 > Race condition: KeeperException$NoNodeException will cause RM shutdown during > ZK

[jira] [Updated] (YARN-3878) AsyncDispatcher can hang while stopping if it is configured for draining events on stop

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3878: - Fix Version/s: 2.8.0 > AsyncDispatcher can hang while stopping if it is configured for draining > events

[jira] [Updated] (YARN-3487) CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3487: - Fix Version/s: 2.8.0 > CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue >

[jira] [Updated] (YARN-3464) Race condition in LocalizerRunner kills localizer before localizing all resources

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3464: - Fix Version/s: 2.8.0 > Race condition in LocalizerRunner kills localizer before localizing all >

[jira] [Updated] (YARN-3893) Both RM in active state when Admin#transitionToActive failure from refeshAll()

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3893: - Fix Version/s: 2.8.0 > Both RM in active state when Admin#transitionToActive failure from refeshAll() >

[jira] [Updated] (YARN-3925) ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3925: - Fix Version/s: 2.8.0 > ContainerLogsUtils#getContainerLogFile fails to read container log files from >

[jira] [Updated] (YARN-3585) NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3585: - Fix Version/s: 2.8.0 > NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled >

[jira] [Updated] (YARN-4760) proxy redirect to history server uses wrong URL

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4760: - Fix Version/s: 2.8.0 > proxy redirect to history server uses wrong URL >

[jira] [Updated] (YARN-5008) LeveldbRMStateStore database can grow substantially leading to long recovery times

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-5008: - Fix Version/s: 2.8.0 > LeveldbRMStateStore database can grow substantially leading to long recovery >

[jira] [Updated] (YARN-3905) Application History Server UI NPEs when accessing apps run after RM restart

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3905: - Fix Version/s: 2.8.0 > Application History Server UI NPEs when accessing apps run after RM restart >

[jira] [Updated] (YARN-3382) Some of UserMetricsInfo metrics are incorrectly set to root queue metrics

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3382: - Fix Version/s: 2.8.0 > Some of UserMetricsInfo metrics are incorrectly set to root queue metrics >

[jira] [Updated] (YARN-3694) Fix dead link for TimelineServer REST API

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3694: - Fix Version/s: 2.8.0 > Fix dead link for TimelineServer REST API >

[jira] [Updated] (YARN-3677) Fix findbugs warnings in yarn-server-resourcemanager

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3677: - Fix Version/s: 2.8.0 > Fix findbugs warnings in yarn-server-resourcemanager >

[jira] [Updated] (YARN-3711) Documentation of ResourceManager HA should explain configurations about listen addresses

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3711: - Fix Version/s: 2.8.0 > Documentation of ResourceManager HA should explain configurations about > listen

[jira] [Updated] (YARN-3516) killing ContainerLocalizer action doesn't take effect when private localizer receives FETCH_FAILURE status.

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3516: - Fix Version/s: 2.8.0 > killing ContainerLocalizer action doesn't take effect when private localizer >

[jira] [Updated] (YARN-6056) Yarn NM using LCE shows a failure when trying to delete a non-existing dir

2017-01-05 Thread Varun Saxena (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Saxena updated YARN-6056: --- Attachment: YARN-6056-branch-2.6.01.patch > Yarn NM using LCE shows a failure when trying to delete a

[jira] [Commented] (YARN-5556) Support for deleting queues without requiring a RM restart

2017-01-05 Thread Xuan Gong (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15802851#comment-15802851 ] Xuan Gong commented on YARN-5556: - sounds good. Will update the design doc > Support for deleting queues

[jira] [Updated] (YARN-4180) AMLauncher does not retry on failures when talking to NM

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4180: - Fix Version/s: 2.8.0 > AMLauncher does not retry on failures when talking to NM >

[jira] [Updated] (YARN-4344) NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4344: - Fix Version/s: 2.8.0 > NMs reconnecting with changed capabilities can lead to wrong cluster resource >

[jira] [Updated] (YARN-4209) RMStateStore FENCED state doesn't work due to updateFencedState called by stateMachine.doTransition

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4209: - Fix Version/s: 2.8.0 > RMStateStore FENCED state doesn't work due to updateFencedState called by >

[jira] [Updated] (YARN-6061) Add a customized uncaughtexceptionhandler for fair scheduler

2017-01-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu updated YARN-6061: --- Component/s: fairscheduler > Add a customized uncaughtexceptionhandler for fair scheduler >

[jira] [Moved] (YARN-6061) Add a customized uncaughtexceptionhandler for fair scheduler

2017-01-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-6061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yufei Gu moved MAPREDUCE-6830 to YARN-6061: --- Component/s: (was: yarn) yarn Key: YARN-6061

[jira] [Commented] (YARN-5798) Handle FSPreemptionThread crashing due to a RuntimeException

2017-01-05 Thread Yufei Gu (JIRA)
[ https://issues.apache.org/jira/browse/YARN-5798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15803084#comment-15803084 ] Yufei Gu commented on YARN-5798: Thanks [~kasha] for the review. I created YARN-6061 to create a customized

[jira] [Updated] (YARN-4556) TestFifoScheduler.testResourceOverCommit fails

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-4556: - Fix Version/s: 2.8.0 > TestFifoScheduler.testResourceOverCommit fails >

[jira] [Updated] (YARN-3457) NPE when NodeManager.serviceInit fails and stopRecoveryStore called

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3457: - Fix Version/s: 2.8.0 > NPE when NodeManager.serviceInit fails and stopRecoveryStore called >

[jira] [Updated] (YARN-3469) ZKRMStateStore: Avoid setting watches that are not required

2017-01-05 Thread Junping Du (JIRA)
[ https://issues.apache.org/jira/browse/YARN-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated YARN-3469: - Fix Version/s: 2.8.0 > ZKRMStateStore: Avoid setting watches that are not required >

  1   2   3   >