[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225721#comment-16225721 ] Jason Lowe commented on YARN-7244: -- The ASF warnings are unrelated. +1 for the branch-2.8 patch.

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225477#comment-16225477 ] Kuhu Shukla commented on YARN-7244: --- [~jlowe], request for comments on the 2.8 version of the patch.

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16225413#comment-16225413 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16223006#comment-16223006 ] Hudson commented on YARN-7244: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13150 (See

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16222874#comment-16222874 ] Jason Lowe commented on YARN-7244: -- +1 for the latest patch. The unit test failure is unrelated and

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218980#comment-16218980 ] Sunil G commented on YARN-7244: --- I think latest patch seems fine to me. > ShuffleHandler is not aware of

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16218956#comment-16218956 ] Kuhu Shukla commented on YARN-7244: --- [~jlowe]/[~sunilg] appreciate any comments on the latest patch!

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16209152#comment-16209152 ] Kuhu Shukla commented on YARN-7244: --- [~jlowe], [~sunilg] request for comments/review. Thanks a lot! >

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208749#comment-16208749 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208630#comment-16208630 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208456#comment-16208456 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208336#comment-16208336 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16208089#comment-16208089 ] Kuhu Shukla commented on YARN-7244: --- Thank you [~sunilg] for the comments! bq. Do you think is it better

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16207983#comment-16207983 ] Sunil G commented on YARN-7244: --- Thanks [~kshukla] and [~jlowe] Usage of {{AuxiliaryLocalPathHandler}} looks

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206852#comment-16206852 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206560#comment-16206560 ] Jason Lowe commented on YARN-7244: -- Thanks for updating the patch! Nit: The whitespace separating the

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205971#comment-16205971 ] Kuhu Shukla commented on YARN-7244: --- [~jlowe], request for review/comments on the latest patch. Thanks

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205969#comment-16205969 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205423#comment-16205423 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205338#comment-16205338 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16204107#comment-16204107 ] Jason Lowe commented on YARN-7244: -- I forgot to mention that AuxiliaryLocalPathHandler should be marked

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16203720#comment-16203720 ] Jason Lowe commented on YARN-7244: -- Thanks for updating the patch! The test failure appears to be

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16202595#comment-16202595 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201883#comment-16201883 ] Kuhu Shukla commented on YARN-7244: --- Test failures are related. Will update shortly. > ShuffleHandler is

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201392#comment-16201392 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201163#comment-16201163 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184319#comment-16184319 ] Jason Lowe commented on YARN-7244: -- bq. rather a new api as you mentioned in LocalDirAllocator named

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184239#comment-16184239 ] Kuhu Shukla commented on YARN-7244: --- Thank you [~jlowe], [~sunilg] for the review/comments. bq. We could

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184092#comment-16184092 ] Sunil G commented on YARN-7244: --- bq.the aux service doesn't even have to manage the directories itself if all

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183000#comment-16183000 ] Jason Lowe commented on YARN-7244: -- bq. Only potential issue which I see is that, once a set of dirs are

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182720#comment-16182720 ] Sunil G commented on YARN-7244: --- Thanks [~jlowe] for adding more clarity on this. 'pull' model may be better

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182678#comment-16182678 ] Jason Lowe commented on YARN-7244: -- Thanks for the patch! The core issue here is that the NM is handing

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182450#comment-16182450 ] Kuhu Shukla commented on YARN-7244: --- Thank you [~sunilg] for the review comments! bq. We could push this

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181173#comment-16181173 ] Sunil G commented on YARN-7244: --- Thanks [~kshukla] for this improvement. Some doubts here. # ShuffleHandler

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181059#comment-16181059 ] Kuhu Shukla commented on YARN-7244: --- bq. Better to check directory exists first if we are not concerned

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16180516#comment-16180516 ] Bibin A Chundatt commented on YARN-7244: Thank you [~kshukla] for patch. Minor comments from my

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16177218#comment-16177218 ] Hadoop QA commented on YARN-7244: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-7244) ShuffleHandler is not aware of disks that are added

[ https://issues.apache.org/jira/browse/YARN-7244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16176739#comment-16176739 ] Hadoop QA commented on YARN-7244: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||