[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-06-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512924#comment-16512924 ] Haibo Chen commented on YARN-8250: -- Thanks [~kkaranasos] for your comments and suggestions! {quote}I

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-06-12 Thread Konstantinos Karanasos (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510299#comment-16510299 ] Konstantinos Karanasos commented on YARN-8250: -- Just went through your comments. I am

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-06-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508385#comment-16508385 ] Haibo Chen commented on YARN-8250: -- In the meantime, I'll try to see how well YARN-6675 works with some

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-06-11 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508382#comment-16508382 ] Haibo Chen commented on YARN-8250: -- My apologies, Arun. I did not mean to indicate in any way you are

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-06-08 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506607#comment-16506607 ] Arun Suresh commented on YARN-8250: --- Apologize for the late reply. {quote} 3) Upon any container

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-06-08 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16506296#comment-16506296 ] Haibo Chen commented on YARN-8250: -- Ping [~asuresh].  > Create another implementation of

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-31 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496872#comment-16496872 ] Haibo Chen commented on YARN-8250: -- [~asuresh], [~leftnoteasy] and I had an offline discussion about this

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476384#comment-16476384 ] Haibo Chen commented on YARN-8250: -- [~leftnoteasy] I can introduce a plug-gable policy that encapsulates

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-15 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476376#comment-16476376 ] Haibo Chen commented on YARN-8250: -- Thanks [~asuresh] for the response. {quote} the whole point of

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476312#comment-16476312 ] Arun Suresh commented on YARN-8250: --- Also with regard to this: {quote} Minimize impact on GUARANTEED

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-15 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476292#comment-16476292 ] Arun Suresh commented on YARN-8250: --- [~haibochen], with regard to this: {quote} avoid aggressive

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-15 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476088#comment-16476088 ] Wangda Tan commented on YARN-8250: -- Thanks [~haibochen] for the detailed explanation, bq. Not sure what

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475100#comment-16475100 ] Haibo Chen commented on YARN-8250: -- [~leftnoteasy] Thanks for your comments. I agree that we should avoid

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-14 Thread Wangda Tan (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475062#comment-16475062 ] Wangda Tan commented on YARN-8250: -- [~haibochen],  I took a very brief look at implemented code since I

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475011#comment-16475011 ] Haibo Chen commented on YARN-8250: -- My understanding of SHED_QUEUED_CONTAINERS is to notify container

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-14 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474808#comment-16474808 ] Arun Suresh commented on YARN-8250: --- [~haibochen], I am not entirely convinced we really need to make the

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-14 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474715#comment-16474715 ] Haibo Chen commented on YARN-8250: -- [~asuresh] Did you get a change to look at the patch? > Create

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Arun Suresh (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469628#comment-16469628 ] Arun Suresh commented on YARN-8250: --- [~haibochen], [~miklos.szeg...@cloudera.com].. do give me a day or

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469577#comment-16469577 ] Haibo Chen commented on YARN-8250: -- Not sure how to address the last checkstyle issue, because we'd ignore

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469553#comment-16469553 ] Miklos Szegedi commented on YARN-8250: -- I think the first and the last checkstyle comments could be

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469548#comment-16469548 ] genericqa commented on YARN-8250: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469398#comment-16469398 ] Haibo Chen commented on YARN-8250: -- This is based on UpdateContainerTokenEvent. There are four

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469378#comment-16469378 ] Miklos Szegedi commented on YARN-8250: -- Thank you for the updated patch [~haibochen]. {code:java} if

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469356#comment-16469356 ] Haibo Chen commented on YARN-8250: -- The unit test failures are unrelated. The ones in

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469315#comment-16469315 ] genericqa commented on YARN-8250: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-09 Thread Haibo Chen (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16469001#comment-16469001 ] Haibo Chen commented on YARN-8250: -- Thanks for the review, [~miklos.szeg...@cloudera.com]!

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-08 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468120#comment-16468120 ] genericqa commented on YARN-8250: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation

2018-05-08 Thread Miklos Szegedi (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468020#comment-16468020 ] Miklos Szegedi commented on YARN-8250: -- Thanks for the patch, [~haibochen]. ContainerScheduler could