[jira] [Commented] (YARN-614) Retry attempts automatically for hardware failures or YARN issues and set default app retries to 1

2013-04-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642401#comment-13642401 ] Bikas Saha commented on YARN-614: - 1) Node lost 2) ContainerExitStatus of AM denotes hardwar

[jira] [Commented] (YARN-591) RM recovery related records do not belong to the API

2013-04-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642399#comment-13642399 ] Bikas Saha commented on YARN-591: - Its a clean move. Thanks! I should have done it correct t

[jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container

2013-04-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642387#comment-13642387 ] Bikas Saha commented on YARN-613: - Looks related to YARN-617 but not a duplicate. [~vinodkv]

[jira] [Created] (YARN-614) Retry attempts automatically for hardware failures or YARN issues and set default app retries to 1

2013-04-25 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-614: --- Summary: Retry attempts automatically for hardware failures or YARN issues and set default app retries to 1 Key: YARN-614 URL: https://issues.apache.org/jira/browse/YARN-614 Pr

[jira] [Commented] (YARN-542) Change the default global AM max-attempts value to be not one

2013-04-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642067#comment-13642067 ] Bikas Saha commented on YARN-542: - I think that the global default should be one because an

[jira] [Created] (YARN-613) Create NM proxy per NM instead of per container

2013-04-25 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-613: --- Summary: Create NM proxy per NM instead of per container Key: YARN-613 URL: https://issues.apache.org/jira/browse/YARN-613 Project: Hadoop YARN Issue Type: Sub-task

[jira] [Commented] (YARN-422) Add AM-NM client library

2013-04-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13642038#comment-13642038 ] Bikas Saha commented on YARN-422: - Overall the approach looks fine. The patch looks like wor

[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart

2013-04-24 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641315#comment-13641315 ] Bikas Saha commented on YARN-579: - appToken is attempt specific. So it needs to be store per

[jira] [Commented] (YARN-562) NM should reject containers allocated by previous RM

2013-04-24 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641308#comment-13641308 ] Bikas Saha commented on YARN-562: - Shouldnt the new exception be inheriting from YarnExcepti

[jira] [Updated] (YARN-556) RM Restart phase 2 - Design for work preserving restart

2013-04-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-556: Labels: gsoc2013 (was: ) > RM Restart phase 2 - Design for work preserving restart > ---

[jira] [Commented] (YARN-556) RM Restart phase 2 - Design for work preserving restart

2013-04-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640012#comment-13640012 ] Bikas Saha commented on YARN-556: - Adding brief description of proposal from YARN-128 design

[jira] [Commented] (YARN-422) Add AM-NM client library

2013-04-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639834#comment-13639834 ] Bikas Saha commented on YARN-422: - Sounds good to me to skip the first wrapper step if there

[jira] [Commented] (YARN-422) Add AM-NM client library

2013-04-23 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639660#comment-13639660 ] Bikas Saha commented on YARN-422: - I was hoping that the final AMNMClient library would be a

[jira] [Commented] (YARN-549) YarnClient.submitApplication should wait for application to be accepted by the RM

2013-04-22 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638504#comment-13638504 ] Bikas Saha commented on YARN-549: - Can you please update the patch to add javadoc to the Cli

[jira] [Updated] (YARN-549) YarnClient.submitApplication should wait for application to be accepted by the RM

2013-04-22 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-549: Description: Currently, when submitting an application, storeApplication will be called for recovery. However

[jira] [Updated] (YARN-549) YarnClient.submitApplication should wait for application to be accepted by the RM

2013-04-22 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-549: Summary: YarnClient.submitApplication should wait for application to be accepted by the RM (was: Asynchronou

[jira] [Commented] (YARN-549) Asynchronous application submission

2013-04-22 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638498#comment-13638498 ] Bikas Saha commented on YARN-549: - Looks good. +1. The simpler test made it a cheaper to tes

[jira] [Commented] (YARN-562) NM should reject containers allocated by previous RM

2013-04-22 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638480#comment-13638480 ] Bikas Saha commented on YARN-562: - Please define a YARN constant or something instead of har

[jira] [Commented] (YARN-549) Asynchronous application submission

2013-04-22 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638327#comment-13638327 ] Bikas Saha commented on YARN-549: - The sleep/blocking is already happening inside yarnclieti

[jira] [Commented] (YARN-549) Asynchronous application submission

2013-04-20 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637400#comment-13637400 ] Bikas Saha commented on YARN-549: - This is user facing. Replace with "application has been s

[jira] [Commented] (YARN-594) Update test and add comments in YARN-534

2013-04-19 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637040#comment-13637040 ] Bikas Saha commented on YARN-594: - ok makes sense. +1 > Update test and add

[jira] [Commented] (YARN-594) Update test and add comments in YARN-534

2013-04-19 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637001#comment-13637001 ] Bikas Saha commented on YARN-594: - Arent store operations happening on the store internal di

[jira] [Commented] (YARN-594) Update test and add comments in YARN-534

2013-04-19 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636923#comment-13636923 ] Bikas Saha commented on YARN-594: - Why do we need to change rm1.stop()? {code} -rm1.stop

[jira] [Commented] (YARN-549) Asynchronous application submission

2013-04-19 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636890#comment-13636890 ] Bikas Saha commented on YARN-549: - Rename RM_CLIENT_STATE_POLL_INTERVAL to app submission po

[jira] [Resolved] (YARN-593) container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources

2013-04-19 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha resolved YARN-593. - Resolution: Fixed Fix Version/s: 3.0.0 +1. Committed to trunk. > container launch o

[jira] [Commented] (YARN-593) container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources

2013-04-19 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636706#comment-13636706 ] Bikas Saha commented on YARN-593: - >From >https://issues.apache.org/jira/browse/MAPREDUCE-4

[jira] [Commented] (YARN-562) NM should reject containers allocated by previous RM

2013-04-18 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636102#comment-13636102 ] Bikas Saha commented on YARN-562: - The name clusterTimeStamp confuses me. What time is it? T

[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

2013-04-18 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636077#comment-13636077 ] Bikas Saha commented on YARN-392: - bq. would it not make sense to go with the convention use

[jira] [Commented] (YARN-585) TestFairScheduler#testNotAllowSubmitApplication is broken due to YARN-514

2013-04-18 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635454#comment-13635454 ] Bikas Saha commented on YARN-585: - It looks like the test is completely bypassing the state

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-17 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634210#comment-13634210 ] Bikas Saha commented on YARN-514: - Looks great. Thanks for being patient with my comments. O

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-16 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633072#comment-13633072 ] Bikas Saha commented on YARN-514: - I am thinking of the following RMAppManager.recover can c

[jira] [Resolved] (YARN-248) Security related work for RM restart

2013-04-16 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha resolved YARN-248. - Resolution: Duplicate > Security related work for RM restart > > >

[jira] [Commented] (YARN-248) Security related work for RM restart

2013-04-16 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633007#comment-13633007 ] Bikas Saha commented on YARN-248: - Closing this because YARN-581, YARN-582 etc have been ope

[jira] [Created] (YARN-581) Test and verify that app delegation tokens are restored after RM restart

2013-04-16 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-581: --- Summary: Test and verify that app delegation tokens are restored after RM restart Key: YARN-581 URL: https://issues.apache.org/jira/browse/YARN-581 Project: Hadoop YARN

[jira] [Created] (YARN-582) Restore appToken and clientToken for app attempt after RM restart

2013-04-16 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-582: --- Summary: Restore appToken and clientToken for app attempt after RM restart Key: YARN-582 URL: https://issues.apache.org/jira/browse/YARN-582 Project: Hadoop YARN Issu

[jira] [Commented] (YARN-579) Make ApplicationToken part of Container's token list to help RM-restart

2013-04-16 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632992#comment-13632992 ] Bikas Saha commented on YARN-579: - In order to help restart, creation of appToken needs to m

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-16 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632975#comment-13632975 ] Bikas Saha commented on YARN-514: - StartAppAttemptTransition and RMAppStartingOrSavingTransi

[jira] [Commented] (YARN-45) Scheduler feedback to AM to release containers

2013-04-15 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632542#comment-13632542 ] Bikas Saha commented on YARN-45: I took a quick look at this patch and the others and from wh

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-15 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632536#comment-13632536 ] Bikas Saha commented on YARN-514: - Sorry for not catching this earlier. RMAppEventType.APP_S

[jira] [Commented] (YARN-445) Ability to signal containers

2013-04-15 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632178#comment-13632178 ] Bikas Saha commented on YARN-445: - IMO it would be great if the API allows YARN/NM to figure

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-15 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632081#comment-13632081 ] Bikas Saha commented on YARN-514: - If we are going to change the enum values then we might a

[jira] [Commented] (YARN-45) Scheduler feedback to AM to release containers

2013-04-14 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631400#comment-13631400 ] Bikas Saha commented on YARN-45: My personal preference would be to not have an API that is n

[jira] [Commented] (YARN-445) Ability to signal containers

2013-04-14 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13631399#comment-13631399 ] Bikas Saha commented on YARN-445: - Sounds like an enhancement in the NM API. Moving under YA

[jira] [Updated] (YARN-445) Ability to signal containers

2013-04-14 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-445: Issue Type: Sub-task (was: New Feature) Parent: YARN-386 > Ability to signal containers > --

[jira] [Updated] (YARN-386) [Umbrella] YARN API Changes

2013-04-14 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-386: Summary: [Umbrella] YARN API Changes (was: [Umbrella] YARN API cleanup) > [Umbrella] YARN API Changes >

[jira] [Commented] (YARN-45) Scheduler feedback to AM to release containers

2013-04-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630769#comment-13630769 ] Bikas Saha commented on YARN-45: I like the idea of the RM giving information to the AM about

[jira] [Commented] (YARN-513) Verify all clients will wait for RM to restart

2013-04-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630677#comment-13630677 ] Bikas Saha commented on YARN-513: - What about other interactions with the RM such as allocat

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-12 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13630320#comment-13630320 ] Bikas Saha commented on YARN-514: - you only need to add the new field in the enum. I dont th

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629745#comment-13629745 ] Bikas Saha commented on YARN-514: - For MAPREDUCE-5140 please check for uses of both NEW and

[jira] [Commented] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629742#comment-13629742 ] Bikas Saha commented on YARN-514: - Looks good overall. Minor tab issues in the patch. I don

[jira] [Commented] (YARN-45) Scheduler feedback to AM to release containers

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629662#comment-13629662 ] Bikas Saha commented on YARN-45: Moved to sub-task of YARN-397 for scheduler API changes.

[jira] [Updated] (YARN-45) Scheduler feedback to AM to release containers

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-45: --- Issue Type: Sub-task (was: Improvement) Parent: YARN-397 > Scheduler feedback to AM to release cont

[jira] [Updated] (YARN-45) Scheduler feedback to AM to release containers

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-45: --- Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-386) > Scheduler feedback to AM to r

[jira] [Commented] (YARN-45) Scheduler feedback to AM to release containers

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13629620#comment-13629620 ] Bikas Saha commented on YARN-45: All API changes at this point are being tracked under YARN-3

[jira] [Updated] (YARN-45) Scheduler feedback to AM to release containers

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-45?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-45: --- Issue Type: Sub-task (was: Improvement) Parent: YARN-386 > Scheduler feedback to AM to release cont

[jira] [Updated] (YARN-563) Add application type to ApplicationReport

2013-04-11 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-563: Issue Type: Sub-task (was: Improvement) Parent: YARN-386 > Add application type to ApplicationRe

[jira] [Commented] (YARN-534) AM max attempts is not checked when RM restart and try to recover attempts

2013-04-10 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628015#comment-13628015 ] Bikas Saha commented on YARN-534: - There needs to be a comment here that this logic needs to

[jira] [Updated] (YARN-561) Nodemanager should set some key information into the environment of every container that it launches.

2013-04-10 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-561: Assignee: Omkar Vinit Joshi > Nodemanager should set some key information into the environment of every

[jira] [Created] (YARN-559) Make all YARN API and libraries available through an api jar

2013-04-09 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-559: --- Summary: Make all YARN API and libraries available through an api jar Key: YARN-559 URL: https://issues.apache.org/jira/browse/YARN-559 Project: Hadoop YARN Issue Typ

[jira] [Commented] (YARN-534) AM max attempts is not checked when RM restart and try to recover attempts

2013-04-08 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625802#comment-13625802 ] Bikas Saha commented on YARN-534: - Looks good overall. Can the message be improved a bit? e

[jira] [Created] (YARN-556) RM Restart phase 2 - Design for work preserving restart

2013-04-08 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-556: --- Summary: RM Restart phase 2 - Design for work preserving restart Key: YARN-556 URL: https://issues.apache.org/jira/browse/YARN-556 Project: Hadoop YARN Issue Type: Sub

[jira] [Commented] (YARN-479) NM retry behavior for connection to RM should be similar for lost heartbeats

2013-04-08 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13625688#comment-13625688 ] Bikas Saha commented on YARN-479: - Thanks! +1. > NM retry behavior for conn

[jira] [Commented] (YARN-549) Asynchronous application submission

2013-04-06 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624526#comment-13624526 ] Bikas Saha commented on YARN-549: - I dont think its an option for a client (eg YARNRunner) t

[jira] [Updated] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-04-06 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-193: Fix Version/s: 2.0.5-beta > Scheduler.normalizeRequest does not account for allocation requests that > e

[jira] [Commented] (YARN-479) NM retry behavior for connection to RM should be similar for lost heartbeats

2013-04-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13623979#comment-13623979 ] Bikas Saha commented on YARN-479: - Patch looks good overall! why is this line showing as a

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-04-05 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13623848#comment-13623848 ] Bikas Saha commented on YARN-193: - +1. Thanks Zhijie. Committed to trunk and branch-2.

[jira] [Created] (YARN-545) NodeResourceMonitor and its Impl are emty and may be removed

2013-04-04 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-545: --- Summary: NodeResourceMonitor and its Impl are emty and may be removed Key: YARN-545 URL: https://issues.apache.org/jira/browse/YARN-545 Project: Hadoop YARN Issue Typ

[jira] [Commented] (YARN-196) Nodemanager should be more robust in handling connection failure to ResourceManager when a cluster is started

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622785#comment-13622785 ] Bikas Saha commented on YARN-196: - here is a finally block which will make the code sleeping

[jira] [Commented] (YARN-479) NM retry behavior for connection to RM should be similar for lost heartbeats

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622783#comment-13622783 ] Bikas Saha commented on YARN-479: - I dont see the value of waitForever if we can specify a l

[jira] [Commented] (YARN-534) AM max attempts is not checked when RM restart and try to recover attempts

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622685#comment-13622685 ] Bikas Saha commented on YARN-534: - Turns out that the max attempts limit is checked when job

[jira] [Commented] (YARN-540) RM state store not cleaned if job succeeds but RM shutdown and restart-dispatcher stopped before it can process REMOVE_APP event

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622681#comment-13622681 ] Bikas Saha commented on YARN-540: - This is a known issue. The problem here is that the rm st

[jira] [Updated] (YARN-529) MR job succeeds and exits even when unregister with RM fails

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-529: Summary: MR job succeeds and exits even when unregister with RM fails (was: Succeeded MR job is retried by R

[jira] [Updated] (YARN-495) Change NM behavior of reboot to resync

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-495: Summary: Change NM behavior of reboot to resync (was: Containers are not terminated when the NM is rebooted)

[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622584#comment-13622584 ] Bikas Saha commented on YARN-392: - bq. I'm against using timers for specifying locality dela

[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

2013-04-04 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622544#comment-13622544 ] Bikas Saha commented on YARN-392: - I dont think its possible for location specific and non-l

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-04-03 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13621073#comment-13621073 ] Bikas Saha commented on YARN-193: - These values need to be on the conservative side so that

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620638#comment-13620638 ] Bikas Saha commented on YARN-193: - Default value of max-vcores of 32 might be too high. Why

[jira] [Updated] (YARN-529) Succeeded MR job is retried by RM if finishApplicationMaster() call fails

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-529: Issue Type: Improvement (was: Sub-task) Parent: (was: YARN-128) > Succeeded MR job is retrie

[jira] [Commented] (YARN-529) Succeeded MR job is retried by RM if finishApplicationMaster() call fails

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620235#comment-13620235 ] Bikas Saha commented on YARN-529: - This problem is related to RM Restart but independent of

[jira] [Updated] (YARN-529) Succeeded MR job is retried by RM if finishApplicationMaster() call fails

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-529: Summary: Succeeded MR job is retried by RM if finishApplicationMaster() call fails (was: Succeeded RM job is

[jira] [Updated] (YARN-529) Succeeded RM job is retried by RM if finishApplicationMaster() call fails

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-529: Summary: Succeeded RM job is retried by RM if finishApplicationMaster() call fails (was: MR app master clean

[jira] [Commented] (YARN-529) MR app master clean staging dir when reboot command sent from RM while the MR job succeeded

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620230#comment-13620230 ] Bikas Saha commented on YARN-529: - By 1) you mean let RM accept finishApplicationAttempt() f

[jira] [Updated] (YARN-529) MR app master clean staging dir when reboot command sent from RM while the MR job succeeded

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated YARN-529: Description: MR app master will clean staging dir, if the job is already succeeded and asked to reboot. If th

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620091#comment-13620091 ] Bikas Saha commented on YARN-193: - Also, why are there so many normalize functions and why a

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620066#comment-13620066 ] Bikas Saha commented on YARN-193: - Can we check that we are getting the expected exception a

[jira] [Commented] (YARN-382) SchedulerUtils improve way normalizeRequest sets the resource capabilities

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620057#comment-13620057 ] Bikas Saha commented on YARN-382: - +1 looks good to me. > SchedulerUtils im

[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13620008#comment-13620008 ] Bikas Saha commented on YARN-392: - Yes YARN-398 but not the proposal currently in there. The

[jira] [Commented] (YARN-444) Move special container exit codes from YarnConfiguration to API

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619584#comment-13619584 ] Bikas Saha commented on YARN-444: - IMO when the container exits because YARN took some speci

[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

2013-04-02 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13619578#comment-13619578 ] Bikas Saha commented on YARN-392: - Sorry, I did not see that patch carefully and assumed tha

[jira] [Commented] (YARN-444) Move special container exit codes from YarnConfiguration to API

2013-04-01 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618991#comment-13618991 ] Bikas Saha commented on YARN-444: - You can look QueueState, YarnApplicationState, NodeState,

[jira] [Commented] (YARN-291) Dynamic resource configuration on NM

2013-04-01 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618681#comment-13618681 ] Bikas Saha commented on YARN-291: - bq. because NM doesn't really care about what resource it

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-03-31 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618493#comment-13618493 ] Bikas Saha commented on YARN-193: - Also, do we really need to create a new Resource object e

[jira] [Commented] (YARN-486) Change startContainer NM API to accept Container as a parameter and make ContainerLaunchContext user land

2013-03-31 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618491#comment-13618491 ] Bikas Saha commented on YARN-486: - Once this change is made there is no need to copy the amC

[jira] [Commented] (YARN-382) SchedulerUtils improve way normalizeRequest sets the resource capabilities

2013-03-31 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618489#comment-13618489 ] Bikas Saha commented on YARN-382: - Given, YARN-193 is only fixing validation and this copyin

[jira] [Commented] (YARN-193) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2013-03-31 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618487#comment-13618487 ] Bikas Saha commented on YARN-193: - This and others like it are back-incompatible but might b

[jira] [Commented] (YARN-392) Make it possible to schedule to specific nodes without dropping locality

2013-03-31 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618475#comment-13618475 ] Bikas Saha commented on YARN-392: - How about calling is disableLocalityRelaxation as thats w

[jira] [Commented] (YARN-444) Move special container exit codes from YarnConfiguration to API

2013-03-31 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13618470#comment-13618470 ] Bikas Saha commented on YARN-444: - Patch looks mostly good. I wonder why the plural has been

[jira] [Commented] (YARN-450) Define value for * in the scheduling protocol

2013-03-28 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/YARN-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13616584#comment-13616584 ] Bikas Saha commented on YARN-450: - +1. Committed to trunk and branch-2. Thanks Zhijie!

[jira] [Created] (YARN-514) Delayed store operations should not result in RM unavailability for app submission

2013-03-28 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-514: --- Summary: Delayed store operations should not result in RM unavailability for app submission Key: YARN-514 URL: https://issues.apache.org/jira/browse/YARN-514 Project: Hadoop YA

[jira] [Created] (YARN-513) Verify all clients will wait for RM to restart

2013-03-28 Thread Bikas Saha (JIRA)
Bikas Saha created YARN-513: --- Summary: Verify all clients will wait for RM to restart Key: YARN-513 URL: https://issues.apache.org/jira/browse/YARN-513 Project: Hadoop YARN Issue Type: Sub-task

<    5   6   7   8   9   10   11   12   13   14   >