[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144668#comment-14144668 ] Hudson commented on YARN-1372: -- FAILURE: Integrated in Hadoop-Yarn-trunk #689 (See

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144823#comment-14144823 ] Hudson commented on YARN-1372: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1880 (See

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14144800#comment-14144800 ] Hudson commented on YARN-1372: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1905 (See

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14143473#comment-14143473 ] Jian He commented on YARN-1372: --- +1 for the latest patch, committing Ensure all completed

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14140052#comment-14140052 ] Jian He commented on YARN-1372: --- Thanks for updating! looks good overall, some comments: - we

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141070#comment-14141070 ] Anubhav Dhoot commented on YARN-1372: - Addressed everything Regarding

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141337#comment-14141337 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141405#comment-14141405 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141418#comment-14141418 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141534#comment-14141534 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14141597#comment-14141597 ] Hadoop QA commented on YARN-1372: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14139887#comment-14139887 ] Hadoop QA commented on YARN-1372: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14138083#comment-14138083 ] Jian He commented on YARN-1372: --- We can probably do this: - Transfer both

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14134762#comment-14134762 ] Anubhav Dhoot commented on YARN-1372: - A finishedContainer that was sent to previous AM

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131865#comment-14131865 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131986#comment-14131986 ] Hadoop QA commented on YARN-1372: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132206#comment-14132206 ] Jian He commented on YARN-1372: --- Thanks for updating the patch! I think we are very close, -

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132271#comment-14132271 ] Anubhav Dhoot commented on YARN-1372: - About the finishedContainersSentToAM in

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14132294#comment-14132294 ] Jian He commented on YARN-1372: --- bq. Why not ack the finishedContainersSentToAM containers to

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130554#comment-14130554 ] Anubhav Dhoot commented on YARN-1372: - why adding context.getContainers().remove(cid);

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14130691#comment-14130691 ] Jian He commented on YARN-1372: --- Thanks for your explanation, bq. Is is possible that for

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14131097#comment-14131097 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129359#comment-14129359 ] Hadoop QA commented on YARN-1372: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14129459#comment-14129459 ] Jian He commented on YARN-1372: --- Thanks for updating the patch. Some comments and naming

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14118436#comment-14118436 ] Jian He commented on YARN-1372: --- looked at patch YARN-1372.002_NMHandlesCompletedApp, I

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14112830#comment-14112830 ] Jian He commented on YARN-1372: --- bq. I looked into an alternate approach where the RM acks

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111235#comment-14111235 ] Jian He commented on YARN-1372: --- [~adhoot], Any thoughts on my last comments ? bq. the same

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111713#comment-14111713 ] Anubhav Dhoot commented on YARN-1372: - bq. I meant is it possible for NM at

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14111718#comment-14111718 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14102058#comment-14102058 ] Anubhav Dhoot commented on YARN-1372: - The tests that failed are all passing

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14101568#comment-14101568 ] Jian He commented on YARN-1372: --- bq. Not sure if there is an easier way to link the two right

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099545#comment-14099545 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14099639#comment-14099639 ] Hadoop QA commented on YARN-1372: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14095049#comment-14095049 ] Jian He commented on YARN-1372: --- bq. maybe we can just use

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14082967#comment-14082967 ] Jian He commented on YARN-1372: --- Thanks for working on this. Scan through the patch: - If

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068879#comment-14068879 ] Anubhav Dhoot commented on YARN-1372: - Yes. Working on this now. Ensure all completed

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14068074#comment-14068074 ] Jian He commented on YARN-1372: --- {code} NM informs RM and holds on to the information

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14001563#comment-14001563 ] Steve Loughran commented on YARN-1372: -- how long is AM restart likely to take? Should

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995354#comment-13995354 ] Jian He commented on YARN-1372: --- An alternative would be to make NM remember the current

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995516#comment-13995516 ] Bikas Saha commented on YARN-1372: -- what happens when they are 100's of long jobs running

[jira] [Commented] (YARN-1372) Ensure all completed containers are reported to the AMs across RM restart

[ https://issues.apache.org/jira/browse/YARN-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13995247#comment-13995247 ] Karthik Kambatla commented on YARN-1372: Based on offline discussion with Anubhav,