[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047563#comment-14047563
]
Hudson commented on YARN-2052:
--
FAILURE: Integrated in Hadoop-Yarn-trunk #599 (See
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047665#comment-14047665
]
Hudson commented on YARN-2052:
--
FAILURE: Integrated in Hadoop-Mapreduce-trunk #1817 (See
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047686#comment-14047686
]
Hudson commented on YARN-2052:
--
SUCCESS: Integrated in Hadoop-Hdfs-trunk #1790 (See
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048216#comment-14048216
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for the review and comments, Jian,
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14048225#comment-14048225
]
Tsuyoshi OZAWA commented on YARN-2052:
--
I'm planning to define epoch format on
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047202#comment-14047202
]
Hudson commented on YARN-2052:
--
SUCCESS: Integrated in Hadoop-trunk-Commit #5799 (See
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047207#comment-14047207
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
Shouldn't epoch have a
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047210#comment-14047210
]
Jian He commented on YARN-2052:
---
bq. For numeric types, the default value is zero.
Copied
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047000#comment-14047000
]
Jian He commented on YARN-2052:
---
found that may be we can change epochProto to use int64
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047020#comment-14047020
]
Jian He commented on YARN-2052:
---
looks good, pending jenkins.
ContainerId creation after
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047026#comment-14047026
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14047031#comment-14047031
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for the review, Jian. The test
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045681#comment-14045681
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045720#comment-14045720
]
Tsuyoshi OZAWA commented on YARN-2052:
--
The test failure of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046419#comment-14046419
]
Jian He commented on YARN-2052:
---
Patch looks good overall, can you update MemoryStateStore
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046504#comment-14046504
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
Not related to this patch,
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046530#comment-14046530
]
Jian He commented on YARN-2052:
---
- Actually, FileSystem and ZK state store has separate
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046544#comment-14046544
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046557#comment-14046557
]
Jian He commented on YARN-2052:
---
can you rename RMEpoch.java to Epoch and similar
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046703#comment-14046703
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14046707#comment-14046707
]
Tsuyoshi OZAWA commented on YARN-2052:
--
The test failure is not related.
ContainerId
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045164#comment-14045164
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for the review, Jian! Updated a
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045240#comment-14045240
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045265#comment-14045265
]
Jian He commented on YARN-2052:
---
hmm, we need to think about toString compatibility too,
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045298#comment-14045298
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
I cursorily looked. Some
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045313#comment-14045313
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Thank you for suggestions, Vinod! I'll update
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045345#comment-14045345
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045373#comment-14045373
]
Tsuyoshi OZAWA commented on YARN-2052:
--
* Renamed RMEpochProto to EpochProto.
* Bump
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14045412#comment-14045412
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14043264#comment-14043264
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14043599#comment-14043599
]
Hadoop QA commented on YARN-2052:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14044059#comment-14044059
]
Jian He commented on YARN-2052:
---
- we can update the structure graph in ZKRMStateStore to
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041494#comment-14041494
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Jian, thank you for clarifying. I'm working to
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041537#comment-14041537
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Brief design is as follows:
1. Adding getter
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041548#comment-14041548
]
Jian He commented on YARN-2052:
---
sounds good
ContainerId creation after work preserving
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041565#comment-14041565
]
Tsuyoshi OZAWA commented on YARN-2052:
--
OK!
ContainerId creation after work
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037463#comment-14037463
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Updated patch to address the comments by
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037481#comment-14037481
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14037875#comment-14037875
]
Jian He commented on YARN-2052:
---
I think the conclusion was to not add any new fields into
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034876#comment-14034876
]
Bikas Saha commented on YARN-2052:
--
With 32 bits for epoch number we have 4 billion
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14036488#comment-14036488
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~bikassaha], [~vinodkv], Sure, I'll update
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034448#comment-14034448
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
bq. BTW, I think we should
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034452#comment-14034452
]
Jian He commented on YARN-2052:
---
Another question is how are we going to show the containerId
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034474#comment-14034474
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Vinod, OK. I'll create new JIRA to address it.
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034541#comment-14034541
]
Jian He commented on YARN-2052:
---
Seems more problem with the randomId approach if user wants
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034637#comment-14034637
]
Tsuyoshi OZAWA commented on YARN-2052:
--
Basically, I agree with the approach. If we
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034691#comment-14034691
]
Bikas Saha commented on YARN-2052:
--
bq. Had an offline discussion with Vinod. Maybe it's
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034702#comment-14034702
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~bikassaha], Yes, I think it's same.
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034716#comment-14034716
]
Jian He commented on YARN-2052:
---
bq. One simple way is to fallback to RM-restart implemented
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034722#comment-14034722
]
Tsuyoshi OZAWA commented on YARN-2052:
--
I meant starting apps from a clean state after
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034731#comment-14034731
]
Bikas Saha commented on YARN-2052:
--
Why would ContainerId#compareTo fail? Existing
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034732#comment-14034732
]
Bikas Saha commented on YARN-2052:
--
Ah. I did not see the rest of the comment. Yes.
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14034746#comment-14034746
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
We should make it a long in the same
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14033101#comment-14033101
]
Jian He commented on YARN-2052:
---
Application itself may possibly use Container.getId to
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031997#comment-14031997
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
bq. BTW, I found that
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032037#comment-14032037
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032053#comment-14032053
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~vinodkv], thank you for the comment. I agree
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032061#comment-14032061
]
Hadoop QA commented on YARN-2052:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030966#comment-14030966
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
bq. e.g. container_XXX_1000
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14030971#comment-14030971
]
Vinod Kumar Vavilapalli commented on YARN-2052:
---
I forgot to add one more
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031223#comment-14031223
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~jianhe] and [~vinodkv], thank you for the
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14031258#comment-14031258
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
The only problem is users who are
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029539#comment-14029539
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~jianhe], I think it's OK after fencing
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029578#comment-14029578
]
Jian He commented on YARN-2052:
---
bq. but one problem is recover() is invoked before the
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14029619#comment-14029619
]
Tsuyoshi OZAWA commented on YARN-2052:
--
[~jianhe], my bad, you're right. I misread
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028707#comment-14028707
]
Jian He commented on YARN-2052:
---
bq. The monotonically increasing sequence could be a
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028717#comment-14028717
]
Tsuyoshi OZAWA commented on YARN-2052:
--
s/epoch number is/epoch number should be/
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028716#comment-14028716
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
we probably need to persist the epoch
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028728#comment-14028728
]
Jian He commented on YARN-2052:
---
On RM startup, RM may need to read and rewrite this epoch
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028734#comment-14028734
]
Bikas Saha commented on YARN-2052:
--
Dont we already read and write synchronously from the
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14028744#comment-14028744
]
Jian He commented on YARN-2052:
---
bq. Dont we already read and write synchronously from the
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13997781#comment-13997781
]
Tsuyoshi OZAWA commented on YARN-2052:
--
{quote}
e.g. container_XXX_1000 after epoch
[
https://issues.apache.org/jira/browse/YARN-2052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13996675#comment-13996675
]
Bikas Saha commented on YARN-2052:
--
The RM identifier is effectively the epoch for the RM.
73 matches
Mail list logo