[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16146763#comment-16146763 ] Hudson commented on YARN-6323: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12271 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12271/]) YARN-6323. Rolling upgrade/config change is broken on timeline v2. (varunsaxena: rev 9b08f365d749185d7ed8e34dc379b2e415a29e99) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355, 3.0.0-beta1, YARN-5355_branch2 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136941#comment-16136941 ] Haibo Chen commented on YARN-6323: -- I have cherry-picked into YARN-5355_branch2. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355, YARN-5355_branch2 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136606#comment-16136606 ] Varun Saxena commented on YARN-6323: Yeah I will delete it again. I think I checked it in github too when I had deleted it last time. Anyways will do so again. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355, YARN-5355-branch-2 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136599#comment-16136599 ] Rohith Sharma K S commented on YARN-6323: - It seems YARN-5355-branch-2 is not deleted from repository. I do able to see this branch. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355, YARN-5355-branch-2 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136593#comment-16136593 ] Varun Saxena commented on YARN-6323: [~haibo.chen], I noticed that you have committed this to YARN-5355-branch-2 instead of YARN-5355_branch2. We now need to make commits to YARN-5355_branch2. I think I had deleted the branch YARN-5355-branch-2 though. Wonder why it appeared. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355, YARN-5355-branch-2 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135383#comment-16135383 ] Vrushali C commented on YARN-6323: -- Thanks [~haibo.chen] , please go ahead with the commit. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133084#comment-16133084 ] Haibo Chen commented on YARN-6323: -- +1. Will commit it later if no more comments. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch, > YARN-6323-YARN-5355.0002.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[
https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131599#comment-16131599
]
Hadoop QA commented on YARN-6323:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} YARN-5355 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
36s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
51s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
17s{color} | {color:green} YARN-5355 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
55s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in YARN-5355 has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} YARN-5355 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
32s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
0s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 62m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6323 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882476/YARN-6323-YARN-5355.0002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 3316ae1ddfa5 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5355 / dad4163 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/16979/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16979/testReport/ |
| modules | C: hadoop-yarn-project/ha
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16131035#comment-16131035 ] Haibo Chen commented on YARN-6323: -- Agree with [~vrushalic] on that we should focus on fixing the failure on the upgrade path from no ATS at all to ATSv2. The inconsistencies between flow name generated by RM and NM for running applications is probably going to be a limitation. We can file another jira to continue to discuss v1->v2 upgrade path. In RM, the flow name is generated by calling TimelineUtil.generateDefaultFlowName(appName, appId). If appName is null, the resulting flow name will be like flow_app.getClusterTimestamp()_app.getId(). Can we make it consistent here in NM? Otherwise, the patch looks good to me. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Fix For: YARN-5355 > > Attachments: YARN-6323.001.patch, YARN-6323-YARN-5355.0001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[
https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16129676#comment-16129676
]
Hadoop QA commented on YARN-6323:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m
56s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} YARN-5355 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
9s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
49s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} YARN-5355 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
6s{color} | {color:green} YARN-5355 passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in YARN-5355 has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} YARN-5355 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
10s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
2s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 76m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ac17dc |
| JIRA Issue | YARN-6323 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12882225/YARN-6323-YARN-5355.0001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 65410ea93454 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | YARN-5355 / 315ff9b |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/16945/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/16945/testReport/ |
| modules | C: hadoop-yarn-project/h
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121240#comment-16121240 ] Rohith Sharma K S commented on YARN-6323: - Going back through whole discussion on this JIRA, continuing on creating a default flow context and publishing container entities are not at all useful unless RM and NM both creates same flow context. If we still go ahed with default flow context with RM has flowName as appName and NM has flowName as appId then both are written into separate rows. From user perspective, he can't able to retrieve container entities at all anytime unless he gives appId as flowName. Given this is fine for running applications during upgrade, creating default context make sense. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121098#comment-16121098 ] Rohith Sharma K S commented on YARN-6323: - Ahh.. I remember the discussion Vrushali. Thanks for pointed out. Thats true that NM start will fail if flow context is null. Current patch looks reasonable to me. I will take a detailed look at the patch. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[
https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120730#comment-16120730
]
Vrushali C commented on YARN-6323:
--
So this jira YARN-6323 is not for data inconsistencies. It is for dealing with
NM startup failure. If you bring up an NM with atsv2 enabled on a node which
has an app that has been running from before atsv2 was turned on, then NM will
not be able to recover the flow context for this app, since the flow context
never existed before.
Related jira was YARN-6555 in which [~rohithsharma] added the work preserving
flow context storage and retrieval on the NM.
To explain this jira a bit more:
In the patch on YARN-6555
https://issues.apache.org/jira/secure/attachment/12869901/YARN-6555.003.patch
at line 386 in ContainerManagerImpl , if the p.getFlowContext() != null then we
create the Flow Context correctly and pass it in as an argument to
ApplicationImpl on line 393. But if it is null (when it does not exist), then
null FlowContext will be passed to ApplicationImpl and ApplicationImpl
constructor will throw new IllegalArgumentException("flow context cannot be
null");
> Rolling upgrade/config change is broken on timeline v2.
>
>
> Key: YARN-6323
> URL: https://issues.apache.org/jira/browse/YARN-6323
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
>Reporter: Li Lu
>Assignee: Vrushali C
> Labels: yarn-5355-merge-blocker
> Attachments: YARN-6323.001.patch
>
>
> Found this issue when deploying on real clusters. If there are apps running
> when we enable timeline v2 (with work preserving restart enabled), node
> managers will fail to start due to missing app context data. We should
> probably assign some default names to these "left over" apps. I believe it's
> suboptimal to let users clean up the whole cluster before enabling timeline
> v2.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120346#comment-16120346 ] Varun Saxena commented on YARN-6323: bq. this is very hard to enforce it from RM. RM can't differentiate between recovered apps and newly submitted apps. Yeah, we will have to write code to ensure this happens i.e. store a flag in state store (non-existence of which indicates data being written to v1). Just wanted to point out another possibility if we wanted to ensure incomplete app data does not exist. However, as I said this approach has the drawback that we may lose data from v1 if user decides to not take up v2 and its unlikely to be a user scenario too, so I do not suggest following this approach. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120223#comment-16120223 ] Rohith Sharma K S commented on YARN-6323: - bq. we can possibly write entities for running apps only to v1 and from new apps to v2 so we do not get incomplete app data for some apps from both v1 and v2. this is very hard to enforce it from RM. RM can't differentiate between recovered apps and newly submitted apps. RM can write into time line server in non exclusive mode for some time period. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119931#comment-16119931 ] Varun Saxena commented on YARN-6323: Just to put it out there, should we differentiate between these 2 scenarios. I mean between momentary writing of entities to both v1 and v2 for rolling upgrade and writing to v1 and v2 for comparison or evaluation. For rolling upgrade scenario, we can possibly write entities for running apps only to v1 and from new apps to v2 so we do not get incomplete app data for some apps from both v1 and v2. However, most users may want to try out v2 for a while before they fully switch to it. And if we adopt the approach in the sentence above, we may lose data from v1, if the user decides to not take up v2. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119911#comment-16119911 ] Varun Saxena commented on YARN-6323: bq. However YARN-6736 is planning to write into both v1 and v2 timelines during upgrade. I think we should make use of it during rolling upgrade so that RM will publish data into v1 and v2. Makes sense to me if we are doing rolling upgrade from v1 to v2. This way transition would be seamless as the user can switch back to only v2 once he is sure that all the applications running at the time of previous switchover (when both v1 and v2 were configured) have completed. And he can trust the data in v2 completely from a particular application onwards. This would also be useful for those who want to try out v2 before they make a final decision to switch over to v2. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119508#comment-16119508 ] Rohith Sharma K S commented on YARN-6323: - thanks Vrushali for reinitiating this thread. However YARN-6736 is planning to write into both v1 and v2 timelines during upgrade. I think we should make use of it during rolling upgrade so that RM will publish data into v1 and v2. On NM restart during upgrade, it is OK for inconsistencies for running applications. But for newer applications, it should be published properly. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119165#comment-16119165 ] Vrushali C commented on YARN-6323: -- Any thoughts on this [~rohithsharma] [~varun_saxena] [~haibo.chen] > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095682#comment-16095682 ] Vrushali C commented on YARN-6323: -- Ping on this jira. To summarize: - new NM fails to recover apps since the timeline flow context is missing for old apps on the NM. This patch will put in a default flow context to help NM proceed. To answer Rohith's questions: bq Application is NOT submitted with tags. So default values are created by YARN. RM creates default FlowContext with FlowName as appName. On NM restart, we are creating FlowContex with appId. So, there will be a inconsistencies when entities are published during rolling upgrade. Yes, inconsistencies would be there but it is not possible to upgrade the RM and the all the NMs at exactly the time, unless we take a downtime. bq. Assume that Application is submitted with some tags. RM recover the application and start publishing with tags as flow context. Again there is inconsistencies in published entity. Yes, but how to synchronize RM and NM across restarts? We could use app id in both cases but this turns out to be strange default data. This patch will ensure the NM does not fail to start up. I thought of adding in some default values for dropping the data but that will be an expensive check to do each time we want to write to the backend. ping [~rohithsharma] [~varun_saxena] [~haibo.chen] any other ideas? At the very least, the NM can't be crashing during an upgrade due to missing flow context. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16033183#comment-16033183 ] Vrushali C commented on YARN-6323: -- Hmm, using the same thing for flow name at both RM and NM does sound good. But I think using the app name as the default on the RM side is good because if the client does not set the tags, the app name is more meaningful than the app id. I wonder if at the RM side, there is a way to distinguish between regular use of default for flow name and the use during restart with ATSv2 on. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032185#comment-16032185 ] Haibo Chen commented on YARN-6323: -- In rolling upgrade cases where the newer version of YARN cluster has ATSv2 enabled, I think we should try to keep the default flow context for all applications that did not specify flow information consistent across all NMs and RM. Otherwise, RM will think application A is in flow X, whereas NMs would think application A is in flow B, and the result in HBase can be confusing. I wonder how much value it adds for RM to assign the default flow context based on application name (in case application name is absent, it falls back to application id). If that does not matter much, I'd propose both RM and NMs assign default flow context based on application id. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16032023#comment-16032023 ] Vrushali C commented on YARN-6323: -- Hmm, I have been thinking over this and I think we all discussed a bit in the last weekly call too. During upgrade, in any case, there won't be complete information for that flow since some containers would have already finished, some might be running on older nodes, some might start on newer ones. The NM does not have the app name but needs to create a default flow context upon restart. The only thing that I can see it can use is the app id. We could put in a special case to drop the data in the writer if a particular flow context is being used. What I mean is, when the NM restarts with atsv2 enabled for the first time and does not find an existing flow context, we create a specific dummy flow context and we check for that in the writer. If it matches this "drop data" flow context, we simply do not write the data to the backend. With YARN-6555, the work preserving restart will ensure that flow context is written and thus will be available when the NM restarts at later occasions, so the dummy flow context won't be used in the future cases. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024242#comment-16024242 ] Rohith Sharma K S commented on YARN-6323: - Thanks Vrushali and Haibo for discussing on rolling upgrade. I have specific scenario to discuss apart from YARN-6555. If default context is considered from NM then. # Application is NOT submitted with tags. So default values are created by YARN. ## RM creates default FlowContext with FlowName as appName. On NM restart, we are creating FlowContex with appId. So, there will be a inconsistencies when entities are published during rolling upgrade. # Assume that Application is submitted with some tags. ## RM recover the application and start publishing with tags as flow context. Again there is inconsistencies in published entity. How are we going to deal with above cases? > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011262#comment-16011262 ] Haibo Chen commented on YARN-6323: -- Thanks for pointing YARN-6555! I forgot that. A few comments: bq. /** default version for any flow. */ The semantic for the default flow version will change in YARN-6555. Let's update it in YARN-6555 Per comment of ApplicationImpl.flowContext, the flow context for an application is set only if the timeline service v.2 is enabled. With your change, even when ATSv2 is disabled, a default flow context is set. How about we return null from recoverFlowContextFromStateStore if ATSv2 is disabled? > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16011130#comment-16011130 ] Vrushali C commented on YARN-6323: -- Thanks [~haibochen] for the review! Yes, you are right, we do want to store the flow context for the work preserving restart and hence YARN-6555. This jira addresses the case when there just isn't any previous flow context stored, which will happen the first time the cluster is upgraded to atsv2 (as the title suggests). I also added in some comments in the code to reflect this. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16010938#comment-16010938 ] Haibo Chen commented on YARN-6323: -- Thanks for the patch [~vrushalic]! I wonder if we should, to include as part of this jira, store the flow context in the NM state store for a given application. If NM has always been running with work preserving and ATSv2 on, because its flow context is not stored, any application upon recovery will get a default flow context rather than their original flow context. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[
https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008779#comment-16008779
]
Hadoop QA commented on YARN-6323:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
39s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
56s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
33s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
9s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
30s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6323 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867835/YARN-6323.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 8d1d8ba4831b 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 6c35001 |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15917/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15917/testReport/ |
| modules | C: hadoop-ya
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008676#comment-16008676 ] Vrushali C commented on YARN-6323: -- The NM Findbug warnings are unrelated to the patch. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > Attachments: YARN-6323.001.patch > > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[
https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008669#comment-16008669
]
Hadoop QA commented on YARN-6323:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
36s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
57s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
in trunk has 5 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
35s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m
1s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 72m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | YARN-6323 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12867835/YARN-6323.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a5237d5bbcd8 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20
09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a9e24a1 |
| Default Java | 1.8.0_121 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-YARN-Build/15915/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/15915/testReport/ |
| modules | C: hadoop-ya
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16003803#comment-16003803 ] Vrushali C commented on YARN-6323: -- bq. However, in the upgrade path, we could choose to not write anything or have a null writer. So while testing on an NM, I was pleasantly surprised to see this message in the NM logs. We do have defensive code right at the writer end so that we don't end up trying to write nulls to the backend as part of row keys. https://github.com/apache/hadoop/blob/YARN-5355/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/HBaseTimelineWriterImpl.java#L131 That said, the NPE when trying to read previous app state still needs to be fixed. https://github.com/apache/hadoop/blob/YARN-5355/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java#L387 I have patch on that, will upload it shortly. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001570#comment-16001570 ] Joep Rottinghuis commented on YARN-6323: Linking to YARN-6555. However, in the upgrade path, we could choose to not write anything or have a null writer. [~vrushalic] pointed out that some code change is needed to allow null flow context in any case when timeline v2 is enabled. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15959365#comment-15959365 ] Vrushali C commented on YARN-6323: -- We would like to fix this so that we can deploy atsv2 without disruption. > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu >Assignee: Vrushali C > Labels: yarn-5355-merge-blocker > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6323) Rolling upgrade/config change is broken on timeline v2.
[ https://issues.apache.org/jira/browse/YARN-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15905875#comment-15905875 ] Varun Saxena commented on YARN-6323: [~gtCarrera9], should this be a merge blocker? And can you elaborate on the issue? > Rolling upgrade/config change is broken on timeline v2. > > > Key: YARN-6323 > URL: https://issues.apache.org/jira/browse/YARN-6323 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Li Lu > > Found this issue when deploying on real clusters. If there are apps running > when we enable timeline v2 (with work preserving restart enabled), node > managers will fail to start due to missing app context data. We should > probably assign some default names to these "left over" apps. I believe it's > suboptimal to let users clean up the whole cluster before enabling timeline > v2. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
