[jira] [Commented] (YARN-9851) Make execution type check compatiable
[
https://issues.apache.org/jira/browse/YARN-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941032#comment-16941032
]
zhoukang commented on YARN-9851:
It looks like that patch fix this issue [~bibinchundatt]thanks!
I will try later
> Make execution type check compatiable
> -
>
> Key: YARN-9851
> URL: https://issues.apache.org/jira/browse/YARN-9851
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.1.2
>Reporter: zhoukang
>Assignee: zhoukang
>Priority: Major
> Attachments: YARN-9851-001.patch
>
>
> During upgrade from 2.6 to 3.1, we encountered a problem:
> {code:java}
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568719110875_6460_08_01, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_11172_01_62, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_11172_01_63, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_11172_01_64, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_30617_01_06, status: RUNNING,
> execution type: null
> for (ContainerStatus remoteContainer : containerStatuses) {
> if (remoteContainer.getState() == ContainerState.RUNNING
> && remoteContainer.getExecutionType() == ExecutionType.GUARANTEED) {
> nodeContainers.add(remoteContainer.getContainerId());
> } else {
> LOG.warn("Lost container " + remoteContainer.getContainerId()
> + ", status: " + remoteContainer.getState()
> + ", execution type: " + remoteContainer.getExecutionType());
> }
> }
> {code}
> The cause is that we has nm with version 2.6, which do not have executionType
> for container status.
> We should check here make the upgrade process more tranparently
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9851) Make execution type check compatiable
[
https://issues.apache.org/jira/browse/YARN-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937117#comment-16937117
]
Bibin A Chundatt commented on YARN-9851:
YARN-9547 didn't fix this issue ?
> Make execution type check compatiable
> -
>
> Key: YARN-9851
> URL: https://issues.apache.org/jira/browse/YARN-9851
> Project: Hadoop YARN
> Issue Type: Improvement
>Affects Versions: 3.1.2
>Reporter: zhoukang
>Assignee: zhoukang
>Priority: Major
> Attachments: YARN-9851-001.patch
>
>
> During upgrade from 2.6 to 3.1, we encountered a problem:
> {code:java}
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568719110875_6460_08_01, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_11172_01_62, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_11172_01_63, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_11172_01_64, status: RUNNING,
> execution type: null
> 2019-09-23,19:29:05,303 WARN
> org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Lost
> container container_e35_1568886618758_30617_01_06, status: RUNNING,
> execution type: null
> for (ContainerStatus remoteContainer : containerStatuses) {
> if (remoteContainer.getState() == ContainerState.RUNNING
> && remoteContainer.getExecutionType() == ExecutionType.GUARANTEED) {
> nodeContainers.add(remoteContainer.getContainerId());
> } else {
> LOG.warn("Lost container " + remoteContainer.getContainerId()
> + ", status: " + remoteContainer.getState()
> + ", execution type: " + remoteContainer.getExecutionType());
> }
> }
> {code}
> The cause is that we has nm with version 2.6, which do not have executionType
> for container status.
> We should check here make the upgrade process more tranparently
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9851) Make execution type check compatiable
[
https://issues.apache.org/jira/browse/YARN-9851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16935989#comment-16935989
]
Hadoop QA commented on YARN-9851:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 22m
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 10s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
50s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 30s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 27 unchanged - 0 fixed = 28 total (was 27) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 19s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 83m 7s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}167m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerOvercommit |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/hadoop-multibranch/job/PR-1502/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/1502 |
| JIRA Issue | YARN-9851 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 9f8d1f7121d2 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git r
