[jira] [Created] (YARN-11671) Tests in hadoop-yarn-server-router are not running
Ayush Saxena created YARN-11671:
---
Summary: Tests in hadoop-yarn-server-router are not running
Key: YARN-11671
URL: https://issues.apache.org/jira/browse/YARN-11671
Project: Hadoop YARN
Issue Type: Bug
Reporter: Ayush Saxena
[https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1549/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt]
{noformat}
[INFO] --- maven-surefire-plugin:3.0.0-M1:test (default-test) @
hadoop-yarn-server-router ---
[INFO]
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11563) Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment
[ https://issues.apache.org/jira/browse/YARN-11563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11563: Summary: Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment (was: Fix word misspellings from CSAssignemnt to CSAssignment) > Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment > > > Key: YARN-11563 > URL: https://issues.apache.org/jira/browse/YARN-11563 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.3.3, 3.3.4, 3.3.6 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Trivial > Attachments: image-2023-12-11-17-55-21-930.png > > > misspellings of CSAssignemnt which shoud be CSAssignment in > AbstractContainerAllocator.java > !image-2023-12-11-17-55-21-930.png|width=389,height=159! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11563) Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment
[ https://issues.apache.org/jira/browse/YARN-11563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17798662#comment-17798662 ] Ayush Saxena commented on YARN-11563: - Committed to trunk. Thanx [~zhongwei11] for the contribution!!! > Fix typo in AbstractContainerAllocator from CSAssignemnt to CSAssignment > > > Key: YARN-11563 > URL: https://issues.apache.org/jira/browse/YARN-11563 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.3.3, 3.3.4, 3.3.6 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Trivial > Labels: pull-request-available > Attachments: image-2023-12-11-17-55-21-930.png > > > misspellings of CSAssignemnt which shoud be CSAssignment in > AbstractContainerAllocator.java > !image-2023-12-11-17-55-21-930.png|width=389,height=159! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11563) Fix word misspellings from CSAssignemnt to CSAssignment
[ https://issues.apache.org/jira/browse/YARN-11563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11563: --- Assignee: wangzhongwei > Fix word misspellings from CSAssignemnt to CSAssignment > --- > > Key: YARN-11563 > URL: https://issues.apache.org/jira/browse/YARN-11563 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacityscheduler >Affects Versions: 3.3.3, 3.3.4, 3.3.6 >Reporter: wangzhongwei >Assignee: wangzhongwei >Priority: Trivial > Attachments: image-2023-12-11-17-55-21-930.png > > > misspellings of CSAssignemnt which shoud be CSAssignment in > AbstractContainerAllocator.java > !image-2023-12-11-17-55-21-930.png|width=389,height=159! -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9568) NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover
[
https://issues.apache.org/jira/browse/YARN-9568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17783707#comment-17783707
]
Ayush Saxena commented on YARN-9568:
I believe this fix breaks the Windows test(HDFS-17246)
> NPE in MiniYarnCluster during FileSystemNodeAttributeStore.recover
> --
>
> Key: YARN-9568
> URL: https://issues.apache.org/jira/browse/YARN-9568
> Project: Hadoop YARN
> Issue Type: Bug
> Components: resourcemanager, test
>Affects Versions: 3.3.0
> Environment: macos
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Fix For: 3.3.0
>
> Attachments: YARN-9568.001.patch, YARN-9568.002.patch, npe.log
>
>
> This seems new in trunk. As in "wasn't happening a couple of weeks ago". Its
> surfacing in the S3A committer tests which are trying to create
> MiniYarnClusters: all such tests are failing as the mini yarn cluster won't
> come up with an NPE in {{FileSystemNodeAttributeStore.recover}}
> I'm not sure why node labels are needed on test clusters; the default implies
> they should be off anyway.
> At the same time, I can't seem to find one specific change in the git log to
> say "this is causing the problem".
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11607) TestTimelineAuthFilterForV2 fails intermittently
Ayush Saxena created YARN-11607:
---
Summary: TestTimelineAuthFilterForV2 fails intermittently
Key: YARN-11607
URL: https://issues.apache.org/jira/browse/YARN-11607
Project: Hadoop YARN
Issue Type: Improvement
Reporter: Ayush Saxena
Ref:
https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1398/testReport/junit/org.apache.hadoop.yarn.server.timelineservice.security/TestTimelineAuthFilterForV2/testPutTimelineEntities_boolean__boolean__3_/
{noformat}
org.opentest4j.AssertionFailedError: expected: <2> but was: <1>
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:55)
at
org.junit.jupiter.api.AssertionUtils.failNotEqual(AssertionUtils.java:62)
at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
at
org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:527)
at
org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.publishAndVerifyEntity(TestTimelineAuthFilterForV2.java:324)
at
org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.publishWithRetries(TestTimelineAuthFilterForV2.java:337)
at
org.apache.hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2.testPutTimelineEntities(TestTimelineAuthFilterForV2.java:383)
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11540) Fix typo: form -> from
[
https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved YARN-11540.
-
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Fix typo: form -> from
> --
>
> Key: YARN-11540
> URL: https://issues.apache.org/jira/browse/YARN-11540
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Seokchan Yoon
>Assignee: Seokchan Yoon
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> There is a typo in a log message that NodeManager prints:
> > Retrieved credentials form RM for {}: {}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11540) Fix typo: form -> from
[
https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745234#comment-17745234
]
Ayush Saxena commented on YARN-11540:
-
Committed to trunk.
Thanx [~yoonsch217] for the contribution & [~tomscut] for the review!!!
> Fix typo: form -> from
> --
>
> Key: YARN-11540
> URL: https://issues.apache.org/jira/browse/YARN-11540
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Seokchan Yoon
>Assignee: Seokchan Yoon
>Priority: Trivial
> Labels: pull-request-available
>
> There is a typo in a log message that NodeManager prints:
> > Retrieved credentials form RM for {}: {}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11540) Fix typo: form -> from
[
https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-11540:
Priority: Trivial (was: Minor)
> Fix typo: form -> from
> --
>
> Key: YARN-11540
> URL: https://issues.apache.org/jira/browse/YARN-11540
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Seokchan Yoon
>Assignee: Seokchan Yoon
>Priority: Trivial
> Labels: pull-request-available
>
> There is a typo in a log message that NodeManager prints:
> > Retrieved credentials form RM for {}: {}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11540) Fix typo: form -> from
[
https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena reassigned YARN-11540:
---
Assignee: Seokchan Yoon
> Fix typo: form -> from
> --
>
> Key: YARN-11540
> URL: https://issues.apache.org/jira/browse/YARN-11540
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Seokchan Yoon
>Assignee: Seokchan Yoon
>Priority: Minor
> Labels: pull-request-available
>
> There is a typo in a log message that NodeManager prints:
> > Retrieved credentials form RM for {}: {}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-11540) Fix typo: form -> from
[
https://issues.apache.org/jira/browse/YARN-11540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena moved HADOOP-18813 to YARN-11540:
--
Component/s: (was: common)
Key: YARN-11540 (was: HADOOP-18813)
Project: Hadoop YARN (was: Hadoop Common)
> Fix typo: form -> from
> --
>
> Key: YARN-11540
> URL: https://issues.apache.org/jira/browse/YARN-11540
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Seokchan Yoon
>Priority: Minor
> Labels: pull-request-available
>
> There is a typo in a log message that NodeManager prints:
> > Retrieved credentials form RM for {}: {}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11528) Lock triple-beam to the version compatible with node.js 12 to avoid compilation error
[
https://issues.apache.org/jira/browse/YARN-11528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741872#comment-17741872
]
Ayush Saxena commented on YARN-11528:
-
Committed to trunk & branch-3.3
Thanx [~iwasakims] for the contribution & [~slfan1989] for the review!!!
> Lock triple-beam to the version compatible with node.js 12 to avoid
> compilation error
> -
>
> Key: YARN-11528
> URL: https://issues.apache.org/jira/browse/YARN-11528
> Project: Hadoop YARN
> Issue Type: Bug
> Components: build
>Affects Versions: 3.4.0
>Reporter: Ayush Saxena
>Assignee: Masatake Iwasaki
>Priority: Major
> Labels: pull-request-available
>
> {code:java}
> 2023-07-10T07:32:20.1069856Z [INFO] yarn install v1.22.5
> 2023-07-10T07:32:20.1435049Z [INFO] info No lockfile found.
> 2023-07-10T07:32:20.1489685Z [INFO] [1/4] Resolving packages...
> 2023-07-10T07:32:20.9138642Z [INFO] warning angular-route@1.6.10: For the
> actively supported Angular, see https://www.npmjs.com/package/@angular/core.
> AngularJS support has officially ended. For extended AngularJS support
> options, see https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:20.9433934Z [INFO] warning angular@1.6.10: For the actively
> supported Angular, see https://www.npmjs.com/package/@angular/core. AngularJS
> support has officially ended. For extended AngularJS support options, see
> https://goo.gle/angularjs-path-forward.
> 2023-07-10T07:32:21.5051267Z [INFO] [2/4] Fetching packages...
> 2023-07-10T07:32:23.2554282Z [INFO] error triple-beam@1.4.0: The engine
> "node" is incompatible with this module. Expected version ">= 16.0.0". Got
> "12.22.1"
> 2023-07-10T07:32:23.2651719Z [INFO] error Found incompatible module.
> 2023-07-10T07:32:23.2664307Z [INFO] info Visit
> https://yarnpkg.com/en/docs/cli/install for documentation about this command.
> {code}
> Ref:
> https://pipelines.actions.githubusercontent.com/serviceHosts/a3203f99-c5b3-4def-b81d-c5a8ebd0356d/_apis/pipelines/1/runs/294/signedlogcontent/2?urlExpires=2023-07-10T09%3A08%3A57.5185178Z=HMACV1=OHImbS%2FdsTBFKRxwRnV6jFPnumE0GxbB%2BalrVpAgmyw%3D
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11506) The formatted yarn queue list is displayed on the command line
[ https://issues.apache.org/jira/browse/YARN-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11506. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > The formatted yarn queue list is displayed on the command line > -- > > Key: YARN-11506 > URL: https://issues.apache.org/jira/browse/YARN-11506 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > h4. In the issue list YARN-11474, the printed information is not aligned,This > issue fixes the yarn queue display formatting problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11506) The formatted yarn queue list is displayed on the command line
[ https://issues.apache.org/jira/browse/YARN-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733883#comment-17733883 ] Ayush Saxena commented on YARN-11506: - Committed to trunk. Thanx [~shaqyl09099] for the contribution & [~slfan1989] for the review!!! > The formatted yarn queue list is displayed on the command line > -- > > Key: YARN-11506 > URL: https://issues.apache.org/jira/browse/YARN-11506 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > > h4. In the issue list YARN-11474, the printed information is not aligned,This > issue fixes the yarn queue display formatting problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11506) The formatted yarn queue list is displayed on the command line
[ https://issues.apache.org/jira/browse/YARN-11506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11506: --- Assignee: Lu Yuan > The formatted yarn queue list is displayed on the command line > -- > > Key: YARN-11506 > URL: https://issues.apache.org/jira/browse/YARN-11506 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > > h4. In the issue list YARN-11474, the printed information is not aligned,This > issue fixes the yarn queue display formatting problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11492) Improve createJerseyClient#setConnectTimeout Code
[ https://issues.apache.org/jira/browse/YARN-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11492. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve createJerseyClient#setConnectTimeout Code > - > > Key: YARN-11492 > URL: https://issues.apache.org/jira/browse/YARN-11492 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > createJerseyClient#setConnectTimeout There is a type conversion code that > converts long to int, we should be careful with this part of code, I added > some verification rules. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11492) Improve createJerseyClient#setConnectTimeout Code
[ https://issues.apache.org/jira/browse/YARN-11492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729277#comment-17729277 ] Ayush Saxena commented on YARN-11492: - Committed to trunk. Thanx [~slfan1989] for the contribution & [~elgoiri] for the review!!! > Improve createJerseyClient#setConnectTimeout Code > - > > Key: YARN-11492 > URL: https://issues.apache.org/jira/browse/YARN-11492 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > createJerseyClient#setConnectTimeout There is a type conversion code that > converts long to int, we should be careful with this part of code, I added > some verification rules. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11498) Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml
[ https://issues.apache.org/jira/browse/YARN-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725172#comment-17725172 ] Ayush Saxena commented on YARN-11498: - code change is not in Hadoop Common but in Yarn, have moved it to Yarn. Good to link the tickets which induced this and the affected hadoop version > Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml > -- > > Key: YARN-11498 > URL: https://issues.apache.org/jira/browse/YARN-11498 > Project: Hadoop YARN > Issue Type: Task > Components: build >Reporter: Devaspati Krishnatri >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-11498) Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml
[ https://issues.apache.org/jira/browse/YARN-11498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena moved HADOOP-18732 to YARN-11498: -- Component/s: build (was: build) Key: YARN-11498 (was: HADOOP-18732) Project: Hadoop YARN (was: Hadoop Common) > Exclude Jettison from jersey-json artifact in hadoop-yarn-common's pom.xml > -- > > Key: YARN-11498 > URL: https://issues.apache.org/jira/browse/YARN-11498 > Project: Hadoop YARN > Issue Type: Task > Components: build >Reporter: Devaspati Krishnatri >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11496) Improve TimelineService log format
[ https://issues.apache.org/jira/browse/YARN-11496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724517#comment-17724517 ] Ayush Saxena commented on YARN-11496: - Committed to trunk. Thanx [~15217812...@163.com] for the contribution & [~slfan1989] for the review!!! > Improve TimelineService log format > -- > > Key: YARN-11496 > URL: https://issues.apache.org/jira/browse/YARN-11496 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineservice >Affects Versions: 3.3.4 >Reporter: Xianming Lei >Assignee: Xianming Lei >Priority: Minor > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11496) Improve TimelineService log format
[ https://issues.apache.org/jira/browse/YARN-11496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11496. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve TimelineService log format > -- > > Key: YARN-11496 > URL: https://issues.apache.org/jira/browse/YARN-11496 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineservice >Affects Versions: 3.3.4 >Reporter: Xianming Lei >Assignee: Xianming Lei >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11496) Improve TimelineService log format
[ https://issues.apache.org/jira/browse/YARN-11496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11496: --- Assignee: Xianming Lei > Improve TimelineService log format > -- > > Key: YARN-11496 > URL: https://issues.apache.org/jira/browse/YARN-11496 > Project: Hadoop YARN > Issue Type: Improvement > Components: timelineservice >Affects Versions: 3.3.4 >Reporter: Xianming Lei >Assignee: Xianming Lei >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11495) Fix typos in hadoop-yarn-server-web-proxy
[ https://issues.apache.org/jira/browse/YARN-11495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722329#comment-17722329 ] Ayush Saxena commented on YARN-11495: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix typos in hadoop-yarn-server-web-proxy > - > > Key: YARN-11495 > URL: https://issues.apache.org/jira/browse/YARN-11495 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Minor > > While reading the code, I found some typo issues. This JIRA will fix these > issues in hadoop-yarn-server-web-proxy. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11495) Fix typos in hadoop-yarn-server-web-proxy
[ https://issues.apache.org/jira/browse/YARN-11495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11495. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix typos in hadoop-yarn-server-web-proxy > - > > Key: YARN-11495 > URL: https://issues.apache.org/jira/browse/YARN-11495 > Project: Hadoop YARN > Issue Type: Improvement > Components: webapp >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Minor > Fix For: 3.4.0 > > > While reading the code, I found some typo issues. This JIRA will fix these > issues in hadoop-yarn-server-web-proxy. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11465) Improved YarnClient Log Format
[ https://issues.apache.org/jira/browse/YARN-11465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714729#comment-17714729 ] Ayush Saxena commented on YARN-11465: - Added [~shaqyl09099] as Yarn contributor to assign the ticket > Improved YarnClient Log Format > -- > > Key: YARN-11465 > URL: https://issues.apache.org/jira/browse/YARN-11465 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.4.0 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > When reading the YarnClient module code, I found some logging irregularities. > This PR fixes the problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11465) Improved YarnClient Log Format
[ https://issues.apache.org/jira/browse/YARN-11465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11465: --- Assignee: Lu Yuan > Improved YarnClient Log Format > -- > > Key: YARN-11465 > URL: https://issues.apache.org/jira/browse/YARN-11465 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.4.0 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > When reading the YarnClient module code, I found some logging irregularities. > This PR fixes the problem -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11472) Normalizes log records on the Yarn client
[ https://issues.apache.org/jira/browse/YARN-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-11472: Fix Version/s: (was: 3.3.4) > Normalizes log records on the Yarn client > - > > Key: YARN-11472 > URL: https://issues.apache.org/jira/browse/YARN-11472 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Priority: Major > Labels: pull-request-available > > Modify the code format based on YARN-11465 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11472) Normalizes log records on the Yarn client
[ https://issues.apache.org/jira/browse/YARN-11472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11472: --- Assignee: Lu Yuan > Normalizes log records on the Yarn client > - > > Key: YARN-11472 > URL: https://issues.apache.org/jira/browse/YARN-11472 > Project: Hadoop YARN > Issue Type: Improvement > Components: client >Affects Versions: 3.3.4 >Reporter: Lu Yuan >Assignee: Lu Yuan >Priority: Major > Labels: pull-request-available > > Modify the code format based on YARN-11465 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-11404:
Fix Version/s: (was: 3.4.0)
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Reopened] (YARN-11404) Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test failure
[
https://issues.apache.org/jira/browse/YARN-11404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena reopened YARN-11404:
-
> Add junit5 dependency to hadoop-mapreduce-client-app to fix few unit test
> failure
> -
>
> Key: YARN-11404
> URL: https://issues.apache.org/jira/browse/YARN-11404
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Reporter: Susheel Gupta
>Assignee: Susheel Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments:
> patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
>
>
> We need to add Junit 5 dependency in
> {code:java}
> /hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/pom.xml{code}
> as the testcase TestAMWebServicesJobConf, TestAMWebServicesJobs,
> TestAMWebServices, TestAMWebServicesAttempts, TestAMWebServicesTasks were
> passing locally but failed at jenkins build in this
> [link|https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5119/7/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt]
> for YARN-5607
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11380) Fix hadoop-yarn-api module Java Doc Errors
[
https://issues.apache.org/jira/browse/YARN-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved YARN-11380.
-
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Fix hadoop-yarn-api module Java Doc Errors
> --
>
> Key: YARN-11380
> URL: https://issues.apache.org/jira/browse/YARN-11380
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 3.4.0
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> When finishing YARN-11373, ps has java-doc compilation errors, I will fix all
> java-doc compilation errors of hadoop-yarn-api module. I will pay attention
> to the java doc compilation of jdk8 & jdk11 at the same time.
> {code:java}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
> (default-cli) on project hadoop-yarn-api: An error has occurred in Javadoc
> report generation:
> [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version
> as HTML 4.01 by using the -html4 option.
> [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
> removed
> [ERROR] in a future release. To suppress this warning, please ensure that any
> HTML constructs
> [ERROR] in your comments are valid in HTML5, and remove the -html4 option.
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:97:
> warning: no description for @throws
> [ERROR]* @throws YarnException
> [ERROR] ^
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:98:
> warning: no description for @throws
> [ERROR]* @throws IOException
> [ERROR] ^
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:129:
> warning: no description for @throws
> [ERROR]* @throws YarnException
> [ERROR] ^
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:130:
> warning: no description for @throws
> [ERROR]* @throws IOException {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11380) Fix hadoop-yarn-api module Java Doc Errors
[
https://issues.apache.org/jira/browse/YARN-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640033#comment-17640033
]
Ayush Saxena commented on YARN-11380:
-
Committed to trunk.
Thanx [~slfan1989] for the contribution!!!
> Fix hadoop-yarn-api module Java Doc Errors
> --
>
> Key: YARN-11380
> URL: https://issues.apache.org/jira/browse/YARN-11380
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
>Affects Versions: 3.4.0
>Reporter: Shilun Fan
>Assignee: Shilun Fan
>Priority: Major
> Labels: pull-request-available
>
> When finishing YARN-11373, ps has java-doc compilation errors, I will fix all
> java-doc compilation errors of hadoop-yarn-api module. I will pay attention
> to the java doc compilation of jdk8 & jdk11 at the same time.
> {code:java}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
> (default-cli) on project hadoop-yarn-api: An error has occurred in Javadoc
> report generation:
> [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version
> as HTML 4.01 by using the -html4 option.
> [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
> removed
> [ERROR] in a future release. To suppress this warning, please ensure that any
> HTML constructs
> [ERROR] in your comments are valid in HTML5, and remove the -html4 option.
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:97:
> warning: no description for @throws
> [ERROR]* @throws YarnException
> [ERROR] ^
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:98:
> warning: no description for @throws
> [ERROR]* @throws IOException
> [ERROR] ^
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:129:
> warning: no description for @throws
> [ERROR]* @throws YarnException
> [ERROR] ^
> [ERROR]
> /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-5146/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ApplicationBaseProtocol.java:130:
> warning: no description for @throws
> [ERROR]* @throws IOException {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11381) Fix hadoop-yarn-common module Java Doc Errors
[ https://issues.apache.org/jira/browse/YARN-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11381. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix hadoop-yarn-common module Java Doc Errors > - > > Key: YARN-11381 > URL: https://issues.apache.org/jira/browse/YARN-11381 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > When finishing YARN-11373, ps has java-doc compilation errors, I will fix > all java-doc compilation errors of hadoop-yarn-common module. I will pay > attention to the java doc compilation of jdk8 & jdk11 at the same time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11381) Fix hadoop-yarn-common module Java Doc Errors
[ https://issues.apache.org/jira/browse/YARN-11381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17639484#comment-17639484 ] Ayush Saxena commented on YARN-11381: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix hadoop-yarn-common module Java Doc Errors > - > > Key: YARN-11381 > URL: https://issues.apache.org/jira/browse/YARN-11381 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.4.0 >Reporter: Shilun Fan >Assignee: Shilun Fan >Priority: Major > Labels: pull-request-available > > When finishing YARN-11373, ps has java-doc compilation errors, I will fix > all java-doc compilation errors of hadoop-yarn-common module. I will pay > attention to the java doc compilation of jdk8 & jdk11 at the same time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11334) [Federation] Improve SubClusterState#fromString parameter and LogMessage
[
https://issues.apache.org/jira/browse/YARN-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved YARN-11334.
-
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> [Federation] Improve SubClusterState#fromString parameter and LogMessage
>
>
> Key: YARN-11334
> URL: https://issues.apache.org/jira/browse/YARN-11334
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Trivial
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> The SubClusterState object has a fromString method, which can be improved in
> the following 2 places
> {code:java}
> /**
> * Convert a string into {@code SubClusterState}.
> *
> * @param x the string to convert in SubClusterState
> * @return the respective {@code SubClusterState}
> */
> public static SubClusterState fromString(String x) {
> try {
> return SubClusterState.valueOf(x);
> } catch (Exception e) {
> LOG.error("Invalid SubCluster State value in the StateStore does not"
> + " match with the YARN Federation standard.");
> return null;
> }
> } {code}
> * The parameter is named x, which cannot well express the meaning of the
> input parameter.
> * The error log does not print the error value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11334) [Federation] Improve SubClusterState#fromString parameter and LogMessage
[
https://issues.apache.org/jira/browse/YARN-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17616804#comment-17616804
]
Ayush Saxena commented on YARN-11334:
-
Committed to trunk.
Thanx [~slfan1989] for the contribution!!!
> [Federation] Improve SubClusterState#fromString parameter and LogMessage
>
>
> Key: YARN-11334
> URL: https://issues.apache.org/jira/browse/YARN-11334
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Trivial
> Labels: pull-request-available
>
> The SubClusterState object has a fromString method, which can be improved in
> the following 2 places
> {code:java}
> /**
> * Convert a string into {@code SubClusterState}.
> *
> * @param x the string to convert in SubClusterState
> * @return the respective {@code SubClusterState}
> */
> public static SubClusterState fromString(String x) {
> try {
> return SubClusterState.valueOf(x);
> } catch (Exception e) {
> LOG.error("Invalid SubCluster State value in the StateStore does not"
> + " match with the YARN Federation standard.");
> return null;
> }
> } {code}
> * The parameter is named x, which cannot well express the meaning of the
> input parameter.
> * The error log does not print the error value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11334) [Federation] Improve SubClusterState#fromString parameter and LogMessage
[
https://issues.apache.org/jira/browse/YARN-11334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-11334:
Priority: Trivial (was: Major)
> [Federation] Improve SubClusterState#fromString parameter and LogMessage
>
>
> Key: YARN-11334
> URL: https://issues.apache.org/jira/browse/YARN-11334
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Trivial
> Labels: pull-request-available
>
> The SubClusterState object has a fromString method, which can be improved in
> the following 2 places
> {code:java}
> /**
> * Convert a string into {@code SubClusterState}.
> *
> * @param x the string to convert in SubClusterState
> * @return the respective {@code SubClusterState}
> */
> public static SubClusterState fromString(String x) {
> try {
> return SubClusterState.valueOf(x);
> } catch (Exception e) {
> LOG.error("Invalid SubCluster State value in the StateStore does not"
> + " match with the YARN Federation standard.");
> return null;
> }
> } {code}
> * The parameter is named x, which cannot well express the meaning of the
> input parameter.
> * The error log does not print the error value.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11240) Fix incorrect placeholder in yarn-module
[ https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11240. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix incorrect placeholder in yarn-module > > > Key: YARN-11240 > URL: https://issues.apache.org/jira/browse/YARN-11240 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Try to deal with the moudle problem at a time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11305) Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After YARN-11241(#4703)
[
https://issues.apache.org/jira/browse/YARN-11305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved YARN-11305.
-
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
Committed to trunk.
Thanx [~slfan1989] for the contribution!!!
> Fix TestLogAggregationService#testLocalFileDeletionAfterUpload Failed After
> YARN-11241(#4703)
> -
>
> Key: YARN-11305
> URL: https://issues.apache.org/jira/browse/YARN-11305
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Yarn's unit tests, with parallel mode enabled
> {code:java}
> /usr/bin/mvn --batch-mode
> -Dmaven.repo.local=/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/yetus-m2/hadoop-trunk-patch-0
> -Ptest-patch -Dsurefire.rerunFailingTestsCount=2 -Pparallel-tests
> -P!shelltest -Pnative -Drequire.fuse -Drequire.openssl -Drequire.snappy
> -Drequire.valgrind -Drequire.zstd -Drequire.test.libhadoop -Pyarn-ui clean
> test -fae{code}
> Because the automatically generated application_id is the same,
> *testLocalFileDeletionAfterUpload* is affected by
> *testLocalFileRemainsAfterUploadOnCleanupDisable*
> testLocalFileDeletionAfterUpload needs to check that the log of an
> application is deleted, and testLocalFileRemainsAfterUploadOnCleanupDisable
> regenerates the log of the same application.
> Error Message
> {code:java}
> Directory
> [/home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-4846/ubuntu-focal/src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestLogAggregationService-localLogDir/application_1234_0001]
> was not deleted {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[ https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11286. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter > configurable > - > > Key: YARN-11286 > URL: https://issues.apache.org/jira/browse/YARN-11286 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > AsyncDispatcher#printEventDetailsExecutor thread pool parameters are > hard-coded, extract this part of hard-coded configuration parameters to the > configuration file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[ https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602736#comment-17602736 ] Ayush Saxena commented on YARN-11286: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter > configurable > - > > Key: YARN-11286 > URL: https://issues.apache.org/jira/browse/YARN-11286 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > AsyncDispatcher#printEventDetailsExecutor thread pool parameters are > hard-coded, extract this part of hard-coded configuration parameters to the > configuration file. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11274. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[ https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602734#comment-17602734 ] Ayush Saxena commented on YARN-11274: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Improve Nodemanager#NodeStatusUpdaterImpl Log > - > > Key: YARN-11274 > URL: https://issues.apache.org/jira/browse/YARN-11274 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > > 1.When RM performs master-slave switching, NM will re-register, report the > list of running apps, and print the number of running apps > 2.registerWithRM#containerReports is enough to print the number of prints, > because the above calling process has already printed the container list, so > there is no need to repeat printing > 3.Use placeholders to display logs to avoid string concatenation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9127) Create more tests to verify GpuDeviceInformationParser
[
https://issues.apache.org/jira/browse/YARN-9127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17598654#comment-17598654
]
Ayush Saxena commented on YARN-9127:
[~pbacsko] / [~snemeth]
There were some xml added as part of this, seems they create issues in the
daily build:
{noformat}
Reason | Tests
XML | Parsing Error(s):
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
{noformat}
Do you folks have any idea about this?
Can check here:
[https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/965/console]
At the very end
> Create more tests to verify GpuDeviceInformationParser
> --
>
> Key: YARN-9127
> URL: https://issues.apache.org/jira/browse/YARN-9127
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.3.0, 3.2.1, 3.1.3
>
> Attachments: YARN-9127.001.patch, YARN-9127.002.patch,
> YARN-9127.003.patch, YARN-9127.004.patch, YARN-9127.005.patch,
> YARN-9127.branch-3.1.001.patch, YARN-9127.branch-3.2.001.patch
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11240) Fix incorrect placeholder in yarn-module
[ https://issues.apache.org/jira/browse/YARN-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17584288#comment-17584288 ] Ayush Saxena commented on YARN-11240: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix incorrect placeholder in yarn-module > > > Key: YARN-11240 > URL: https://issues.apache.org/jira/browse/YARN-11240 > Project: Hadoop YARN > Issue Type: Improvement >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Try to deal with the moudle problem at a time. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9425) Make initialDelay configurable for FederationStateStoreService#scheduledExecutorService
[ https://issues.apache.org/jira/browse/YARN-9425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582641#comment-17582641 ] Ayush Saxena commented on YARN-9425: Committed to trunk. Thanx [~groot] and [~shenyinjie] for the contribution!!! > Make initialDelay configurable for > FederationStateStoreService#scheduledExecutorService > --- > > Key: YARN-9425 > URL: https://issues.apache.org/jira/browse/YARN-9425 > Project: Hadoop YARN > Issue Type: Bug > Components: federation >Affects Versions: 3.1.0 >Reporter: Shen Yinjie >Assignee: groot >Priority: Major > Labels: pull-request-available > Attachments: YARN-9425_1.patch, YARN-9425_2.patch, YARN-9425_3.patch, > YARN-9425_4.patch > > > When enable YARN federation, subclusters info in Router Web UI cannot be > loaded immediately, and client cannot find any active subclusters after 5mins > by default ,which is configured by > "yarn.federation.state-store.heartbeat-interval-secs". > IMA,we should seperate 'initialDely' and 'delay' for > FederationStateStoreService#scheduledExecutorService. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10793) Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice
[
https://issues.apache.org/jira/browse/YARN-10793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17582204#comment-17582204
]
Ayush Saxena commented on YARN-10793:
-
Hi [~aajisaka] ,
Can you give a check if these failures are related to this patch:
[https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/957/testReport/junit/org.apache.hadoop.yarn.server.timeline/TestLevelDBCacheTimelineStore/testGetDomains/]
The error seems to be NoClassDef for the import introduced in this patch:
{noformat}
java.lang.NoClassDefFoundError: org/junit/jupiter/api/Assertions
at
org.apache.hadoop.yarn.server.timeline.TimelineStoreTestUtils.loadTestEntityData(TimelineStoreTestUtils.java:148){noformat}
> Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice
> -
>
> Key: YARN-10793
> URL: https://issues.apache.org/jira/browse/YARN-10793
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: ANANDA G B
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11203) Fix typo in hadoop-yarn-server-router module
[ https://issues.apache.org/jira/browse/YARN-11203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11203. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Fix typo in hadoop-yarn-server-router module > > > Key: YARN-11203 > URL: https://issues.apache.org/jira/browse/YARN-11203 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Fix typo in hadoop-yarn-server-router module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11203) Fix typo in hadoop-yarn-server-router module
[ https://issues.apache.org/jira/browse/YARN-11203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570318#comment-17570318 ] Ayush Saxena commented on YARN-11203: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! > Fix typo in hadoop-yarn-server-router module > > > Key: YARN-11203 > URL: https://issues.apache.org/jira/browse/YARN-11203 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Fix typo in hadoop-yarn-server-router module. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11192) TestRouterWebServicesREST failing after YARN-9827
[
https://issues.apache.org/jira/browse/YARN-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved YARN-11192.
-
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> TestRouterWebServicesREST failing after YARN-9827
> -
>
> Key: YARN-11192
> URL: https://issues.apache.org/jira/browse/YARN-11192
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In YARN-9827, the following modifications:
> {code:java}
> GenericExceptionHandler should respond with SERVICE_UNAVAILABLE in case of
> connection and service unavailable exception instead of
> INTERNAL_SERVICE_ERROR. {code}
> This modification caused all of YARN Federation's TestRouterWebServicesREST
> unit tests to fail
> {code:java}
> [ERROR] Tests run: 201, Failures: 15, Errors: 0, Skipped: 0, Flakes: 2
> .
> [ERROR]
> org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppStateXML(org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST)
> [ERROR] Run 1: TestRouterWebServicesREST.testUpdateAppStateXML:774
> expected:<500> but was:<503>
> [ERROR] Run 2: TestRouterWebServicesREST.testUpdateAppStateXML:774
> expected:<500> but was:<503>
> [ERROR] Run 3: TestRouterWebServicesREST.testUpdateAppStateXML:774
> expected:<500> but was:<503> {code}
> Report-URL:
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4464/5/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11192) TestRouterWebServicesREST failing after YARN-9827
[
https://issues.apache.org/jira/browse/YARN-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17557872#comment-17557872
]
Ayush Saxena commented on YARN-11192:
-
Committed to trunk.
Thanx [~slfan1989] for the contribution!!!
> TestRouterWebServicesREST failing after YARN-9827
> -
>
> Key: YARN-11192
> URL: https://issues.apache.org/jira/browse/YARN-11192
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In YARN-9827, the following modifications:
> {code:java}
> GenericExceptionHandler should respond with SERVICE_UNAVAILABLE in case of
> connection and service unavailable exception instead of
> INTERNAL_SERVICE_ERROR. {code}
> This modification caused all of YARN Federation's TestRouterWebServicesREST
> unit tests to fail
> {code:java}
> [ERROR] Tests run: 201, Failures: 15, Errors: 0, Skipped: 0, Flakes: 2
> .
> [ERROR]
> org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppStateXML(org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST)
> [ERROR] Run 1: TestRouterWebServicesREST.testUpdateAppStateXML:774
> expected:<500> but was:<503>
> [ERROR] Run 2: TestRouterWebServicesREST.testUpdateAppStateXML:774
> expected:<500> but was:<503>
> [ERROR] Run 3: TestRouterWebServicesREST.testUpdateAppStateXML:774
> expected:<500> but was:<503> {code}
> Report-URL:
> https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4464/5/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11142) Remove unused Imports in Hadoop YARN project
[ https://issues.apache.org/jira/browse/YARN-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17544533#comment-17544533 ] Ayush Saxena commented on YARN-11142: - Merged PR to trunk. Thanx [~groot] for the contribution!!! > Remove unused Imports in Hadoop YARN project > > > Key: YARN-11142 > URL: https://issues.apache.org/jira/browse/YARN-11142 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Resolved] (YARN-11142) Remove unused Imports in Hadoop YARN project
[ https://issues.apache.org/jira/browse/YARN-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved YARN-11142. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove unused Imports in Hadoop YARN project > > > Key: YARN-11142 > URL: https://issues.apache.org/jira/browse/YARN-11142 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 50m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11130) RouterClientRMService Has Unused import
[ https://issues.apache.org/jira/browse/YARN-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-11130: --- Assignee: fanshilun > RouterClientRMService Has Unused import > --- > > Key: YARN-11130 > URL: https://issues.apache.org/jira/browse/YARN-11130 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: RouterClientRMService Has Unused Import.png, > YARN-11130.01.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > RouterClientRMService Has Unused import, keep the code clean and remove > unused. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11130) RouterClientRMService Has Unused import
[ https://issues.apache.org/jira/browse/YARN-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534706#comment-17534706 ] Ayush Saxena commented on YARN-11130: - Committed to trunk. Thanx [~slfan1989] for the contribution!!! Note: Added [~slfan1989] as Yarn-Contributor-1 to assign this jira > RouterClientRMService Has Unused import > --- > > Key: YARN-11130 > URL: https://issues.apache.org/jira/browse/YARN-11130 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Attachments: RouterClientRMService Has Unused Import.png, > YARN-11130.01.patch > > Time Spent: 1.5h > Remaining Estimate: 0h > > RouterClientRMService Has Unused import, keep the code clean and remove > unused. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11026) Make AppPlacementAllocator configurable in AppSchedulingInfo
[
https://issues.apache.org/jira/browse/YARN-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17501147#comment-17501147
]
Ayush Saxena commented on YARN-11026:
-
This breaks
TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml
{{yarn.scheduler.app-placement-allocator.class}} is added in the
YarnConfiguration, but not in {{yarn-default.xml}}
Ref:
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/testReport/org.apache.hadoop.yarn.conf/TestYarnConfigurationFields/testCompareConfigurationClassAgainstXml/
> Make AppPlacementAllocator configurable in AppSchedulingInfo
>
>
> Key: YARN-11026
> URL: https://issues.apache.org/jira/browse/YARN-11026
> Project: Hadoop YARN
> Issue Type: Task
>Reporter: Minni Mittal
>Assignee: Minni Mittal
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10788) TestCsiClient fails
[
https://issues.apache.org/jira/browse/YARN-10788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487495#comment-17487495
]
Ayush Saxena commented on YARN-10788:
-
>From the error it looks OS specific:
{noformat}
Caused by: io.netty.channel.unix.Errors$NativeIoException: bind(..) failed:
File name too long
{noformat}
Reproduces in the docker:
{noformat}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.yarn.csi.client.TestCsiClient
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.041 s
<<< FAILURE! - in org.apache.hadoop.yarn.csi.client.TestCsiClient
[ERROR] testIdentityService(org.apache.hadoop.yarn.csi.client.TestCsiClient)
Time elapsed: 1.701 s <<< ERROR!
java.io.IOException: Failed to bind
at io.grpc.netty.NettyServer.start(NettyServer.java:257)
at io.grpc.internal.ServerImpl.start(ServerImpl.java:184)
at io.grpc.internal.ServerImpl.start(ServerImpl.java:90)
at
org.apache.hadoop.yarn.csi.client.FakeCsiDriver.start(FakeCsiDriver.java:56)
at
org.apache.hadoop.yarn.csi.client.TestCsiClient.testIdentityService(TestCsiClient.java:72)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)
Caused by: io.netty.channel.unix.Errors$NativeIoException: bind(..) failed:
File name too long
{noformat}
So, the problem is with the path.
I just added a line to set the path to {{/tmp}} to confirm the root cause like:
{code:java}
public static void setUp() throws IOException {
System.setProperty(GenericTestUtils.SYSPROP_TEST_DATA_DIR, "/tmp");
{code}
And the test passed,
{noformat}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.yarn.csi.client.TestCsiClient
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.982 s
- in org.apache.hadoop.yarn.csi.client.TestCsiClient
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
{noformat}
Just tried to find the root cause, Didn't work on finding the best solution, if
someone is interested can take this ahead, I will be happy to
[jira] [Commented] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
[
https://issues.apache.org/jira/browse/YARN-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17252338#comment-17252338
]
Ayush Saxena commented on YARN-10540:
-
I too gave it a try, On Mac the UI seems to be broken :
!Mac-Yarn-UI.png!
But on Ubuntu-20, everything works fine :
!Yarn-UI-Ubuntu.png!
Even the http link is working for me, seems a mac only issue
> Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
>
>
> Key: YARN-10540
> URL: https://issues.apache.org/jira/browse/YARN-10540
> Project: Hadoop YARN
> Issue Type: Task
> Components: webapp
>Affects Versions: 3.2.2
>Reporter: Sunil G
>Priority: Critical
> Attachments: Mac-Yarn-UI.png, Screenshot 2020-12-19 at 11.01.43
> PM.png, Screenshot 2020-12-19 at 11.02.14 PM.png, Yarn-UI-Ubuntu.png,
> yarnodes.png, yarnui2onubuntu.png
>
>
> YARN-10450 added changes in NodeInfo class.
> Various exceptions are showing while accessing UI2 and UI1 NODE pages.
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at
> org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70)
> {code}
> {code:java}
> 2020-12-19 22:55:54,846 WARN
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
[
https://issues.apache.org/jira/browse/YARN-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-10540:
Attachment: Mac-Yarn-UI.png
> Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
>
>
> Key: YARN-10540
> URL: https://issues.apache.org/jira/browse/YARN-10540
> Project: Hadoop YARN
> Issue Type: Task
> Components: webapp
>Affects Versions: 3.2.2
>Reporter: Sunil G
>Priority: Critical
> Attachments: Mac-Yarn-UI.png, Screenshot 2020-12-19 at 11.01.43
> PM.png, Screenshot 2020-12-19 at 11.02.14 PM.png, Yarn-UI-Ubuntu.png,
> yarnodes.png, yarnui2onubuntu.png
>
>
> YARN-10450 added changes in NodeInfo class.
> Various exceptions are showing while accessing UI2 and UI1 NODE pages.
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at
> org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70)
> {code}
> {code:java}
> 2020-12-19 22:55:54,846 WARN
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10540) Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
[
https://issues.apache.org/jira/browse/YARN-10540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-10540:
Attachment: Yarn-UI-Ubuntu.png
> Node page is broken in YARN UI1 and UI2 including RMWebService api for nodes
>
>
> Key: YARN-10540
> URL: https://issues.apache.org/jira/browse/YARN-10540
> Project: Hadoop YARN
> Issue Type: Task
> Components: webapp
>Affects Versions: 3.2.2
>Reporter: Sunil G
>Priority: Critical
> Attachments: Mac-Yarn-UI.png, Screenshot 2020-12-19 at 11.01.43
> PM.png, Screenshot 2020-12-19 at 11.02.14 PM.png, Yarn-UI-Ubuntu.png,
> yarnodes.png, yarnui2onubuntu.png
>
>
> YARN-10450 added changes in NodeInfo class.
> Various exceptions are showing while accessing UI2 and UI1 NODE pages.
> {code:java}
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.NodesPage$NodesBlock.render(NodesPage.java:164)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
> at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
> at
> org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
> at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
> at
> org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
> at
> org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
> at
> org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:216)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RmController.nodes(RmController.java:70)
> {code}
> {code:java}
> 2020-12-19 22:55:54,846 WARN
> org.apache.hadoop.yarn.webapp.GenericExceptionHandler: INTERNAL_SERVER_ERROR
> java.lang.NullPointerException
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.dao.NodeInfo.(NodeInfo.java:103)
> at
> org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebServices.getNodes(RMWebServices.java:450)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10040) DistributedShell test failure on X86 and ARM
[ https://issues.apache.org/jira/browse/YARN-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227548#comment-17227548 ] Ayush Saxena commented on YARN-10040: - [~abmodi] any pointers about this? Is the code only broken or just the test. If the functionality itself has some issue we should consider reverting YARN-9697, else if this is only a test issue, we should wrap this up, if there isn't a fix available we can disable this test for time being. Let me know what is the actual situation. I can try help in whichever way possible. [~aajisaka] [~elgoiri] any pointers about this. Or can you point to someone who has some context about this part of code. > DistributedShell test failure on X86 and ARM > > > Key: YARN-10040 > URL: https://issues.apache.org/jira/browse/YARN-10040 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell > Environment: X86/ARM > OS: ubuntu1804 > Java 8 >Reporter: zhao bo >Assignee: Abhishek Modi >Priority: Major > Attachments: YARN-10040.001.patch > > > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > Please see the Apache Jenkins Test result: > [https://builds.apache.org/job/hadoop-multibranch/job/PR-1767/1/testReport/] > > These 2 tests are failed on both X86 and ARM platform. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10347) Fix double locking in CapacityScheduler#reinitialize in branch-3.1
[
https://issues.apache.org/jira/browse/YARN-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17154062#comment-17154062
]
Ayush Saxena commented on YARN-10347:
-
hmm, need to see what triggered this at jenkins.
Anyway the fix seems pretty straight forward. The original trunk patch didn't
had it in YARN-10022, got added during backport.
+1, Thanx [~iwasakims] for the find. This should ideally unblock 3.1.4 release
as well?
cc. [~gabor.bota]
> Fix double locking in CapacityScheduler#reinitialize in branch-3.1
> --
>
> Key: YARN-10347
> URL: https://issues.apache.org/jira/browse/YARN-10347
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.1.4
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Critical
> Attachments: YARN-10347-branch-3.1.001.patch
>
>
> Double locking blocks another threads in ResourceManager waiting for the lock.
> I found the issue on testing hadoop-3.1.4-RC2 with RM-HA enabled deployment.
> ResourceManager blocks on {{submitApplication}} waiting for the lock when I
> run example MR applications.
> {noformat}
> "IPC Server handler 45 on default port 8032" #211 daemon prio=5 os_prio=0
> tid=0x7f0e45a40200 nid=0x418 waiting on condition [0x7f0e14abe000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x85d56510> (a
> java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283)
> at
> java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.checkAndGetApplicationPriority(CapacityScheduler.java:2521)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:417)
> at
> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:342)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:678)
> at
> org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:277)
> at
> org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:563)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:527)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1015)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:943)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2943)
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10347) Fix double locking in CapacityScheduler#reinitialize in branch-3.1
[ https://issues.apache.org/jira/browse/YARN-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17154009#comment-17154009 ] Ayush Saxena edited comment on YARN-10347 at 7/8/20, 8:59 PM: -- Thanx [~iwasakims] for the fix. Changes seems fair enough, but the jenkins doesn't seems behaving as expected. Any idea? was (Author: ayushtkn): Thanx [~iwasakims] for the fix. Changes seems fair enough, but the jenkins doesn't seems behaving expected. Any idea? > Fix double locking in CapacityScheduler#reinitialize in branch-3.1 > -- > > Key: YARN-10347 > URL: https://issues.apache.org/jira/browse/YARN-10347 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.4 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Critical > Attachments: YARN-10347-branch-3.1.001.patch > > > Double locking blocks another threads in ResourceManager waiting for the lock. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10347) Fix double locking in CapacityScheduler#reinitialize in branch-3.1
[ https://issues.apache.org/jira/browse/YARN-10347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17154009#comment-17154009 ] Ayush Saxena commented on YARN-10347: - Thanx [~iwasakims] for the fix. Changes seems fair enough, but the jenkins doesn't seems behaving expected. Any idea? > Fix double locking in CapacityScheduler#reinitialize in branch-3.1 > -- > > Key: YARN-10347 > URL: https://issues.apache.org/jira/browse/YARN-10347 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.4 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Critical > Attachments: YARN-10347-branch-3.1.001.patch > > > Double locking blocks another threads in ResourceManager waiting for the lock. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10265) Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue
[ https://issues.apache.org/jira/browse/YARN-10265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17107092#comment-17107092 ] Ayush Saxena commented on YARN-10265: - +1 > Upgrade Netty-all dependency to latest version 4.1.50 to fix ARM support issue > -- > > Key: YARN-10265 > URL: https://issues.apache.org/jira/browse/YARN-10265 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Priority: Major > Attachments: YARN-10265.001.patch > > > In the issue: https://issues.apache.org/jira/browse/YARN-9898 we have added a > workaround using an non-official released netty-4.1.48 to fix the ARM support > issue. but just a few hours agon, Netty has release the 4.1.50 version which > is officially support ARM platform, please see: > [https://github.com/netty/netty/pull/9804] > > netty-4.1.50.Final release: > [https://github.com/netty/netty/releases/tag/netty-4.1.50.Final] > commits from netty-4.1.48 to netty-4.1.50: > [https://github.com/netty/netty/compare/netty-4.1.48.Final...netty-4.1.50.Final] > So, now it is better to upgrade the netty-dependency version of Hadoop to > 4.1.50 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17106603#comment-17106603
]
Ayush Saxena commented on YARN-9898:
Committed to trunk, branch-3.3 and 3.3.0
Thanx [~seanlau] for the contribution!!!
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch,
> YARN-9898.003.patch, YARN-9898.004.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105425#comment-17105425
]
Ayush Saxena commented on YARN-9898:
Thanx [~seanlau] for the update. The build seems happy now.
v004 LGTM +1
if no further comments, will commit by tomorrow EOD.
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch,
> YARN-9898.003.patch, YARN-9898.004.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17103669#comment-17103669
]
Ayush Saxena commented on YARN-9898:
Well the second build also crashed with the same errors :
Is this related :
{code:java}
[ERROR] Plugin org.apache.hadoop:hadoop-maven-plugins:3.4.0-SNAPSHOT or one of
its dependencies could not be resolved: Failed to read artifact descriptor for
org.apache.hadoop:hadoop-maven-plugins:jar:3.4.0-SNAPSHOT: 1 problem was
encountered while building the effective model for
org.apache.hadoop:hadoop-maven-plugins:3.4.0-SNAPSHOT
[ERROR] [ERROR] 'dependencyManagement.dependencies.dependency.groupId' for
${netty4.group}:netty-all:jar with value '${netty4.group}' does not match a
valid id pattern. @
{code}
https://builds.apache.org/job/PreCommit-YARN-Build/26014/artifact/out/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
The errors are due to dependencies, and doesn't fail at my local. Can you split
the HDFS part of changes into a separate HDFS jira?
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9898) Dependency netty-all-4.1.27.Final doesn't support ARM platform
[
https://issues.apache.org/jira/browse/YARN-9898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17102750#comment-17102750
]
Ayush Saxena commented on YARN-9898:
Thanx [~seanlau] for the patch.
I have re triggered the build here, I don't think the failure is related to the
patch :
https://builds.apache.org/job/PreCommit-YARN-Build/26007/
Have verified the fix in AARCH-64 :
{noformat}
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO] Running org.apache.hadoop.yarn.csi.client.TestCsiClient
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.835 s
- in org.apache.hadoop.yarn.csi.client.TestCsiClient
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
{noformat}
{noformat}
[INFO]
[INFO] Detecting the operating system and CPU architecture
[INFO]
[INFO] os.detected.name: linux
[INFO] os.detected.arch: aarch_64
[INFO] os.detected.version: 4.15
[INFO] os.detected.version.major: 4
[INFO] os.detected.version.minor: 15
[INFO] os.detected.release: ubuntu
[INFO] os.detected.release.version: 16.04
[INFO] os.detected.release.like.ubuntu: true
[INFO] os.detected.release.like.debian: true
[INFO] os.detected.classifier: linux-aarch_64
[INFO]
{noformat}
[~weichiu] Let me know for any concerns, will hold this up for you a couple of
days to have a look.
> Dependency netty-all-4.1.27.Final doesn't support ARM platform
> --
>
> Key: YARN-9898
> URL: https://issues.apache.org/jira/browse/YARN-9898
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: liusheng
>Assignee: liusheng
>Priority: Major
> Attachments: YARN-9898.001.patch, YARN-9898.002.patch
>
>
> Hadoop dependent the Netty package, but the *netty-all-4.1.27.Final* of
> io.netty maven repo, cannot support ARM platform.
> When run the test *TestCsiClient.testIdentityService* on ARM server, it will
> raise error like following:
> {code:java}
> Caused by: java.io.FileNotFoundException:
> META-INF/native/libnetty_transport_native_epoll_aarch_64.so
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:161)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:243)
> at
> io.netty.util.internal.NativeLibraryLoader.load(NativeLibraryLoader.java:124)
> ... 45 more
> Suppressed: java.lang.UnsatisfiedLinkError: no
> netty_transport_native_epoll_aarch_64 in java.library.path
> at
> java.lang.ClassLoader.loadLibrary(ClassLoader.java:1867)
> at java.lang.Runtime.loadLibrary0(Runtime.java:870)
> at java.lang.System.loadLibrary(System.java:1122)
> at
> io.netty.util.internal.NativeLibraryUtil.loadLibrary(NativeLibraryUtil.java:38)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> io.netty.util.internal.NativeLibraryLoader$1.run(NativeLibraryLoader.java:263)
> at java.security.AccessController.doPrivileged(Native
> Method)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(NativeLibraryLoader.java:255)
> at
> io.netty.util.internal.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:233)
> ... 46 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10040) DistributedShell test failure on X86 and ARM
[
https://issues.apache.org/jira/browse/YARN-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17064057#comment-17064057
]
Ayush Saxena commented on YARN-10040:
-
Seems one test got fixed but {{testDSShellWithEnforceExecutionType}} still stays
> DistributedShell test failure on X86 and ARM
>
>
> Key: YARN-10040
> URL: https://issues.apache.org/jira/browse/YARN-10040
> Project: Hadoop YARN
> Issue Type: Bug
> Components: applications/distributed-shell
> Environment: X86/ARM
> OS: ubuntu1804
> Java 8
>Reporter: zhao bo
>Assignee: Abhishek Modi
>Priority: Major
> Attachments: YARN-10040.001.patch
>
>
> *
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers
> *
> org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType
> Please see the Apache Jenkins Test result:
> [https://builds.apache.org/job/hadoop-multibranch/job/PR-1767/1/testReport/]
>
> These 2 tests are failed on both X86 and ARM platform.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9831) NMTokenSecretManagerInRM#createNMToken blocks ApplicationMasterService allocate flow
[ https://issues.apache.org/jira/browse/YARN-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046489#comment-17046489 ] Ayush Saxena commented on YARN-9831: Thanx [~BilwaST] for the patch. On a quick look. Looks good Please fix the checkstyle warnings. [~maniraj...@gmail.com] Give a check, if this is fine with you.. > NMTokenSecretManagerInRM#createNMToken blocks ApplicationMasterService > allocate flow > > > Key: YARN-9831 > URL: https://issues.apache.org/jira/browse/YARN-9831 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Bibin Chundatt >Assignee: Bilwa S T >Priority: Critical > Attachments: YARN-9831.001.patch, YARN-9831.002.patch > > > Currently attempt's NMToken cannot be generated independently. > Each attempts allocate flow blocks each other. We should improve the same -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8982) [Router] Add locality policy
[ https://issues.apache.org/jira/browse/YARN-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-8982: --- Fix Version/s: (was: 3.4.0) 3.3.0 > [Router] Add locality policy > - > > Key: YARN-8982 > URL: https://issues.apache.org/jira/browse/YARN-8982 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Giovanni Matteo Fumarola >Assignee: Young Chen >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-8982.v1.patch, YARN-8982.v2.patch, > YARN-8982.v3.patch, YARN-8982.v4.patch, YARN-8982.v5.patch > > > This jira tracks the effort to add a new policy in the Router. > This policy will allow the Router to pick the SubCluster based on the node > that the client requested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Moved] (YARN-10146) [Federation] Add missing REST APIs for Router
[ https://issues.apache.org/jira/browse/YARN-10146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena moved HDFS-15179 to YARN-10146: Key: YARN-10146 (was: HDFS-15179) Project: Hadoop YARN (was: Hadoop HDFS) > [Federation] Add missing REST APIs for Router > - > > Key: YARN-10146 > URL: https://issues.apache.org/jira/browse/YARN-10146 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Bilwa S T >Priority: Major > > In FederationInterceptorREST following API are not implemented: > |getClusterInfo| > |getSchedulerInfo| > |dumpSchedulerLogs| > |getActivities| > |getAppActivities| > |getAppStatistics| > |getAppState| > |getNodeToLabels| > |getLabelsToNodes| > |replaceLabelsOnNodes| > |replaceLabelsOnNode| > |getClusterNodeLabels| > |addToClusterNodeLabels| > |removeFromCluserNodeLabels| > |getLabelsOnNode| > |getAppPriority| > |updateApplicationPriority| > |getAppQueue| > |updateAppQueue| > |postDelegationToken| > |postDelegationTokenExpiration| > |cancelDelegationToken| > |createNewReservation| > |submitReservation| > |updateReservation| > |deleteReservation| > |listReservation| > |getAppTimeout| > |getAppTimeouts| > |updateApplicationTimeout| > |getAppAttempts| > |checkUserAccessToQueue| > |getAppAttempt| > |getContainers| > |getContainer| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9624) Use switch case for ProtoUtils#convertFromProtoFormat containerState
[
https://issues.apache.org/jira/browse/YARN-9624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17026427#comment-17026427
]
Ayush Saxena commented on YARN-9624:
+1
> Use switch case for ProtoUtils#convertFromProtoFormat containerState
>
>
> Key: YARN-9624
> URL: https://issues.apache.org/jira/browse/YARN-9624
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin Chundatt
>Assignee: Bilwa S T
>Priority: Major
> Labels: performance
> Attachments: YARN-9624.001.patch, YARN-9624.002.patch,
> YARN-9624.003.patch
>
>
> On large cluster with 100K+ containers on every heartbeat
> {{ContainerState.valueOf(e.name().replace(CONTAINER_STATE_PREFIX, ""))}} will
> be too costly. Update with switch case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10040) DistributedShell test failure on X86 and ARM
[ https://issues.apache.org/jira/browse/YARN-10040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17004554#comment-17004554 ] Ayush Saxena commented on YARN-10040: - Thanx [~bzhaoopenstack] for the report. Seems YARN-9697, could be the reason, I reverted it and the tests passes for me. [~abmodi] [~bibinchundatt] [~elgoiri] you worked on YARN-9697, mind giving a check. > DistributedShell test failure on X86 and ARM > > > Key: YARN-10040 > URL: https://issues.apache.org/jira/browse/YARN-10040 > Project: Hadoop YARN > Issue Type: Bug > Components: applications/distributed-shell > Environment: X86/ARM > OS: ubuntu1804 > Java 8 >Reporter: zhao bo >Priority: Major > > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers > * > org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType > Please see the Apache Jenkins Test result: > [https://builds.apache.org/job/hadoop-multibranch/job/PR-1767/1/testReport/] > > These 2 tests are failed on both X86 and ARM platform. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-10042) Uupgrade grpc-xxx depdencies to 1.26.0
[ https://issues.apache.org/jira/browse/YARN-10042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena reassigned YARN-10042: --- Assignee: liusheng > Uupgrade grpc-xxx depdencies to 1.26.0 > -- > > Key: YARN-10042 > URL: https://issues.apache.org/jira/browse/YARN-10042 > Project: Hadoop YARN > Issue Type: Bug >Reporter: liusheng >Assignee: liusheng >Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-10042.001.patch, > hadoop_build_aarch64_grpc_1.26.0.log, hadoop_build_x86_64_grpc_1.26.0.log, > yarn_csi_tests_aarch64_grpc_1.26.0.log, yarn_csi_tests_x86_64_grpc_1.26.0.log > > > For now, Hadoop YARN use grpc-context, grpc-core, grpc-netty, grpc-protobuf, > grpc-protobuf-lite, grpc-stub and protoc-gen-grpc-java of version 1.15.1, but > the "protoc-gen-grpc-java" cannot support on aarch64 platform. Now the > grpc-java repo has support aarch64 platform and release in 1.26.0 in maven > central. > see: > [https://github.com/grpc/grpc-java/pull/6496] > [https://search.maven.org/search?q=g:io.grpc] > It is better to upgrade the version of grpc-xxx dependencies to 1.26.0 > version. both x86_64 and aarch64 server are building OK accroding to my > testing, please see the attachment, they are: log of building on aarch64, log > of building on x86_64, log of running tests of yarn csi on aarch64, log of > running tests of yarn csi on x86_64. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986089#comment-16986089
]
Ayush Saxena commented on YARN-9985:
[~aajisaka] [~tasanuma] can you help review.
> Unsupported "transitionToObserver" option displaying for rmadmin command
>
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Assignee: Ayush Saxena
>Priority: Minor
> Attachments: YARN-9985-01.patch, YARN-9985-02.patch,
> image-2019-11-18-18-31-17-755.png, image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
> -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout])
> or
> [NodeID]
[jira] [Commented] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984999#comment-16984999
]
Ayush Saxena commented on YARN-9985:
Found Failover too mentioned in two places along with transitionToObserver,
Removed this as that too wasn't supposed to be there as per YARN-3397
> Unsupported "transitionToObserver" option displaying for rmadmin command
>
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Assignee: Ayush Saxena
>Priority: Minor
> Attachments: YARN-9985-01.patch, YARN-9985-02.patch,
> image-2019-11-18-18-31-17-755.png, image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
>
[jira] [Updated] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-9985:
---
Attachment: YARN-9985-02.patch
> Unsupported "transitionToObserver" option displaying for rmadmin command
>
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Assignee: Ayush Saxena
>Priority: Minor
> Attachments: YARN-9985-01.patch, YARN-9985-02.patch,
> image-2019-11-18-18-31-17-755.png, image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
> -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout])
> or
> [NodeID] [resourcetypes] ([OvercommitTimeout]). : Update resource on
>
[jira] [Updated] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-9985:
---
Summary: Unsupported "transitionToObserver" option displaying for rmadmin
command (was: YARN: Unsupported "transitionToObserver" option displaying for
rmadmin command)
> Unsupported "transitionToObserver" option displaying for rmadmin command
>
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Assignee: Ayush Saxena
>Priority: Minor
> Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png,
> image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
> -updateNodeResource [NodeID]
[jira] [Commented] (YARN-9985) Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984617#comment-16984617
]
Ayush Saxena commented on YARN-9985:
Thanx [~SouryakantaDwivedy] for the report.
Uploaded patch with the fix,
Pls Review!!!
> Unsupported "transitionToObserver" option displaying for rmadmin command
>
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Assignee: Ayush Saxena
>Priority: Minor
> Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png,
> image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
> -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout])
> or
>
[jira] [Updated] (YARN-9985) YARN: Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-9985:
---
Attachment: YARN-9985-01.patch
> YARN: Unsupported "transitionToObserver" option displaying for rmadmin command
> --
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Priority: Minor
> Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png,
> image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
> -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout])
> or
> [NodeID] [resourcetypes] ([OvercommitTimeout]). : Update resource on
> specific node.
> -transitionToActive
[jira] [Assigned] (YARN-9985) YARN: Unsupported "transitionToObserver" option displaying for rmadmin command
[
https://issues.apache.org/jira/browse/YARN-9985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena reassigned YARN-9985:
--
Assignee: Ayush Saxena
> YARN: Unsupported "transitionToObserver" option displaying for rmadmin command
> --
>
> Key: YARN-9985
> URL: https://issues.apache.org/jira/browse/YARN-9985
> Project: Hadoop YARN
> Issue Type: Bug
> Components: RM, yarn
>Affects Versions: 3.2.1
>Reporter: Souryakanta Dwivedy
>Assignee: Ayush Saxena
>Priority: Minor
> Attachments: YARN-9985-01.patch, image-2019-11-18-18-31-17-755.png,
> image-2019-11-18-18-35-54-688.png
>
>
> Unsupported "transitionToObserver" option displaying for rmadmin command
> Check the options for Yarn rmadmin command
> It will display the "-transitionToObserver " option which is not
> supported
> by yarn rmadmin command which is wrong behavior.
> But if you check the yarn rmadmin -help it will not display any option
> "-transitionToObserver "
>
> !image-2019-11-18-18-31-17-755.png!
>
> ==
> install/hadoop/resourcemanager/bin> ./yarn rmadmin -help
> rmadmin is the command to execute YARN administrative commands.
> The full syntax is:
> yarn rmadmin [-refreshQueues] [-refreshNodes [-g|graceful [timeout in
> seconds] -client|server]] [-refreshNodesResources]
> [-refreshSuperUserGroupsConfiguration] [-refreshUserToGroupsMappings]
> [-refreshAdminAcls] [-refreshServiceAcl] [-getGroup [username]]
> [-addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">]
> [-removeFromClusterNodeLabels ] [-replaceLabelsOnNode
> <"node1[:port]=label1,label2 node2[:port]=label1"> [-failOnUnknownNodes]]
> [-directlyAccessNodeLabelStore] [-refreshClusterMaxPriority]
> [-updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout]) or
> -updateNodeResource [NodeID] [ResourceTypes] ([OvercommitTimeout])]
> *{color:#FF}[-transitionToActive [--forceactive] ]{color}
> {color:#FF}[-transitionToStandby ]{color}* [-getServiceState
> ] [-getAllServiceState] [-checkHealth ] [-help [cmd]]
> -refreshQueues: Reload the queues' acls, states and scheduler specific
> properties.
> ResourceManager will reload the mapred-queues configuration file.
> -refreshNodes [-g|graceful [timeout in seconds] -client|server]: Refresh the
> hosts information at the ResourceManager. Here [-g|graceful [timeout in
> seconds] -client|server] is optional, if we specify the timeout then
> ResourceManager will wait for timeout before marking the NodeManager as
> decommissioned. The -client|server indicates if the timeout tracking should
> be handled by the client or the ResourceManager. The client-side tracking is
> blocking, while the server-side tracking is not. Omitting the timeout, or a
> timeout of -1, indicates an infinite timeout. Known Issue: the server-side
> tracking will immediately decommission if an RM HA failover occurs.
> -refreshNodesResources: Refresh resources of NodeManagers at the
> ResourceManager.
> -refreshSuperUserGroupsConfiguration: Refresh superuser proxy groups mappings
> -refreshUserToGroupsMappings: Refresh user-to-groups mappings
> -refreshAdminAcls: Refresh acls for administration of ResourceManager
> -refreshServiceAcl: Reload the service-level authorization policy file.
> ResourceManager will reload the authorization policy file.
> -getGroups [username]: Get the groups which given user belongs to.
> -addToClusterNodeLabels
> <"label1(exclusive=true),label2(exclusive=false),label3">: add to cluster
> node labels. Default exclusivity is true
> -removeFromClusterNodeLabels (label splitted by ","):
> remove from cluster node labels
> -replaceLabelsOnNode <"node1[:port]=label1,label2
> node2[:port]=label1,label2"> [-failOnUnknownNodes] : replace labels on nodes
> (please note that we do not support specifying multiple labels on a single
> host for now.)
> [-failOnUnknownNodes] is optional, when we set this option, it will fail if
> specified nodes are unknown.
> -directlyAccessNodeLabelStore: This is DEPRECATED, will be removed in future
> releases. Directly access node label store, with this option, all node label
> related operations will not connect RM. Instead, they will access/modify
> stored node labels directly. By default, it is false (access via RM). AND
> PLEASE NOTE: if you configured yarn.node-labels.fs-store.root-dir to a local
> directory (instead of NFS or HDFS), this option will only work when the
> command run on the machine where RM is running.
> -refreshClusterMaxPriority: Refresh cluster max priority
> -updateNodeResource [NodeID] [MemSize] [vCores] ([OvercommitTimeout])
> or
> [NodeID] [resourcetypes] ([OvercommitTimeout]). : Update resource on
> specific
[jira] [Commented] (YARN-9624) Use switch case for ProtoUtils#convertFromProtoFormat containerState
[
https://issues.apache.org/jira/browse/YARN-9624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16958057#comment-16958057
]
Ayush Saxena commented on YARN-9624:
For the test instead of having if-else checks and explicitly passing one elemt,
it can be done in one line using streams, Something like this :
{code:java}
@Test
public void testConvertFromOrToProtoFormat() {
// Check if utility has all enum values
Stream.of(ContainerState.values())
.forEach(a -> ProtoUtils.convertToProtoFormat(a));
Stream.of(ContainerSubState.values())
.forEach(a -> ProtoUtils.convertToProtoFormat(a));
Stream.of(ContainerSubStateProto.values())
.forEach(a -> ProtoUtils.convertFromProtoFormat(a));
Stream.of(ContainerStateProto.values())
.forEach(a -> ProtoUtils.convertFromProtoFormat(a));
}
{code}
Give a check if it sounds better.
> Use switch case for ProtoUtils#convertFromProtoFormat containerState
>
>
> Key: YARN-9624
> URL: https://issues.apache.org/jira/browse/YARN-9624
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Bibin Chundatt
>Assignee: Bilwa S T
>Priority: Major
> Labels: performance
> Attachments: YARN-9624.001.patch, YARN-9624.002.patch
>
>
> On large cluster with 100K+ containers on every heartbeat
> {{ContainerState.valueOf(e.name().replace(CONTAINER_STATE_PREFIX, ""))}} will
> be too costly. Update with switch case.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16776636#comment-16776636 ] Ayush Saxena commented on YARN-9287: Test Failures Seems Not Related. [~bibinchundatt] [~giovanni.fumarola] Can you give a look. :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch, > YARN-9287-05.patch, YARN-9287-06.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-06.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch, > YARN-9287-05.patch, YARN-9287-06.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-05.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch, YARN-9287-05.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16775310#comment-16775310 ] Ayush Saxena commented on YARN-9287: Thanx [~giovanni.fumarola] for the review. Have uploaded the rebased version. Pls Review :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-05.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch, YARN-9287-05.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773139#comment-16773139 ] Ayush Saxena commented on YARN-9287: Ping [~bibinchundatt] [~giovanni.fumarola] Can Someone Help With The Review :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16770297#comment-16770297 ] Ayush Saxena commented on YARN-9287: Fixed CheckStyle in v4. Other Warnings for pathlen and all seems not related. Pls Review :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-04.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch, YARN-9287-04.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-03.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch, > YARN-9287-03.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-02.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch, YARN-9287-02.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16762986#comment-16762986 ] Ayush Saxena commented on YARN-9287: Hi [~giovanni.fumarola]!!! Can you help review. :) > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9287) Consecutive String Builder Append Should Reuse
[ https://issues.apache.org/jira/browse/YARN-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated YARN-9287: --- Attachment: YARN-9287-01.patch > Consecutive String Builder Append Should Reuse > -- > > Key: YARN-9287 > URL: https://issues.apache.org/jira/browse/YARN-9287 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: YARN-9287-01.patch > > > Consecutive calls to StringBuffer/StringBuilder .append should be chained, > reusing the target object. This can improve the performance by producing a > smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9287) Consecutive String Builder Append Should Reuse
Ayush Saxena created YARN-9287: -- Summary: Consecutive String Builder Append Should Reuse Key: YARN-9287 URL: https://issues.apache.org/jira/browse/YARN-9287 Project: Hadoop YARN Issue Type: Improvement Reporter: Ayush Saxena Assignee: Ayush Saxena Consecutive calls to StringBuffer/StringBuilder .append should be chained, reusing the target object. This can improve the performance by producing a smaller bytecode, reducing overhead and improving inlining. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748635#comment-16748635
]
Ayush Saxena edited comment on YARN-8498 at 1/23/19 9:43 AM:
-
[~vinayrpet] [~sunilg] [~surendrasingh]
Mind giving a check
was (Author: ayushtkn):
[~vinayrpet] [~sunilg]
Mind giving a check
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16748635#comment-16748635
]
Ayush Saxena commented on YARN-8498:
[~vinayrpet] [~sunilg]
Mind giving a check
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738672#comment-16738672
]
Ayush Saxena commented on YARN-8498:
Yetus ran on older version.
After patch Native compilation passes for me on UBUNTU-18.04/GCC 7.3.0
{noformat}
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop YARN . SUCCESS [ 3.270 s]
[INFO] Apache Hadoop YARN API . SUCCESS [ 13.977 s]
[INFO] Apache Hadoop YARN Common .. SUCCESS [ 11.779 s]
[INFO] Apache Hadoop YARN Server .. SUCCESS [ 0.610 s]
[INFO] Apache Hadoop YARN Server Common ... SUCCESS [ 7.117 s]
[INFO] Apache Hadoop YARN NodeManager . SUCCESS [ 54.988 s]
[INFO] Apache Hadoop YARN Web Proxy ... SUCCESS [ 6.875 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ... SUCCESS [ 5.056 s]
[INFO] Apache Hadoop YARN Timeline Service SUCCESS [ 3.475 s]
[INFO] Apache Hadoop YARN ResourceManager . SUCCESS [ 14.097 s]
[INFO] Apache Hadoop YARN Server Tests SUCCESS [ 3.539 s]
[INFO] Apache Hadoop YARN Client .. SUCCESS [ 4.673 s]
[INFO] Apache Hadoop YARN SharedCacheManager .. SUCCESS [ 3.990 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage . SUCCESS [ 3.146 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [ 0.638 s]
[INFO] Apache Hadoop YARN TimelineService HBase Common SUCCESS [ 6.824 s]
[INFO] Apache Hadoop YARN TimelineService HBase Client SUCCESS [ 7.431 s]
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [ 0.578 s]
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SUCCESS [ 5.473 s]
[INFO] Apache Hadoop YARN TimelineService HBase tests . SUCCESS [ 6.380 s]
[INFO] Apache Hadoop YARN Router .. SUCCESS [ 5.389 s]
[INFO] Apache Hadoop YARN Applications SUCCESS [ 0.661 s]
[INFO] Apache Hadoop YARN DistributedShell SUCCESS [ 3.212 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ... SUCCESS [ 2.647 s]
[INFO] Apache Hadoop YARN Services SUCCESS [ 0.683 s]
[INFO] Apache Hadoop YARN Services Core ... SUCCESS [ 9.123 s]
[INFO] Apache Hadoop YARN Services API SUCCESS [ 4.718 s]
[INFO] Yet Another Learning Platform .. SUCCESS [ 3.505 s]
[INFO] Apache Hadoop YARN Site SUCCESS [ 0.612 s]
[INFO] Apache Hadoop YARN Registry SUCCESS [ 1.621 s]
[INFO] Apache Hadoop YARN UI .. SUCCESS [ 1.163 s]
[INFO] Apache Hadoop YARN CSI . SUCCESS [ 14.102 s]
[INFO] Apache Hadoop YARN Project . SUCCESS [ 3.415 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 03:35 min
[INFO] Finished at: 2019-01-10T02:44:39+05:30
[INFO] Final Memory: 210M/870M
[INFO]
{noformat}
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error:
