[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602622#comment-17602622
]
ASF GitHub Bot commented on YARN-11296:
---
slfan1989 commented on code in PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#discussion_r967601774
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreTables.sql:
##
@@ -140,7 +140,7 @@ IF NOT EXISTS ( SELECT * FROM
[FederationStateStore].sys.tables
CREATE TABLE [dbo].[reservationsHomeSubCluster](
reservationId VARCHAR(128) COLLATE Latin1_General_100_BIN2 NOT
NULL,
homeSubCluster VARCHAR(256) NOT NULL,
-createTime DATETIME2 NOT NULL CONSTRAINT ts_createAppTime
DEFAULT GETUTCDATE(),
+createTime DATETIME2 NOT NULL CONSTRAINT ts_createResTime
DEFAULT GETUTCDATE(),
Review Comment:
This is the name of the constraint (newly specified when the table was
created), for reservation I think it might be better to prefix it with res.
> Fix SQLFederationStateStore#Sql script bug
> --
>
> Key: YARN-11296
> URL: https://issues.apache.org/jira/browse/YARN-11296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> *MySQL:*
> FederationStateStoreTables.sql cannot create *membership* table, both in
> MySQL 5.7 and MySQL 8.0.
>
> {code:java}
> CREATE TABLE membership(
>subClusterId varchar(256) NOT NULL,
>amRMServiceAddress varchar(256) NOT NULL,
>clientRMServiceAddress varchar(256) NOT NULL,
>rmAdminServiceAddress varchar(256) NOT NULL,
>rmWebServiceAddress varchar(256) NOT NULL,
>lastHeartBeat datetime NOT NULL,
>state varchar(32) NOT NULL,
>lastStartTime bigint NULL,
>capability varchar(6000),
>CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId), -- missing comma
> UNIQUE(lastStartTime)
> ); {code}
>
> *SQLServer:*
> FederationStateStoreTables.sql
>
> {code:java}
> IF NOT EXISTS ( SELECT * FROM [FederationStateStore].sys.tables
> WHERE name = 'membership'
> AND schema_id = SCHEMA_ID('dbo'))
> BEGIN
> PRINT 'Table membership does not exist, create it...'
> SET ANSI_NULLS ON
> SET QUOTED_IDENTIFIER ON
> SET ANSI_PADDING ON
> CREATE TABLE [dbo].[membership](
> [subClusterId]VARCHAR(256) COLLATE
> Latin1_General_100_BIN2 NOT NULL,
> [amRMServiceAddress] VARCHAR(256) NOT NULL,
> [clientRMServiceAddress] VARCHAR(256) NOT NULL,
> [rmAdminServiceAddress] VARCHAR(256) NOT NULL,
> [rmWebServiceAddress] VARCHAR(256) NOT NULL,
> [lastHeartBeat] DATETIME2 NOT NULL,
> [state] VARCHAR(32) NOT NULL,
> [lastStartTime] BIGINT NOT NULL,
> [capability] VARCHAR(6000) NOT NULL,
> CONSTRAINT [pk_subClusterId] PRIMARY KEY
> (
> [subClusterId]
> ), -- missing comma
> CONSTRAINT [uc_lastStartTime] UNIQUE
> (
> [lastStartTime]
> )
> )
> SET ANSI_PADDING OFF
> PRINT 'Table membership created.'
> END
> ELSE
> PRINT 'Table membership exists, no operation required...'
> GO
> GO {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602621#comment-17602621
]
ASF GitHub Bot commented on YARN-11296:
---
slfan1989 commented on code in PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#discussion_r967601401
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreTables.sql:
##
@@ -36,7 +36,7 @@ CREATE TABLE membership(
state varchar(32) NOT NULL,
lastStartTime bigint NULL,
capability varchar(6000),
- CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId)
+ CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId),
Review Comment:
Thank you very much for your help in reviewing the code. This part of the
modification has been verified. I used the compatibility mode of HSQLDB's Mysql
and SqlServer to verify the table and stored procedure scripts respectively.
> Fix SQLFederationStateStore#Sql script bug
> --
>
> Key: YARN-11296
> URL: https://issues.apache.org/jira/browse/YARN-11296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> *MySQL:*
> FederationStateStoreTables.sql cannot create *membership* table, both in
> MySQL 5.7 and MySQL 8.0.
>
> {code:java}
> CREATE TABLE membership(
>subClusterId varchar(256) NOT NULL,
>amRMServiceAddress varchar(256) NOT NULL,
>clientRMServiceAddress varchar(256) NOT NULL,
>rmAdminServiceAddress varchar(256) NOT NULL,
>rmWebServiceAddress varchar(256) NOT NULL,
>lastHeartBeat datetime NOT NULL,
>state varchar(32) NOT NULL,
>lastStartTime bigint NULL,
>capability varchar(6000),
>CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId), -- missing comma
> UNIQUE(lastStartTime)
> ); {code}
>
> *SQLServer:*
> FederationStateStoreTables.sql
>
> {code:java}
> IF NOT EXISTS ( SELECT * FROM [FederationStateStore].sys.tables
> WHERE name = 'membership'
> AND schema_id = SCHEMA_ID('dbo'))
> BEGIN
> PRINT 'Table membership does not exist, create it...'
> SET ANSI_NULLS ON
> SET QUOTED_IDENTIFIER ON
> SET ANSI_PADDING ON
> CREATE TABLE [dbo].[membership](
> [subClusterId]VARCHAR(256) COLLATE
> Latin1_General_100_BIN2 NOT NULL,
> [amRMServiceAddress] VARCHAR(256) NOT NULL,
> [clientRMServiceAddress] VARCHAR(256) NOT NULL,
> [rmAdminServiceAddress] VARCHAR(256) NOT NULL,
> [rmWebServiceAddress] VARCHAR(256) NOT NULL,
> [lastHeartBeat] DATETIME2 NOT NULL,
> [state] VARCHAR(32) NOT NULL,
> [lastStartTime] BIGINT NOT NULL,
> [capability] VARCHAR(6000) NOT NULL,
> CONSTRAINT [pk_subClusterId] PRIMARY KEY
> (
> [subClusterId]
> ), -- missing comma
> CONSTRAINT [uc_lastStartTime] UNIQUE
> (
> [lastStartTime]
> )
> )
> SET ANSI_PADDING OFF
> PRINT 'Table membership created.'
> END
> ELSE
> PRINT 'Table membership exists, no operation required...'
> GO
> GO {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602598#comment-17602598
]
ASF GitHub Bot commented on YARN-11274:
---
hadoop-yetus commented on PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#issuecomment-1242600230
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 19s | | trunk passed |
| +1 :green_heart: | compile | 1m 39s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 35s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 33s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 0s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 122m 22s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4783 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 29cf11697cc4 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f08532694e8c3b679fb462c10fe18c7b531a9aff |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/4/testReport/ |
| Max. process+thread count | 556 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
[jira] [Commented] (YARN-7614) [RESERVATION] Support ListReservation APIs in Federation Router
[
https://issues.apache.org/jira/browse/YARN-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602590#comment-17602590
]
ASF GitHub Bot commented on YARN-7614:
--
slfan1989 commented on code in PR #4843:
URL: https://github.com/apache/hadoop/pull/4843#discussion_r967555986
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServicesREST.java:
##
@@ -198,6 +202,7 @@ public Boolean get() {
}
}, 1000, 10 * 1000);
} catch (Exception e) {
+ LOG.error("Web app not running", e);
Review Comment:
We should add some more information, because the current log is not enough
for analysis after an error is reported.
The error stack is as follows:
```
Error Message
Web app not running
Stacktrace
java.lang.AssertionError: Web app not running
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.waitWebAppRunning(TestRouterWebServicesREST.java:201)
at
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.setUp(TestRouterWebServicesREST.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
```
We can also view the following test report
[Test-Report](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4843/12/testReport/org.apache.hadoop.yarn.server.router.webapp/TestRouterWebServicesREST/org_apache_hadoop_yarn_server_router_webapp_TestRouterWebServicesREST/)
Just looking at the error message of the stack, it seems impossible to
analyze why the `Web app is not running`.
> [RESERVATION] Support ListReservation APIs in Federation Router
> ---
>
> Key: YARN-7614
> URL: https://issues.apache.org/jira/browse/YARN-7614
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation, reservation system
>Reporter: Carlo Curino
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7614) [RESERVATION] Support ListReservation APIs in Federation Router
[
https://issues.apache.org/jira/browse/YARN-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602589#comment-17602589
]
ASF GitHub Bot commented on YARN-7614:
--
slfan1989 commented on code in PR #4843:
URL: https://github.com/apache/hadoop/pull/4843#discussion_r967555986
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServicesREST.java:
##
@@ -198,6 +202,7 @@ public Boolean get() {
}
}, 1000, 10 * 1000);
} catch (Exception e) {
+ LOG.error("Web app not running", e);
Review Comment:
We should also add some more information, because the current log is not
enough for analysis after an error is reported.
The error stack is as follows:
```
Error Message
Web app not running
Stacktrace
java.lang.AssertionError: Web app not running
at org.junit.Assert.fail(Assert.java:89)
at
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.waitWebAppRunning(TestRouterWebServicesREST.java:201)
at
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.setUp(TestRouterWebServicesREST.java:225)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
```
We can also view the following test report
[Test-Report](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4843/12/testReport/org.apache.hadoop.yarn.server.router.webapp/TestRouterWebServicesREST/org_apache_hadoop_yarn_server_router_webapp_TestRouterWebServicesREST/)
Just looking at the error message of the stack, it seems impossible to
analyze why the `Web app is not running`.
> [RESERVATION] Support ListReservation APIs in Federation Router
> ---
>
> Key: YARN-7614
> URL: https://issues.apache.org/jira/browse/YARN-7614
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation, reservation system
>Reporter: Carlo Curino
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11297) Improve Yarn Router Reservation Submission Code
[ https://issues.apache.org/jira/browse/YARN-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602584#comment-17602584 ] ASF GitHub Bot commented on YARN-11297: --- slfan1989 commented on PR #4863: URL: https://github.com/apache/hadoop/pull/4863#issuecomment-1242573530 @goiri Thank you very much for helping to review the code! > Improve Yarn Router Reservation Submission Code > --- > > Key: YARN-11297 > URL: https://issues.apache.org/jira/browse/YARN-11297 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > The same reservation may be submitted repeatedly. At this time, we should use > the reserved results first. If the reserved results are not available, > consider applying from other RMs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602564#comment-17602564
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1242502110
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 44s | | trunk passed |
| +1 :green_heart: | compile | 10m 38s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 9s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 10m 15s | | trunk passed |
| +1 :green_heart: | javadoc | 7m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 6m 14s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 9s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 31s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 30s | | the patch passed |
| +1 :green_heart: | compile | 9m 56s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 56s | | the patch passed |
| +1 :green_heart: | javac | 9m 56s | | the patch passed |
| +1 :green_heart: | compile | 8m 59s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 8m 59s | | the patch passed |
| +1 :green_heart: | javac | 8m 59s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/16/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/16/artifact/out/blanks-tabs.txt)
| The patch 10 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 36s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 55s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 261m 23s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 21s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 7s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 24s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 5s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 13s | | The patch does not
generate ASF License warnings. |
| | | 503m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602553#comment-17602553
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1242477787
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 55s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 22s | | trunk passed |
| +1 :green_heart: | compile | 10m 47s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 25s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 32s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 37s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 43s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 8s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 35s | | the patch passed |
| +1 :green_heart: | compile | 9m 51s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 51s | | the patch passed |
| +1 :green_heart: | javac | 9m 51s | | the patch passed |
| +1 :green_heart: | compile | 9m 18s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 18s | | the patch passed |
| +1 :green_heart: | javac | 9m 18s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/15/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/15/artifact/out/blanks-tabs.txt)
| The patch 10 line(s) with tabs. |
| +1 :green_heart: | checkstyle | 1m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 17s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 19m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 237m 30s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 27s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 21s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 33s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 9s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 473m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41
[jira] [Commented] (YARN-11290) Improve Query Condition of FederationStateStore#getApplicationsHomeSubCluster
[
https://issues.apache.org/jira/browse/YARN-11290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602510#comment-17602510
]
ASF GitHub Bot commented on YARN-11290:
---
hadoop-yetus commented on PR #4846:
URL: https://github.com/apache/hadoop/pull/4846#issuecomment-1242351273
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 46s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 41s | | trunk passed |
| +1 :green_heart: | compile | 9m 57s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 47s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 8m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 40s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 5m 59s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 18m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 3s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 27s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 35s | | the patch passed |
| +1 :green_heart: | compile | 9m 21s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 21s | | the patch passed |
| +1 :green_heart: | javac | 9m 21s | | the patch passed |
| +1 :green_heart: | compile | 9m 4s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 4s | | the patch passed |
| +1 :green_heart: | javac | 9m 4s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/14/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 52s | | the patch passed |
| +1 :green_heart: | mvnsite | 8m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 38s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 57s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 20m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 14s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 1m 30s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 29s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 32s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 4m 13s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 19s | | The patch does not
generate ASF License warnings. |
| | | 469m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4846/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4846 |
| Optional
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602505#comment-17602505
]
ASF GitHub Bot commented on YARN-11296:
---
hadoop-yetus commented on PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#issuecomment-1242322388
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 33s | | trunk passed |
| +1 :green_heart: | compile | 10m 39s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 15s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 3s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 8s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 7s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 25s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 14m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 23s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 28s | | the patch passed |
| +1 :green_heart: | compile | 10m 3s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| -1 :x: | javac | 10m 3s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/6/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 2 new + 741
unchanged - 0 fixed = 743 total (was 741) |
| +1 :green_heart: | compile | 9m 8s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 9m 8s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/6/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 2 new
+ 654 unchanged - 0 fixed = 656 total (was 654) |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 51s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/6/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 9 new + 0 unchanged -
1 fixed = 9 total (was 1) |
| +1 :green_heart: | mvnsite | 5m 55s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 53s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 14s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 14m 33s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 242m 52s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 3m 19s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| |
[jira] [Resolved] (YARN-11297) Improve Yarn Router Reservation Submission Code
[ https://issues.apache.org/jira/browse/YARN-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved YARN-11297. Resolution: Fixed > Improve Yarn Router Reservation Submission Code > --- > > Key: YARN-11297 > URL: https://issues.apache.org/jira/browse/YARN-11297 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > The same reservation may be submitted repeatedly. At this time, we should use > the reserved results first. If the reserved results are not available, > consider applying from other RMs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11297) Improve Yarn Router Reservation Submission Code
[ https://issues.apache.org/jira/browse/YARN-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602486#comment-17602486 ] ASF GitHub Bot commented on YARN-11297: --- goiri merged PR #4863: URL: https://github.com/apache/hadoop/pull/4863 > Improve Yarn Router Reservation Submission Code > --- > > Key: YARN-11297 > URL: https://issues.apache.org/jira/browse/YARN-11297 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > The same reservation may be submitted repeatedly. At this time, we should use > the reserved results first. If the reserved results are not available, > consider applying from other RMs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602483#comment-17602483
]
ASF GitHub Bot commented on YARN-11296:
---
goiri commented on code in PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#discussion_r967322160
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/SQLServer/FederationStateStoreTables.sql:
##
@@ -140,7 +140,7 @@ IF NOT EXISTS ( SELECT * FROM
[FederationStateStore].sys.tables
CREATE TABLE [dbo].[reservationsHomeSubCluster](
reservationId VARCHAR(128) COLLATE Latin1_General_100_BIN2 NOT
NULL,
homeSubCluster VARCHAR(256) NOT NULL,
-createTime DATETIME2 NOT NULL CONSTRAINT ts_createAppTime
DEFAULT GETUTCDATE(),
+createTime DATETIME2 NOT NULL CONSTRAINT ts_createResTime
DEFAULT GETUTCDATE(),
Review Comment:
This is also used in line 37. Where does this field come from?
##
hadoop-yarn-project/hadoop-yarn/bin/FederationStateStore/MySQL/FederationStateStoreTables.sql:
##
@@ -36,7 +36,7 @@ CREATE TABLE membership(
state varchar(32) NOT NULL,
lastStartTime bigint NULL,
capability varchar(6000),
- CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId)
+ CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId),
Review Comment:
Is this validated by the unit test?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/sql/TestFDMySQLAccuracy.java:
##
@@ -0,0 +1,73 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.sql;
+
+import org.apache.hadoop.yarn.conf.YarnConfiguration;
+import org.apache.hadoop.yarn.server.federation.store.FederationStateStore;
+import
org.apache.hadoop.yarn.server.federation.store.impl.MySQLFederationStateStore;
+import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.sql.SQLException;
+import java.util.List;
+
+public class TestFDMySQLAccuracy extends FederationSQLAccuracyTest {
Review Comment:
What is FD?
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/SQLServerFederationStateStore.java:
##
@@ -0,0 +1,114 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ *http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.yarn.server.federation.store.impl;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.yarn.exceptions.YarnException;
+import org.apache.hadoop.yarn.server.federation.store.FederationStateStore;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.sql.Connection;
+import java.sql.SQLException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+/**
+ * SQLServerFederationStateStore implementation of {@link
FederationStateStore}.
+ */
+public class SQLServerFederationStateStore extends SQLFederationStateStore {
+
+ private static final Logger LOG =
+
[jira] [Commented] (YARN-7614) [RESERVATION] Support ListReservation APIs in Federation Router
[
https://issues.apache.org/jira/browse/YARN-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602481#comment-17602481
]
ASF GitHub Bot commented on YARN-7614:
--
goiri commented on code in PR #4843:
URL: https://github.com/apache/hadoop/pull/4843#discussion_r967317594
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServicesREST.java:
##
@@ -198,6 +202,7 @@ public Boolean get() {
}
}, 1000, 10 * 1000);
} catch (Exception e) {
+ LOG.error("Web app not running", e);
Review Comment:
Why do we do this? Isn't it clear enough with the "fail()" message?
> [RESERVATION] Support ListReservation APIs in Federation Router
> ---
>
> Key: YARN-7614
> URL: https://issues.apache.org/jira/browse/YARN-7614
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: federation, reservation system
>Reporter: Carlo Curino
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11293) [Federation] Router Support DelegationToken storeNewMasterKey/removeStoredMasterKey With MemoryStateStore
[ https://issues.apache.org/jira/browse/YARN-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602417#comment-17602417 ] ASF GitHub Bot commented on YARN-11293: --- slfan1989 commented on PR #4852: URL: https://github.com/apache/hadoop/pull/4852#issuecomment-1242113989 @goiri Can you help merge to trunk branch? Thank you very much! I will continue to develop YARN-11294. > [Federation] Router Support DelegationToken > storeNewMasterKey/removeStoredMasterKey With MemoryStateStore > - > > Key: YARN-11293 > URL: https://issues.apache.org/jira/browse/YARN-11293 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > Support storing storeNewMasterKey/removeStoredMasterKey in MemoryStore mode. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7614) [RESERVATION] Support ListReservation APIs in Federation Router
[ https://issues.apache.org/jira/browse/YARN-7614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602414#comment-17602414 ] ASF GitHub Bot commented on YARN-7614: -- slfan1989 commented on PR #4843: URL: https://github.com/apache/hadoop/pull/4843#issuecomment-1242106562 @goiri Can you help merge this pr into trunk branch? I am continuing to finish YARN-11226 on the basis of this pr. Thank you very much! `Hadoop.yarn.server.router.webapp.TestRouterWebServicesREST` has failed to run Junit Test, but there is no clear reason for the error in the log. I added some logs. If there is an exception in TestRouterWebServicesREST in the later development, we can locate and solve it. From a personal point of view, the change of this pr should not affect the operation of TestRouterWebServicesREST. > [RESERVATION] Support ListReservation APIs in Federation Router > --- > > Key: YARN-7614 > URL: https://issues.apache.org/jira/browse/YARN-7614 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation, reservation system >Reporter: Carlo Curino >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602408#comment-17602408
]
ASF GitHub Bot commented on YARN-11296:
---
slfan1989 commented on PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#issuecomment-1242091483
@goiri Please help to review this pr, thank you very much!
> Fix SQLFederationStateStore#Sql script bug
> --
>
> Key: YARN-11296
> URL: https://issues.apache.org/jira/browse/YARN-11296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> *MySQL:*
> FederationStateStoreTables.sql cannot create *membership* table, both in
> MySQL 5.7 and MySQL 8.0.
>
> {code:java}
> CREATE TABLE membership(
>subClusterId varchar(256) NOT NULL,
>amRMServiceAddress varchar(256) NOT NULL,
>clientRMServiceAddress varchar(256) NOT NULL,
>rmAdminServiceAddress varchar(256) NOT NULL,
>rmWebServiceAddress varchar(256) NOT NULL,
>lastHeartBeat datetime NOT NULL,
>state varchar(32) NOT NULL,
>lastStartTime bigint NULL,
>capability varchar(6000),
>CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId), -- missing comma
> UNIQUE(lastStartTime)
> ); {code}
>
> *SQLServer:*
> FederationStateStoreTables.sql
>
> {code:java}
> IF NOT EXISTS ( SELECT * FROM [FederationStateStore].sys.tables
> WHERE name = 'membership'
> AND schema_id = SCHEMA_ID('dbo'))
> BEGIN
> PRINT 'Table membership does not exist, create it...'
> SET ANSI_NULLS ON
> SET QUOTED_IDENTIFIER ON
> SET ANSI_PADDING ON
> CREATE TABLE [dbo].[membership](
> [subClusterId]VARCHAR(256) COLLATE
> Latin1_General_100_BIN2 NOT NULL,
> [amRMServiceAddress] VARCHAR(256) NOT NULL,
> [clientRMServiceAddress] VARCHAR(256) NOT NULL,
> [rmAdminServiceAddress] VARCHAR(256) NOT NULL,
> [rmWebServiceAddress] VARCHAR(256) NOT NULL,
> [lastHeartBeat] DATETIME2 NOT NULL,
> [state] VARCHAR(32) NOT NULL,
> [lastStartTime] BIGINT NOT NULL,
> [capability] VARCHAR(6000) NOT NULL,
> CONSTRAINT [pk_subClusterId] PRIMARY KEY
> (
> [subClusterId]
> ), -- missing comma
> CONSTRAINT [uc_lastStartTime] UNIQUE
> (
> [lastStartTime]
> )
> )
> SET ANSI_PADDING OFF
> PRINT 'Table membership created.'
> END
> ELSE
> PRINT 'Table membership exists, no operation required...'
> GO
> GO {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11297) Improve Yarn Router Reservation Submission Code
[ https://issues.apache.org/jira/browse/YARN-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602407#comment-17602407 ] ASF GitHub Bot commented on YARN-11297: --- slfan1989 commented on PR #4863: URL: https://github.com/apache/hadoop/pull/4863#issuecomment-1242090066 @goiri Please help to review this pr again, thank you very much! > Improve Yarn Router Reservation Submission Code > --- > > Key: YARN-11297 > URL: https://issues.apache.org/jira/browse/YARN-11297 > Project: Hadoop YARN > Issue Type: Improvement > Components: federation >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > > The same reservation may be submitted repeatedly. At this time, we should use > the reserved results first. If the reserved results are not available, > consider applying from other RMs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[
https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602358#comment-17602358
]
ASF GitHub Bot commented on YARN-11286:
---
hadoop-yetus commented on PR #4824:
URL: https://github.com/apache/hadoop/pull/4824#issuecomment-1242001512
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 53s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 37s | | trunk passed |
| +1 :green_heart: | compile | 10m 40s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 7s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 20s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 19s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 46s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 18s | | the patch passed |
| +1 :green_heart: | compile | 9m 44s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 9m 44s | | the patch passed |
| +1 :green_heart: | compile | 8m 52s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 8m 52s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 14s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 8s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 13s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 34s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 27s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 164m 25s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4824/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4824 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 301e0b90a5d1 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 11d717a58022e1750feb49d5b160bbded7d0dc70 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4824/3/testReport/ |
| Max.
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602334#comment-17602334
]
ASF GitHub Bot commented on YARN-11274:
---
hadoop-yetus commented on PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#issuecomment-1241965648
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 55s | | trunk passed |
| +1 :green_heart: | compile | 1m 46s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 49s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 1m 34s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 34s | | the patch passed |
| +1 :green_heart: | compile | 1m 22s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 28s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 2 new + 78 unchanged - 0 fixed = 80 total (was 78) |
| +1 :green_heart: | mvnsite | 0m 43s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 2s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 125m 16s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4783 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 445ee071e872 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 58db83935d30ddf6fd5c6c6734b3d22d40231fb9 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4783/3/testReport/ |
| Max. process+thread count
[jira] [Commented] (YARN-11278) Ambiguous error message in mutation API
[
https://issues.apache.org/jira/browse/YARN-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602313#comment-17602313
]
ASF GitHub Bot commented on YARN-11278:
---
9uapaw closed pull request #4841: YARN-11278. Fixed Ambiguous error message in
mutation API
URL: https://github.com/apache/hadoop/pull/4841
> Ambiguous error message in mutation API
> ---
>
> Key: YARN-11278
> URL: https://issues.apache.org/jira/browse/YARN-11278
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Reporter: András Győri
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
>
> In RMWebServices#updateSchedulerConfiguration, we are checking two
> prerequisites:
> {code:java}
> if (scheduler instanceof MutableConfScheduler && ((MutableConfScheduler)
> scheduler).isConfigurationMutable()) { {code}
> However, the error message is misleading in the second case (namely if the
> configuration is not mutable eg. a FILE_CONFIGURATION_STORE)
> {code:java}
> } else {
> return Response.status(Status.BAD_REQUEST)
> .entity("Configuration change only supported by " +
> "MutableConfScheduler.")
> .build(); {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11278) Ambiguous error message in mutation API
[
https://issues.apache.org/jira/browse/YARN-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602311#comment-17602311
]
ASF GitHub Bot commented on YARN-11278:
---
ashutoshcipher commented on PR #4841:
URL: https://github.com/apache/hadoop/pull/4841#issuecomment-1241924624
Thanks for reviewing @9uapaw :)
> Ambiguous error message in mutation API
> ---
>
> Key: YARN-11278
> URL: https://issues.apache.org/jira/browse/YARN-11278
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Reporter: András Győri
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
>
> In RMWebServices#updateSchedulerConfiguration, we are checking two
> prerequisites:
> {code:java}
> if (scheduler instanceof MutableConfScheduler && ((MutableConfScheduler)
> scheduler).isConfigurationMutable()) { {code}
> However, the error message is misleading in the second case (namely if the
> configuration is not mutable eg. a FILE_CONFIGURATION_STORE)
> {code:java}
> } else {
> return Response.status(Status.BAD_REQUEST)
> .entity("Configuration change only supported by " +
> "MutableConfScheduler.")
> .build(); {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11278) Ambiguous error message in mutation API
[
https://issues.apache.org/jira/browse/YARN-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602308#comment-17602308
]
ASF GitHub Bot commented on YARN-11278:
---
9uapaw commented on PR #4841:
URL: https://github.com/apache/hadoop/pull/4841#issuecomment-1241915760
Thanks for the patch @ashutoshcipher. The change looks good to me, will
commit it to trunk.
> Ambiguous error message in mutation API
> ---
>
> Key: YARN-11278
> URL: https://issues.apache.org/jira/browse/YARN-11278
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Reporter: András Győri
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
>
> In RMWebServices#updateSchedulerConfiguration, we are checking two
> prerequisites:
> {code:java}
> if (scheduler instanceof MutableConfScheduler && ((MutableConfScheduler)
> scheduler).isConfigurationMutable()) { {code}
> However, the error message is misleading in the second case (namely if the
> configuration is not mutable eg. a FILE_CONFIGURATION_STORE)
> {code:java}
> } else {
> return Response.status(Status.BAD_REQUEST)
> .entity("Configuration change only supported by " +
> "MutableConfScheduler.")
> .build(); {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11278) Ambiguous error message in mutation API
[
https://issues.apache.org/jira/browse/YARN-11278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602306#comment-17602306
]
ASF GitHub Bot commented on YARN-11278:
---
ashutoshcipher commented on PR #4841:
URL: https://github.com/apache/hadoop/pull/4841#issuecomment-1241910786
@9uapaw - Please help in reviewing the PR. Thanks.
> Ambiguous error message in mutation API
> ---
>
> Key: YARN-11278
> URL: https://issues.apache.org/jira/browse/YARN-11278
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Reporter: András Győri
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
>
> In RMWebServices#updateSchedulerConfiguration, we are checking two
> prerequisites:
> {code:java}
> if (scheduler instanceof MutableConfScheduler && ((MutableConfScheduler)
> scheduler).isConfigurationMutable()) { {code}
> However, the error message is misleading in the second case (namely if the
> configuration is not mutable eg. a FILE_CONFIGURATION_STORE)
> {code:java}
> } else {
> return Response.status(Status.BAD_REQUEST)
> .entity("Configuration change only supported by " +
> "MutableConfScheduler.")
> .build(); {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-11300) [Umbrella] Extract and refactor queue resource calculation
[ https://issues.apache.org/jira/browse/YARN-11300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] groot reassigned YARN-11300: Assignee: groot > [Umbrella] Extract and refactor queue resource calculation > -- > > Key: YARN-11300 > URL: https://issues.apache.org/jira/browse/YARN-11300 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: András Győri >Assignee: groot >Priority: Major > Labels: newbie, ramp-up > > Queue resource calculations are not centralised, scattered through many > classes and a source of dozens of bugs. We need a common place to store these > calculations, simplify them if possible and extend it with proper logging. > Example of such calculations: > [AbstractLeafQueue#assignContainers|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L1147] > [AbstractCSQueue#accept|https://github.com/apache/hadoop/blob/0e6a6d18809c1958e3aaae88f0d3ce5bf380b350/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java#L1045] > [AbstractLeafQueue#calculateAndGetAMResourceLimitPerPartition|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L766] > [AbstractLeafQueue#updateMaximumApplications|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L2344] > ... and many more. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-11300) [Umbrella] Extract and refactor queue resource calculation
[ https://issues.apache.org/jira/browse/YARN-11300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] András Győri updated YARN-11300: Labels: newbie ramp-up (was: ) > [Umbrella] Extract and refactor queue resource calculation > -- > > Key: YARN-11300 > URL: https://issues.apache.org/jira/browse/YARN-11300 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: András Győri >Priority: Major > Labels: newbie, ramp-up > > Queue resource calculations are not centralised, scattered through many > classes and a source of dozens of bugs. We need a common place to store these > calculations, simplify them if possible and extend it with proper logging. > Example of such calculations: > [AbstractLeafQueue#assignContainers|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L1147] > [AbstractCSQueue#accept|https://github.com/apache/hadoop/blob/0e6a6d18809c1958e3aaae88f0d3ce5bf380b350/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java#L1045] > [AbstractLeafQueue#calculateAndGetAMResourceLimitPerPartition|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L766] > [AbstractLeafQueue#updateMaximumApplications|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L2344] > ... and many more. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-11300) [Umbrella] Extract and refactor queue resource calculation
András Győri created YARN-11300: --- Summary: [Umbrella] Extract and refactor queue resource calculation Key: YARN-11300 URL: https://issues.apache.org/jira/browse/YARN-11300 Project: Hadoop YARN Issue Type: Improvement Reporter: András Győri Queue resource calculations are not centralised, scattered through many classes and a source of dozens of bugs. We need a common place to store these calculations, simplify them if possible and extend it with proper logging. Example of such calculations: [AbstractLeafQueue#assignContainers|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L1147] [AbstractCSQueue#accept|https://github.com/apache/hadoop/blob/0e6a6d18809c1958e3aaae88f0d3ce5bf380b350/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java#L1045] [AbstractLeafQueue#calculateAndGetAMResourceLimitPerPartition|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L766] [AbstractLeafQueue#updateMaximumApplications|https://github.com/apache/hadoop/blob/3a66348fda29b18054853baed6664fec92dc5cc5/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractLeafQueue.java#L2344] ... and many more. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602269#comment-17602269
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r966945891
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -1287,10 +1280,8 @@ public void verifyRMHeartbeatResponseForNodeLabels(
} else {
// case where updated labels from NodeLabelsProvider is sent to RM
and
// RM rejected the labels
- LOG.error(
- "NM node labels {" + StringUtils.join(",", getPreviousValue())
- + "} were not accepted by RM and message from RM : "
- + response.getDiagnosticsMessage());
+ LOG.error("NM node labels {{}} were not accepted by RM and message
from RM : {}.",
+ StringUtils.join(",", getPreviousValue()),
response.getDiagnosticsMessage());
Review Comment:
I will fix it.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602268#comment-17602268
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r966945533
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -1136,9 +1130,8 @@ public void verifyRMHeartbeatResponseForNodeAttributes(
} else {
// case where updated node attributes from NodeAttributesProvider
// is sent to RM and RM rejected the attributes
- LOG.error("NM node attributes {" + getPreviousValue()
- + "} were not accepted by RM and message from RM : " + response
- .getDiagnosticsMessage());
+ LOG.error("NM node attributes {{}} were not accepted by RM and
message from RM : {}." ,
+ getPreviousValue(), response.getDiagnosticsMessage() );
Review Comment:
I agree with you, I will modify the code.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602267#comment-17602267
]
ASF GitHub Bot commented on YARN-11274:
---
slfan1989 commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r966944820
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdater.java:
##
@@ -625,7 +625,7 @@ public RegisterNodeManagerResponse registerNodeManager(
@Override
public NodeHeartbeatResponse nodeHeartbeat(NodeHeartbeatRequest request)
throws YarnException, IOException {
- LOG.info("Got heartBeatId: [" + heartBeatID +"]");
+ LOG.info("Got heartBeatId: [{}}]", heartBeatID);
Review Comment:
Thanks for your help reviewing the code. I will modify the code, the last }
is redundant.
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602262#comment-17602262
]
ASF GitHub Bot commented on YARN-11296:
---
slfan1989 commented on PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#issuecomment-1241825833
The implementation idea of SQL script Check is as follows:
1. From the script directory, synchronize Mysql and SqlServer scripts for
creating tables and creating stored procedures.
2. Use HSQLDB's Mysql and SqlServer compatibility modes respectively to
verify the creation of tables and stored procedures.
> Fix SQLFederationStateStore#Sql script bug
> --
>
> Key: YARN-11296
> URL: https://issues.apache.org/jira/browse/YARN-11296
> Project: Hadoop YARN
> Issue Type: Bug
> Components: federation
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
>
> *MySQL:*
> FederationStateStoreTables.sql cannot create *membership* table, both in
> MySQL 5.7 and MySQL 8.0.
>
> {code:java}
> CREATE TABLE membership(
>subClusterId varchar(256) NOT NULL,
>amRMServiceAddress varchar(256) NOT NULL,
>clientRMServiceAddress varchar(256) NOT NULL,
>rmAdminServiceAddress varchar(256) NOT NULL,
>rmWebServiceAddress varchar(256) NOT NULL,
>lastHeartBeat datetime NOT NULL,
>state varchar(32) NOT NULL,
>lastStartTime bigint NULL,
>capability varchar(6000),
>CONSTRAINT pk_subClusterId PRIMARY KEY (subClusterId), -- missing comma
> UNIQUE(lastStartTime)
> ); {code}
>
> *SQLServer:*
> FederationStateStoreTables.sql
>
> {code:java}
> IF NOT EXISTS ( SELECT * FROM [FederationStateStore].sys.tables
> WHERE name = 'membership'
> AND schema_id = SCHEMA_ID('dbo'))
> BEGIN
> PRINT 'Table membership does not exist, create it...'
> SET ANSI_NULLS ON
> SET QUOTED_IDENTIFIER ON
> SET ANSI_PADDING ON
> CREATE TABLE [dbo].[membership](
> [subClusterId]VARCHAR(256) COLLATE
> Latin1_General_100_BIN2 NOT NULL,
> [amRMServiceAddress] VARCHAR(256) NOT NULL,
> [clientRMServiceAddress] VARCHAR(256) NOT NULL,
> [rmAdminServiceAddress] VARCHAR(256) NOT NULL,
> [rmWebServiceAddress] VARCHAR(256) NOT NULL,
> [lastHeartBeat] DATETIME2 NOT NULL,
> [state] VARCHAR(32) NOT NULL,
> [lastStartTime] BIGINT NOT NULL,
> [capability] VARCHAR(6000) NOT NULL,
> CONSTRAINT [pk_subClusterId] PRIMARY KEY
> (
> [subClusterId]
> ), -- missing comma
> CONSTRAINT [uc_lastStartTime] UNIQUE
> (
> [lastStartTime]
> )
> )
> SET ANSI_PADDING OFF
> PRINT 'Table membership created.'
> END
> ELSE
> PRINT 'Table membership exists, no operation required...'
> GO
> GO {code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[
https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602236#comment-17602236
]
ASF GitHub Bot commented on YARN-11286:
---
slfan1989 commented on code in PR #4824:
URL: https://github.com/apache/hadoop/pull/4824#discussion_r966873332
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java:
##
@@ -189,10 +189,24 @@ protected void serviceInit(Configuration conf) throws
Exception{
ThreadFactory threadFactory = new ThreadFactoryBuilder()
.setNameFormat("PrintEventDetailsThread #%d")
.build();
+
Review Comment:
Thank you very much for helping to review the code, I will modify the code.
> Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter
> configurable
> -
>
> Key: YARN-11286
> URL: https://issues.apache.org/jira/browse/YARN-11286
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> AsyncDispatcher#printEventDetailsExecutor thread pool parameters are
> hard-coded, extract this part of hard-coded configuration parameters to the
> configuration file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602211#comment-17602211
]
ASF GitHub Bot commented on YARN-11296:
---
hadoop-yetus commented on PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#issuecomment-1241686561
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 52s | | trunk passed |
| +1 :green_heart: | compile | 10m 57s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 9m 38s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 12s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 1s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 30s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 16m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 24s | | the patch passed |
| +1 :green_heart: | compile | 9m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| -1 :x: | javac | 9m 57s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
with JDK Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 generated 2 new + 740
unchanged - 0 fixed = 742 total (was 740) |
| +1 :green_heart: | compile | 9m 6s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 9m 6s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 generated 2 new
+ 653 unchanged - 0 fixed = 655 total (was 653) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 53s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/5/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 26 new + 0 unchanged -
1 fixed = 26 total (was 1) |
| +1 :green_heart: | mvnsite | 6m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 5s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 15s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 13m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 238m 25s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 3m 41s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 33s | | The patch does not
generate ASF License warnings. |
| |
[jira] [Commented] (YARN-11274) Improve Nodemanager#NodeStatusUpdaterImpl Log
[
https://issues.apache.org/jira/browse/YARN-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602175#comment-17602175
]
ASF GitHub Bot commented on YARN-11274:
---
ayushtkn commented on code in PR #4783:
URL: https://github.com/apache/hadoop/pull/4783#discussion_r966720430
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -1287,10 +1280,8 @@ public void verifyRMHeartbeatResponseForNodeLabels(
} else {
// case where updated labels from NodeLabelsProvider is sent to RM
and
// RM rejected the labels
- LOG.error(
- "NM node labels {" + StringUtils.join(",", getPreviousValue())
- + "} were not accepted by RM and message from RM : "
- + response.getDiagnosticsMessage());
+ LOG.error("NM node labels {{}} were not accepted by RM and message
from RM : {}.",
+ StringUtils.join(",", getPreviousValue()),
response.getDiagnosticsMessage());
Review Comment:
same as above for ``{{}}``
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java:
##
@@ -1136,9 +1130,8 @@ public void verifyRMHeartbeatResponseForNodeAttributes(
} else {
// case where updated node attributes from NodeAttributesProvider
// is sent to RM and RM rejected the attributes
- LOG.error("NM node attributes {" + getPreviousValue()
- + "} were not accepted by RM and message from RM : " + response
- .getDiagnosticsMessage());
+ LOG.error("NM node attributes {{}} were not accepted by RM and
message from RM : {}." ,
+ getPreviousValue(), response.getDiagnosticsMessage() );
Review Comment:
I am not sure about this ``{{}}``, give a check if it works they way you
want, in general it is very confusing can explore using `[{}]` as well
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdater.java:
##
@@ -625,7 +625,7 @@ public RegisterNodeManagerResponse registerNodeManager(
@Override
public NodeHeartbeatResponse nodeHeartbeat(NodeHeartbeatRequest request)
throws YarnException, IOException {
- LOG.info("Got heartBeatId: [" + heartBeatID +"]");
+ LOG.info("Got heartBeatId: [{}}]", heartBeatID);
Review Comment:
there are two `}}`, is that intentional?
> Improve Nodemanager#NodeStatusUpdaterImpl Log
> -
>
> Key: YARN-11274
> URL: https://issues.apache.org/jira/browse/YARN-11274
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> 1.When RM performs master-slave switching, NM will re-register, report the
> list of running apps, and print the number of running apps
> 2.registerWithRM#containerReports is enough to print the number of prints,
> because the above calling process has already printed the container list, so
> there is no need to repeat printing
> 3.Use placeholders to display logs to avoid string concatenation
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11286) Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter configurable
[
https://issues.apache.org/jira/browse/YARN-11286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602169#comment-17602169
]
ASF GitHub Bot commented on YARN-11286:
---
ayushtkn commented on code in PR #4824:
URL: https://github.com/apache/hadoop/pull/4824#discussion_r966710909
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/event/AsyncDispatcher.java:
##
@@ -189,10 +189,24 @@ protected void serviceInit(Configuration conf) throws
Exception{
ThreadFactory threadFactory = new ThreadFactoryBuilder()
.setNameFormat("PrintEventDetailsThread #%d")
.build();
+
Review Comment:
nit: avoid this
> Make AsyncDispatcher#printEventDetailsExecutor thread pool parameter
> configurable
> -
>
> Key: YARN-11286
> URL: https://issues.apache.org/jira/browse/YARN-11286
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
>
> AsyncDispatcher#printEventDetailsExecutor thread pool parameters are
> hard-coded, extract this part of hard-coded configuration parameters to the
> configuration file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-11296) Fix SQLFederationStateStore#Sql script bug
[
https://issues.apache.org/jira/browse/YARN-11296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17602158#comment-17602158
]
ASF GitHub Bot commented on YARN-11296:
---
hadoop-yetus commented on PR #4858:
URL: https://github.com/apache/hadoop/pull/4858#issuecomment-1241578400
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 56s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 48s | | trunk passed |
| +1 :green_heart: | compile | 9m 49s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 50s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 10s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 8s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 33s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 13m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 15s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 5m 25s | | the patch passed |
| +1 :green_heart: | compile | 10m 12s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 10m 12s | | the patch passed |
| +1 :green_heart: | compile | 9m 12s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 9m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 48s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt)
| hadoop-yarn-project/hadoop-yarn: The patch generated 21 new + 0 unchanged -
1 fixed = 21 total (was 1) |
| +1 :green_heart: | mvnsite | 5m 49s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 59s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 22s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 14m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 17s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 239m 56s | | hadoop-yarn in the patch
passed. |
| +1 :green_heart: | unit | 3m 35s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 437m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4858/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4858 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient xmllint spotbugs checkstyle |
| uname | Linux 827eb3b9e857 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2c1b4a541f37f3c6286643504c60863502dd7fb0 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
