Re: [PR] HADOOP-18889. use getDuration() on http connection settings [hadoop]
virajjasani commented on code in PR #6180:
URL: https://github.com/apache/hadoop/pull/6180#discussion_r1363235836
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java:
##
@@ -378,17 +388,39 @@ private static void initSigner(Configuration conf,
*/
private static void initRequestTimeout(Configuration conf,
ClientOverrideConfiguration.Builder clientConfig) {
-long requestTimeoutMillis = conf.getTimeDuration(REQUEST_TIMEOUT,
-DEFAULT_REQUEST_TIMEOUT, TimeUnit.SECONDS, TimeUnit.MILLISECONDS);
+final Duration requestTimeoutMillis = getDuration(conf, REQUEST_TIMEOUT,
+DEFAULT_REQUEST_TIMEOUT, TimeUnit.SECONDS);
-if (requestTimeoutMillis > Integer.MAX_VALUE) {
- LOG.debug("Request timeout is too high({} ms). Setting to {} ms instead",
- requestTimeoutMillis, Integer.MAX_VALUE);
- requestTimeoutMillis = Integer.MAX_VALUE;
+if (requestTimeoutMillis.toMillis() > 0) {
+ clientConfig.apiCallAttemptTimeout(requestTimeoutMillis);
}
+ }
-if(requestTimeoutMillis > 0) {
-
clientConfig.apiCallAttemptTimeout(Duration.ofMillis(requestTimeoutMillis));
+ /**
+ * Get duration. This may be negative; callers must check.
+ * If the config option is greater than {@code Integer.MAX_VALUE}
milliseconds,
+ * it is set to that max.
+ * Logs the value for diagnostics.
+ * @param conf config
+ * @param name option name
+ * @param defVal default value
+ * @param defaultUnit unit of default value
+ * @return duration. may be negative.
+ */
+ private static Duration getDuration(final Configuration conf,
Review Comment:
nit: `getDurationInMillis` or `getDurationWithMillis` might be better?
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/AWSClientConfig.java:
##
@@ -378,17 +388,39 @@ private static void initSigner(Configuration conf,
*/
private static void initRequestTimeout(Configuration conf,
ClientOverrideConfiguration.Builder clientConfig) {
-long requestTimeoutMillis = conf.getTimeDuration(REQUEST_TIMEOUT,
-DEFAULT_REQUEST_TIMEOUT, TimeUnit.SECONDS, TimeUnit.MILLISECONDS);
+final Duration requestTimeoutMillis = getDuration(conf, REQUEST_TIMEOUT,
+DEFAULT_REQUEST_TIMEOUT, TimeUnit.SECONDS);
-if (requestTimeoutMillis > Integer.MAX_VALUE) {
- LOG.debug("Request timeout is too high({} ms). Setting to {} ms instead",
- requestTimeoutMillis, Integer.MAX_VALUE);
- requestTimeoutMillis = Integer.MAX_VALUE;
+if (requestTimeoutMillis.toMillis() > 0) {
+ clientConfig.apiCallAttemptTimeout(requestTimeoutMillis);
}
+ }
-if(requestTimeoutMillis > 0) {
-
clientConfig.apiCallAttemptTimeout(Duration.ofMillis(requestTimeoutMillis));
+ /**
+ * Get duration. This may be negative; callers must check.
+ * If the config option is greater than {@code Integer.MAX_VALUE}
milliseconds,
+ * it is set to that max.
+ * Logs the value for diagnostics.
+ * @param conf config
+ * @param name option name
+ * @param defVal default value
+ * @param defaultUnit unit of default value
+ * @return duration. may be negative.
Review Comment:
nit: `Get duration` -> `Get duration object representing value in
milliseconds`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-16791 Add getEnclosingRoot API to filesystem interface and all i… [hadoop]
hadoop-yetus commented on PR #6198:
URL: https://github.com/apache/hadoop/pull/6198#issuecomment-1767687124
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 27s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 11 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 25s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 13s | | trunk passed |
| +1 :green_heart: | compile | 9m 49s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 8m 56s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 31s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 6m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 24s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 20s | | the patch passed |
| +1 :green_heart: | compile | 9m 1s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 9m 1s | | the patch passed |
| +1 :green_heart: | javac | 9m 1s | | the patch passed |
| +1 :green_heart: | compile | 8m 56s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | cc | 8m 56s | | the patch passed |
| +1 :green_heart: | javac | 8m 56s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 2m 28s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6198/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 23 new + 735 unchanged - 0 fixed = 758 total (was
735) |
| +1 :green_heart: | mvnsite | 3m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 24s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 33s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 7m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 38s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 37s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 15s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 196m 10s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 19m 23s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 401m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6198/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6198 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
cc buflint bufcompat |
| uname | Linux bffdf13de26d 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8f4430928d9023071643b8bd0b15f0cffecbc143 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776492#comment-17776492 ] ASF GitHub Bot commented on HADOOP-18942: - iwasakims commented on PR #6200: URL: https://github.com/apache/hadoop/pull/6200#issuecomment-1767683237 I built it with `-Dhbase.profile=2.0` and ran relevant tests as done in #5988. ``` mvn clean install -DskipTests -DskipShade -Dhbase.profile=2.0 cd hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase mvn clean test -Dhbase.profile=2.0 cd ../hadoop-yarn-server-timelineservice-hbase-tests mvn clean test -Dhbase.profile=2.0 ``` > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18942. Upgrade ZooKeeper to 3.7.2. [hadoop]
iwasakims commented on PR #6200: URL: https://github.com/apache/hadoop/pull/6200#issuecomment-1767683237 I built it with `-Dhbase.profile=2.0` and ran relevant tests as done in #5988. ``` mvn clean install -DskipTests -DskipShade -Dhbase.profile=2.0 cd hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase mvn clean test -Dhbase.profile=2.0 cd ../hadoop-yarn-server-timelineservice-hbase-tests mvn clean test -Dhbase.profile=2.0 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776491#comment-17776491 ] Masatake Iwasaki commented on HADOOP-18942: --- Thanks, [~groot]. I've already worked on it and submitted a PR. > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776490#comment-17776490 ] ASF GitHub Bot commented on HADOOP-18942: - iwasakims commented on PR #6200: URL: https://github.com/apache/hadoop/pull/6200#issuecomment-1767678688 jetty-client must be excluded to address dependency convergence error. ``` [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided paths to dependency are: +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:test +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided and +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:jar:3.7.2:compile +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:provided +-org.eclipse.jetty:jetty-http:jar:9.4.52.v20230823:provided [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided paths to dependency are: +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:test +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided and +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:test +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided and +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:jar:3.7.2:compile +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:provided +-org.eclipse.jetty:jetty-io:jar:9.4.52.v20230823:provided ``` ``` [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile paths to dependency are: +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile and +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:test-jar:tests:3.7.2:test +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:test +-org.eclipse.jetty:jetty-http:jar:9.4.52.v20230823:test [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile paths to dependency are: +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile and +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile and +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:test-jar:tests:3.7.2:test +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:test +-org.eclipse.jetty:jetty-io:jar:9.4.52.v20230823:test ``` > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18942: Labels: pull-request-available (was: ) > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18942. Upgrade ZooKeeper to 3.7.2. [hadoop]
iwasakims commented on PR #6200: URL: https://github.com/apache/hadoop/pull/6200#issuecomment-1767678688 jetty-client must be excluded to address dependency convergence error. ``` [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided paths to dependency are: +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:test +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided and +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:jar:3.7.2:compile +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:provided +-org.eclipse.jetty:jetty-http:jar:9.4.52.v20230823:provided [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided paths to dependency are: +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:test +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:provided +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided and +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:test +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:provided and +-org.apache.hadoop:hadoop-auth:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:jar:3.7.2:compile +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:provided +-org.eclipse.jetty:jetty-io:jar:9.4.52.v20230823:provided ``` ``` [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile paths to dependency are: +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile and +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:test-jar:tests:3.7.2:test +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:test +-org.eclipse.jetty:jetty-http:jar:9.4.52.v20230823:test [WARNING] Dependency convergence error for org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile paths to dependency are: +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-http:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile and +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.eclipse.jetty:jetty-server:jar:9.4.51.v20230217:compile +-org.eclipse.jetty:jetty-io:jar:9.4.51.v20230217:compile and +-org.apache.hadoop:hadoop-common:jar:3.4.0-SNAPSHOT +-org.apache.zookeeper:zookeeper:test-jar:tests:3.7.2:test +-org.eclipse.jetty:jetty-client:jar:9.4.52.v20230823:test +-org.eclipse.jetty:jetty-io:jar:9.4.52.v20230823:test ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] HADOOP-18942. Upgrade ZooKeeper to 3.7.2. [hadoop]
iwasakims opened a new pull request, #6200: URL: https://github.com/apache/hadoop/pull/6200 https://issues.apache.org/jira/browse/HADOOP-18942 While [HADOOP-18613](https://issues.apache.org/jira/browse/HADOOP-18613) is proposing to upgrade ZooKeeper to 3.8, it will bring dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative short-term fix for addressing CVEs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18889. use getDuration() on http connection settings [hadoop]
virajjasani commented on code in PR #6180: URL: https://github.com/apache/hadoop/pull/6180#discussion_r1363203939 ## hadoop-common-project/hadoop-common/src/main/resources/core-default.xml: ## @@ -1596,14 +1596,15 @@ fs.s3a.connection.establish.timeout - 5000 - Socket connection setup timeout in milliseconds. + 5s + Socket connection setup timeout in milliseconds; this will be retried +more than once. fs.s3a.connection.timeout - 20 - Socket connection timeout in milliseconds. + 200s + Socket connection timeout. Review Comment: For both of them, is it valuable to mention in the description that milliseconds is the default unit unless user specifies any other unit explicitly? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18889) S3A: V2 SDK client does not work with third-party store
[ https://issues.apache.org/jira/browse/HADOOP-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776488#comment-17776488 ] ASF GitHub Bot commented on HADOOP-18889: - virajjasani commented on code in PR #6180: URL: https://github.com/apache/hadoop/pull/6180#discussion_r1363203939 ## hadoop-common-project/hadoop-common/src/main/resources/core-default.xml: ## @@ -1596,14 +1596,15 @@ fs.s3a.connection.establish.timeout - 5000 - Socket connection setup timeout in milliseconds. + 5s + Socket connection setup timeout in milliseconds; this will be retried +more than once. fs.s3a.connection.timeout - 20 - Socket connection timeout in milliseconds. + 200s + Socket connection timeout. Review Comment: For both of them, is it valuable to mention in the description that milliseconds is the default unit unless user specifies any other unit explicitly? > S3A: V2 SDK client does not work with third-party store > --- > > Key: HADOOP-18889 > URL: https://issues.apache.org/jira/browse/HADOOP-18889 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Critical > Labels: pull-request-available > Fix For: 3.4.0 > > > testing against an external store without specifying region now blows up > because the region is queried off eu-west-1. > What are we do to here? require the region setting *which wasn't needed > before? what even region do we provide for third party stores? -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy [hadoop]
tomscut commented on PR #6174: URL: https://github.com/apache/hadoop/pull/6174#issuecomment-1767646253 Thanks @GuoPhilipse for your contribution! Thanks @Hexiaoqiao and @zhangshuyan0 for your review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17220. Fix same available space policy in AvailableSpaceVolumeChoosingPolicy [hadoop]
tomscut merged PR #6174: URL: https://github.com/apache/hadoop/pull/6174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17679) Upgrade Protobuf from 3.7.1 to 3.17.3
[
https://issues.apache.org/jira/browse/HADOOP-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776477#comment-17776477
]
Colm O hEigeartaigh commented on HADOOP-17679:
--
Note we have many CVEs in protobuf 3.7.1:
{code:java}
protobuf-java 3.7.1 java-archive
CVE-2021-22569 Medium
protobuf-java 3.7.1 java-archive
CVE-2021-22570 Medium
protobuf-java 3.7.1 java-archive
CVE-2022-3171 High
protobuf-java 3.7.1 3.15.0 java-archive
GHSA-77rm-9x9h-xj3g High
protobuf-java 3.7.1 3.16.1 java-archive
GHSA-wrvw-hg22-4m67 High
protobuf-java 3.7.1 3.16.3 java-archive
GHSA-4gg5-vx3j-xwc7 High
protobuf-java 3.7.1 3.16.3 java-archive
GHSA-g5ww-5jh7-63cx High
protobuf-java 3.7.1 3.16.3 java-archive
GHSA-h4h5-3hr4-j3g2 Medium {code}
> Upgrade Protobuf from 3.7.1 to 3.17.3
> -
>
> Key: HADOOP-17679
> URL: https://issues.apache.org/jira/browse/HADOOP-17679
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.0
>Reporter: Igor Dvorzhak
>Assignee: Igor Dvorzhak
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Protobuf 3.17.0 is the latest Protobuf release now.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] CVE-2023-2976 [hadoop-thirdparty]
coheigea commented on PR #25: URL: https://github.com/apache/hadoop-thirdparty/pull/25#issuecomment-1767618510 Note we already have a JIRA and PR https://github.com/apache/hadoop-thirdparty/pull/23 for this CVE -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[
https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776475#comment-17776475
]
ASF GitHub Bot commented on HADOOP-18931:
-
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767612330
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 12s | | trunk passed |
| +1 :green_heart: | compile | 18m 4s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 24s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 18s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| +1 :green_heart: | compile | 17m 25s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 25s | | the patch passed |
| +1 :green_heart: | compile | 16m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 38s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 226m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5b030286faee 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 348b63a32cae7ed686a6f9910379d43301075844 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/4/testReport/ |
| Max. process+thread count | 3072 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[
https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776476#comment-17776476
]
ASF GitHub Bot commented on HADOOP-18931:
-
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767612709
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 26s | | trunk passed |
| +1 :green_heart: | compile | 18m 11s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 18s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 25s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| +1 :green_heart: | compile | 18m 4s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 18m 4s | | the patch passed |
| +1 :green_heart: | compile | 17m 34s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 37m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 42s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 233m 20s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a6a8ce21920b 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 348b63a32cae7ed686a6f9910379d43301075844 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/3/testReport/ |
| Max. process+thread count | 2318 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0
Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767612709
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 45m 26s | | trunk passed |
| +1 :green_heart: | compile | 18m 11s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 18s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 25s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| +1 :green_heart: | compile | 18m 4s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 18m 4s | | the patch passed |
| +1 :green_heart: | compile | 17m 34s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 37m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 42s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 233m 20s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a6a8ce21920b 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 348b63a32cae7ed686a6f9910379d43301075844 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/3/testReport/ |
| Max. process+thread count | 2318 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767612330
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 12s | | trunk passed |
| +1 :green_heart: | compile | 18m 4s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 24s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 18s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| +1 :green_heart: | compile | 17m 25s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 25s | | the patch passed |
| +1 :green_heart: | compile | 16m 53s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 53s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 17s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 22s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 38s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 3s | | The patch does not
generate ASF License warnings. |
| | | 226m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5b030286faee 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 348b63a32cae7ed686a6f9910379d43301075844 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/4/testReport/ |
| Max. process+thread count | 3072 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
[PR] YARN-11593. [Federation] Improve command line help information. [hadoop]
slfan1989 opened a new pull request, #6199: URL: https://github.com/apache/hadoop/pull/6199 ### Description of PR JIRA: YARN-11593. [Federation] Improve command line help information. ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-18867:
--
Fix Version/s: 3.3.7
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.7
>
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
[ https://issues.apache.org/jira/browse/HADOOP-18942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776447#comment-17776447 ] Ashutosh Gupta commented on HADOOP-18942: - [~iwasakims] let me know if you not working on it. I can take this up. Thanks. > Upgrade ZooKeeper to 3.7.2 > -- > > Key: HADOOP-18942 > URL: https://issues.apache.org/jira/browse/HADOOP-18942 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring > dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative > short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki resolved HADOOP-18867.
---
Fix Version/s: 3.4.0
Hadoop Flags: Reviewed
Resolution: Fixed
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776445#comment-17776445
]
ASF GitHub Bot commented on HADOOP-18867:
-
iwasakims commented on PR #5988:
URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1767457496
Thanks, @ayushtkn and @pjfanning. I filed HADOOP-18942 and merging this.
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]
iwasakims merged PR #5988: URL: https://github.com/apache/hadoop/pull/5988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776446#comment-17776446
]
ASF GitHub Bot commented on HADOOP-18867:
-
iwasakims merged PR #5988:
URL: https://github.com/apache/hadoop/pull/5988
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]
iwasakims commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1767457496 Thanks, @ayushtkn and @pjfanning. I filed HADOOP-18942 and merging this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18942) Upgrade ZooKeeper to 3.7.2
Masatake Iwasaki created HADOOP-18942: - Summary: Upgrade ZooKeeper to 3.7.2 Key: HADOOP-18942 URL: https://issues.apache.org/jira/browse/HADOOP-18942 Project: Hadoop Common Issue Type: Improvement Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki While HADOOP-18613 is proposing to upgrade ZooKeeper to 3.8, it will bring dependency conflicts. Upgrading to ZooKeeper 3.7 could be alternative short-term fix for addressing CVEs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18918) ITestS3GuardTool fails if SSE/DSSE encryption is used
[
https://issues.apache.org/jira/browse/HADOOP-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776436#comment-17776436
]
ASF GitHub Bot commented on HADOOP-18918:
-
virajjasani commented on PR #6165:
URL: https://github.com/apache/hadoop/pull/6165#issuecomment-1767407129
@ahmarsuhail @mehakmeet @mukund-thakur could you please review this PR?
> ITestS3GuardTool fails if SSE/DSSE encryption is used
> -
>
> Key: HADOOP-18918
> URL: https://issues.apache.org/jira/browse/HADOOP-18918
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 3.3.6
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Labels: pull-request-available
>
> {code:java}
> [ERROR] Tests run: 15, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 25.989 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool
> [ERROR]
> testLandsatBucketRequireUnencrypted(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool)
> Time elapsed: 0.807 s <<< ERROR!
> 46: Bucket s3a://landsat-pds: required encryption is none but actual
> encryption is DSSE-KMS
> at
> org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.exitException(S3GuardTool.java:915)
> at
> org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.badState(S3GuardTool.java:881)
> at
> org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$BucketInfo.run(S3GuardTool.java:511)
> at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:283)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82)
> at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:963)
> at
> org.apache.hadoop.fs.s3a.s3guard.S3GuardToolTestHelper.runS3GuardCommand(S3GuardToolTestHelper.java:147)
> at
> org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.run(AbstractS3GuardToolTestBase.java:114)
> at
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool.testLandsatBucketRequireUnencrypted(ITestS3GuardTool.java:74)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:750)
> {code}
> Since landsat requires none encryption, the test should be skipped for any
> encryption algorithm.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18918. ITestS3GuardTool fails if SSE/DSSE encryption is used [hadoop]
virajjasani commented on PR #6165: URL: https://github.com/apache/hadoop/pull/6165#issuecomment-1767407129 @ahmarsuhail @mehakmeet @mukund-thakur could you please review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] Add getEnclosingRoot API to FileSystem interface and all implementations [hadoop]
hadoop-yetus commented on PR #6196:
URL: https://github.com/apache/hadoop/pull/6196#issuecomment-1767405119
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 5m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 11 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 22s | | branch-3.3 passed |
| +1 :green_heart: | compile | 13m 21s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 2m 2s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 5m 4s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 4m 38s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 8m 58s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 26m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 16s | | the patch passed |
| +1 :green_heart: | compile | 11m 26s | | the patch passed |
| +1 :green_heart: | cc | 11m 26s | | the patch passed |
| +1 :green_heart: | javac | 11m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 46s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6196/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 296 new + 756 unchanged - 0 fixed = 1052 total
(was 756) |
| +1 :green_heart: | mvnsite | 4m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 22s | | the patch passed |
| -1 :x: | spotbugs | 1m 37s |
[/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6196/1/artifact/out/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) |
| +1 :green_heart: | shadedclient | 24m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 16m 0s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 7s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 179m 56s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6196/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 15m 22s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | unit | 77m 56s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| -1 :x: | asflicense | 1m 1s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6196/1/artifact/out/results-asflicense.txt)
| The patch generated 2 ASF License warnings. |
| | | 465m 36s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
|
| | Comparison of String objects using == or != in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule.equals(Object)
At Rule.java:== or != in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule.equals(Object)
At Rule.java:[line 211] |
| | Comparison of String objects using == or != in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule.equals(Object)
At Rule.java:== or != in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.placement.schema.Rule.equals(Object)
At Rule.java:[line 211] |
| | Comparison of String objects using == or !=
Re: [PR] Branch 3.2 - hadoop per bucket endpoint configuration is ignored [hadoop]
ayushtkn closed pull request #4861: Branch 3.2 - hadoop per bucket endpoint configuration is ignored URL: https://github.com/apache/hadoop/pull/4861 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17228.Add documentation related to BlockManager. [hadoop]
ayushtkn merged PR #6195: URL: https://github.com/apache/hadoop/pull/6195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[
https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776426#comment-17776426
]
ASF GitHub Bot commented on HADOOP-18931:
-
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767343446
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 8s | | trunk passed |
| +1 :green_heart: | compile | 19m 2s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 54s | | the patch passed |
| +1 :green_heart: | compile | 17m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 15s | | the patch passed |
| +1 :green_heart: | compile | 17m 36s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 15s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 1 new + 66
unchanged - 0 fixed = 67 total (was 66) |
| +1 :green_heart: | mvnsite | 1m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 36m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 35s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 230m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8b894d7807e8 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc12e290af66acf1df52083e58aec66e0aa4ddbb |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/testReport/ |
| Max. process+thread count | 1276 (vs. ulimit of 5500) |
| modules | C:
Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767343446
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 8s | | trunk passed |
| +1 :green_heart: | compile | 19m 2s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 17m 25s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 54s | | the patch passed |
| +1 :green_heart: | compile | 17m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 15s | | the patch passed |
| +1 :green_heart: | compile | 17m 36s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 17m 36s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 15s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 1 new + 66
unchanged - 0 fixed = 67 total (was 66) |
| +1 :green_heart: | mvnsite | 1m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 36m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 35s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 230m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 8b894d7807e8 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc12e290af66acf1df52083e58aec66e0aa4ddbb |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/testReport/ |
| Max. process+thread count | 1276 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered
[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[
https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776425#comment-17776425
]
ASF GitHub Bot commented on HADOOP-18931:
-
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767338765
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 13m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 49s | | trunk passed |
| +1 :green_heart: | compile | 17m 55s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 27s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| +1 :green_heart: | compile | 18m 1s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 18m 1s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 22s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 1 new + 66
unchanged - 0 fixed = 67 total (was 66) |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 10s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 37s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 7s | | The patch does not
generate ASF License warnings. |
| | | 239m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 6632c61d4835 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc12e290af66acf1df52083e58aec66e0aa4ddbb |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/testReport/ |
| Max. process+thread count | 2152 (vs. ulimit of 5500) |
| modules | C:
Re: [PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]
hadoop-yetus commented on PR #6197:
URL: https://github.com/apache/hadoop/pull/6197#issuecomment-1767338765
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 13m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 49s | | trunk passed |
| +1 :green_heart: | compile | 17m 55s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 27s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 36m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| +1 :green_heart: | compile | 18m 1s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 18m 1s | | the patch passed |
| +1 :green_heart: | compile | 16m 23s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 23s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 22s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 1 new + 66
unchanged - 0 fixed = 67 total (was 66) |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 10s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 53s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 37s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 7s | | The patch does not
generate ASF License warnings. |
| | | 239m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6197 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 6632c61d4835 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc12e290af66acf1df52083e58aec66e0aa4ddbb |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/testReport/ |
| Max. process+thread count | 2152 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6197/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered
[PR] HDFS-16791 Add getEnclosingRoot API to filesystem interface and all i… [hadoop]
mccormickt12 opened a new pull request, #6198: URL: https://github.com/apache/hadoop/pull/6198 …mplementations ### Description of PR Add getEnclosingRoot API to filesystem interface. This API is needed in ViewFileSystem and RBF setups to get around the rename constraint. Renames (like custom trash and staging directories) should be done using this directory to ensure that each mount point is a self contained entity ### How was this patch tested? unit tests ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[
https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776423#comment-17776423
]
ASF GitHub Bot commented on HADOOP-18867:
-
pjfanning commented on PR #5988:
URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1767324560
@ayushtkn I agree this can be merged but ideally, a 2nd PR should be done
soon to bring zookeeper up to 3..7.2 (or later).
> Upgrade ZooKeeper to 3.6.4
> --
>
> Key: HADOOP-18867
> URL: https://issues.apache.org/jira/browse/HADOOP-18867
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
>Affects Versions: 3.3.6
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Labels: pull-request-available
>
> While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of
> the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later.
> Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}.
> {noformat}
> $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests
> clean install
> Dependency convergence error for
> org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency
> are:
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test
> +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.13.0:compile
> and
> +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT
> +-org.apache.hbase:hbase-common:jar:2.2.4:compile
> +-org.apache.yetus:audience-annotations:jar:0.5.0:compile
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]
pjfanning commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1767324560 @ayushtkn I agree this can be merged but ideally, a 2nd PR should be done soon to bring zookeeper up to 3..7.2 (or later). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket
[
https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776398#comment-17776398
]
ASF GitHub Bot commented on HADOOP-18930:
-
hadoop-yetus commented on PR #6168:
URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1767214969
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 28s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 22s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 4 new + 3 unchanged - 0 fixed
= 7 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 3m 0s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 129m 30s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.s3a.impl.TestErrorTranslation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6168 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1c98e744b912 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 18b4628eab1c6cccdc53b5c59448cd47908bddd5 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
Re: [PR] HADOOP-18930. Make fs.s3a.create.performance a bucket-wide setting [hadoop]
hadoop-yetus commented on PR #6168:
URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1767214969
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 28s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 33m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 22s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 4 new + 3 unchanged - 0 fixed
= 7 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 3m 0s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 129m 30s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.s3a.impl.TestErrorTranslation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6168 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 1c98e744b912 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 18b4628eab1c6cccdc53b5c59448cd47908bddd5 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/3/testReport/ |
| Max. process+thread count | 699 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
[jira] [Commented] (HADOOP-18930) make fs.s3a.create.performance an option you can set for the entire bucket
[
https://issues.apache.org/jira/browse/HADOOP-18930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776395#comment-17776395
]
ASF GitHub Bot commented on HADOOP-18930:
-
hadoop-yetus commented on PR #6168:
URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1767191203
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 3 unchanged - 0 fixed
= 4 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 51s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 134m 6s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.s3a.impl.TestErrorTranslation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6168 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 387fc5cf7fb2 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f234e28951e43e8bf6cdfbaae898e20b4611d4ce |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
Re: [PR] HADOOP-18930. Make fs.s3a.create.performance a bucket-wide setting [hadoop]
hadoop-yetus commented on PR #6168:
URL: https://github.com/apache/hadoop/pull/6168#issuecomment-1767191203
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 33s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 12s | | trunk passed |
| +1 :green_heart: | shadedclient | 34m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 3 unchanged - 0 fixed
= 4 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 35m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 51s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 134m 6s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.s3a.impl.TestErrorTranslation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6168 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 387fc5cf7fb2 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f234e28951e43e8bf6cdfbaae898e20b4611d4ce |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6168/2/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 5500) |
| modules | C:
[jira] [Commented] (HADOOP-18925) S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable CopyFromLocalOperation
[
https://issues.apache.org/jira/browse/HADOOP-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776390#comment-17776390
]
ASF GitHub Bot commented on HADOOP-18925:
-
hadoop-yetus commented on PR #6163:
URL: https://github.com/apache/hadoop/pull/6163#issuecomment-1767175080
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 39s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 2 new + 3 unchanged - 0 fixed
= 5 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 47s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 126m 12s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.s3a.impl.TestErrorTranslation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6163 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 2bf462ce5a4e 5.15.0-86-generic #96-Ubuntu SMP Wed Sep 20
08:23:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6e7c97550af0c9f003faa8f1d5a10e550ab00103 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Re: [PR] HADOOP-18925. S3A: option "fs.s3a.optimized.copy.from.local.enabled" to control CopyFromLocalOperation [hadoop]
hadoop-yetus commented on PR #6163:
URL: https://github.com/apache/hadoop/pull/6163#issuecomment-1767175080
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 39s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 32m 29s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 2 new + 3 unchanged - 0 fixed
= 5 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 5s | | the patch passed |
| +1 :green_heart: | shadedclient | 32m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 2m 47s |
[/patch-unit-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/artifact/out/patch-unit-hadoop-tools_hadoop-aws.txt)
| hadoop-aws in the patch passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 126m 12s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.fs.s3a.impl.TestErrorTranslation |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6163 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 2bf462ce5a4e 5.15.0-86-generic #96-Ubuntu SMP Wed Sep 20
08:23:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6e7c97550af0c9f003faa8f1d5a10e550ab00103 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6163/3/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U:
Re: [PR] CVE-2023-2976 [hadoop-thirdparty]
goiri commented on PR #25: URL: https://github.com/apache/hadoop-thirdparty/pull/25#issuecomment-1767154645 Approved but waiting for the CI run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18843. Guava version 32.0.1 bump to fix CVE-2023-2976 [hadoop-thirdparty]
Grimoren commented on PR #23: URL: https://github.com/apache/hadoop-thirdparty/pull/23#issuecomment-1767114027 > I've hit the approve and run button to see what the ci builds say. if things are good I'll merge Any chance you could merge this soon. Got a CVE reporting against hadoop-shaded-guava. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] CVE-2023-2976 [hadoop-thirdparty]
Grimoren opened a new pull request, #25: URL: https://github.com/apache/hadoop-thirdparty/pull/25 Published Vulnerabilities CVE-2023-2976 Use of Java's default temporary directory for file creation in `FileBackedOutputStream` in Google Guava versions 1.0 to 31.1 on Unix systems and Android Ice Cream Sandwich allows other users and apps on the machine with access to the default Java temporary directory to be able to access the files created by the class. Even though the security vulnerability is fixed in version 32.0.0, we recommend using version 32.0.1 as version 32.0.0 breaks some functionality under Windows. CWE-552 Files or Directories Accessible to External Parties CVSSv3: Base Score: HIGH (7.1) Vector: CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:N References: MISC - https://github.com/google/guava/issues/2575 MISC - https://security.netapp.com/advisory/ntap-20230818-0008/ OSSINDEX - [CVE-2023-2976] CWE-552: Files or Directories Accessible to External Parties OSSIndex - http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2023-2976 OSSIndex - https://github.com/google/guava/issues/2575 OSSIndex - https://github.com/google/guava/releases/tag/v32.0.0 Vulnerable Software & Versions: cpe:2.3:a:google:guava:*:*:*:*:*:*:*:* versions up to (excluding) 32.0.0 CVE-2020-8908 suppress A temp directory creation vulnerability exists in all versions of Guava, allowing an attacker with access to the machine to potentially access data in a temporary directory created by the Guava API com.google.common.io.Files.createTempDir(). By default, on unix-like systems, the created directory is world-readable (readable by an attacker with access to the system). The method in question has been marked @Deprecated in versions 30.0 and later and should not be used. For Android developers, we recommend choosing a temporary directory API provided by Android, such as context.getCacheDir(). For other Java developers, we recommend migrating to the Java 7 API java.nio.file.Files.createTempDirectory() which explicitly configures permissions of 700, or configuring the Java runtime's java.io.tmpdir system property to point to a location whose permissions are appropriately configured. CWE-732 Incorrect Permission Assignment for Critical Resource CVSSv2: Base Score: LOW (2.1) Vector: /AV:L/AC:L/Au:N/C:P/I:N/A:N CVSSv3: Base Score: LOW (3.3) Vector: CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:L/I:N/A:N References: CONFIRM - https://github.com/google/guava/commit/fec0dbc4634006a6162cfd4d0d09c962073ddf40 CONFIRM - https://github.com/google/guava/issues/4011 CONFIRM - https://security.netapp.com/advisory/ntap-20220210-0003/ MISC - https://lists.apache.org/thread.html/r007add131977f4f576c232b25e024249a3d16f66aad14a4b52819d21%40%3Ccommon-issues.hadoop.apache.org%3E MISC - https://lists.apache.org/thread.html/r07ed3e4417ad043a27bee7bb33322e9bfc7d7e6d1719b8e3dfd95c14%40%3Cdev.drill.apache.org%3E MISC - https://lists.apache.org/thread.html/r161b87f8037bbaff400194a63cd2016c9a69f5949f06dcc79beeab54%40%3Cdev.drill.apache.org%3E MISC - https://lists.apache.org/thread.html/r215b3d50f56faeb2f9383505f3e62faa9f549bb23e8a9848b78a968e%40%3Ccommits.ws.apache.org%3E MISC - https://lists.apache.org/thread.html/r294be9d31c0312d2c0837087204b5d4bf49d0552890e6eec716fa6a6%40%3Cyarn-issues.hadoop.apache.org%3E MISC - https://lists.apache.org/thread.html/r2fe45d96eea8434b91592ca08109118f6308d60f6d0e21d52438cfb4%40%3Cd ev.drill.apache.org%3E MISC - https://lists.apache.org/thread.html/r3c3b33ee5bef0c67391d27a97cbfd89d44f328cf072b601b58d4e748%40%3Ccommits.pulsar.apache.org%3E MISC - https://lists.apache.org/thread.html/r3dd8881de891598d67e9840dd7c2ef1d08abbb49e9690c7ae1bc%40%3Cissues.geode.apache.org%3E MISC - https://lists.apache.org/thread.html/r4776f62dfae4a0006658542f43034a7fc199350e35a66d4e18164ee6%40%3Ccommits.cxf.apache.org%3E MISC - https://lists.apache.org/thread.html/r49549a8322f62cd3acfa4490d25bfba0be04f3f9ff4d14fe36199d27%40%3Cyarn-dev.hadoop.apache.org%3E MISC - https://lists.apache.org/thread.html/r58a8775205ab1839dba43054b09a9ab3b25b423a4170b2413c4067ac%40%3Ccommon-issues.hadoop.apache.org%3E MISC - https://lists.apache.org/thread.html/r5b3d93dfdfb7708e796e8762ab40edbde8ff8add48aba53e5ea26f44%40%3Cissues.geode.apache.org%3E MISC - https://lists.apache.org/thread.html/r5d61b98ceb7bba939a651de5900dbd67be3817db6bfcc41c6e04e199%40%3Cyarn-issues.hadoop.apache.org%3E MISC - https://lis ts.apache.org/thread.html/r6874dfe26eefc41b7c9a5e4a0487846fc4accf8c78ff948b24a1104a%40%3Cdev.drill.apache.org%3E MISC - https://lists.apache.org/thread.html/r68d86f4b06c808204f62bcb254fcb5b0432528ee8d37a07ef4bc8222%40%3Ccommits.ws.apache.org%3E MISC - https://lists.apache.org/thread.html/r79e47ed555bdb1180e528420a7a2bb898541367a29a3bc6bbf0baf2c%40%3Cissues.hive.apache.org%3E MISC -
[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC
[
https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776371#comment-17776371
]
ASF GitHub Bot commented on HADOOP-18919:
-
hadoop-yetus commented on PR #6194:
URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1767075251
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 47s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 39m 12s | | trunk passed |
| +1 :green_heart: | compile | 18m 29s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 50s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 33s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 8m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 54s | | the patch passed |
| +1 :green_heart: | compile | 17m 39s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 39s | | the patch passed |
| +1 :green_heart: | compile | 16m 44s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 35s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 21 new + 294 unchanged - 1 fixed = 315 total (was
295) |
| +1 :green_heart: | mvnsite | 4m 15s | | the patch passed |
| -1 :x: | javadoc | 1m 5s |
[/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 generated 4 new + 0
unchanged - 0 fixed = 4 total (was 0) |
| +1 :green_heart: | javadoc | 3m 30s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 8m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 8s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 239m 39s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 103m 8s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 1m 6s | | The patch does not
generate ASF License warnings. |
| | | 625m 6s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/Dockerfile
|
|
Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]
hadoop-yetus commented on PR #6194:
URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1767075251
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 47s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 39m 12s | | trunk passed |
| +1 :green_heart: | compile | 18m 29s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 16m 50s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 40s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 18s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 3m 33s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 8m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 54s | | the patch passed |
| +1 :green_heart: | compile | 17m 39s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 17m 39s | | the patch passed |
| +1 :green_heart: | compile | 16m 44s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 16m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 35s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 21 new + 294 unchanged - 1 fixed = 315 total (was
295) |
| +1 :green_heart: | mvnsite | 4m 15s | | the patch passed |
| -1 :x: | javadoc | 1m 5s |
[/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 generated 4 new + 0
unchanged - 0 fixed = 4 total (was 0) |
| +1 :green_heart: | javadoc | 3m 30s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 8m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 8s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 239m 39s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 103m 8s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 1m 6s | | The patch does not
generate ASF License warnings. |
| | | 625m 6s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6194/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6194 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 5086b12781f2
[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776359#comment-17776359 ] ASF GitHub Bot commented on HADOOP-18931: - virajjasani opened a new pull request, #6197: URL: https://github.com/apache/hadoop/pull/6197 Jira: HADOOP-18931 > FileSystem.getFileSystemClass() to log at debug the jar the .class came from > > > Key: HADOOP-18931 > URL: https://issues.apache.org/jira/browse/HADOOP-18931 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.6 >Reporter: Steve Loughran >Priority: Minor > > we want to be able to log the jar the filesystem implementation class, so > that we can identify which version of a module the class came from. > this is to help track down problems where different machines in the cluster > or the .tar.gz bundle is out of date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18931: Labels: pull-request-available (was: ) > FileSystem.getFileSystemClass() to log at debug the jar the .class came from > > > Key: HADOOP-18931 > URL: https://issues.apache.org/jira/browse/HADOOP-18931 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.6 >Reporter: Steve Loughran >Priority: Minor > Labels: pull-request-available > > we want to be able to log the jar the filesystem implementation class, so > that we can identify which version of a module the class came from. > this is to help track down problems where different machines in the cluster > or the .tar.gz bundle is out of date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776356#comment-17776356 ] Viraj Jasani edited comment on HADOOP-18931 at 10/17/23 7:16 PM: - sounds good, it makes sense to log for all fs invocation by keeping the log separate from the heavy service load. was (Author: vjasani): sounds good, it makes sense to log for all fs invocation > FileSystem.getFileSystemClass() to log at debug the jar the .class came from > > > Key: HADOOP-18931 > URL: https://issues.apache.org/jira/browse/HADOOP-18931 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.6 >Reporter: Steve Loughran >Priority: Minor > > we want to be able to log the jar the filesystem implementation class, so > that we can identify which version of a module the class came from. > this is to help track down problems where different machines in the cluster > or the .tar.gz bundle is out of date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] HADOOP-18931. FileSystem.getFileSystemClass() to log at debug the jar the .class came from [hadoop]
virajjasani opened a new pull request, #6197: URL: https://github.com/apache/hadoop/pull/6197 Jira: HADOOP-18931 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18931) FileSystem.getFileSystemClass() to log at debug the jar the .class came from
[ https://issues.apache.org/jira/browse/HADOOP-18931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776356#comment-17776356 ] Viraj Jasani commented on HADOOP-18931: --- sounds good, it makes sense to log for all fs invocation > FileSystem.getFileSystemClass() to log at debug the jar the .class came from > > > Key: HADOOP-18931 > URL: https://issues.apache.org/jira/browse/HADOOP-18931 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Affects Versions: 3.3.6 >Reporter: Steve Loughran >Priority: Minor > > we want to be able to log the jar the filesystem implementation class, so > that we can identify which version of a module the class came from. > this is to help track down problems where different machines in the cluster > or the .tar.gz bundle is out of date. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] Add getEnclosingRoot API to FileSystem interface and all implementations [hadoop]
mccormickt12 commented on PR #6196: URL: https://github.com/apache/hadoop/pull/6196#issuecomment-1766980795 cc @simbadzina @goiri (This is related to the [old pr](https://github.com/apache/hadoop/pull/4967/files#diff-a07f38553cfc52416c1a9bc82110837c8d0c078a0f1ef9c62c290d7de5b3e745) I tagged you on yesterday -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776333#comment-17776333 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran commented on PR #6185: URL: https://github.com/apache/hadoop/pull/6185#issuecomment-1766974645 @apurtell can you review & approve the final bit of the protobuf 2.5 work -this cuts it out from the distributions > Make protobuf 2.5 an optional runtime dependency. > - > > Key: HADOOP-18487 > URL: https://issues.apache.org/jira/browse/HADOOP-18487 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ipc >Affects Versions: 3.3.4 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > uses of protobuf 2.5 and RpcEnginej have been deprecated since 3.3.0 in > HADOOP-17046 > while still keeping those files around (for a long time...), how about we > make the protobuf 2.5.0 export off hadoop common and hadoop-hdfs *provided*, > rather than *compile* > that way, if apps want it for their own apis, they have to explicitly ask for > it, but at least our own scans don't break. > i have no idea what will happen to the rest of the stack at this point, it > will be "interesting" to see -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18487. Protobuf 2.5 removal part 2: stop exporting protobuf-2.5" [hadoop]
steveloughran commented on PR #6185: URL: https://github.com/apache/hadoop/pull/6185#issuecomment-1766974645 @apurtell can you review & approve the final bit of the protobuf 2.5 work -this cuts it out from the distributions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18487) Make protobuf 2.5 an optional runtime dependency.
[ https://issues.apache.org/jira/browse/HADOOP-18487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776328#comment-17776328 ] ASF GitHub Bot commented on HADOOP-18487: - steveloughran merged PR #6184: URL: https://github.com/apache/hadoop/pull/6184 > Make protobuf 2.5 an optional runtime dependency. > - > > Key: HADOOP-18487 > URL: https://issues.apache.org/jira/browse/HADOOP-18487 > Project: Hadoop Common > Issue Type: Improvement > Components: build, ipc >Affects Versions: 3.3.4 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > uses of protobuf 2.5 and RpcEnginej have been deprecated since 3.3.0 in > HADOOP-17046 > while still keeping those files around (for a long time...), how about we > make the protobuf 2.5.0 export off hadoop common and hadoop-hdfs *provided*, > rather than *compile* > that way, if apps want it for their own apis, they have to explicitly ask for > it, but at least our own scans don't break. > i have no idea what will happen to the rest of the stack at this point, it > will be "interesting" to see -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18487. Make protobuf 2.5 an optional runtime dependency. (#4996) [hadoop]
steveloughran merged PR #6184: URL: https://github.com/apache/hadoop/pull/6184 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17228.Add documentation related to BlockManager. [hadoop]
hadoop-yetus commented on PR #6195:
URL: https://github.com/apache/hadoop/pull/6195#issuecomment-1766944233
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 54s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 48s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 46s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 56s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 2m 0s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| -1 :x: | javadoc | 0m 37s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/2/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt)
| hadoop-hdfs in the patch failed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04. |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 51s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 197m 22s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 291m 8s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6195 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux c56ad6a20513 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a0dbe541f1ad24cdb582af436705b82ceb4fe727 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/2/testReport/ |
| Max. process+thread count | 3745 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache
[jira] [Commented] (HADOOP-18925) S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable CopyFromLocalOperation
[ https://issues.apache.org/jira/browse/HADOOP-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776319#comment-17776319 ] ASF GitHub Bot commented on HADOOP-18925: - jojochuang commented on PR #6163: URL: https://github.com/apache/hadoop/pull/6163#issuecomment-1766909835 There are conflicts in the PR > S3A: add option "fs.s3a.copy.from.local.enabled" to enable/disable > CopyFromLocalOperation > - > > Key: HADOOP-18925 > URL: https://issues.apache.org/jira/browse/HADOOP-18925 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 >Affects Versions: 3.3.6 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > > reported failure of CopyFromLocalOperation.getFinalPath() during job > submission with s3a declared as cluster fs. > add an emergency option to disable this optimised uploader and revert to the > superclass implementation -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18925. S3A: option "fs.s3a.optimized.copy.from.local.enabled" to control CopyFromLocalOperation [hadoop]
jojochuang commented on PR #6163: URL: https://github.com/apache/hadoop/pull/6163#issuecomment-1766909835 There are conflicts in the PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-9013. [BackPort] [GPG] fix order of steps cleaning Registry entries in ApplicationCleaner. [hadoop]
goiri commented on code in PR #6147:
URL: https://github.com/apache/hadoop/pull/6147#discussion_r1362534634
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java:
##
@@ -4367,7 +4367,7 @@ public static boolean isAclEnabled(Configuration conf) {
* the default value is 0s.
*/
public static final long
DEFAULT_ROUTER_USER_CLIENT_THREAD_POOL_KEEP_ALIVE_TIME =
- TimeUnit.SECONDS.toMillis(0); // 0s
+ TimeUnit.SECONDS.toMillis(30); // 0s
Review Comment:
The comment needs to be updated.
Also not sure what's the agreement on changing default values.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/test/java/org/apache/hadoop/yarn/server/globalpolicygenerator/applicationcleaner/TestDefaultApplicationCleaner.java:
##
@@ -159,7 +163,39 @@ public class TestableDefaultApplicationCleaner
extends DefaultApplicationCleaner {
@Override
public Set getAppsFromRouter() throws YarnRuntimeException {
+ if (appIdToAddConcurrently != null) {
+SubClusterId scId = SubClusterId.newInstance("MySubClusterId");
+try {
+ ApplicationHomeSubCluster appHomeSubCluster =
+ ApplicationHomeSubCluster.newInstance(appIdToAddConcurrently,
scId);
+ AddApplicationHomeSubClusterRequest request =
+
AddApplicationHomeSubClusterRequest.newInstance(appHomeSubCluster);
+ stateStore.addApplicationHomeSubCluster(request);
+} catch (YarnException e) {
+ throw new YarnRuntimeException(e);
+}
+registryClient.writeAMRMTokenForUAM(appIdToAddConcurrently,
scId.toString(),
+new Token<>());
+ }
return routerAppIds;
}
}
+
+ @Test
+ public void testConcurrentNewApp() throws YarnException {
+appIdToAddConcurrently = ApplicationId.newInstance(1, 1);
+
+appCleaner.run();
+
+// The concurrently added app should be still there
+GetApplicationsHomeSubClusterRequest appHomeSubClusterRequest =
+ GetApplicationsHomeSubClusterRequest.newInstance();
+GetApplicationsHomeSubClusterResponse applicationsHomeSubCluster =
+stateStore.getApplicationsHomeSubCluster(appHomeSubClusterRequest);
+int size = applicationsHomeSubCluster.getAppsHomeSubClusters().size();
+Assert.assertEquals(1, size);
Review Comment:
For consistency, either extract both or none.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11592. Add timeout to GPGUtils#invokeRMWebService. [hadoop]
goiri commented on code in PR #6189: URL: https://github.com/apache/hadoop/pull/6189#discussion_r1362528994 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-globalpolicygenerator/src/main/java/org/apache/hadoop/yarn/server/globalpolicygenerator/GPGUtils.java: ## @@ -128,4 +130,16 @@ public static Map createUniformWeights( } return weights; } + + @VisibleForTesting Review Comment: Why visible for testing? We are not testing it, right? It'd be good to have a unit test though. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC
[
https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776298#comment-17776298
]
ASF GitHub Bot commented on HADOOP-18919:
-
jojochuang commented on code in PR #6194:
URL: https://github.com/apache/hadoop/pull/6194#discussion_r1362493253
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java:
##
@@ -786,4 +789,96 @@ public static List
getZKAuthInfos(Configuration conf,
throw e;
}
}
+
+ public static void validateSslConfiguration(TruststoreKeystore
truststoreKeystore) throws ConfigurationException {
+if
(org.apache.commons.lang3.StringUtils.isEmpty(truststoreKeystore.keystoreLocation))
{
Review Comment:
would be nice to import org.apache.commons.lang3.StringUtils in the
beginning and get rid of the prefix in the code. I believe all the old
commons-lang library code is removed from Hadoop now we we only use
commons-lang3.
> Zookeeper SSL/TLS support in HDFS ZKFC
> --
>
> Key: HADOOP-18919
> URL: https://issues.apache.org/jira/browse/HADOOP-18919
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Zita Dombi
>Assignee: Zita Dombi
>Priority: Major
> Labels: pull-request-available
>
> HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled
> in hadoop-common. With those changes we have the necessary parameters, that
> we need to set to enable SSL/TLS in a ZK Client.
> In YARN-11468 the SSL communication can be set in Yarn, now we need to
> similar changes in HDFS to enable it correctly. In HDFS ZK Client is used in
> the Failover Controller. In this improvement we need to create the ZK client
> with the necessary SSL configs if we enable it, which we can track under a
> new HDFS config.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776297#comment-17776297 ] ASF GitHub Bot commented on HADOOP-18919: - jojochuang commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766842223 Unrelated question... in [HADOOP-18709](https://github.com/apache/hadoop/pull/5638/files#top) a new dependency was added (netty-transport-native-epoll) it's a native lib for Netty to run on Linux. I wonder if it could trouble running on other platforms. > Zookeeper SSL/TLS support in HDFS ZKFC > -- > > Key: HADOOP-18919 > URL: https://issues.apache.org/jira/browse/HADOOP-18919 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Zita Dombi >Assignee: Zita Dombi >Priority: Major > Labels: pull-request-available > > HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled > in hadoop-common. With those changes we have the necessary parameters, that > we need to set to enable SSL/TLS in a ZK Client. > In YARN-11468 the SSL communication can be set in Yarn, now we need to > similar changes in HDFS to enable it correctly. In HDFS ZK Client is used in > the Failover Controller. In this improvement we need to create the ZK client > with the necessary SSL configs if we enable it, which we can track under a > new HDFS config. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]
jojochuang commented on code in PR #6194:
URL: https://github.com/apache/hadoop/pull/6194#discussion_r1362493253
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java:
##
@@ -786,4 +789,96 @@ public static List
getZKAuthInfos(Configuration conf,
throw e;
}
}
+
+ public static void validateSslConfiguration(TruststoreKeystore
truststoreKeystore) throws ConfigurationException {
+if
(org.apache.commons.lang3.StringUtils.isEmpty(truststoreKeystore.keystoreLocation))
{
Review Comment:
would be nice to import org.apache.commons.lang3.StringUtils in the
beginning and get rid of the prefix in the code. I believe all the old
commons-lang library code is removed from Hadoop now we we only use
commons-lang3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]
jojochuang commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766842223 Unrelated question... in [HADOOP-18709](https://github.com/apache/hadoop/pull/5638/files#top) a new dependency was added (netty-transport-native-epoll) it's a native lib for Netty to run on Linux. I wonder if it could trouble running on other platforms. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17227. EC: Fix bug in choosing targets when racks is not enough. [hadoop]
hadoop-yetus commented on PR #6192:
URL: https://github.com/apache/hadoop/pull/6192#issuecomment-1766828501
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 56s | | trunk passed |
| +1 :green_heart: | compile | 0m 52s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 55s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 33s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 54s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 198m 31s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 291m 28s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestErasureCodingMultipleRacks |
| | hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6192 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ee50156ecd10 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 226a9b0ed829be81046612d626d19eaacec094bd |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/3/testReport/ |
| Max. process+thread count | 4038 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/3/console |
|
[PR] staging [hadoop]
mccormickt12 opened a new pull request, #6196: URL: https://github.com/apache/hadoop/pull/6196 ### Description of PR Original PR and comments https://github.com/apache/hadoop/pull/4967/files#diff-a07f38553cfc52416c1a9bc82110837c8d0c078a0f1ef9c62c290d7de5b3e745 ### How was this patch tested? Unit tersts ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17228.Add documentation related to BlockManager. [hadoop]
hadoop-yetus commented on PR #6195:
URL: https://github.com/apache/hadoop/pull/6195#issuecomment-1766784532
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 16m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 16s | | trunk passed |
| +1 :green_heart: | compile | 1m 26s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 15s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 38s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 10s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| -1 :x: | javadoc | 0m 54s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/1/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04.txt)
| hadoop-hdfs in the patch failed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04. |
| +1 :green_heart: | javadoc | 1m 27s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 36m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 257m 58s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 414m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6195 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux bc73cdd183af 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19
13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 00080df4c25d5c4c43c2dc31453b680b0f2f00e0 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/1/testReport/ |
| Max. process+thread count | 3224 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6195/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache
Re: [PR] HADOOP-18711. Upgrade nimbus jwt jar due to issues in its embedded shaded json-smart code [hadoop]
pjfanning commented on PR #5639: URL: https://github.com/apache/hadoop/pull/5639#issuecomment-1766712078 @rohit-kb could you rebase this? The issue with the module-info classes has recently been resolved in branch-3.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18938) S3A region logic to handle vpce and non standard endpoints
[ https://issues.apache.org/jira/browse/HADOOP-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776243#comment-17776243 ] Steve Loughran commented on HADOOP-18938: - fail fast may not be the right strategy as the region could be set in AWS_REGION or the system property. So leave to the SDK > S3A region logic to handle vpce and non standard endpoints > --- > > Key: HADOOP-18938 > URL: https://issues.apache.org/jira/browse/HADOOP-18938 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Priority: Major > > For non standard endpoints such as VPCE the region parsing added in > HADOOP-18908 doesn't work. This is expected as that logic is only meant to be > used for standard endpoints. > If you are using a non-standard endpoint, check if a region is also provided, > else fail fast. > Also update documentation to explain to region and endpoint behaviour with > SDK V2. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18908) Improve s3a region handling, including determining from endpoint
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18908. - Fix Version/s: 3.4.0 Resolution: Fixed > Improve s3a region handling, including determining from endpoint > > > Key: HADOOP-18908 > URL: https://issues.apache.org/jira/browse/HADOOP-18908 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Ahmar Suhail >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > S3A region logic improved for better inference and > to be compatible with previous releases > 1. If you are using an AWS S3 AccessPoint, its region is determined >from the ARN itself. > 2. If fs.s3a.endpoint.region is set and non-empty, it is used. > 3. If fs.s3a.endpoint is an s3.*.amazonaws.com url, >the region is determined by by parsing the URL >Note: vpce endpoints are not handled by this. > 4. If fs.s3a.endpoint.region==null, and none could be determined >from the endpoint, use us-east-2 as default. > 5. If fs.s3a.endpoint.region=="" then it is handed off to >The default AWS SDK resolution process. > Consult the AWS SDK documentation for the details on its resolution > process, knowing that it is complicated and may use environment variables, > entries in ~/.aws/config, IAM instance information within > EC2 deployments and possibly even JSON resources on the classpath. > Put differently: it is somewhat brittle across deployments. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18908) Improve s3a region handling, including determining from endpoint
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18908: Description: S3A region logic improved for better inference and to be compatible with previous releases 1. If you are using an AWS S3 AccessPoint, its region is determined from the ARN itself. 2. If fs.s3a.endpoint.region is set and non-empty, it is used. 3. If fs.s3a.endpoint is an s3.*.amazonaws.com url, the region is determined by by parsing the URL Note: vpce endpoints are not handled by this. 4. If fs.s3a.endpoint.region==null, and none could be determined from the endpoint, use us-east-2 as default. 5. If fs.s3a.endpoint.region=="" then it is handed off to The default AWS SDK resolution process. Consult the AWS SDK documentation for the details on its resolution process, knowing that it is complicated and may use environment variables, entries in ~/.aws/config, IAM instance information within EC2 deployments and possibly even JSON resources on the classpath. Put differently: it is somewhat brittle across deployments. was: s3a now requires the fs.s3a.endpoint.region to be set; and while it can determine it from a network call, this takes time and doesn't work for third party stores. proposed * reinstate parsing of the fs.3a.endpoint url to automatically determine region from well known endoints (and vplink ones) * don't try to talk to AWS S3 if endpoint isn't an aws one: for that caller must declare (HADOOP-18673) * document this in v2 migration, including stack traces of falures > Improve s3a region handling, including determining from endpoint > > > Key: HADOOP-18908 > URL: https://issues.apache.org/jira/browse/HADOOP-18908 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Ahmar Suhail >Priority: Major > Labels: pull-request-available > > S3A region logic improved for better inference and > to be compatible with previous releases > 1. If you are using an AWS S3 AccessPoint, its region is determined >from the ARN itself. > 2. If fs.s3a.endpoint.region is set and non-empty, it is used. > 3. If fs.s3a.endpoint is an s3.*.amazonaws.com url, >the region is determined by by parsing the URL >Note: vpce endpoints are not handled by this. > 4. If fs.s3a.endpoint.region==null, and none could be determined >from the endpoint, use us-east-2 as default. > 5. If fs.s3a.endpoint.region=="" then it is handed off to >The default AWS SDK resolution process. > Consult the AWS SDK documentation for the details on its resolution > process, knowing that it is complicated and may use environment variables, > entries in ~/.aws/config, IAM instance information within > EC2 deployments and possibly even JSON resources on the classpath. > Put differently: it is somewhat brittle across deployments. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18908) Improve s3a region handling, including determining from endpoint
[ https://issues.apache.org/jira/browse/HADOOP-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776241#comment-17776241 ] ASF GitHub Bot commented on HADOOP-18908: - steveloughran merged PR #6187: URL: https://github.com/apache/hadoop/pull/6187 > Improve s3a region handling, including determining from endpoint > > > Key: HADOOP-18908 > URL: https://issues.apache.org/jira/browse/HADOOP-18908 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Ahmar Suhail >Priority: Major > Labels: pull-request-available > > s3a now requires the fs.s3a.endpoint.region to be set; and while it can > determine it from a network call, this takes time and doesn't work for third > party stores. > proposed > * reinstate parsing of the fs.3a.endpoint url to automatically determine > region from well known endoints (and vplink ones) > * don't try to talk to AWS S3 if endpoint isn't an aws one: for that caller > must declare (HADOOP-18673) > * document this in v2 migration, including stack traces of falures -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18908. Improve S3A region handling. [hadoop]
steveloughran merged PR #6187: URL: https://github.com/apache/hadoop/pull/6187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]
ferdelyi commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766546738 > Could you take a look at this @ferdelyi? I saw that you worked on both #5638 and #6114. > > I moved some code that you added from ZKCuratorManager to SecurityUtil, so I could use it when we are using ZooKeeper without Curator in ZKFC. I also did some changes in Yarn in this patch, where I am creating an ActiveStandbyElector with a TruststoreKeystore based on the Yarn SSL configs. Do you think it is the right way to do that? Thank you @dombizita for reaching out. We discussed it in a call. Looks good to me and great work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776240#comment-17776240 ] ASF GitHub Bot commented on HADOOP-18919: - ferdelyi commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766546738 > Could you take a look at this @ferdelyi? I saw that you worked on both #5638 and #6114. > > I moved some code that you added from ZKCuratorManager to SecurityUtil, so I could use it when we are using ZooKeeper without Curator in ZKFC. I also did some changes in Yarn in this patch, where I am creating an ActiveStandbyElector with a TruststoreKeystore based on the Yarn SSL configs. Do you think it is the right way to do that? Thank you @dombizita for reaching out. We discussed it in a call. Looks good to me and great work! > Zookeeper SSL/TLS support in HDFS ZKFC > -- > > Key: HADOOP-18919 > URL: https://issues.apache.org/jira/browse/HADOOP-18919 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Zita Dombi >Assignee: Zita Dombi >Priority: Major > Labels: pull-request-available > > HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled > in hadoop-common. With those changes we have the necessary parameters, that > we need to set to enable SSL/TLS in a ZK Client. > In YARN-11468 the SSL communication can be set in Yarn, now we need to > similar changes in HDFS to enable it correctly. In HDFS ZK Client is used in > the Failover Controller. In this improvement we need to create the ZK client > with the necessary SSL configs if we enable it, which we can track under a > new HDFS config. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18939. NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() [hadoop]
steveloughran merged PR #6193: URL: https://github.com/apache/hadoop/pull/6193 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()
[
https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776233#comment-17776233
]
ASF GitHub Bot commented on HADOOP-18939:
-
steveloughran merged PR #6193:
URL: https://github.com/apache/hadoop/pull/6193
> NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()
> -
>
> Key: HADOOP-18939
> URL: https://issues.apache.org/jira/browse/HADOOP-18939
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> NPE in error handling code of RetryOnErrorCodeCondition.shouldRetry(); in
> bundle-2.20.128.jar
> This is AWS SDK code; fix needs to go there.
> {code}
> Caused by: java.lang.NullPointerException
> at
> software.amazon.awssdk.awscore.retry.conditions.RetryOnErrorCodeCondition.shouldRetry(RetryOnErrorCodeCondition.java:45)
> ~[bundle-2.20.128.jar:?]
> at
> software.amazon.awssdk.core.retry.conditions.OrRetryCondition.lambda$shouldRetry$0(OrRetryCondition.java:46)
> ~[bundle-2.20.128.jar:?]
> at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
> ~[?:1.8.0_382]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18939) NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()
[
https://issues.apache.org/jira/browse/HADOOP-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776232#comment-17776232
]
ASF GitHub Bot commented on HADOOP-18939:
-
steveloughran commented on PR #6193:
URL: https://github.com/apache/hadoop/pull/6193#issuecomment-1766508017
> The NPE will get fixed in the SDK.
yes, but that's a future tense event. we are having problems with our
deployments already
> NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry()
> -
>
> Key: HADOOP-18939
> URL: https://issues.apache.org/jira/browse/HADOOP-18939
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Priority: Critical
> Labels: pull-request-available
>
> NPE in error handling code of RetryOnErrorCodeCondition.shouldRetry(); in
> bundle-2.20.128.jar
> This is AWS SDK code; fix needs to go there.
> {code}
> Caused by: java.lang.NullPointerException
> at
> software.amazon.awssdk.awscore.retry.conditions.RetryOnErrorCodeCondition.shouldRetry(RetryOnErrorCodeCondition.java:45)
> ~[bundle-2.20.128.jar:?]
> at
> software.amazon.awssdk.core.retry.conditions.OrRetryCondition.lambda$shouldRetry$0(OrRetryCondition.java:46)
> ~[bundle-2.20.128.jar:?]
> at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
> ~[?:1.8.0_382]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18939. NPE in AWS v2 SDK RetryOnErrorCodeCondition.shouldRetry() [hadoop]
steveloughran commented on PR #6193: URL: https://github.com/apache/hadoop/pull/6193#issuecomment-1766508017 > The NPE will get fixed in the SDK. yes, but that's a future tense event. we are having problems with our deployments already -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]
Hexiaoqiao commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1766458976 Back to this PR. > IMO, adding a timeout mechanism may not add much pressure on NameNode. However, it seems that the implementation of that solution is more complex than the current patch and requires more comprehensive design and consideration. The good aspect is that the timeout mechanism can completely solve the problem of excess replica leakage, after all, the situation where datanodes fail to successfully delete replicas according to commands may not be limited to the scenario described in this JIRA. I totally support the solution @zhangshuyan0 mentioned here. https://issues.apache.org/jira/browse/HDFS-17218?focusedCommentId=17774766=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17774766 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18830. Cut S3 Select [hadoop]
ahmarsuhail commented on PR #6144: URL: https://github.com/apache/hadoop/pull/6144#issuecomment-1766391559 And agree with your comment, should cut `org.apache.hadoop.fs.s3a.select.BlockingEnumeration` and add the commit to the package-info. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18830) Cut S3 Select
[
https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776209#comment-17776209
]
ASF GitHub Bot commented on HADOOP-18830:
-
ahmarsuhail commented on PR #6144:
URL: https://github.com/apache/hadoop/pull/6144#issuecomment-1766391559
And agree with your comment, should cut
`org.apache.hadoop.fs.s3a.select.BlockingEnumeration` and add the commit to the
package-info.
> Cut S3 Select
> -
>
> Key: HADOOP-18830
> URL: https://issues.apache.org/jira/browse/HADOOP-18830
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
>
> getting s3 select to work with the v2 sdk is tricky, we need to add extra
> libraries to the classpath beyond just bundle.jar. we can do this but
> * AFAIK nobody has ever done CSV predicate pushdown, as it breaks split logic
> completely
> * CSV is a bad format
> * one-line JSON more structured but also way less efficient
> ORC/Parquet benefit from vectored IO and work spanning the cluster.
> accordingly, I'm wondering what to do about s3 select
> # cut?
> # downgrade to optional and document the extra classes on the classpath
> Option #2 is straightforward and effectively the default. we can also declare
> the feature deprecated.
> {code}
> [ERROR]
> testReadLandsatRecordsNoMatch(org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat)
> Time elapsed: 147.958 s <<< ERROR!
> java.io.IOException: java.lang.NoClassDefFoundError:
> software/amazon/eventstream/MessageDecoder
> at
> org.apache.hadoop.fs.s3a.select.SelectObjectContentHelper.select(SelectObjectContentHelper.java:75)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$select$10(WriteOperationHelper.java:660)
> at
> org.apache.hadoop.fs.store.audit.AuditingFunctions.lambda$withinAuditSpan$0(AuditingFunctions.java:62)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18830) Cut S3 Select
[
https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776207#comment-17776207
]
ASF GitHub Bot commented on HADOOP-18830:
-
ahmarsuhail commented on code in PR #6144:
URL: https://github.com/apache/hadoop/pull/6144#discussion_r1362087831
##
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md:
##
@@ -35,7 +35,6 @@ full details.
* [Encryption](./encryption.html)
* [Performance](./performance.html)
-* [S3Guard](./s3guard.html)
Review Comment:
why are we removing this? think the S3Guard page is still there
> Cut S3 Select
> -
>
> Key: HADOOP-18830
> URL: https://issues.apache.org/jira/browse/HADOOP-18830
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
>
> getting s3 select to work with the v2 sdk is tricky, we need to add extra
> libraries to the classpath beyond just bundle.jar. we can do this but
> * AFAIK nobody has ever done CSV predicate pushdown, as it breaks split logic
> completely
> * CSV is a bad format
> * one-line JSON more structured but also way less efficient
> ORC/Parquet benefit from vectored IO and work spanning the cluster.
> accordingly, I'm wondering what to do about s3 select
> # cut?
> # downgrade to optional and document the extra classes on the classpath
> Option #2 is straightforward and effectively the default. we can also declare
> the feature deprecated.
> {code}
> [ERROR]
> testReadLandsatRecordsNoMatch(org.apache.hadoop.fs.s3a.select.ITestS3SelectLandsat)
> Time elapsed: 147.958 s <<< ERROR!
> java.io.IOException: java.lang.NoClassDefFoundError:
> software/amazon/eventstream/MessageDecoder
> at
> org.apache.hadoop.fs.s3a.select.SelectObjectContentHelper.select(SelectObjectContentHelper.java:75)
> at
> org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$select$10(WriteOperationHelper.java:660)
> at
> org.apache.hadoop.fs.store.audit.AuditingFunctions.lambda$withinAuditSpan$0(AuditingFunctions.java:62)
> at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:122)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18830. Cut S3 Select [hadoop]
ahmarsuhail commented on code in PR #6144: URL: https://github.com/apache/hadoop/pull/6144#discussion_r1362087831 ## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md: ## @@ -35,7 +35,6 @@ full details. * [Encryption](./encryption.html) * [Performance](./performance.html) -* [S3Guard](./s3guard.html) Review Comment: why are we removing this? think the S3Guard page is still there -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776205#comment-17776205 ] ASF GitHub Bot commented on HADOOP-18919: - dombizita commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766365751 Could you take a look at this @ferdelyi? I saw that you worked on both #5638 and #6114. I moved some code that you added from ZKCuratorManager to SecurityUtil, so I could use it when we are using ZooKeeper without Curator in ZKFC. I also did some changes in Yarn in this patch, where I am creating an ActiveStandbyElector with a TruststoreKeystore based on the Yarn SSL configs. Do you think it is the right way to do that? > Zookeeper SSL/TLS support in HDFS ZKFC > -- > > Key: HADOOP-18919 > URL: https://issues.apache.org/jira/browse/HADOOP-18919 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Zita Dombi >Assignee: Zita Dombi >Priority: Major > Labels: pull-request-available > > HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled > in hadoop-common. With those changes we have the necessary parameters, that > we need to set to enable SSL/TLS in a ZK Client. > In YARN-11468 the SSL communication can be set in Yarn, now we need to > similar changes in HDFS to enable it correctly. In HDFS ZK Client is used in > the Failover Controller. In this improvement we need to create the ZK client > with the necessary SSL configs if we enable it, which we can track under a > new HDFS config. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18919. Zookeeper SSL/TLS support in HDFS ZKFC [hadoop]
dombizita commented on PR #6194: URL: https://github.com/apache/hadoop/pull/6194#issuecomment-1766365751 Could you take a look at this @ferdelyi? I saw that you worked on both #5638 and #6114. I moved some code that you added from ZKCuratorManager to SecurityUtil, so I could use it when we are using ZooKeeper without Curator in ZKFC. I also did some changes in Yarn in this patch, where I am creating an ActiveStandbyElector with a TruststoreKeystore based on the Yarn SSL configs. Do you think it is the right way to do that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]
zhangshuyan0 commented on code in PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#discussion_r1362023878
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java:
##
@@ -1007,6 +1013,7 @@ public void updateRegInfo(DatanodeID nodeReg) {
for(DatanodeStorageInfo storage : getStorageInfos()) {
if (storage.getStorageType() != StorageType.PROVIDED) {
storage.setBlockReportCount(0);
+storage.setBlockContentsStale(true);
Review Comment:
> StaleStorage is used to prevent the namenode from deleting replicas of
blocks whose replicas are indeterminate.
About StaleStorage we can say comments in code:
https://github.com/apache/hadoop/blob/42e695d510befabf3b7cfc52349b0d78faadb249/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java#L142-L149
> At startup or at failover, the storages in the cluster may have pending
block deletions from a previous incarnation of the NameNode.
From this, it can be seen that, the design of the "stale content" is to
address the "indeterminate" caused by pending deletions. By the way, if the
information provided by ExcessRedundancyMap is accurate, there will be no
"indeterminate" caused by pending deletions.
See also: https://issues.apache.org/jira/browse/HDFS-1972
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17227. EC: Fix bug in choosing targets when racks is not enough. [hadoop]
hadoop-yetus commented on PR #6192:
URL: https://github.com/apache/hadoop/pull/6192#issuecomment-1766291730
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 26s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 28s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 49s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 54s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 49s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 50s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 46s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 34s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 45s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 52s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 31s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 192m 6s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 284m 22s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestErasureCodingMultipleRacks |
| | hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6192 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux f27d0e89ef1f 4.15.0-212-generic #223-Ubuntu SMP Tue May 23
13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 226a9b0ed829be81046612d626d19eaacec094bd |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/2/testReport/ |
| Max. process+thread count | 3266 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6192/2/console |
|
[jira] [Commented] (HADOOP-18932) upgrade AWS v1 and v2 SDKs
[ https://issues.apache.org/jira/browse/HADOOP-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776175#comment-17776175 ] ASF GitHub Bot commented on HADOOP-18932: - steveloughran merged PR #6178: URL: https://github.com/apache/hadoop/pull/6178 > upgrade AWS v1 and v2 SDKs > -- > > Key: HADOOP-18932 > URL: https://issues.apache.org/jira/browse/HADOOP-18932 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Priority: Major > Labels: pull-request-available > > Bump up the sdk versions for both...even if we don't ship v1 it helps us > qualify releases with newer versions, and means that an upgrade of that alone > to branch-3.3 will be in sync. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18932. upgrade AWS v1 and v2 SDKs [hadoop]
steveloughran merged PR #6178: URL: https://github.com/apache/hadoop/pull/6178 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]
zhangshuyan0 commented on code in PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361994647
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java:
##
@@ -1007,6 +1013,7 @@ public void updateRegInfo(DatanodeID nodeReg) {
for(DatanodeStorageInfo storage : getStorageInfos()) {
if (storage.getStorageType() != StorageType.PROVIDED) {
storage.setBlockReportCount(0);
+storage.setBlockContentsStale(true);
Review Comment:
@ZanderXu I think there is a misunderstanding between us. I totally agree
with this change. The difference between us may be that I think it is more
appropriate to merge this change with this PR instead of opening a new issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]
ZanderXu commented on code in PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361993679
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java:
##
@@ -1007,6 +1013,7 @@ public void updateRegInfo(DatanodeID nodeReg) {
for(DatanodeStorageInfo storage : getStorageInfos()) {
if (storage.getStorageType() != StorageType.PROVIDED) {
storage.setBlockReportCount(0);
+storage.setBlockContentsStale(true);
Review Comment:
About the relationship between stale storage and ExcessRedundancyMap.
- StaleStorage is used to prevent the namenode from deleting replicas of
blocks whose replicas are indeterminate.
- ExcessRedundancyMap is used to mark the replicas of blocks that namenode
is deleting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]
ZanderXu commented on code in PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#discussion_r1361950390
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java:
##
@@ -1007,6 +1013,7 @@ public void updateRegInfo(DatanodeID nodeReg) {
for(DatanodeStorageInfo storage : getStorageInfos()) {
if (storage.getStorageType() != StorageType.PROVIDED) {
storage.setBlockReportCount(0);
+storage.setBlockContentsStale(true);
Review Comment:
> Actually, this situation can happen at any time, not just between
"registerDataNode" and "blockReport". Why do you think that after the DN is
re-registered, the probability of the above situation happening will increase,
and it needs to be dealt with specifically?
Yes, there are some other situations can cause this case. And I don't think
the probability of the above situation happening will increase.
I just think we have a chance to reduce this case. So I think we need to do
it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18919) Zookeeper SSL/TLS support in HDFS ZKFC
[ https://issues.apache.org/jira/browse/HADOOP-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zita Dombi updated HADOOP-18919: Summary: Zookeeper SSL/TLS support in HDFS ZKFC (was: Zookeeper SSL/TLS support in HDFS ) > Zookeeper SSL/TLS support in HDFS ZKFC > -- > > Key: HADOOP-18919 > URL: https://issues.apache.org/jira/browse/HADOOP-18919 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Zita Dombi >Assignee: Zita Dombi >Priority: Major > Labels: pull-request-available > > HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled > in hadoop-common. With those changes we have the necessary parameters, that > we need to set to enable SSL/TLS in a ZK Client. > In YARN-11468 the SSL communication can be set in Yarn, now we need to > similar changes in HDFS to enable it correctly. In HDFS ZK Client is used in > the Failover Controller. In this improvement we need to create the ZK client > with the necessary SSL configs if we enable it, which we can track under a > new HDFS config. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
