Re: [PR] HDFS-17210. Optimize AvailableSpaceBlockPlacementPolicy. [hadoop]
GuoPhilipse commented on PR #6113: URL: https://github.com/apache/hadoop/pull/6113#issuecomment-1763730309 > /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Thanks @zhangshuyan0 for your notice, It seems the variable name a bit long (over 100), so the checkstyle always complains. do you have better ideas for this case? https://github.com/apache/hadoop/assets/46367746/cb0e894b-208b-4088-b798-784512a0d16c;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18918) ITestS3GuardTool fails if SSE/DSSE encryption is used
[ https://issues.apache.org/jira/browse/HADOOP-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HADOOP-18918: -- Status: Patch Available (was: In Progress) > ITestS3GuardTool fails if SSE/DSSE encryption is used > - > > Key: HADOOP-18918 > URL: https://issues.apache.org/jira/browse/HADOOP-18918 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, test >Affects Versions: 3.3.6 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Labels: pull-request-available > > {code:java} > [ERROR] Tests run: 15, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > 25.989 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool > [ERROR] > testLandsatBucketRequireUnencrypted(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool) > Time elapsed: 0.807 s <<< ERROR! > 46: Bucket s3a://landsat-pds: required encryption is none but actual > encryption is DSSE-KMS > at > org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.exitException(S3GuardTool.java:915) > at > org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.badState(S3GuardTool.java:881) > at > org.apache.hadoop.fs.s3a.s3guard.S3GuardTool$BucketInfo.run(S3GuardTool.java:511) > at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:283) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:82) > at org.apache.hadoop.fs.s3a.s3guard.S3GuardTool.run(S3GuardTool.java:963) > at > org.apache.hadoop.fs.s3a.s3guard.S3GuardToolTestHelper.runS3GuardCommand(S3GuardToolTestHelper.java:147) > at > org.apache.hadoop.fs.s3a.s3guard.AbstractS3GuardToolTestBase.run(AbstractS3GuardToolTestBase.java:114) > at > org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardTool.testLandsatBucketRequireUnencrypted(ITestS3GuardTool.java:74) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) > at > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) > at > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.lang.Thread.run(Thread.java:750) > {code} > Since landsat requires none encryption, the test should be skipped for any > encryption algorithm. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18850) Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS)
[ https://issues.apache.org/jira/browse/HADOOP-18850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HADOOP-18850: -- Status: Patch Available (was: In Progress) > Enable dual-layer server-side encryption with AWS KMS keys (DSSE-KMS) > - > > Key: HADOOP-18850 > URL: https://issues.apache.org/jira/browse/HADOOP-18850 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3, security >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > > Add support for DSSE-KMS > https://docs.aws.amazon.com/AmazonS3/latest/userguide/specifying-dsse-encryption.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17210. Optimize AvailableSpaceBlockPlacementPolicy. [hadoop]
zhangshuyan0 commented on PR #6113: URL: https://github.com/apache/hadoop/pull/6113#issuecomment-1763622513 @GuoPhilipse Would you mind fixing checkstyle based on the CI result? It seems that there are three lines with a length exceeding 100. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers [hadoop]
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1763621982 Hi @Hexiaoqiao @zhangshuyan0 do you have any comments or suggestions about this PR? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11591 Fix some wrong symbols in Federation.md [hadoop]
singer-bin closed pull request #6182: YARN-11591 Fix some wrong symbols in Federation.md URL: https://github.com/apache/hadoop/pull/6182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11591 Fix some wrong symbols in Federation.md [hadoop]
singer-bin commented on PR #6182: URL: https://github.com/apache/hadoop/pull/6182#issuecomment-1763606613 Thank you @slfan1989 for your comment. It may be a problem with my habits. It’s uncomfortable to watch. I will close this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11577. Improve FederationInterceptorREST Method Result. [hadoop]
hadoop-yetus commented on PR #6190: URL: https://github.com/apache/hadoop/pull/6190#issuecomment-1763416821 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 37s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 1s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 3 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 44m 4s | | trunk passed | | +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | checkstyle | 0m 34s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 39s | | trunk passed | | +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | spotbugs | 1m 1s | | trunk passed | | +1 :green_heart: | shadedclient | 33m 44s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 24s | | the patch passed | | +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 0m 25s | | the patch passed | | +1 :green_heart: | compile | 0m 23s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | javac | 0m 23s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | -0 :warning: | checkstyle | 0m 19s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router: The patch generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) | | +1 :green_heart: | mvnsite | 0m 26s | | the patch passed | | +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | spotbugs | 0m 55s | | the patch passed | | +1 :green_heart: | shadedclient | 33m 25s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 30s | | hadoop-yarn-server-router in the patch passed. | | +1 :green_heart: | asflicense | 0m 41s | | The patch does not generate ASF License warnings. | | | | 126m 1s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6190 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint | | uname | Linux 5a9be02598ad 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 856caa9f41171739eeb99b7409c4a017eb06bdf1 | | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/3/testReport/ | | Max. process+thread count | 752 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
Re: [PR] YARN-11577. Improve FederationInterceptorREST Method Result. [hadoop]
hadoop-yetus commented on PR #6190: URL: https://github.com/apache/hadoop/pull/6190#issuecomment-1763416216 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 25s | | Docker mode activated. | _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | | No case conflicting files found. | | +0 :ok: | codespell | 0m 1s | | codespell was not available. | | +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available. | | +0 :ok: | xmllint | 0m 1s | | xmllint was not available. | | +1 :green_heart: | @author | 0m 0s | | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | | The patch appears to include 4 new or modified test files. | _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 33m 26s | | trunk passed | | +1 :green_heart: | compile | 0m 27s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | compile | 0m 25s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | checkstyle | 0m 25s | | trunk passed | | +1 :green_heart: | mvnsite | 0m 29s | | trunk passed | | +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 0m 25s | | trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | spotbugs | 0m 46s | | trunk passed | | +1 :green_heart: | shadedclient | 20m 34s | | branch has no errors when building and testing our client artifacts. | _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 0m 18s | | the patch passed | | +1 :green_heart: | compile | 0m 19s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javac | 0m 19s | | the patch passed | | +1 :green_heart: | compile | 0m 18s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | javac | 0m 18s | | the patch passed | | +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks issues. | | -0 :warning: | checkstyle | 0m 13s | [/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router.txt) | hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router: The patch generated 12 new + 0 unchanged - 0 fixed = 12 total (was 0) | | +1 :green_heart: | mvnsite | 0m 20s | | the patch passed | | +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 | | +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | +1 :green_heart: | spotbugs | 0m 40s | | the patch passed | | +1 :green_heart: | shadedclient | 20m 24s | | patch has no errors when building and testing our client artifacts. | _ Other Tests _ | | +1 :green_heart: | unit | 0m 22s | | hadoop-yarn-server-router in the patch passed. | | +1 :green_heart: | asflicense | 0m 29s | | The patch does not generate ASF License warnings. | | | | 84m 36s | | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/6190 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint | | uname | Linux 11ac86a66594 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 4bac7c813b606b8d880a3a41993f51bee6db2ab2 | | Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 | | Test Results | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/4/testReport/ | | Max. process+thread count | 553 (vs. ulimit of 5500) | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
[jira] [Commented] (HADOOP-18867) Upgrade ZooKeeper to 3.6.4
[ https://issues.apache.org/jira/browse/HADOOP-18867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17775344#comment-17775344 ] ASF GitHub Bot commented on HADOOP-18867: - pjfanning commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1763285460 @iwasakims I don't have commit rights on Hadoop. What you haven't explained is what the benefit a patch upgrade in zookeeper jar achieves (3.6.3 to 3.6.4) - i.e. what change in zookeeper 3.6.3 jar do we need? I am saying that the benefit of upgrading to 3.7.2 is getting a fox for https://github.com/advisories/GHSA-7286-pgfv-vxvh. > Upgrade ZooKeeper to 3.6.4 > -- > > Key: HADOOP-18867 > URL: https://issues.apache.org/jira/browse/HADOOP-18867 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 3.3.6 >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Minor > Labels: pull-request-available > > While ZooKeeper 3.6 is already EOL, we can upgrade to the final release of > the ZooKeeper 3.6 as short-term fix until bumping to ZooKeeper 3.7 or later. > Dependency convergence error must be addressed on {{-Dhbase.profile=2.0}}. > {noformat} > $ mvn clean install -Dzookeeper.version=3.6.4 -Dhbase.profile=2.0 -DskipTests > clean install > Dependency convergence error for > org.apache.yetus:audience-annotations:jar:0.13.0:compile paths to dependency > are: > +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT > +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test > +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile > +-org.apache.zookeeper:zookeeper-jute:jar:3.6.4:compile > +-org.apache.yetus:audience-annotations:jar:0.13.0:compile > and > +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT > +-org.apache.hadoop:hadoop-common:test-jar:tests:3.4.0-SNAPSHOT:test > +-org.apache.zookeeper:zookeeper:jar:3.6.4:compile > +-org.apache.yetus:audience-annotations:jar:0.13.0:compile > and > +-org.apache.hadoop:hadoop-yarn-server-timelineservice-hbase-common:jar:3.4.0-SNAPSHOT > +-org.apache.hbase:hbase-common:jar:2.2.4:compile > +-org.apache.yetus:audience-annotations:jar:0.5.0:compile > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18867. Upgrade ZooKeeper to 3.6.4. [hadoop]
pjfanning commented on PR #5988: URL: https://github.com/apache/hadoop/pull/5988#issuecomment-1763285460 @iwasakims I don't have commit rights on Hadoop. What you haven't explained is what the benefit a patch upgrade in zookeeper jar achieves (3.6.3 to 3.6.4) - i.e. what change in zookeeper 3.6.3 jar do we need? I am saying that the benefit of upgrading to 3.7.2 is getting a fox for https://github.com/advisories/GHSA-7286-pgfv-vxvh. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org