1. we should patch gcs, to ignore those new keys. 2. i may be able to validate RC1 this weekend
On Fri, 16 Jun 2023 at 03:20, Wei-Chiu Chuang <weic...@cloudera.com.invalid> wrote: > Overall so far so good. > > hadoop-api-shim: > built, tested successfully. > > cloudstore: > built successfully. > > Spark: > built successfully. Passed hadoop-cloud tests. > > Ozone: > One test failure due to unrelated Ozone issue. This test is being disabled > in the latest Ozone code. > > org.apache.hadoop.hdds.utils.NativeLibraryNotLoadedException: Unable > to load library ozone_rocksdb_tools from both java.library.path & > resource file libozone_rocksdb_t > ools.so from jar. > at > org.apache.hadoop.hdds.utils.db.managed.ManagedSSTDumpTool.<init>(ManagedSSTDumpTool.java:49) > > > Google gcs: > There are two test failures. The tests were added recently by HADOOP-18724 > <https://issues.apache.org/jira/browse/HADOOP-18724> in Hadoop 3.3.6. This > is okay. Not production code problem. Can be addressed in GCS code. > > [ERROR] Errors: > [ERROR] > > TestInMemoryGoogleContractOpen>AbstractContractOpenTest.testFloatingPointLength:403 > » IllegalArgument Unknown mandatory key for gs://fake-in-memory-test-buck > et/contract-test/testFloatingPointLength "fs.option.openfile.length" > [ERROR] > > TestInMemoryGoogleContractOpen>AbstractContractOpenTest.testOpenFileApplyAsyncRead:341 > » IllegalArgument Unknown mandatory key for gs://fake-in-memory-test-b > ucket/contract-test/testOpenFileApplyAsyncRead "fs.option.openfile.length" > > > > > > On Wed, Jun 14, 2023 at 5:01 PM Wei-Chiu Chuang <weic...@apache.org> > wrote: > > > The hbase-filesystem tests passed after reverting HADOOP-18596 > > <https://issues.apache.org/jira/browse/HADOOP-18596> and HADOOP-18633 > > <https://issues.apache.org/jira/browse/HADOOP-18633> from my local tree. > > So I think it's a matter of the default behavior being changed. It's not > > the end of the world. I think we can address it by adding an incompatible > > change flag and a release note. > > > > On Wed, Jun 14, 2023 at 3:55 PM Wei-Chiu Chuang <weic...@apache.org> > > wrote: > > > >> Cross referenced git history and jira. Changelog needs some update > >> > >> Not in the release > >> > >> 1. HDFS-16858 <https://issues.apache.org/jira/browse/HDFS-16858> > >> > >> > >> 1. HADOOP-18532 <https://issues.apache.org/jira/browse/HADOOP-18532> > >> 2. > >> 1. HDFS-16861 <https://issues.apache.org/jira/browse/HDFS-16861> > >> 2. > >> 1. HDFS-16866 > >> <https://issues.apache.org/jira/browse/HDFS-16866> > >> 2. > >> 1. HADOOP-18320 > >> <https://issues.apache.org/jira/browse/HADOOP-18320> > >> 2. > >> > >> Updated fixed version. Will generate. new Changelog in the next RC. > >> > >> Was able to build HBase and hbase-filesystem without any code change. > >> > >> hbase has one unit test failure. This one is reproducible even with > >> Hadoop 3.3.5, so maybe a red herring. Local env or something. > >> > >> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > >> 9.007 s <<< FAILURE! - in > >> org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker > >> [ERROR] > >> > org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness > >> Time elapsed: 3.13 s <<< ERROR! > >> java.lang.OutOfMemoryError: Java heap space > >> at > >> > org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker$RandomTestData.<init>(TestSyncTimeRangeTracker.java:91) > >> at > >> > org.apache.hadoop.hbase.regionserver.TestSyncTimeRangeTracker.testConcurrentIncludeTimestampCorrectness(TestSyncTimeRangeTracker.java:156) > >> > >> hbase-filesystem has three test failures in TestHBOSSContractDistCp, and > >> is not reproducible with Hadoop 3.3.5. > >> [ERROR] Failures: [ERROR] > >> > TestHBOSSContractDistCp>AbstractContractDistCpTest.testDistCpUpdateCheckFileSkip:976->Assert.fail:88 > >> 10 errors in file of length 10 > >> [ERROR] > >> > TestHBOSSContractDistCp>AbstractContractDistCpTest.testUpdateDeepDirectoryStructureNoChange:270->AbstractContractDistCpTest.assertCounterInRange:290->Assert.assertTrue:41->Assert.fail:88 > >> Files Skipped value 0 too below minimum 1 > >> [ERROR] > >> > TestHBOSSContractDistCp>AbstractContractDistCpTest.testUpdateDeepDirectoryStructureToRemote:259->AbstractContractDistCpTest.distCpUpdateDeepDirectoryStructure:334->AbstractContractDistCpTest.assertCounterInRange:294->Assert.assertTrue:41->Assert.fail:88 > >> Files Copied value 2 above maximum 1 > >> [INFO] > >> [ERROR] Tests run: 240, Failures: 3, Errors: 0, Skipped: 58 > >> > >> > >> Ozone > >> test in progress. Will report back. > >> > >> > >> On Tue, Jun 13, 2023 at 11:27 PM Wei-Chiu Chuang <weic...@apache.org> > >> wrote: > >> > >>> I am inviting anyone to try and vote on this release candidate. > >>> > >>> Note: > >>> This is built off branch-3.3.6 plus PR#5741 (aws sdk update) and > PR#5740 > >>> (LICENSE file update) > >>> > >>> The RC is available at: > >>> https://home.apache.org/~weichiu/hadoop-3.3.6-RC0-amd64/ (for amd64) > >>> https://home.apache.org/~weichiu/hadoop-3.3.6-RC0-arm64/ (for arm64) > >>> > >>> Git tag: release-3.3.6-RC0 > >>> https://github.com/apache/hadoop/releases/tag/release-3.3.6-RC0 > >>> > >>> Maven artifacts is built by x86 machine and are staged at > >>> > https://repository.apache.org/content/repositories/orgapachehadoop-1378/ > >>> > >>> My public key: > >>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > >>> > >>> Changelog: > >>> https://home.apache.org/~weichiu/hadoop-3.3.6-RC0-amd64/CHANGELOG.md > >>> > >>> Release notes: > >>> > https://home.apache.org/~weichiu/hadoop-3.3.6-RC0-amd64/RELEASENOTES.md > >>> > >>> This is a relatively small release (by Hadoop standard) containing > about > >>> 120 commits. > >>> Please give it a try, this RC vote will run for 7 days. > >>> > >>> > >>> Feature highlights: > >>> > >>> SBOM artifacts > >>> ---------------------------------------- > >>> Starting from this release, Hadoop publishes Software Bill of Materials > >>> (SBOM) using > >>> CycloneDX Maven plugin. For more information about SBOM, please go to > >>> [SBOM](https://cwiki.apache.org/confluence/display/COMDEV/SBOM). > >>> > >>> HDFS RBF: RDBMS based token storage support > >>> ---------------------------------------- > >>> HDFS Router-Router Based Federation now supports storing delegation > >>> tokens on MySQL, > >>> [HADOOP-18535](https://issues.apache.org/jira/browse/HADOOP-18535) > >>> which improves token operation through over the original > Zookeeper-based > >>> implementation. > >>> > >>> > >>> New File System APIs > >>> ---------------------------------------- > >>> [HADOOP-18671](https://issues.apache.org/jira/browse/HADOOP-18671) > >>> moved a number of > >>> HDFS-specific APIs to Hadoop Common to make it possible for certain > >>> applications that > >>> depend on HDFS semantics to run on other Hadoop compatible file > systems. > >>> > >>> In particular, recoverLease() and isFileClosed() are exposed through > >>> LeaseRecoverable > >>> interface. While setSafeMode() is exposed through SafeMode interface. > >>> > >>> > >>> >
