Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64

2023-02-23 Thread Apache Jenkins Server
For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/448/

[Feb 21, 2023, 11:18:49 AM] (github) HADOOP-18622. Upgrade ant to 1.10.13 
(#5360). Contributed by Aleksandr Nikolaev.
[Feb 21, 2023, 2:39:32 PM] (github) YARN-11394. Fix 
hadoop-yarn-server-resourcemanager module Java Doc Errors. (#5288)
[Feb 22, 2023, 1:58:13 AM] (Ayush Saxena) Revert "YARN-11404. Add junit5 
dependency to hadoop-mapreduce-client-app to fix few unit test failure. 
Contributed by Susheel Gupta"
[Feb 22, 2023, 8:46:42 AM] (github) YARN-11408. Add a check of 
autoQueueCreation is disabled for emitDefaultUserLimitFactor method (#5278)
[Feb 22, 2023, 11:48:12 AM] (github) HADOOP-18636 LocalDirAllocator cannot 
recover from directory tree deletion (#5412)
[Feb 22, 2023, 6:38:50 PM] (Owen O'Malley) HADOOP-18535. Implement token 
storage solution based on MySQL
[Feb 22, 2023, 7:29:41 PM] (github) HADOOP-18582. Addendum: Skip unnecessary 
cleanup logic in DistCp. (#5409)
[Feb 22, 2023, 8:37:35 PM] (github) YARN-11370. [Federation] Refactor 
MemoryFederationStateStore code. (#5126)
[Feb 22, 2023, 9:58:44 PM] (github) HDFS-16901: RBF: Propagates real user's 
username via the caller context, when a proxy user is being used. (#5346)
[Feb 23, 2023, 12:37:49 AM] (Owen O'Malley) HDFS-16901: Minor fix for unit test.




-1 overall


The following subsystems voted -1:
blanks hadolint mvnsite pathlen spotbugs unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-common-project/hadoop-common/src/test/resources/xml/external-dtd.xml 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
 

spotbugs :

   module:hadoop-hdfs-project/hadoop-hdfs 
   Redundant nullcheck of oldLock, which is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.DataStorage.isPreUpgradableLayout(Storage$StorageDirectory))
 Redundant null check at DataStorage.java:is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.DataStorage.isPreUpgradableLayout(Storage$StorageDirectory))
 Redundant null check at DataStorage.java:[line 695] 
   Redundant nullcheck of metaChannel, which is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.MappableBlockLoader.verifyChecksum(long,
 FileInputStream, FileChannel, String) Redundant null check at 
MappableBlockLoader.java:is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.MappableBlockLoader.verifyChecksum(long,
 FileInputStream, FileChannel, String) Redundant null check at 
MappableBlockLoader.java:[line 138] 
   Redundant nullcheck of blockChannel, which is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.MemoryMappableBlockLoader.load(long,
 FileInputStream, FileInputStream, String, ExtendedBlockId) Redundant null 
check at MemoryMappableBlockLoader.java:is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.MemoryMappableBlockLoader.load(long,
 FileInputStream, FileInputStream, String, ExtendedBlockId) Redundant null 
check at MemoryMappableBlockLoader.java:[line 75] 
   Redundant nullcheck of blockChannel, which is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.NativePmemMappableBlockLoader.load(long,
 FileInputStream, FileInputStream, String, ExtendedBlockId) Redundant null 
check at NativePmemMappableBlockLoader.java:is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.NativePmemMappableBlockLoader.load(long,
 FileInputStream, FileInputStream, String, ExtendedBlockId) Redundant null 
check at NativePmemMappableBlockLoader.java:[line 85] 
   Redundant nullcheck of metaChannel, which is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.NativePmemMappableBlockLoader.verifyChecksumAndMapBlock(NativeIO$POSIX$$PmemMappedRegion,,
 long, FileInputStream, FileChannel, String) Redundant null che

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

2023-02-23 Thread Apache Jenkins Server
For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/947/

No changes


ERROR: File 'out/email-report.txt' does not exist

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Resolved] (MAPREDUCE-7428) Fix failures related to Junit 4 to Junit 5 upgrade in org.apache.hadoop.mapreduce.v2.app.webapp

2023-02-23 Thread Ayush Saxena (Jira)


 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-7428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ayush Saxena resolved MAPREDUCE-7428.
-
Resolution: Fixed

> Fix failures related to Junit 4 to Junit 5 upgrade in 
> org.apache.hadoop.mapreduce.v2.app.webapp
> ---
>
> Key: MAPREDUCE-7428
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7428
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.4.0
>Reporter: Ashutosh Gupta
>Assignee: Akira Ajisaka
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>
> Few test are getting failed due to Junit 4 to Junit 5 upgrade in 
> org.apache.hadoop.mapreduce.v2.app.webapp 
> [https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1071/testReport/]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org



Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

2023-02-23 Thread Apache Jenkins Server
For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1145/

[Feb 22, 2023, 1:58:13 AM] (Ayush Saxena) Revert "YARN-11404. Add junit5 
dependency to hadoop-mapreduce-client-app to fix few unit test failure. 
Contributed by Susheel Gupta"
[Feb 22, 2023, 8:46:42 AM] (github) YARN-11408. Add a check of 
autoQueueCreation is disabled for emitDefaultUserLimitFactor method (#5278)
[Feb 22, 2023, 11:48:12 AM] (github) HADOOP-18636 LocalDirAllocator cannot 
recover from directory tree deletion (#5412)
[Feb 22, 2023, 6:38:50 PM] (Owen O'Malley) HADOOP-18535. Implement token 
storage solution based on MySQL
[Feb 22, 2023, 7:29:41 PM] (github) HADOOP-18582. Addendum: Skip unnecessary 
cleanup logic in DistCp. (#5409)




-1 overall


The following subsystems voted -1:
blanks hadolint pathlen spotbugs unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-common-project/hadoop-common/src/test/resources/xml/external-dtd.xml 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
 

spotbugs :

   module:hadoop-mapreduce-project/hadoop-mapreduce-client 
   Write to static field 
org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:[line 120] 

spotbugs :

   
module:hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
 
   Write to static field 
org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:[line 120] 

spotbugs :

   module:hadoop-mapreduce-project 
   Write to static field 
org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:[line 120] 

spotbugs :

   module:root 
   Write to static field 
org.apache.hadoop.mapreduce.task.reduce.Fetcher.nextId from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:from instance method new 
org.apache.hadoop.mapreduce.task.reduce.Fetcher(JobConf, TaskAttemptID, 
ShuffleSchedulerImpl, MergeManager, Reporter, ShuffleClientMetrics, 
ExceptionReporter, SecretKey) At Fetcher.java:[line 120] 

Failed junit tests :

   hadoop.mapred.TestShuffleHandler 
  

   cc:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1145/artifact/out/results-compile-cc-root.txt
 [96K]

   javac:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1145/artifact/out/results-compile-javac-root.txt
 [528K]

   blanks:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1145/artifact/out/blanks-eol.txt
 [14M]
  
https://ci-hadoop.a

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Steve Loughran
And I've just hit HADOOP-18641. cyclonedx maven plugin breaks on recent
maven releases (3.9.0)

on a new local build with maven updated on homebrew (which i needed for
spark). so a code change too. That issue doesn't surface on our
release dockers, but will hit other people. especially over time. Going to
revert HADOOP-18590. Publish SBOM artifacts (#5281)



On Thu, 23 Feb 2023 at 10:29, Steve Loughran  wrote:

> ok, let me cancel, update those jiras and kick off again. that will save
> anyone else having to do their homework
>
> On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma 
> wrote:
>
>> I'm now -1 as I found the wrong information on the top page (index.md).
>>
>> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
>> Router Based Federation.
>>
>> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
>> though it is actually not in branch-3.3. I corrected the fix version and
>> created HDFS-16889 to backport them to branch-3.3 about a month ago.
>> Unfortunately, it won't be fixed soon. I should have let you know at that
>> time, sorry.  Supporting Observer NameNode in RBF is a prominent feature.
>> So I think we have to delete the description from the top page not to
>> confuse Hadoop users.
>>
>> - Takanobu
>>
>> 2023年2月23日(木) 17:17 Takanobu Asanuma :
>>
>> > Thanks for driving the release, Steve and Mukund.
>> >
>> > I found that there were some jiras with wrong fix versions.
>> >
>> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
>> > - HDFS-16845
>> > - HADOOP-18345
>> >
>> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
>> > (and it is not in release-3.3.4) :
>> > - HADOOP-17276
>> > - HDFS-13293
>> > - HDFS-15630
>> > - HDFS-16266
>> > - HADOOP-18003
>> > - HDFS-16310
>> > - HADOOP-18014
>> >
>> > I corrected all the wrong fix versions just now. I'm not sure we should
>> > revote it since it only affects the changelog.
>> >
>> > - Takanobu
>> >
>> > 2023年2月21日(火) 22:43 Steve Loughran :
>> >
>> >> Apache Hadoop 3.3.5
>> >>
>> >> Mukund and I have put together a release candidate (RC1) for Hadoop
>> 3.3.5.
>> >>
>> >> What we would like is for anyone who can to verify the tarballs,
>> >> especially
>> >> anyone who can try the arm64 binaries as we want to include them too.
>> >>
>> >> The RC is available at:
>> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>> >>
>> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
>> >>
>> >> The maven artifacts are staged at
>> >>
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>> >>
>> >> You can find my public key at:
>> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> >>
>> >> Change log
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>> >>
>> >> Release notes
>> >>
>> >>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>> >>
>> >> This is off branch-3.3 and is the first big release since 3.3.2.
>> >>
>> >> Key changes include
>> >>
>> >> * Big update of dependencies to try and keep those reports of
>> >>   transitive CVEs under control -both genuine and false positives.
>> >> * HDFS RBF enhancements
>> >> * Critical fix to ABFS input stream prefetching for correct reading.
>> >> * Vectored IO API for all FSDataInputStream implementations, with
>> >>   high-performance versions for file:// and s3a:// filesystems.
>> >>   file:// through java native io
>> >>   s3a:// parallel GET requests.
>> >> * This release includes Arm64 binaries. Please can anyone with
>> >>   compatible systems validate these.
>> >>
>> >> Note, because the arm64 binaries are built separately on a different
>> >> platform and JVM, their jar files may not match those of the x86
>> >> release -and therefore the maven artifacts. I don't think this is
>> >> an issue (the ASF actually releases source tarballs, the binaries are
>> >> there for help only, though with the maven repo that's a bit blurred).
>> >>
>> >> The only way to be consistent would actually untar the x86.tar.gz,
>> >> overwrite its binaries with the arm stuff, retar, sign and push out
>> >> for the vote. Even automating that would be risky.
>> >>
>> >> Please try the release and vote. The vote will run for 5 days.
>> >>
>> >> Steve and Mukund
>> >>
>> >
>>
>


Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Steve Loughran
ok, let me cancel, update those jiras and kick off again. that will save
anyone else having to do their homework

On Thu, 23 Feb 2023 at 08:56, Takanobu Asanuma  wrote:

> I'm now -1 as I found the wrong information on the top page (index.md).
>
> > 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
> Router Based Federation.
>
> The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
> though it is actually not in branch-3.3. I corrected the fix version and
> created HDFS-16889 to backport them to branch-3.3 about a month ago.
> Unfortunately, it won't be fixed soon. I should have let you know at that
> time, sorry.  Supporting Observer NameNode in RBF is a prominent feature.
> So I think we have to delete the description from the top page not to
> confuse Hadoop users.
>
> - Takanobu
>
> 2023年2月23日(木) 17:17 Takanobu Asanuma :
>
> > Thanks for driving the release, Steve and Mukund.
> >
> > I found that there were some jiras with wrong fix versions.
> >
> > The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> > - HDFS-16845
> > - HADOOP-18345
> >
> > The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> > (and it is not in release-3.3.4) :
> > - HADOOP-17276
> > - HDFS-13293
> > - HDFS-15630
> > - HDFS-16266
> > - HADOOP-18003
> > - HDFS-16310
> > - HADOOP-18014
> >
> > I corrected all the wrong fix versions just now. I'm not sure we should
> > revote it since it only affects the changelog.
> >
> > - Takanobu
> >
> > 2023年2月21日(火) 22:43 Steve Loughran :
> >
> >> Apache Hadoop 3.3.5
> >>
> >> Mukund and I have put together a release candidate (RC1) for Hadoop
> 3.3.5.
> >>
> >> What we would like is for anyone who can to verify the tarballs,
> >> especially
> >> anyone who can try the arm64 binaries as we want to include them too.
> >>
> >> The RC is available at:
> >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
> >>
> >> The git tag is release-3.3.5-RC1, commit 274f91a3259
> >>
> >> The maven artifacts are staged at
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
> >>
> >> You can find my public key at:
> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>
> >> Change log
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
> >>
> >> Release notes
> >>
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
> >>
> >> This is off branch-3.3 and is the first big release since 3.3.2.
> >>
> >> Key changes include
> >>
> >> * Big update of dependencies to try and keep those reports of
> >>   transitive CVEs under control -both genuine and false positives.
> >> * HDFS RBF enhancements
> >> * Critical fix to ABFS input stream prefetching for correct reading.
> >> * Vectored IO API for all FSDataInputStream implementations, with
> >>   high-performance versions for file:// and s3a:// filesystems.
> >>   file:// through java native io
> >>   s3a:// parallel GET requests.
> >> * This release includes Arm64 binaries. Please can anyone with
> >>   compatible systems validate these.
> >>
> >> Note, because the arm64 binaries are built separately on a different
> >> platform and JVM, their jar files may not match those of the x86
> >> release -and therefore the maven artifacts. I don't think this is
> >> an issue (the ASF actually releases source tarballs, the binaries are
> >> there for help only, though with the maven repo that's a bit blurred).
> >>
> >> The only way to be consistent would actually untar the x86.tar.gz,
> >> overwrite its binaries with the arm stuff, retar, sign and push out
> >> for the vote. Even automating that would be risky.
> >>
> >> Please try the release and vote. The vote will run for 5 days.
> >>
> >> Steve and Mukund
> >>
> >
>


Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Takanobu Asanuma
I'm now -1 as I found the wrong information on the top page (index.md).

> 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
Router Based Federation.

The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
though it is actually not in branch-3.3. I corrected the fix version and
created HDFS-16889 to backport them to branch-3.3 about a month ago.
Unfortunately, it won't be fixed soon. I should have let you know at that
time, sorry.  Supporting Observer NameNode in RBF is a prominent feature.
So I think we have to delete the description from the top page not to
confuse Hadoop users.

- Takanobu

2023年2月23日(木) 17:17 Takanobu Asanuma :

> Thanks for driving the release, Steve and Mukund.
>
> I found that there were some jiras with wrong fix versions.
>
> The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> - HDFS-16845
> - HADOOP-18345
>
> The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> (and it is not in release-3.3.4) :
> - HADOOP-17276
> - HDFS-13293
> - HDFS-15630
> - HDFS-16266
> - HADOOP-18003
> - HDFS-16310
> - HADOOP-18014
>
> I corrected all the wrong fix versions just now. I'm not sure we should
> revote it since it only affects the changelog.
>
> - Takanobu
>
> 2023年2月21日(火) 22:43 Steve Loughran :
>
>> Apache Hadoop 3.3.5
>>
>> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>>
>> What we would like is for anyone who can to verify the tarballs,
>> especially
>> anyone who can try the arm64 binaries as we want to include them too.
>>
>> The RC is available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>
>> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>
>> The maven artifacts are staged at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>
>> You can find my public key at:
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>> Change log
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>
>> Release notes
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>
>> This is off branch-3.3 and is the first big release since 3.3.2.
>>
>> Key changes include
>>
>> * Big update of dependencies to try and keep those reports of
>>   transitive CVEs under control -both genuine and false positives.
>> * HDFS RBF enhancements
>> * Critical fix to ABFS input stream prefetching for correct reading.
>> * Vectored IO API for all FSDataInputStream implementations, with
>>   high-performance versions for file:// and s3a:// filesystems.
>>   file:// through java native io
>>   s3a:// parallel GET requests.
>> * This release includes Arm64 binaries. Please can anyone with
>>   compatible systems validate these.
>>
>> Note, because the arm64 binaries are built separately on a different
>> platform and JVM, their jar files may not match those of the x86
>> release -and therefore the maven artifacts. I don't think this is
>> an issue (the ASF actually releases source tarballs, the binaries are
>> there for help only, though with the maven repo that's a bit blurred).
>>
>> The only way to be consistent would actually untar the x86.tar.gz,
>> overwrite its binaries with the arm stuff, retar, sign and push out
>> for the vote. Even automating that would be risky.
>>
>> Please try the release and vote. The vote will run for 5 days.
>>
>> Steve and Mukund
>>
>


Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Takanobu Asanuma
Thanks for driving the release, Steve and Mukund.

I found that there were some jiras with wrong fix versions.

The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
- HDFS-16845
- HADOOP-18345

The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
(and it is not in release-3.3.4) :
- HADOOP-17276
- HDFS-13293
- HDFS-15630
- HDFS-16266
- HADOOP-18003
- HDFS-16310
- HADOOP-18014

I corrected all the wrong fix versions just now. I'm not sure we should
revote it since it only affects the changelog.

- Takanobu

2023年2月21日(火) 22:43 Steve Loughran :

> Apache Hadoop 3.3.5
>
> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>
> What we would like is for anyone who can to verify the tarballs, especially
> anyone who can try the arm64 binaries as we want to include them too.
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>
> The git tag is release-3.3.5-RC1, commit 274f91a3259
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>
> Release notes
>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>
> This is off branch-3.3 and is the first big release since 3.3.2.
>
> Key changes include
>
> * Big update of dependencies to try and keep those reports of
>   transitive CVEs under control -both genuine and false positives.
> * HDFS RBF enhancements
> * Critical fix to ABFS input stream prefetching for correct reading.
> * Vectored IO API for all FSDataInputStream implementations, with
>   high-performance versions for file:// and s3a:// filesystems.
>   file:// through java native io
>   s3a:// parallel GET requests.
> * This release includes Arm64 binaries. Please can anyone with
>   compatible systems validate these.
>
> Note, because the arm64 binaries are built separately on a different
> platform and JVM, their jar files may not match those of the x86
> release -and therefore the maven artifacts. I don't think this is
> an issue (the ASF actually releases source tarballs, the binaries are
> there for help only, though with the maven repo that's a bit blurred).
>
> The only way to be consistent would actually untar the x86.tar.gz,
> overwrite its binaries with the arm stuff, retar, sign and push out
> for the vote. Even automating that would be risky.
>
> Please try the release and vote. The vote will run for 5 days.
>
> Steve and Mukund
>