[jira] [Resolved] (HADOOP-19663) Add RISC-V build infrastructure and placeholder implementation for CRC32 acceleration

2025-10-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19663. - Resolution: Fixed > Add RISC-V build infrastructure and placeholder implementation

[jira] [Created] (HADOOP-19719) Upgrade to wildfly version with support for openssl 3 and ARM

2025-10-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19719: --- Summary: Upgrade to wildfly version with support for openssl 3 and ARM Key: HADOOP-19719 URL: https://issues.apache.org/jira/browse/HADOOP-19719 Project

[jira] [Resolved] (HADOOP-19707) Surefire upgrade leads to increased report output, can cause Jenkins OOM

2025-10-02 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19707. - Fix Version/s: 3.5.0 Resolution: Fixed > Surefire upgrade leads to increa

[jira] [Resolved] (HADOOP-19282) S3A: STSClientFactory has hard-coded dependency on shaded HttpClient

2025-10-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19282. - Fix Version/s: 3.5.0 Resolution: Fixed > S3A: STSClientFactory has hard-co

[jira] [Created] (HADOOP-19712) S3A: Deadlock observed in IOStatistics EvaluatingStatisticsMap.entryset()

2025-10-01 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19712: --- Summary: S3A: Deadlock observed in IOStatistics EvaluatingStatisticsMap.entryset() Key: HADOOP-19712 URL: https://issues.apache.org/jira/browse/HADOOP-19712

[jira] [Resolved] (HADOOP-19694) Bump guava to 33.4.8-jre due to EOL

2025-09-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19694. - Fix Version/s: thirdparty-1.5.0 Resolution: Fixed > Bump guava to 33.4.8-

[jira] [Created] (HADOOP-19708) volcaono tos: remove shading

2025-09-26 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19708: --- Summary: volcaono tos: remove shading Key: HADOOP-19708 URL: https://issues.apache.org/jira/browse/HADOOP-19708 Project: Hadoop Common Issue Type

[jira] [Resolved] (HADOOP-19700) hadoop-thirdparty build to update maven plugin dependencies

2025-09-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19700. - Fix Version/s: thirdparty-1.5.0 Resolution: Fixed > hadoop-thirdparty build

September 2025 NPM supply chain attacks and builds of Hadoop.

2025-09-24 Thread Steve Loughran
September 2025 NPM supply chain attacks and builds of Hadoop. There have been two attacks on developers through subverted npm artifacts the month. The first npm supply chain attack of September was against files used in the node.js part of our distros -the yarn application catalog. https://www.

[jira] [Created] (HADOOP-19700) hadoop-thirdparty build to use later dependency-check-plugin version

2025-09-22 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19700: --- Summary: hadoop-thirdparty build to use later dependency-check-plugin version Key: HADOOP-19700 URL: https://issues.apache.org/jira/browse/HADOOP-19700 Project

Re: Missing full tarball for Hadoop 3.4.2

2025-09-19 Thread Steve Loughran
ck to using hadoop-3.4.2.tar.gz. > > > > What do others think? > > > > *From: *slfan1989 > *Date: *Friday, 19 September 2025 at 04:20 > *To: *"Suhail, Ahmar" > *Cc: *"Suhail, Ahmar" , "ah...@apache.org" < > ah...@apache.org>, Ayush

[jira] [Resolved] (HADOOP-19698) S3A Analytics-Accelerator: Update LICENSE-binary

2025-09-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19698. - Fix Version/s: 3.5.0 Resolution: Fixed > S3A Analytics-Accelerator: Upd

[jira] [Created] (HADOOP-19697) google gs connector registration failing

2025-09-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19697: --- Summary: google gs connector registration failing Key: HADOOP-19697 URL: https://issues.apache.org/jira/browse/HADOOP-19697 Project: Hadoop Common

[jira] [Created] (HADOOP-19696) hadoop distros to move cloud connectors to hadoop common/lib

2025-09-17 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19696: --- Summary: hadoop distros to move cloud connectors to hadoop common/lib Key: HADOOP-19696 URL: https://issues.apache.org/jira/browse/HADOOP-19696 Project: Hadoop

[jira] [Resolved] (HADOOP-19688) S3A: ITestS3ACommitterMRJob failing on Junit5

2025-09-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19688. - Fix Version/s: 3.5.0 Resolution: Fixed > S3A: ITestS3ACommitterMRJob failing

[jira] [Resolved] (HADOOP-19680) Update non-thirdparty Guava version to 32.0.1

2025-09-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19680. - Fix Version/s: 3.5.0 Resolution: Fixed > Update non-thirdparty Guava version

[jira] [Created] (HADOOP-19688) S3A: ITestS3ACommitterMRJob failing on Junit5

2025-09-15 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19688: --- Summary: S3A: ITestS3ACommitterMRJob failing on Junit5 Key: HADOOP-19688 URL: https://issues.apache.org/jira/browse/HADOOP-19688 Project: Hadoop Common

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-08-25 Thread Steve Loughran
/pull/7888 < > > https://github.com/apache/hadoop/pull/7888> > > PR on branch-3.4: https://github.com/apache/hadoop/pull/7889 < > > https://github.com/apache/hadoop/pull/7889> > > > > > > Feel free to merge them as per the release timelines. > >

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-08-20 Thread Steve Loughran
just discovering as I do the third party compatibility testing of the latest sdk that the fs.s3a.create.checksum.algorithm stuff is a bit broken -nothing to worry about though as its a new feature so not a regression On Wed, 20 Aug 2025 at 16:13, Suhail, Ahmar wrote: > Hey all, > > I've fixed th

[jira] [Created] (HADOOP-19654) Upgrade AWS SDK to 2.32.x

2025-08-18 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19654: --- Summary: Upgrade AWS SDK to 2.32.x Key: HADOOP-19654 URL: https://issues.apache.org/jira/browse/HADOOP-19654 Project: Hadoop Common Issue Type

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-08-15 Thread Steve Loughran
sorry, been behind on this. -1 packaging related. that hadoop-3.4.2-lean.tar.gz file is actually a tar file. Took me a while to work this down as I just assumed my ant build script was at fault. file downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean.tar.gz downloads/hadoop-3.4.2-RC2/hadoop-3.4.2-lean

[jira] [Created] (HADOOP-19650) ABFS: NPE when close() called on uninitialized filesystem

2025-08-15 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19650: --- Summary: ABFS: NPE when close() called on uninitialized filesystem Key: HADOOP-19650 URL: https://issues.apache.org/jira/browse/HADOOP-19650 Project: Hadoop

Re: [DISCUSS] Merge HADOOP-19343 (GCS FileSystem)

2025-08-14 Thread Steve Loughran
I'm in favour of that merge...if there are problems we can fix in the repo. Having it merged is the way to get a broader test environment and so find those problems. Steve (who has the google 3.1.3 branch open in an IDE right now) On Wed, 13 Aug 2025 at 19:21, Chris Nauroth wrote: > Hello ever

[jira] [Resolved] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2025-08-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18296. - Fix Version/s: 3.5.0 Resolution: Fixed > Memory fragmentation

[jira] [Resolved] (HADOOP-19610) S3A: ITests to run under JUnit5

2025-07-28 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19610. - Fix Version/s: 3.5.0 Resolution: Fixed > S3A: ITests to run under JUn

[jira] [Reopened] (HADOOP-19197) S3A: Support AWS KMS Encryption Context

2025-07-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-19197: - > S3A: Support AWS KMS Encryption Cont

[jira] [Resolved] (HADOOP-19627) S3A: testIfMatchOverwriteWithOutdatedEtag() fails when not using SSE-KMS

2025-07-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19627. - Fix Version/s: 3.5.0 Assignee: Ahmar Suhail Resolution: Fixed merged

Re: Object Store Integration Tests in Pre-submit

2025-07-24 Thread Steve Loughran
Didn't know about the ASF credentials. We'd want them to be used somewhere to generate those session credentials, with those credentials the only secrets that a test run would have. I"d thought of somehow generating restricted session credentials to the target bucket only, and with a duration of 6

[jira] [Resolved] (HADOOP-19608) Upgrade to Junit 5.13.3

2025-07-24 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19608. - Fix Version/s: 3.5.0 Resolution: Fixed JUnit has been upgraded to 5.13.3 so

[jira] [Resolved] (HADOOP-19634) acknowledge Guava license on LimitInputStream

2025-07-23 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19634. - Fix Version/s: 3.5.0 3.4.2 Assignee: PJ Fanning

[jira] [Resolved] (HADOOP-19535) S3A : Add WebIdentityTokenFileCredentialsProvider to default S3 credential provider chain

2025-07-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19535. - Resolution: Fixed > S3A : Add WebIdentityTokenFileCredentialsProvider to default

[jira] [Resolved] (HADOOP-19629) S3A: Support of CSE-KMS in version 3.4.1 seems to be broken

2025-07-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19629. - Fix Version/s: 3.4.2 Resolution: Duplicate > S3A: Support of CSE-KMS in vers

Re: [Bug Report] Thread leak in ABFS AbfsClientThrottlingAnalyzer

2025-07-15 Thread Steve Loughran
do the signup form for hadoop and say "as discussed on common dev" as reason ... the signup is there to keep spam accounts out On Wed, 9 Jul 2025 at 05:21, Matt wrote: > CC'ing common-dev as that list has more activity > > On 7/7/25 20:39, Matt wrote: > > Hi Hadoop team! > > > > I'm not sure whe

[jira] [Resolved] (HADOOP-19363) S3A: Support analytics-accelerator-s3 input streams for parquet read performance

2025-07-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19363. - Resolution: Fixed > S3A: Support analytics-accelerator-s3 input streams for parq

[jira] [Resolved] (HADOOP-19573) S3A: ITestS3AConfiguration.testDirectoryAllocatorDefval() failing

2025-07-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19573. - Fix Version/s: 3.4.2 Resolution: Fixed >

[jira] [Resolved] (HADOOP-19384) S3A: Add support for ProfileCredentialsProvider

2025-07-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19384. - Fix Version/s: 3.4.2 Resolution: Fixed > S3A: Add support

[jira] [Resolved] (HADOOP-19576) Insert Overwrite Jobs With MagicCommitter Fails On S3 Express Storage

2025-07-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19576. - Fix Version/s: 3.5.0 3.4.2 Resolution: Fixed > Ins

[jira] [Created] (HADOOP-19610) S3A: ITests to run under JUnit5

2025-07-08 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19610: --- Summary: S3A: ITests to run under JUnit5 Key: HADOOP-19610 URL: https://issues.apache.org/jira/browse/HADOOP-19610 Project: Hadoop Common Issue Type

[jira] [Created] (HADOOP-19608) Upgrade to Junit 5.13.3

2025-07-07 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19608: --- Summary: Upgrade to Junit 5.13.3 Key: HADOOP-19608 URL: https://issues.apache.org/jira/browse/HADOOP-19608 Project: Hadoop Common Issue Type: Sub-task

[jira] [Resolved] (HADOOP-19424) [S3A] Upgrade JUnit from 4 to 5 in hadoop-aws.

2025-07-07 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19424. - Fix Version/s: 3.5.0 Resolution: Fixed > [S3A] Upgrade JUnit from 4 to 5

Re: JUnit5 parameterization pain

2025-07-07 Thread Steve Loughran
ased on > Junit 5.8, removing Junit 4, and then move on to upgrading other components. > > Best, > Shilun Fan. > > On 2025/07/06 19:26:18 Cheng Pan wrote: > > Upgrading JUnit from 5.8 to 5.13 sounds like a straightforward > > version bumping, and should be low risk, so just

JUnit5 parameterization pain

2025-07-04 Thread Steve Loughran
I see that Junit5 completely breaks parameterization because its test running is completely different...the value gets passed as a parameter to the test method, which is too late for any test setup. As a result, all s3a ITests are broken. The good news, JUnit 5.13 appears to restore this functiona

[jira] [Created] (HADOOP-19593) S3A: tests against third party stores failing after latest update & conditional

2025-06-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19593: --- Summary: S3A: tests against third party stores failing after latest update & conditional Key: HADOOP-19593 URL: https://issues.apache.org/jira/browse/HADOOP-1

[jira] [Created] (HADOOP-19592) S3A: S3 Express bucket failure of conditional overwrite of multiparts is 200 + error, not 412

2025-06-19 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19592: --- Summary: S3A: S3 Express bucket failure of conditional overwrite of multiparts is 200 + error, not 412 Key: HADOOP-19592 URL: https://issues.apache.org/jira/browse/HADOOP

[jira] [Resolved] (HADOOP-19583) JDK8: RawLocalFileSystem calls ByteBuffer.flip

2025-06-18 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19583. - Resolution: Won't Fix too many other code snippets do this; need a large PR, or

[jira] [Resolved] (HADOOP-19353) Über-jira: S3A Hadoop 3.4.2 features

2025-06-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19353. - Resolution: Fixed > Über-jira: S3A Hadoop 3.4.2 featu

[jira] [Reopened] (HADOOP-19353) Über-jira: S3A Hadoop 3.4.2 features

2025-06-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-19353: - > Über-jira: S3A Hadoop 3.4.2 featu

[jira] [Created] (HADOOP-19586) Über-jira: S3A Hadoop 3.4.3 features

2025-06-16 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19586: --- Summary: Über-jira: S3A Hadoop 3.4.3 features Key: HADOOP-19586 URL: https://issues.apache.org/jira/browse/HADOOP-19586 Project: Hadoop Common Issue

[jira] [Resolved] (HADOOP-19353) Über-jira: S3A Hadoop 3.4.2 features

2025-06-16 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19353. - Fix Version/s: 3.4.2 Resolution: Fixed > Über-jira: S3A Hadoop 3.4.2 featu

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-11 Thread Steve Loughran
this is my PR for parquet buffers https://github.com/apache/hadoop/pull/7732 doesn't do any complex work, just * adds a capability to declare buffers may be sliced * adds a switch to turn verification off for file:// * adds tests to show that slicing may happen * docs For Parquet I'm fixing the t

[jira] [Resolved] (HADOOP-19295) S3A: fs.s3a.connection.request.timeout too low for large uploads over slow links

2025-06-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19295. - Fix Version/s: 3.4.2 Resolution: Fixed > S3A: fs.s3a.connection.request.time

[jira] [Reopened] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2025-06-09 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-18296: - Assignee: Steve Loughran > Memory fragmentation in ChecksumFileSystem Vectored

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-06 Thread Steve Loughran
regarding parquet, the jira there is : https://github.com/apache/parquet-java/issues/3237 and while the change in that lib is "release the buffers", it hits a problem with coalesced reads as there the releasing doesn't work. my fix will be: disable coalescing, keep the code (for now), and add a p

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-04 Thread Steve Loughran
the overloaded ByteByffer.flip() method, which is absent from very old JDK releases. If you encounter the error " java.lang.NoSuchMethodError: java.nio.ByteBuffer.flip()Ljava/nio/ByteBuffer;" you need to upgrade to a more recent version of java 8 or later. On Wed, 4 Jun 2025 at 16:45, Steve Lo

[jira] [Created] (HADOOP-19583) JDK8: RawLocalFileSystem calls ByteBuffer.flip

2025-06-04 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19583: --- Summary: JDK8: RawLocalFileSystem calls ByteBuffer.flip Key: HADOOP-19583 URL: https://issues.apache.org/jira/browse/HADOOP-19583 Project: Hadoop Common

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-04 Thread Steve Loughran
a.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) this can actually be fixed trivially, casting to java.io.Buffer first: ((Byffer)buff).flip()) I'll do this On Tue, 3 Jun 2025 at 16:04, Steve Loughran wrote: > >

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-03 Thread Steve Loughran
t's indicative of a memory leak. Surprised and Annoyed I hadn't spotted this before...the tests went in last year. On Tue, 3 Jun 2025 at 10:55, Steve Loughran wrote: > > Ahmar, no need for prs there, it's a "commit then review" repo, as in > other people can

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-03 Thread Steve Loughran
t; https://github.com/apache/hadoop-release-support/pull/4 > > > Masatake - Yes, sounds good, will update the documentation for the new RC > if it is created. > > > Mukund - I realised too late that I messed up the commit for that one. > Will figure out how to fix.. > >

[jira] [Reopened] (HADOOP-19516) S3A: SDK reads content twice during PUT to S3 Express store.

2025-06-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-19516: - Assignee: (was: Steve Loughran) > S3A: SDK reads content twice during PUT to

Re: [VOTE] Release Apache Hadoop 3.4.2

2025-06-02 Thread Steve Loughran
can you put up the relevant changes to the hadoop-release-support module; I'd like to use it as part of my validation and I'm assuming you have that src/releases/release-info-3.4.2.properties file On Wed, 28 May 2025 at 13:25, Ahmar Suhail wrote: > Hey all, > > The first release candidate for Ha

[jira] [Created] (HADOOP-19581) Update PR template to ask about AI controbution

2025-05-29 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19581: --- Summary: Update PR template to ask about AI controbution Key: HADOOP-19581 URL: https://issues.apache.org/jira/browse/HADOOP-19581 Project: Hadoop Common

[jira] [Resolved] (HADOOP-19567) S3A: error stack traces printed on analytics stream factory close

2025-05-26 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19567. - Resolution: Fixed > S3A: error stack traces printed on analytics stream factory cl

[jira] [Resolved] (HADOOP-19259) upgrade to jackson 2.14.3

2025-05-21 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19259. - Fix Version/s: 3.5.0 Assignee: PJ Fanning Resolution: Fixed > upgr

[jira] [Created] (HADOOP-19573) S3A: ITestS3AConfiguration.testDirectoryAllocatorDefval() faiilng

2025-05-20 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19573: --- Summary: S3A: ITestS3AConfiguration.testDirectoryAllocatorDefval() faiilng Key: HADOOP-19573 URL: https://issues.apache.org/jira/browse/HADOOP-19573 Project

[jira] [Resolved] (HADOOP-19554) LocalDirAllocator still doesn't always recover from directory tree deletion

2025-05-19 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19554. - Fix Version/s: 3.4.3 3.5.0 Resolution: Fixed

[jira] [Created] (HADOOP-19569) S3A: NPE in S3AFileSystem.lambda$abortMultipartUpload fs closed and multipart upload in progress

2025-05-14 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19569: --- Summary: S3A: NPE in S3AFileSystem.lambda$abortMultipartUpload fs closed and multipart upload in progress Key: HADOOP-19569 URL: https://issues.apache.org/jira/browse

[jira] [Created] (HADOOP-19567) error stack traces printed on analytics stream factory close

2025-05-13 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19567: --- Summary: error stack traces printed on analytics stream factory close Key: HADOOP-19567 URL: https://issues.apache.org/jira/browse/HADOOP-19567 Project: Hadoop

[jira] [Resolved] (HADOOP-19557) S3A: S3ABlockOutputStream to never log/reject hflush(): calls

2025-05-01 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19557. - Fix Version/s: 3.5.0 3.4.2 Resolution: Fixed >

[jira] [Created] (HADOOP-19557) S3A: S3ABlockOutputStream to never log/reject hflush(): calls

2025-04-29 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19557: --- Summary: S3A: S3ABlockOutputStream to never log/reject hflush(): calls Key: HADOOP-19557 URL: https://issues.apache.org/jira/browse/HADOOP-19557 Project

[jira] [Created] (HADOOP-19554) LocalDirAllocator doesn't always recover from directory tree deletion

2025-04-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19554: --- Summary: LocalDirAllocator doesn't always recover from directory tree deletion Key: HADOOP-19554 URL: https://issues.apache.org/jira/browse/HADOOP-

[jira] [Resolved] (HADOOP-19485) S3A: Upgrade AWS V2 SDK to 2.29.52

2025-04-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19485. - Fix Version/s: 3.5.0 3.4.2 Resolution: Fixed > S3A: Upgr

[jira] [Resolved] (HADOOP-19527) S3A: testVectoredReadAfterNormalRead() failing with 412 response from S3

2025-04-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19527. - Fix Version/s: 3.5.0 3.4.2 Resolution: Fixed >

[jira] [Resolved] (HADOOP-19523) Upgrade to hadoop-thirdparty 1.4.0

2025-04-25 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19523. - Fix Version/s: 3.4.2 Resolution: Fixed > Upgrade to hadoop-thirdparty 1.

Re: [DISCUSS] Optimize the invoke retry of async repsonder when some nameservices are slow

2025-04-23 Thread Steve Loughran
think you meant to send this to hdfs-dev, not the -subscribe list. On Wed, 16 Apr 2025 at 14:09, Zhanghaobo wrote: > Hello, everyone, so sorry to bother you and I would like to discuss how > to optimize the invoke retry logic of async repsonder when some > nameservices are slow. > Currently,

Re: HADOOP-19343 feature branch: GCS FileSystem

2025-04-23 Thread Steve Loughran
I like it, even if gcs want to be more agile at first 1. can help testing/debugging against a broadly used store 2. stops us accidentally breaking things (who would do that 🙂) 3. makes it easier to do changes across the modules Because Chris is involved, I'm not worried about this being

upgrai

2025-04-16 Thread Steve Loughran
FYI, I'm backporting HADOOP-19315, Bump avro from 1.9.2 to 1.11.4 to branch-3.4 https://github.com/apache/hadoop/pull/7615# it means there's binary incompatibility with downstream apps using avro 1.9.2 *but the avro bug is so critical they need to stop doing that ASAP* Our release notes will ha

Upgrading Avro on branch-3.4 to one that doesn't do RCEs

2025-04-15 Thread Steve Loughran
I've now got enough knowledge of how vulnerable avro (and to a very much lower extent parquet) is to RCEs from malicious files which trigger classloading. We need to fix this on branch-3.4, even though it means that the release is somewhat incompatible with older versions. https://issues.apache.o

[jira] [Resolved] (HADOOP-19256) S3A: Support S3 Conditional Writes

2025-04-11 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19256. - Fix Version/s: 3.5.0 3.4.2 Resolution: Fixed > S3A: Supp

[jira] [Resolved] (HADOOP-19455) HADOOP-19455. S3A: Enable logging of SDK client metrics

2025-04-05 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19455. - Fix Version/s: 3.5.0 Resolution: Fixed > HADOOP-19455. S3A: Enable logging

[jira] [Created] (HADOOP-19523) Upgrade to hadoop-thirdparty 1.4.0

2025-04-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19523: --- Summary: Upgrade to hadoop-thirdparty 1.4.0 Key: HADOOP-19523 URL: https://issues.apache.org/jira/browse/HADOOP-19523 Project: Hadoop Common Issue

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-04-05 Thread Steve Loughran
gelogs look good. > > > > > > On Fri, Mar 21, 2025 at 4:35 PM Chris Nauroth > wrote: > > > > > > > +1 > > > > > > > > * Verified signatures. > > > > * LICENSE-binary has the expected dependencies and versions. > > >

[jira] [Created] (HADOOP-19527) S3A: testVectoredReadAfterNormalRead() failing with 412 response from S3

2025-04-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19527: --- Summary: S3A: testVectoredReadAfterNormalRead() failing with 412 response from S3 Key: HADOOP-19527 URL: https://issues.apache.org/jira/browse/HADOOP-19527

[jira] [Created] (HADOOP-19512) S3A: analytics acclerator vector IO test skip logic incomplete

2025-04-05 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19512: --- Summary: S3A: analytics acclerator vector IO test skip logic incomplete Key: HADOOP-19512 URL: https://issues.apache.org/jira/browse/HADOOP-19512 Project

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-04-04 Thread Steve Loughran
t; wrote: > > > >> I just did > >> gpg --verify .tar.gz.asc .tar.gz > >> > >> Ok after importing hadoop KEYS > >> <https://downloads.apache.org/hadoop/common/KEYS>, this is the output > >> which > >> looks good to me: > >>

[jira] [Resolved] (HADOOP-19492) S3A: some tests failing on third-party stores

2025-04-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19492. - Fix Version/s: 3.5.0 3.4.2 Resolution: Duplicate > S3A: s

[jira] [Resolved] (HADOOP-19512) S3A: Test failures during AWS SDK upgrade

2025-04-03 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19512. - Fix Version/s: 3.5.0 3.4.2 Assignee: Steve Loughran (was

[jira] [Created] (HADOOP-19524) bulk delete must fully qualify paths before validating them

2025-03-31 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19524: --- Summary: bulk delete must fully qualify paths before validating them Key: HADOOP-19524 URL: https://issues.apache.org/jira/browse/HADOOP-19524 Project: Hadoop

[jira] [Resolved] (HADOOP-19080) S3A to support writing to object lock buckets

2025-03-31 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19080. - Fix Version/s: 3.5.0 Resolution: Fixed > S3A to support writing to object l

[ANNOUNCE] Apache Hadoop Thirdparty 1.4.0 release

2025-03-27 Thread Steve Loughran
On behalf of the Apache Hadoop Project Management Committee, I am pleased to announce the release of Apache Hadoop Thirdparty 1.4.0. For the list of changes, see https://issues.apache.org/jira/browse/HADOOP-19483 Many thanks to everyone who helped in this release. steve

[jira] [Resolved] (HADOOP-19483) Release Hadoop Third-Party 1.4.0

2025-03-27 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-19483. - Resolution: Fixed > Release Hadoop Third-Party 1.

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-03-27 Thread Steve Loughran
The time is up, the votes are in and they are +1 (binding) Ayush Saxena (binding) Chris Nauroth (binding) Steve Loughran (binding) Wei-Chiu Chuang (binding) Xiaoqiao He (binding) = 0: none -1: none Accordingly, the RC is the release; I will do the next steps in the process ASAP Steve On Tue

[jira] [Created] (HADOOP-19516) S3A: SDK reads content twice during PUT to S3 Express store.

2025-03-26 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19516: --- Summary: S3A: SDK reads content twice during PUT to S3 Express store. Key: HADOOP-19516 URL: https://issues.apache.org/jira/browse/HADOOP-19516 Project: Hadoop

[jira] [Created] (HADOOP-19514) SecretManager logs at info in bin/hadoop calls

2025-03-25 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19514: --- Summary: SecretManager logs at info in bin/hadoop calls Key: HADOOP-19514 URL: https://issues.apache.org/jira/browse/HADOOP-19514 Project: Hadoop Common

[jira] [Created] (HADOOP-19511) S3A: AWS4SignerType failing

2025-03-24 Thread Steve Loughran (Jira)
Steve Loughran created HADOOP-19511: --- Summary: S3A: AWS4SignerType failing Key: HADOOP-19511 URL: https://issues.apache.org/jira/browse/HADOOP-19511 Project: Hadoop Common Issue Type: Task

[VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-03-21 Thread Steve Loughran
Apache Hadoop-Thirdparty 1.4.0 I have built a release candidate (RC1) for Hadoop-Thirdparty 1.4.0. The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/thirdparty-1.4.0-RC1/ The git tag is release-1.4.0-RC1, and commit a1eda71bbfa630138b97ecac1a977c18d05f0003 The maven artifact

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-03-17 Thread Steve Loughran
GPG key expired? > > gpg: Signature made Fri Mar 14 08:56:00 2025 PDT > gpg:using RSA key 38237EE425050285077DB57AD22CF846DBB162A0 > gpg: Good signature from "Steve Loughran (ASF code sign key - 2018) < > ste...@apache.org>" [expired] > gpg:

Re: [VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-03-17 Thread Steve Loughran
Mar 2025 at 21:46, Wei-Chiu Chuang wrote: > Looks like your GPG key expired? > > gpg: Signature made Fri Mar 14 08:56:00 2025 PDT > gpg:using RSA key 38237EE425050285077DB57AD22CF846DBB162A0 > gpg: Good signature from "Steve Loughran (ASF code sign key - 2018) &

[jira] [Resolved] (HADOOP-18544) S3A: add option to disable probe for dir marker recreation on delete/rename.

2025-03-15 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-18544. - Resolution: Won't Fix Harshit tried this and too many things broke. pity >

[VOTE] Release Apache Hadoop Thirdparty 1.4.0

2025-03-14 Thread Steve Loughran
I have built a release candidate (RC0) for Hadoop-Thirdparty 1.4.0. The RC is available at: https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.4.0-RC0/ The git tag is release-1.4.0-RC0, and commit 5595f1357eaad02e1cfb660bcce7fd34515197ff The maven artifacts are staged at https://r

Re: Dropping support for builds on Centos 7

2025-03-14 Thread Steve Loughran
I think we really need to push ahead on a 3.5 release for this stuff and for java 17. I'm doing the new thirdparty jar once I've got the docker building *and signing* it...I'm having to do the build in EC2 as it doesn't seem to work on my arm64 laptop right now. Then ahmar can start the 3.4.2 rel

  1   2   3   4   5   6   7   8   9   10   >