Hi,
Yes, sorry about that seem to have messed up the link. Should have been
https://issues.apache.org/jira/browse/HADOOP-18757
Best,
Emil
On 01/08/2023 19:08, Dongjoon Hyun wrote:
Hi, Emil.
HADOOP-18568 is still open and it seems to be never a part of the Hadoop
trunk branch.
Do you mean another JIRA?
Dongjoon.
On Tue, Aug 1, 2023 at 2:59 AM Emil Ejbyfeldt
<eejbyfe...@liveintent.com.invalid> wrote:
Hi,
We previously ran some experiments on builds from the 3.5 branch and
noticed that Hadoop had a regression
(https://issues.apache.org/jira/browse/HADOOP-18568
<https://issues.apache.org/jira/browse/HADOOP-18568>) in their s3a
committer affecting 3.3.5 and 3.3.6 (Spark 3.4 uses hadoop 3.3.4). This
fix has been merged into Hadoop and will be part the next release of
Hadoop.
From our testing the regression when writing data to S3 with large
number of tasks S3 is severe enough that we would need to revert to
hadoop 3.3.4 in order to use spark 3.5 release.
Since it only for S3 I am not sure it warrants action changes in Spark
(e.g rolling back hadoop to 3.3.4). But it probably something people
testing the rc against s3 should be aware of.
Best,
Emil
On 29/07/2023 10:29, Yuanjian Li wrote:
> Hi everyone,
>
> Following the release timeline, I will cut the RC on*Tuesday, Aug
1st at
> 1 pm PST* as scheduled.
>
> Date Event
> July 17th 2023
> Late July
> 2023 Code freeze. Release branch cut.
> QA period. Focus on bug fixes, tests, stability and docs.
> Generally, no new features merged.
>
>
> August 2023 Release candidates (RC), voting, etc. until final
release passes
>
>
> Best,
> Yuanjian
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
<mailto:dev-unsubscr...@spark.apache.org>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
