For those that are interested, I have a draft PR open for this
https://github.com/apache/flink/pull/21128 - Feel free to have a look.
I'm not sure yet why the Flink CI fails, since these tests are passing
locally.
Thanks,
Martijn
On Tue, Oct 25, 2022 at 10:21 AM Matthias Pohl
wrote:
> >
> Additionally, having code that hasn't been touch for a while increases
the risk of it
Sorry about this incomplete confusing sentence. I was about to remove it
when accidentally pushing the shortcut for sending the message out to the
mailing list.
On Tue, Oct 25, 2022 at 10:18 AM Matthias Pohl
I guess upgrading the minimal version should also mean cleaning up the
codebase, i.e. removing code segments that have been around to allow
support for older versions. The overall goal should be to improve the Flink
codebase in my opinion. Considering what David said in the old thread about
Hadoop
I believe there are some reflection based approaches in the `flink-yarn`
module, for supporting outdated APIs in early Hadoop versions.
I haven't done a thorough check, and these are what I get.
- AMRMClientAsyncReflector
- ApplicationSubmissionContextReflector
- ContainerRequestReflector
-
Given that we do not bundle any hadoop classes in the Flink binary, do you
mean simply bump the hadoop version in the parent pom?
If it is, why do not we use the latest stable hadoop version 3.3.4? It
seems that our cron build has verified that hadoop3 could work.
Best,
Yang
David Morávek
+1; anything below 2.10.x seems to be EOL
Best,
D.
On Mon, Oct 17, 2022 at 10:48 AM Márton Balassi
wrote:
> Hi Martjin,
>
> +1 for 2.10.2. Do you expect to have bandwidth in the near term to
> implement the bump?
>
> On Wed, Oct 5, 2022 at 5:00 PM Gabor Somogyi
> wrote:
>
> > Hi Martin,
> >
>
Hi Martjin,
+1 for 2.10.2. Do you expect to have bandwidth in the near term to
implement the bump?
On Wed, Oct 5, 2022 at 5:00 PM Gabor Somogyi
wrote:
> Hi Martin,
>
> Thanks for bringing this up! Lately I was thinking about to bump the hadoop
> version to at least 2.6.1 to clean up issues
Hi Martin,
Thanks for bringing this up! Lately I was thinking about to bump the hadoop
version to at least 2.6.1 to clean up issues like this:
Hi everyone,
Little over a year ago a discussion thread was opened on changing the
minimal supported version of Hadoop and bringing that to 2.8.5. [1] In this
discussion thread, I would like to propose to bring that minimal supported
version of Hadoop to 2.10.2.
Hadoop 2.8.5 is vulnerable for
+1
Thanks for driving this, David.
Thank you~
Xintong Song
On Tue, Jan 4, 2022 at 4:28 AM Thomas Weise wrote:
> +1 for bumping minimum supported Hadoop version to 2.8.5
>
> On Mon, Jan 3, 2022 at 12:25 AM David Morávek wrote:
> >
> > As there were no strong objections, we'll proceed with
+1 for bumping minimum supported Hadoop version to 2.8.5
On Mon, Jan 3, 2022 at 12:25 AM David Morávek wrote:
>
> As there were no strong objections, we'll proceed with bumping the Hadoop
> version to 2.8.5 and removing the safeguards and the CI for any earlier
> versions. This will effectively
As there were no strong objections, we'll proceed with bumping the Hadoop
version to 2.8.5 and removing the safeguards and the CI for any earlier
versions. This will effectively make the Hadoop 2.8.5 the least supported
version in Flink 1.15.
Best,
D.
On Thu, Dec 23, 2021 at 11:03 AM Till
If there are no users strongly objecting to dropping Hadoop support for <
2.8, then I am +1 for this since otherwise we won't gain a lot as Xintong
said.
Cheers,
Till
On Wed, Dec 22, 2021 at 10:33 AM David Morávek wrote:
> Agreed, if we drop the CI for lower versions, there is actually no
Agreed, if we drop the CI for lower versions, there is actually no point of
having safeguards as we can't really test for them.
Maybe one more thought (it's more of a feeling), I feel that users running
really old Hadoop versions are usually slower to adopt (they most likely
use what the current
Sorry to join the discussion late.
+1 for dropping support for hadoop versions < 2.8 from my side.
TBH, warping the reflection based logic with safeguards sounds a bit
neither fish nor fowl to me. It weakens the major benefits that we look for
by dropping support for early versions.
- The
CC user@f.a.o
Is anyone aware of something that blocks us from doing the upgrade?
D.
On Tue, Dec 21, 2021 at 5:50 PM David Morávek
wrote:
> Hi Martijn,
>
> from person experience, most Hadoop users are lagging behind the release
> lines by a lot, because upgrading a Hadoop cluster is not
Hi Martijn,
from person experience, most Hadoop users are lagging behind the release
lines by a lot, because upgrading a Hadoop cluster is not really a simply
task to achieve. I think for now, we can stay a bit conservative, nothing
blocks us for using 2.8.5 as we don't use any "newer" APIs in
Hi David,
Thanks for bringing this up for discussion! Given that Hadoop 2.8 is
considered EOL, shouldn't we bump the version to Hadoop 2.10? [1]
Best regards,
Martijn
[1]
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Active+Release+Lines
On Tue, 14 Dec 2021 at 10:28, Till Rohrmann
Hi David,
I think we haven't updated our Hadoop dependencies in a long time. Hence,
it is probably time to do so. So +1 for upgrading to the latest patch
release.
If newer 2.x Hadoop versions are compatible with 2.y with x >= y, then I
don't see a problem with dropping support for pre-bundled
Hi,
I'd like to start a discussion about upgrading a minimal Hadoop version
that Flink supports.
Even though the default value for `hadoop.version` property is set to
2.8.3, we're still ensuring both runtime and compile compatibility with
Hadoop 2.4.x with the scheduled pipeline[1].
Here is
20 matches
Mail list logo
