Thanks for constantly driving these maintenance topics, Chesnay. +1 from my
side for deprecating Java 8. I see the point Jingsong is raising. But I
agree with what David already said here. Deprecating the Java version is a
tool to make users aware of it (same as starting this discussion thread).
Thank you Chesnay for starting the discussion! This will generate bit of a
work for some users, but it's a good thing to keep moving the project
forward. Big +1 for this.
Jingsong:
Receiving this signal, the user may be unhappy because his application
> may be all on Java 8. Upgrading is a big
Hi,
For the likelihood of never deleting some SST files by RocksDB. Unfortunately,
it could happen as current level compaction strategy in RocksDB is triggered by
upper input level size reached to the threshold and the compaction priority
cannot guarantee all files would be choosed during
Hi Wenlong,
Thanks for the update. And +1 for this proposal.
Best
Yun Tang
On 2021/11/22 07:28:25 "wenlong.lwl" wrote:
> Hi, Yun Gao, Yun Tang, and Aitozi,
> thanks for the suggestion again, I have added following section in proposed
> changes at
>
Lijie Wang created FLINK-25011:
--
Summary: Introduce VertexParallelismDecider
Key: FLINK-25011
URL: https://issues.apache.org/jira/browse/FLINK-25011
Project: Flink
Issue Type: Sub-task
Liu created FLINK-25010:
---
Summary: Speed up hive's createMRSplits by multi thread
Key: FLINK-25010
URL: https://issues.apache.org/jira/browse/FLINK-25010
Project: Flink
Issue Type: Improvement
Hi Timo,
Thanks for driving this discussion, the sql job's upgrading compatibility always
is a big pain point. In the last version we completed some work, this FLIP will
make the whole upgrade story possible.
I have a few comments:
1) "EXPLAIN PLAN EXECUTE STATEMENT SET BEGIN ... END" is
Hi Chesnay,
Thanks for bringing this for discussion.
We should dig deeper into the current Java version of Flink users. At
least make sure Java 8 is not a mainstream version.
Receiving this signal, the user may be unhappy because his application
may be all on Java 8. Upgrading is a big job,
> This is what I don't really understand here: how adding a configuration
option causes issues here?
This is why: for most Flink production use cases I see, it's not like a
couple of people manage ~5 Flink
jobs, so they can easily track all the big changes in every minor Flink
version. Typically
Xinbin Huang created FLINK-25009:
Summary: Output slotSharingGroup as part of JsonGraph
Key: FLINK-25009
URL: https://issues.apache.org/jira/browse/FLINK-25009
Project: Flink
Issue Type:
There is one more fundamental issue with either of your two
proposals that've just came to my mind.
What happens if you have externalized checkpoints and the job fails
before the initial checkpoint can be safely removed?
You start the job from the latest created checkpoint and
> If you assume the 1st checkpoint needs to be "full" you know you are not
> allowed to use any shared files.
> It's true you should know about the shared files of the previous checkpoint,
> but e.g. RocksDB already tracks that.
I mean that the design described by FLIP implies the following
Marios Trivyzas created FLINK-25008:
---
Summary: Improve behaviour of NotNullEnforcer when dropping records
Key: FLINK-25008
URL: https://issues.apache.org/jira/browse/FLINK-25008
Project: Flink
Hi all,
Given I see generally consensus around having a convention and using
assertj, I propose to merge these 2 PRs:
* Add the explanation of this convention in our code quality guide:
https://github.com/apache/flink-web/pull/482
* Add assertj to dependency management in the parent pom and link
There is one more fundamental issue with either of your two proposals
that've just came to my mind. What happens if you have externalized
checkpoints and the job fails before the initial checkpoint can be
safely removed? You have a situation where you have a retained
checkpoint that was built on
Hi,
also a +1 from me because of everything Chesnay already said.
Ingo
On Mon, Nov 22, 2021 at 4:41 PM Martijn Visser
wrote:
> Hi Chesnay,
>
> Thanks for bringing this up for discussion. Big +1 for dropping Java 8 and
> deprecating it in 1.15, given that Java 8 support will end. We already
Hi Chesnay,
Thanks for bringing this up for discussion. Big +1 for dropping Java 8 and
deprecating it in 1.15, given that Java 8 support will end. We already see
other dependencies that Flink use either have dropped support for Java 8
(Trino) or are going to drop it (Kafka).
Best regards,
> NULL in SQL essentially means "UNKNOWN", it's not as scary as a null in
java which will easily cause a NPE or some random behavior with a c++
function call.
This is true from the user point of view, except our runtime doesn't treat
null as some value where you can safely execute operations and
As with any Java version there comes a time when one needs to ask
themselves for how long one intends to stick with it.
With Java 17 being released 2 months ago, and the active support for
Java 8 ending in 4 months, it is time for us to think about that with
regard to Java 8.
As such I'd
Ori Popowski created FLINK-25007:
Summary: Session window with dynamic gap doesn't work
Key: FLINK-25007
URL: https://issues.apache.org/jira/browse/FLINK-25007
Project: Flink
Issue Type: Bug
I don't think there is anything wrong with the API. My comment was about
what's behind the API. With tracking the shared files on JM you can not
say if you can clear the files after couple of checkpoints or 10s, 100s
or 1000s, which translates into minutes/hours/days/weeks of processing.
I think
Chesnay Schepler created FLINK-25006:
Summary: Investigate memory issues on CI
Key: FLINK-25006
URL: https://issues.apache.org/jira/browse/FLINK-25006
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-25005:
Summary: Add java17-target profile
Key: FLINK-25005
URL: https://issues.apache.org/jira/browse/FLINK-25005
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-25004:
Summary: Disable spotless on Java 17
Key: FLINK-25004
URL: https://issues.apache.org/jira/browse/FLINK-25004
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-25003:
Summary: RestClientTest#testConnectionTimeout fails on Java 17
Key: FLINK-25003
URL: https://issues.apache.org/jira/browse/FLINK-25003
Project: Flink
Chesnay Schepler created FLINK-25002:
Summary: Setup required --add-opens/--add-exports
Key: FLINK-25002
URL: https://issues.apache.org/jira/browse/FLINK-25002
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-25001:
Summary: Zookeeper 3.4 fails on Java 17
Key: FLINK-25001
URL: https://issues.apache.org/jira/browse/FLINK-25001
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-25000:
Summary: Scala 2.12.7 doesn't compile on Java 17
Key: FLINK-25000
URL: https://issues.apache.org/jira/browse/FLINK-25000
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-24999:
Summary: flink-python doesn't work on Java 17
Key: FLINK-24999
URL: https://issues.apache.org/jira/browse/FLINK-24999
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-24998:
Summary: SIGSEGV in Kryo / C2 CompilerThread on Java 17
Key: FLINK-24998
URL: https://issues.apache.org/jira/browse/FLINK-24998
Project: Flink
Issue
zouyunhe created FLINK-24997:
Summary: count(null) not supported in flink sql query
Key: FLINK-24997
URL: https://issues.apache.org/jira/browse/FLINK-24997
Project: Flink
Issue Type: Bug
Chesnay Schepler created FLINK-24996:
Summary: Update CI image to contain Java 17
Key: FLINK-24996
URL: https://issues.apache.org/jira/browse/FLINK-24996
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-24995:
Summary: EnumValueSerializerCompatibilityTest fails to compile on
later Scala versions
Key: FLINK-24995
URL: https://issues.apache.org/jira/browse/FLINK-24995
bx123 created FLINK-24994:
-
Summary: WindowedStream#reduce(ReduceFunction function) gives
useless suggestion when reducefunction is richfunction.
Key: FLINK-24994
URL: https://issues.apache.org/jira/browse/FLINK-24994
Thanks Dawid,
Regarding clarity,
I think that all proposals require waiting for some event: re-upload /
checkpoint completion / api response.
But with the current one, there is an assumption: "initial checkpoint
can be deleted once a new one completes" (instead of just "initial
checkpoint can be
Chesnay Schepler created FLINK-24993:
Summary: ExceptionUtils#isDirectOutOfMemoryError doesn't work on
Java 17
Key: FLINK-24993
URL: https://issues.apache.org/jira/browse/FLINK-24993
Project:
Hi, Timo, thanks for driving the discussion and the preparation on the
FLIP. This is a pain point of Flink SQL complaining by our users badly. I
have seen many cases where our users suffer while trying to upgrade the
flink version in order to take advantage of the bug fixes and performance
Chesnay Schepler created FLINK-24992:
Summary: StreamingWithStateTestBase does not compile on later
Scala versions
Key: FLINK-24992
URL: https://issues.apache.org/jira/browse/FLINK-24992
Project:
Chesnay Schepler created FLINK-24991:
Summary: FlinkStatistic does not compile on later Scala versions
Key: FLINK-24991
URL: https://issues.apache.org/jira/browse/FLINK-24991
Project: Flink
Chesnay Schepler created FLINK-24990:
Summary: LookupJoinITCase fails on Java 17
Key: FLINK-24990
URL: https://issues.apache.org/jira/browse/FLINK-24990
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-24989:
Summary: Upgrade shade-plugin to 3.2.4
Key: FLINK-24989
URL: https://issues.apache.org/jira/browse/FLINK-24989
Project: Flink
Issue Type: Sub-task
@Yun
I think it is a good comment with I agree in principal. However, we use
--fromSavepoint (cli), savepointPath (REST API),
SavepointRestoreSettings for both restoring from a savepoint and an
externalized checkpoint already. I wanted to voice that concern.
Nevertheless I am fine with changing
Chesnay Schepler created FLINK-24988:
Summary: Upgrade lombok to 1.18.22
Key: FLINK-24988
URL: https://issues.apache.org/jira/browse/FLINK-24988
Project: Flink
Issue Type: Sub-task
Nicolaus Weidner created FLINK-24987:
Summary: Enhance ExternalizedCheckpointCleanup enum
Key: FLINK-24987
URL: https://issues.apache.org/jira/browse/FLINK-24987
Project: Flink
Issue
Chesnay Schepler created FLINK-24986:
Summary: StateHandleSerializationTest fails on Java 17
Key: FLINK-24986
URL: https://issues.apache.org/jira/browse/FLINK-24986
Project: Flink
Issue
Chesnay Schepler created FLINK-24985:
Summary: LocalBufferPoolDestroyTest#isInBlockingBufferRequest does
not work on Java 17
Key: FLINK-24985
URL: https://issues.apache.org/jira/browse/FLINK-24985
Chesnay Schepler created FLINK-24984:
Summary: Rename MAVEN_OPTS variable in azure yaml
Key: FLINK-24984
URL: https://issues.apache.org/jira/browse/FLINK-24984
Project: Flink
Issue Type:
Hi,
Thanks for the proposal Dawid, I have some questions and remarks:
1. How will stop-with-savepoint be handled?
Shouldn't side effects be enforced in this case? (i.e. send
notifyCheckpointComplete)
2. Instead of forcing re-upload, can we "inverse control" in no-claim mode?
Anyways, any
Chesnay Schepler created FLINK-24983:
Summary: Upgrade surefire to 3.0.0-M5
Key: FLINK-24983
URL: https://issues.apache.org/jira/browse/FLINK-24983
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-24982:
Summary: File leak in AbstractRecoverableWriterTest
Key: FLINK-24982
URL: https://issues.apache.org/jira/browse/FLINK-24982
Project: Flink
Issue
Chesnay Schepler created FLINK-24981:
Summary: Fix JDK12+ profile activations
Key: FLINK-24981
URL: https://issues.apache.org/jira/browse/FLINK-24981
Project: Flink
Issue Type: Sub-task
Lijie Wang created FLINK-24980:
--
Summary: Collect sizes of result partitions when tasks finish
Key: FLINK-24980
URL: https://issues.apache.org/jira/browse/FLINK-24980
Project: Flink
Issue Type:
Chesnay Schepler created FLINK-24979:
Summary: Remove MaxPermSize configuration in HBase surefire config
Key: FLINK-24979
URL: https://issues.apache.org/jira/browse/FLINK-24979
Project: Flink
I also like the checklist provided by our current PR template. One annoying
thing in my opinion, though, is that we do not rely on checkboxes. Ingo
already proposed such a change in [1]. Chesnay had some good points on
certain items that were meant to be added to the template but are actually
Chesnay Schepler created FLINK-24978:
Summary: Upgrade Asm to 9.2
Key: FLINK-24978
URL: https://issues.apache.org/jira/browse/FLINK-24978
Project: Flink
Issue Type: Sub-task
Jingsong Lee created FLINK-24977:
Summary: validatePKConstraints in KafkaDynamicTableFactory throws
wrong exception message
Key: FLINK-24977
URL: https://issues.apache.org/jira/browse/FLINK-24977
Hi,
Very thanks Dawid for proposing the FLIP to clarify the ownership for the
states. +1 for the overall changes since it makes the behavior clear and
provide users a determined method to finally cleanup savepoints / retained
checkpoints.
Regarding the changes to the public interface, it
Congratulations!
Matthias Pohl 于2021年11月22日周一 下午4:10写道：
> Congratulations :-)
>
> On Thu, Nov 18, 2021 at 3:23 AM Jingsong Li
> wrote:
>
> > Congratulations, Jing! Well deserved!
> >
> > On Wed, Nov 17, 2021 at 3:00 PM Lincoln Lee
> > wrote:
> > >
> > > Congratulations, Jing!
> > >
> > >
Congratulations :-)
On Thu, Nov 18, 2021 at 3:23 AM Jingsong Li wrote:
> Congratulations, Jing! Well deserved!
>
> On Wed, Nov 17, 2021 at 3:00 PM Lincoln Lee
> wrote:
> >
> > Congratulations, Jing!
> >
> > Best,
> > Lincoln Lee
> >
> >
> > Jing Zhang 于2021年11月17日周三 上午10:24写道：
> >
> > >
xiaodao created FLINK-24976:
---
Summary: sink utils not check the schema info between query and
sink table
Key: FLINK-24976
URL: https://issues.apache.org/jira/browse/FLINK-24976
Project: Flink
Congratulations, Fabian. :-)
On Tue, Nov 16, 2021 at 5:08 PM Zakelly Lan wrote:
> Congratulations, Fabian
>
> Best,
> Zakelly
>
> On Tue, Nov 16, 2021 at 11:21 PM liwei li wrote:
>
> > Congratulations Fabian :-)
> >
> > Piotr Nowojski 于2021年11月16日周二 下午11:16写道：
> >
> > > Congratulations :)
> >
Congratulations :-)
On Tue, Nov 16, 2021 at 2:33 PM Benchao Li wrote:
> Congratulations Yangze~
>
> Dawid Wysakowicz 于2021年11月15日周一 下午10:28写道：
>
> > Congrats!
> >
> > On 15/11/2021 15:22, Marios Trivyzas wrote:
> > > Congrats Yangze!
> > >
> > > On Mon, Nov 15, 2021 at 1:34 PM Martijn Visser
62 matches
Mail list logo
