Hi, Sergey.
Thanks for the quick reply.
I try to package it in other pc with jdk8 and it succeeds. Please ignore
it. It seems like some errors in my environment.
Best,
Hang
Sergey Nuyanzin 于2024年1月11日周四 14:31写道：
> Hi Hang
>
> thanks for checking
> yes, it could be packaged with jdk8,
Thanks for driving this effort, xiangyu!
The proposal overall LGTM.
I just have a small question. There are other places in Flink that interact
with external storage. Should we consider adding a general retry mechanism
to them?
xiangyu feng 于2024年1月8日周一 11:31写道：
> Hi devs,
>
> I'm opening this
Hi Yu,
I haven't thought too much about the compatibility design before. By the nature
of the problem, it's impossible to make V3 compatible with V2, what we can do
is to somewhat better inform users when switching the hasher, but I don't have
any good idea so far. Do you have any suggestions
Hi devs,
I'd like to start a discussion on FLIP-416: Deprecate and remove the
RestoreMode#LEGACY[1].
The FLIP-193[2] introduced two modes of state file ownership during
checkpoint restoration: RestoreMode#CLAIM and RestoreMode#NO_CLAIM. The
LEGACY mode, which was how Flink worked until 1.15, has
Thanks for the comments, Zhu and Matthias.
@Zhu Zhu
> How about disabling the checkpoint to avoid the cost? I know the cost is
> there even if we disable the checkpoint at the moment. But I think it can be
> fixed.
> If HA is disabled, the jobmanager needs to directly participate in all blob
Hi Leonard,
Thanks for your help very much !
I have already started the discussion about FLIP-415: Introduce a new join
operator to support minibatch[1].
And I’m looking forward to your feedback if you have some spare time to take a
look.
[1]
Hi Qingsheng,
I agree with you that it would be clearer to have a new interface that
extracts the SplitFetcher creation and management logic from the current
SplitFetcherManager. However, extensive modifications to the interface may
influence a lot and cause compatibility issues. Perhaps we can
Hi Hang
thanks for checking
yes, it could be packaged with jdk8, moreover jdk8 is checked in ci
for instance here ci for the commit tagged with v3.0.0-rc1 [1]
the strange thing in the output that you've provided is
>org.apache.flink:flink-connector-hive_2.12:jar:3.0.0: Could not find
> artifact
Hi Jane,
Thanks for your reminder! I missed this.
I updated the FLIP with the UML of MiniBatchStreamingJoinOperator and linking
my POC implementation as reference.
They are placed in the part of Proposed Changes.
Best,
Xu Shuai
> 2024年1月11日 11:18，Jane Chan 写道：
>
> Hi shuai,
>
> Thanks
+1 (non-binding)
On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su wrote:
> +1 (non-binding)
>
> Best,
> Xuannan
>
> On Thu, Jan 11, 2024 at 10:28 AM Xuyang wrote:
> >
> > +1 (non-binding)--
> >
> > Best！
> > Xuyang
> >
> >
> >
> >
> >
> > 在 2024-01-11 10:00:11，"Yang Wang" 写道：
> > >+1
Hi Hongshun and Becket,
Sorry for being late in the discussion! I went through the entire FLIP but
I still have some concerns about the new SplitFetcherManager.
First of all I agree that we should hide the elementQueue from connector
developers. This could simplify the interface exposed to
Hi Zhanghao,
Thanks for driving this, that’s really painful for us when we need to switch
config `pipeline.operator-chaining`.
But I have a Concern, according to FLIP description, modifying `isChainable`
related code in `StreamGraphHasherV2` will cause the generated operator id to
be changed,
Thanks for your response, Benchao.
Here is my thought on the newly added option.
Users' current jobs are running on a version without minibatch join. If the
existing option to enable minibatch join is utilized, then when users' jobs are
migrated to the new version, the internal behavior of the
Jane Chan created FLINK-34059:
-
Summary: Add documentation on how to use state TTL hint
Key: FLINK-34059
URL: https://issues.apache.org/jira/browse/FLINK-34059
Project: Flink
Issue Type:
Hi, Sergey Nuyanzin.
Thanks for driving this.
I try to package the source with jdk8 and it will cause an error as follows.
[INFO]
[INFO] BUILD FAILURE
[INFO]
+1 (non-binding)
- Validated checksum hash
- Verified signature
- Verified web PR
- Verified tags
Best,
Jiabao
> 2024年1月11日 11:25，Hang Ruan 写道：
>
> Sorry that I make a mistake. I build the source with Maven and jdk11.
>
> Best,
> Hang
>
> Hang Ruan 于2024年1月11日周四 11:13写道：
>
>> +1
Sorry that I make a mistake. I build the source with Maven and jdk11.
Best,
Hang
Hang Ruan 于2024年1月11日周四 11:13写道：
> +1 (non-binding)
>
> - Validated checksum hash
> - Verified signature
> - Verified that no binaries exist in the source archive
> - Build the source with Maven and jdk8
> -
Hi shuai,
Thanks for initiating the discussion. The mini-batch join optimization is
very helpful, particularly for optimizing outer join conditions in CDC
sources and handling cascade joins. And +1 for the proposal.
However, I don't see any details on the proposed
+1 (non-binding)
Best,
Xuannan
On Thu, Jan 11, 2024 at 10:28 AM Xuyang wrote:
>
> +1 (non-binding)--
>
> Best！
> Xuyang
>
>
>
>
>
> 在 2024-01-11 10:00:11，"Yang Wang" 写道：
> >+1 (binding)
> >
> >
> >Best,
> >Yang
> >
> >On Thu, Jan 11, 2024 at 9:53 AM liu ron wrote:
> >
> >> +1
+1 (non-binding)
- Validated checksum hash
- Verified signature
- Verified that no binaries exist in the source archive
- Build the source with Maven and jdk8
- Verified web PR
- Verified that the flink-connector-base is not packaged in hive connector
Best,
Hang
Sergey Nuyanzin 于2024年1月11日周四
Thanks everyone for discussing this topic!
My question is could we make a trade-off between Flink users
and Flink maintainers?
1. From the perspective of a Flink maintainer
I strongly agree with Martin's point of view, such as:
- Allowing backporting of new features to Flink 1.x will result in
Feng Jin created FLINK-34058:
Summary: Support optional parameters for named parameters
Key: FLINK-34058
URL: https://issues.apache.org/jira/browse/FLINK-34058
Project: Flink
Issue Type:
Hi Zakelly,
I am fine with either Option 2 or Option 3. I think the naming in
Option 2 makes it clear that it is a boolean configuration. However,
most of the currently available boolean configurations do not use
"enable" as a suffix. Therefore, Option 3 looks good to me as it
follows the current
Feng Jin created FLINK-34057:
Summary: Support named parameters for functions
Key: FLINK-34057
URL: https://issues.apache.org/jira/browse/FLINK-34057
Project: Flink
Issue Type: Sub-task
Feng Jin created FLINK-34056:
Summary: Support named parameters for procedures
Key: FLINK-34056
URL: https://issues.apache.org/jira/browse/FLINK-34056
Project: Flink
Issue Type: Sub-task
+1 (non-binding)--
Best！
Xuyang
在 2024-01-11 10:00:11，"Yang Wang" 写道：
>+1 (binding)
>
>
>Best,
>Yang
>
>On Thu, Jan 11, 2024 at 9:53 AM liu ron wrote:
>
>> +1 non-binding
>>
>> Best
>> Ron
>>
>> Matthias Pohl 于2024年1月10日周三 23:05写道：
>>
>> > +1 (binding)
>> >
>> > On Wed, Jan 10,
Feng Jin created FLINK-34055:
Summary: Introduce a new annotation for named parameters.
Key: FLINK-34055
URL: https://issues.apache.org/jira/browse/FLINK-34055
Project: Flink
Issue Type:
Feng Jin created FLINK-34054:
Summary: FLIP-387: Support named parameters for functions and call
procedures
Key: FLINK-34054
URL: https://issues.apache.org/jira/browse/FLINK-34054
Project: Flink
Jane Chan created FLINK-34053:
-
Summary: Support state TTL hint for group aggregate
Key: FLINK-34053
URL: https://issues.apache.org/jira/browse/FLINK-34053
Project: Flink
Issue Type: Sub-task
+1 (binding)
Best,
Yang
On Thu, Jan 11, 2024 at 9:53 AM liu ron wrote:
> +1 non-binding
>
> Best
> Ron
>
> Matthias Pohl 于2024年1月10日周三 23:05写道：
>
> > +1 (binding)
> >
> > On Wed, Jan 10, 2024 at 3:35 PM ConradJam wrote:
> >
> > > +1 non-binding
> > >
> > > Dawid Wysakowicz 于2024年1月10日周三
+1 non-binding
Best
Ron
Matthias Pohl 于2024年1月10日周三 23:05写道：
> +1 (binding)
>
> On Wed, Jan 10, 2024 at 3:35 PM ConradJam wrote:
>
> > +1 non-binding
> >
> > Dawid Wysakowicz 于2024年1月10日周三 21:06写道：
> >
> > > +1 (binding)
> > > Best,
> > > Dawid
> > >
> > > On Wed, 10 Jan 2024 at 11:54, Piotr
>
> That's a very good point. I realize that the word 'recovery' means way too
> many things. So I suggest picking a more specific word here, how about
> 'execution.state-recovery.*' ? Checkpointing and state recovery are
> corresponding terms and won't make ambiguity.
>
This makes the
Hi everyone,
Please review and vote on the release candidate #1 for the version 3.0.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)
This version is compatible with Flink 1.18.x
The complete staging area is available for your
Thanks for driving this Martijn
based on the info in MANIFEST.MF of jars it seems it was built with jdk11
shouldn't we still use jdk8 for that?
On Fri, Jan 5, 2024 at 4:03 PM Martijn Visser
wrote:
> Hi everyone,
> Please review and vote on the release candidate #1 for the version
> 3.0.1, as
Hi Dong,
> On Jan 4, 2024, at 10:18 PM, Dong Lin wrote:
>
> Hi Ken,
>
> Sorry for the late reply. I didn't notice this email from you until now.
>
> In this scenario you described above, I don't think operator2 will see the
> result modified by operato1. Note that object re-use applies only
Thanks for joining the discussion, everyone and sorry for picking it up
that late. Here are a few points, I want to add to this discussion:
- FLINK-24038 [1] led to a reduction of the curator/k8s client leader
election requests by having a single leader election per JM rather than
individual once
+1 (binding)
On Wed, Jan 10, 2024 at 3:35 PM ConradJam wrote:
> +1 non-binding
>
> Dawid Wysakowicz 于2024年1月10日周三 21:06写道：
>
> > +1 (binding)
> > Best,
> > Dawid
> >
> > On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski
> wrote:
> >
> > > +1 (binding)
> > >
> > > śr., 10 sty 2024 o 11:25 Martijn
Just to give more context, my setup uses Apache Flink 1.18 with the
adaptive scheduler enabled, issues happen randomly particularly
post-restart behaviors.
After each restart, the system logs indicate "Adding split(s) to reader:",
signifying the reassignment of partitions across different
Thanks shuai for driving this, mini-batch Join is a very useful
optimization, +1 for the general idea.
Regarding the configuration
"table.exec.stream.join.mini-batch-enabled", I'm not sure it's really
necessary. The semantic of changelog emitted by the Join operator is
eventual consistency, so
Junrui Li created FLINK-34052:
-
Summary: Missing TopSpeedWindowing and SessionWindowing JARs in
Flink Maven Repository
Key: FLINK-34052
URL: https://issues.apache.org/jira/browse/FLINK-34052
Project:
+1 non-binding
Dawid Wysakowicz 于2024年1月10日周三 21:06写道：
> +1 (binding)
> Best,
> Dawid
>
> On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski wrote:
>
> > +1 (binding)
> >
> > śr., 10 sty 2024 o 11:25 Martijn Visser
> > napisał(a):
> >
> > > +1 (binding)
> > >
> > > On Wed, Jan 10, 2024 at 4:43 AM
Hangxiang Yu created FLINK-34051:
Summary: Fix equals/hashCode/toString for SavepointRestoreSettings
Key: FLINK-34051
URL: https://issues.apache.org/jira/browse/FLINK-34051
Project: Flink
Hi Alex,
I saw that I missed replying to this topic. I do think that Xintong
touched on an important topic when he mentioned that we should define
what an LTS version means. From my point of view, I would state that
an LTS version for Apache Flink means that bug fixes only will be made
available
Jinzhong Li created FLINK-34050:
---
Summary: Rocksdb state has space amplification after rescaling
with DeleteRange
Key: FLINK-34050
URL: https://issues.apache.org/jira/browse/FLINK-34050
Project: Flink
Hi devs,
I’d like to start a discussion on FLIP-415: Introduce a new join operator to
support minibatch[1].
Currently, when performing cascading connections in Flink, there is a pain
point of record amplification. Every record join operator receives would
trigger join process. However, if
+1 (binding)
Best,
Dawid
On Wed, 10 Jan 2024 at 11:54, Piotr Nowojski wrote:
> +1 (binding)
>
> śr., 10 sty 2024 o 11:25 Martijn Visser
> napisał(a):
>
> > +1 (binding)
> >
> > On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang wrote:
> > >
> > > +1 (binding)
> > >
> > > Best,
> > > Xingbo
> > >
>
+1 (non-binding)
-verified checksums
-verified signatures
-checked release tag
-verified that there is no binary in source
-built from sources
On Wed, Jan 10, 2024 at 12:30 PM Leonard Xu wrote:
> +1 (binding)
>
> - verified signatures
> - verified hashsums
> - checked Github release tag
> -
+1 (binding)
- verified signatures
- verified hashsums
- checked Github release tag
- started SQL Client, used MySQL CDC connector to capture data change from
database , the result is expected
- reviewed the web PR
- reviewed the release notes PR
Best,
Leonard
> 2024年1月10日 上午10:48，Qingsheng
+1 (binding)
śr., 10 sty 2024 o 11:25 Martijn Visser
napisał(a):
> +1 (binding)
>
> On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang wrote:
> >
> > +1 (binding)
> >
> > Best,
> > Xingbo
> >
> > Dian Fu 于2024年1月10日周三 11:35写道：
> >
> > > +1 (binding)
> > >
> > > Regards,
> > > Dian
> > >
> > > On
+1 (binding)
On Wed, Jan 10, 2024 at 11:22 AM Martijn Visser
wrote:
>
> +1 (binding)
>
> On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang wrote:
> >
> > +1 (binding)
> >
> > Best,
> > Xingbo
> >
> > Dian Fu 于2024年1月10日周三 11:35写道：
> >
> > > +1 (binding)
> > >
> > > Regards,
> > > Dian
> > >
> > >
Hi everyone,
It seems we still don't have a consensus on the rules for boolean type
options. Let me recap the alternatives we have:
Option 1: Use enumeration options instead if possible. But this may cause
some name collisions or confusion as we discussed and we should unify the
statement
+1 (binding)
On Wed, Jan 10, 2024 at 4:43 AM Xingbo Huang wrote:
>
> +1 (binding)
>
> Best,
> Xingbo
>
> Dian Fu 于2024年1月10日周三 11:35写道：
>
> > +1 (binding)
> >
> > Regards,
> > Dian
> >
> > On Wed, Jan 10, 2024 at 5:09 AM Sharath wrote:
> > >
> > > +1 (non-binding)
> > >
> > > Best,
> > >
Hi all,
After several rounds of offline discussions with Xingtong and Jinhao,
we have decided to narrow the scope of the FLIP. It will now focus on
introducing OperatorAttributes that indicate whether an operator emits
records only after inputs have ended. We will also use the attribute
to
Hey, shuai
I’ve added wiki permission for you, looking forward your streaming join
optimization.
Best,
Leonard
Hi all,
Currently, when performing cascading connections in Flink, there is a pain
point of record amplification as mentioned in discussion similar to
https://lists.apache.org/thread/2021fmwhtotl0okmtyc5b7tndlp3khf9,
I have implemented the optimization of POC and it works. I wonder to forward a
xuyang created FLINK-34049:
--
Summary: Refactor classes related to window TVF aggregation to
prepare for non-aligned windows
Key: FLINK-34049
URL: https://issues.apache.org/jira/browse/FLINK-34049
Project:
xuyang created FLINK-34048:
--
Summary: Support Session Window Aggregate in table runtime
Key: FLINK-34048
URL: https://issues.apache.org/jira/browse/FLINK-34048
Project: Flink
Issue Type: Sub-task
Matthias Pohl created FLINK-34047:
-
Summary: Inject GitHub Actions/Azure Pipelines env variables in
uploading_watchdog.sh
Key: FLINK-34047
URL: https://issues.apache.org/jira/browse/FLINK-34047
58 matches
Mail list logo
