Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[ConradJam]

+1 (no-binding)

Zhu Zhu  于2023年7月25日周二 10:07写道：

> +1 (binding)
>
> Thanks,
> Zhu
>
> Xintong Song  于2023年7月25日周二 09:46写道：
> >
> > +1 (binding)
> >
> > Best,
> >
> > Xintong
> >
> >
> >
> > On Mon, Jul 24, 2023 at 11:58 PM Jing Ge 
> wrote:
> >
> > > +1
> > >
> > > On Mon, Jul 24, 2023 at 11:43 PM Chesnay Schepler 
> > > wrote:
> > >
> > > > I update the endpoint in the FLIP.
> > > >
> > > > On 24/07/2023 14:28, Matthias Pohl wrote:
> > > > > I should have mentioned it in the discussion thread but I missed
> going
> > > > over
> > > > > that ML thread earlier: We might want to update the FLIP to refer
> to
> > > the
> > > > > actual endpoint /jobs/:jobid/rescaling (AFAIU) with the
> corresponding
> > > > cause
> > > > > being FLINK-12312 [1].
> > > > >
> > > > > But that's just a minor thing.
> > > > >
> > > > > +1 (binding)
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/FLINK-12312
> > > > >
> > > > > On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf <
> kna...@apache.org>
> > > > wrote:
> > > > >
> > > > >> +1 (binding)
> > > > >>
> > > > >> Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> > > > >> martijnvis...@apache.org>:
> > > > >>
> > > > >>> +1 (binding)
> > > > >>>
> > > > >>> On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler <
> ches...@apache.org
> > > >
> > > > >>> wrote:
> > > > >>>
> > > >  Hello,
> > > > 
> > > >  I'd like to start a vote on FLIP-340.
> > > > 
> > > >  Discussion thread:
> > > > 
> https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
> > > >  FLIP:
> > > > 
> > > > 
> > > > >>
> > > >
> > >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
> > > >  Regards,
> > > >  Chesnay
> > > > 
> > > > >>
> > > > >> --
> > > > >> https://twitter.com/snntrable
> > > > >> https://github.com/knaufk
> > > > >>
> > > >
> > > >
> > >
>


-- 
Best

ConradJam

Re: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[ConradJam]

Congratulations, Yong Fang

Mang Zhang  于2023年7月25日周二 12:08写道：

> Congratulations, Yong Fang!
>
>
> --
>
> Best regards,
> Mang Zhang
>
>
>
>
>
> 在 2023-07-25 10:30:24，"Jark Wu"  写道：
> >Congratulations, Yong Fang!
> >
> >Best,
> >Jark
> >
> >On Mon, 24 Jul 2023 at 22:11, Wencong Liu  wrote:
> >
> >> Congratulations!
> >>
> >> Best,
> >> Wencong Liu
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> 在 2023-07-24 11:03:30，"Paul Lam"  写道：
> >> >Congrats, Shammon!
> >> >
> >> >Best,
> >> >Paul Lam
> >> >
> >> >> 2023年7月24日 10:56，Jingsong Li  写道：
> >> >>
> >> >> Shammon
> >> >
> >>
>


-- 
Best

ConradJam

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

[Xintong Song]

Hi Alex,

Providing a longer supporting period for the last 1.x minor release makes
sense to me.

I think we need to be more specific about what LTS means here.

   - IIUC, that means for the last 1.x minor release, we will keep
   providing 1.x.y / 1.x.z bugfix release. This is a stronger support compared
   to regular minor releases which by default are only supported for 2 minor
   release cycles.
   - Do we only provide bug fixes for the LTS release, or do we also allow
   backporting features to that release?
   - How long exactly shall we support the LTS release?

And maybe we can make this a general convention for last minor releases for
all major releases, rather than only discuss it for the 2.0 version bump.

@Leonard,

I'd like to clarify that there are no community decisions yet on release
2.0 after 1.19. It is possible to have 1.20 before 2.0.

Best,

Xintong



On Tue, Jul 25, 2023 at 11:54 AM Leonard Xu  wrote:

> +1, it’s pretty necessary especially we deprecated so many APIs in 1.18
> and plan to remove in 2.0.
>
> The 1.19 should be a proper version for LTS Release.
>
> Best,
> Leonard
>
>
> > On Jul 25, 2023, at 3:30 AM, Alexander Fedulov <
> alexander.fedu...@gmail.com> wrote:
> >
> > Hello everyone,
> >
> > Recently, there were a lot of discussions about the deprecation of
> various
> > APIs for the upcoming 2.0 release. It appears there are two main
> motivations
> > with opposing directions, causing these discussions to remain unsettled.
> On
> > one hand, there's a desire to finally trim a wide range of legacy APIs,
> some
> > lingering around since the beginning of the 1.x release line (as far
> back as
> > 2016). On the other hand, there is a commitment to uphold our guarantees
> to
> > the users, ensuring a smooth transition.
> >
> > I believe we could reconcile these two motivations. My proposition is to
> > designate the final release of the 1.x timeline as a Long-Term Support
> (LTS)
> > release. By doing so, we would:
> >
> > 1. Enable more efficient cleanup and be liberated to introduce more
> breaking
> >   changes, paving the way for greater innovation in the 2.0 release.
> > 2. Sustain a positive user experience by granting enough time for the
> > changes
> >   introduced in 2.0 to stabilize, allowing users to confidently
> transition
> >   their production code to the new release.
> >
> > I look forward to hearing your thoughts on this proposal.
> >
> > Best Regards,
> > Alex
>
>

Re: [DISCUSS][2.0] FLIP-350: Remove query parameters from Jar handlers

[Xintong Song]

Same as I proposed in the FLIP-352 thread, the phasing out of the query
parameters may also benefit from a REST API version bump.

Best,

Xintong



On Mon, Jul 24, 2023 at 6:43 PM Chesnay Schepler  wrote:

> The jar handlers currently accept a variety of parameters both as query
> parameters and via the request body.
>
> While it is primarily a problem for sending program args as query
> parameters, because it's a nightmare with escaping parameters, the
> remaining parameters should follow suite for consistency.
>
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-350%3A+Remove+query+parameters+from+Jar+handlers
>
>

Re: [DISCUSS][2.0] FLIP-349: Move RocksDB statebackend classes to o.a.f.state.rocksdb package

[Xintong Song]

+1

Best,

Xintong



On Mon, Jul 24, 2023 at 6:25 PM Chesnay Schepler  wrote:

> To properly reflect the state of the rocksdb statebackend I propose to
> move all classes in the state-backend-rocksdb module under the classes
> to o.a.f.state.rocksdb package.
>
>
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-349%3A+Move+RocksDB+statebackend+classes+to+o.a.f.state.rocksdb+package
>
>

Re:Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Mang Zhang]

Congratulations, Yong Fang!


--

Best regards,
Mang Zhang





在 2023-07-25 10:30:24，"Jark Wu"  写道：
>Congratulations, Yong Fang!
>
>Best,
>Jark
>
>On Mon, 24 Jul 2023 at 22:11, Wencong Liu  wrote:
>
>> Congratulations!
>>
>> Best,
>> Wencong Liu
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> 在 2023-07-24 11:03:30，"Paul Lam"  写道：
>> >Congrats, Shammon!
>> >
>> >Best,
>> >Paul Lam
>> >
>> >> 2023年7月24日 10:56，Jingsong Li  写道：
>> >>
>> >> Shammon
>> >
>>

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

[Leonard Xu]

+1, it’s pretty necessary especially we deprecated so many APIs in 1.18 and 
plan to remove in 2.0.

The 1.19 should be a proper version for LTS Release.

Best,
Leonard


> On Jul 25, 2023, at 3:30 AM, Alexander Fedulov  
> wrote:
> 
> Hello everyone,
> 
> Recently, there were a lot of discussions about the deprecation of various
> APIs for the upcoming 2.0 release. It appears there are two main motivations
> with opposing directions, causing these discussions to remain unsettled. On
> one hand, there's a desire to finally trim a wide range of legacy APIs, some
> lingering around since the beginning of the 1.x release line (as far back as
> 2016). On the other hand, there is a commitment to uphold our guarantees to
> the users, ensuring a smooth transition.
> 
> I believe we could reconcile these two motivations. My proposition is to
> designate the final release of the 1.x timeline as a Long-Term Support (LTS)
> release. By doing so, we would:
> 
> 1. Enable more efficient cleanup and be liberated to introduce more breaking
>   changes, paving the way for greater innovation in the 2.0 release.
> 2. Sustain a positive user experience by granting enough time for the
> changes
>   introduced in 2.0 to stabilize, allowing users to confidently transition
>   their production code to the new release.
> 
> I look forward to hearing your thoughts on this proposal.
> 
> Best Regards,
> Alex

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

[Dong Lin]

Hi David,

Thank you for the detailed comments and the suggestion of this alternative
approach.

I agree with you that this alternative can also address the target use-case
with the same correctness. In comparison to the current FLIP, this
alternative indeed introduces much less complexity to the Flink runtime
internal implementation.

At a high level, this alternative is simulating a one-time emission of
Watermark(useProcessingTime=true) with periodic emission of
Watermark(timestamp=wall-lock-time).

One downside of this alternative is that it can introduce a bit of extra
per-record runtime overhead. This is because the ingestion time watermark
will be emitted periodically according to pipeline.auto-watermark-interval
(200 ms by default). Thus there is still a short period where the watermark
from the HybridSource can be lagging behind wall-clock time. For operators
whose logic depends on the watermark, such as TemporalRowTimeJoinOperator,
they will need to check build-side watermark and delay/buffer records on
the probe-side until it receives the next ingestion-time watermark.

The impact of this overhead probably depends on the throughput/watermark of
the probe-side records. On the other hand, given that join operator is
typically already heavy (due to state backend access and build-side
buffer), and the watermark from probe-side (e.g. Kafka) is probably also
lagging behind wall-clock time, it is probably not an issue in most cases.
Therefore I agree that it is worth trying this approach. We can revisit
this issue if we any issues around performance or usability of this
approach.

Another potential concern is that it requires the user to use ingestion
time. I am not sure we are able to do this in a backward-compatible way
yet. We probably need to go through the existing APIs around ingestion time
watermark to validate this.

BTW, with the introduction of RecordAttributes(isBacklog=true/false) from
FLIP-327
,
another short-term approach is to let TemporalProcessTimeJoinOperator keep
buffering records from MySQL/HybridSource as long as isBacklog=true, and
process them in a processing-time manner once it receives isBacklog=false.
This should also address the use-case targeted by FLIP-326. The only caveat
with this approach is that it is a bit hacky, because it requires
JoinOpertor to always buffer records when isBacklog=true, whereas
isBacklog's semantics only says it is "optional" to buffer records, which
can be an issue in the long term.

Thanks,
Dong

On Tue, Jul 25, 2023 at 2:37 AM David Anderson  wrote:

> I'm delighted to see interest in developing support for
> processing-time temporal joins.
>
> The proposed implementation seems rather complex, and I'm not
> convinced this complexity is justified/necessary. I'd like to outline
> a simpler alternative that I think would satisfy the key objectives.
>
> Key ideas:
>
> 1. Limit support to the HybridSource (or a derivative thereof). (E.g.,
> I'm guessing the MySQL CDC Source could be reworked to be a hybrid
> source.)
> 2. Have this HybridSource wait to begin emitting watermarks until it
> has handled all events from the bounded sources. (I'm not sure how the
> HybridSource handles this now; if this is an incompatible change, we
> can find a way to deal with that.)
> 3. Instruct users to use an ingestion time watermarking strategy for
> their unbounded source (the source the HybridSource handles last) if
> they want to do something like a processing time temporal join.
>
> One objection to this is the limitation of only supporting the
> HybridSource -- what about cases where the user has a single source,
> e.g., a Kafka topic? I'm suggesting the user would divide their
> build-side stream into two parts -- a bounded component that is fully
> ingested by the hybrid source before watermarking begins, followed by
> an unbounded component.
>
> I think this alternative handles use cases like processing-time
> temporal join rather nicely, without requiring any changes to
> watermarks or the core runtime.
>
> David
>
> On Thu, Jun 29, 2023 at 1:39 AM Martijn Visser 
> wrote:
> >
> > Hi Dong and Xuannan,
> >
> > I'm excited to see this FLIP. I think support for processing-time
> > temporal joins is something that the Flink users will greatly benefit
> > off. I specifically want to call-out that it's great to see the use
> > cases that this enables. From a technical implementation perspective,
> > I defer to the opinion of others with expertise on this topic.
> >
> > Best regards,
> >
> > Martijn
> >
> > On Sun, Jun 25, 2023 at 9:03 AM Xuannan Su 
> wrote:
> > >
> > > Hi all,
> > >
> > > Dong(cc'ed) and I are opening this thread to discuss our proposal to
> > > enhance the watermark to properly support processing-time temporal
> > > join, which has been documented in FLIP-326 [1].
> > >
> > > We want to 

Re: [DISCUSS] Add missing visibility annotation for Table APIs

[Lincoln Lee]

Hi Jane,

Thanks for driving this! Overall the proposed annotations looks good to me.
Some comments for the table[1]:

For the `DynamicFilteringEvent`, tend to keep it 'internal' since it's a
concreate implementation of `SourceEvent`(and other two implementers are
not public ones) .

For the `LookupOptions` and `Trigger`s, because they're all in the public
interfaces of the flip[2], I'm fine with making them all public ones or
just excluding the Trigger implemantors, cc @Qingsheng can you also help to
check this?

For the `BuiltInFunctionDefinitions$Builder`, I think it should be
`BuiltInFunctionDefinition$Builder`.


[1].
https://docs.google.com/spreadsheets/d/1e8M0tUtKkZXEd8rCZtZ0C6Ty9QkNaPySsrCgz0vEID4/edit?usp=sharing
[2].
https://cwiki.apache.org/confluence/display/FLINK/FLIP-221%3A+Abstraction+for+lookup+source+cache+and+metric#FLIP221:Abstractionforlookupsourcecacheandmetric-PublicInterfaces

Best,
Lincoln Lee

Jark Wu  于2023年7月25日周二 10:47写道：

> Hi Jane,
>
> Thanks for kicking off this work and collecting the detailed list.
>
> +1 to add the missing annotation.
>
> This often confuses me whether the class can be modified without breaking
> the compatibility
>  when looking at classes in table-common and table-api. Explicitly mark the
> visibility can be
> helpful in this case.
>
> I have some additional suggestions wrt the class annotations:
> - classes in org.apache.flink.table.catalog.stats.* can be @PublicEvolving,
> because all the classes in there are needed to build the @PublicEvolving
>  CatalogColumnStatistics.
> - PeriodicCacheReloadTrigger and TimedCacheReloadTrigger can be
> @PublicEvolving,
> they are built-in implementations of cache reload trigger and are exposed
> to connectors.
> - CoreModule can be @PublicEvolving to allow users use it in
> TableEnv#loadModule(name, Module).
>
> Best,
> Jark
>
>
> On Fri, 21 Jul 2023 at 00:34, Jane Chan  wrote:
>
> > Hi, Devs,
> >
> > I would like to start a discussion on adding missing visibility
> annotation
> > (PublicEvolving / Internal etc.) for Table APIs.
> >
> > The motivation for starting this discussion was during the cleanup of
> which
> > Table API to be deprecated for version 2.0, I noticed that some of the
> APIs
> > lack visibility annotations, which may lead to users relying on APIs that
> > should have been marked as internal.
> >
> > Therefore, I have compiled a sheet[1] listing the currently unmarked
> > classes under the table-api-java, table-api-java-bridge, and table-common
> > modules and the recommended annotations to be added.
> >
> > My thought is that all public classes/interfaces within the three modules
> > mentioned above should be explicitly marked, and we might consider
> > introducing a new architectural rule to perform auto-check on newly added
> > APIs in the future.
> >
> > Let me explain the details.
> >
> > 1. Why table-api-java, table-api-java-bridge, and table-common?
> > Because according to Flink's application project configuration doc[2],
> > table-api-java and table-api-java-bridge are the leading dependencies for
> > users to develop a table program. Although flink-table-common is not
> > listed, it is the core dependency for users to implement a User-Defined
> > Function/Connector[3].
> >
> > 2. How are the classes listed in this table compiled?
> > I use a customized Intellij profile to perform the code inspection under
> > these three modules to find all public classes/interfaces without API
> > visibility annotations, along with a manual check.
> >
> > 3. How is the suggested API visibility to be determined?
> > For all APIs suggested as PublicEvolving, I added a comment on the cell
> to
> > explain the reason. The rest APIs, which are indicated as Internal, are
> > either util classes or implementations.
> >
> > I'm looking forward to your ideas, and it would be great if any
> interested
> > developers could help review this list together.
> >
> >
> > [1]
> >
> >
> https://docs.google.com/spreadsheets/d/1e8M0tUtKkZXEd8rCZtZ0C6Ty9QkNaPySsrCgz0vEID4/edit?usp=sharing
> > [2]
> >
> >
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/configuration/overview/#which-dependencies-do-you-need
> > [3]
> >
> >
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sourcessinks/#project-configuration
> >
> >
> > Best regards,
> > Jane
> >
>

Re: [DISCUSS] Add missing visibility annotation for Table APIs

[Jark Wu]

Hi Jane,

Thanks for kicking off this work and collecting the detailed list.

+1 to add the missing annotation.

This often confuses me whether the class can be modified without breaking
the compatibility
 when looking at classes in table-common and table-api. Explicitly mark the
visibility can be
helpful in this case.

I have some additional suggestions wrt the class annotations:
- classes in org.apache.flink.table.catalog.stats.* can be @PublicEvolving,
because all the classes in there are needed to build the @PublicEvolving
 CatalogColumnStatistics.
- PeriodicCacheReloadTrigger and TimedCacheReloadTrigger can be
@PublicEvolving,
they are built-in implementations of cache reload trigger and are exposed
to connectors.
- CoreModule can be @PublicEvolving to allow users use it in
TableEnv#loadModule(name, Module).

Best,
Jark


On Fri, 21 Jul 2023 at 00:34, Jane Chan  wrote:

> Hi, Devs,
>
> I would like to start a discussion on adding missing visibility annotation
> (PublicEvolving / Internal etc.) for Table APIs.
>
> The motivation for starting this discussion was during the cleanup of which
> Table API to be deprecated for version 2.0, I noticed that some of the APIs
> lack visibility annotations, which may lead to users relying on APIs that
> should have been marked as internal.
>
> Therefore, I have compiled a sheet[1] listing the currently unmarked
> classes under the table-api-java, table-api-java-bridge, and table-common
> modules and the recommended annotations to be added.
>
> My thought is that all public classes/interfaces within the three modules
> mentioned above should be explicitly marked, and we might consider
> introducing a new architectural rule to perform auto-check on newly added
> APIs in the future.
>
> Let me explain the details.
>
> 1. Why table-api-java, table-api-java-bridge, and table-common?
> Because according to Flink's application project configuration doc[2],
> table-api-java and table-api-java-bridge are the leading dependencies for
> users to develop a table program. Although flink-table-common is not
> listed, it is the core dependency for users to implement a User-Defined
> Function/Connector[3].
>
> 2. How are the classes listed in this table compiled?
> I use a customized Intellij profile to perform the code inspection under
> these three modules to find all public classes/interfaces without API
> visibility annotations, along with a manual check.
>
> 3. How is the suggested API visibility to be determined?
> For all APIs suggested as PublicEvolving, I added a comment on the cell to
> explain the reason. The rest APIs, which are indicated as Internal, are
> either util classes or implementations.
>
> I'm looking forward to your ideas, and it would be great if any interested
> developers could help review this list together.
>
>
> [1]
>
> https://docs.google.com/spreadsheets/d/1e8M0tUtKkZXEd8rCZtZ0C6Ty9QkNaPySsrCgz0vEID4/edit?usp=sharing
> [2]
>
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/configuration/overview/#which-dependencies-do-you-need
> [3]
>
> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sourcessinks/#project-configuration
>
>
> Best regards,
> Jane
>

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

[Xintong Song]

>
> we should treat these cases as errors
>
Looking at the fields listed in the FLIP, I'd agree with this argument. And
based on this, shouldn't we fail the request with e.g., a status code 500,
rather than responding with a fallback value silently?

Best,

Xintong



On Tue, Jul 25, 2023 at 12:22 AM Jing Ge  wrote:

> We might consider using 0 as null for values that never return 0. But null
> is still not equal to 0 and it will be very difficult to let every
> contributor in this community follow this rule, especially for future
> unknown APIs, which means there will be some cases that still need null.
> Personally, I would choose accuracy over convenience and consistency over
> convenience. Therefore, null is recommended.
>
> Best regards,
> Jing
>
> On Mon, Jul 24, 2023 at 11:48 PM Chesnay Schepler 
> wrote:
>
> > The downside to null is that it again forces users to handle this case
> > themselves.
> >
> > The reality is that there is no good default value.
> >
> > Ideally we fix all cases where we return such values, such that the
> > fallback to 0 isn't even used.
> > Arguably the same could be said for null, but I'd think that 0 is less
> > of a surprise.
> >
> > On 24/07/2023 17:21, Gyula Fóra wrote:
> > > I agree that it's a bit strange to have 0 as a fallback value because
> it
> > > can also naturally occur for many metrics.
> > > If we want to omit the value null would be probably better as Matthias
> > > suggested.
> > >
> > > Gyula
> > >
> > > On Mon, Jul 24, 2023 at 4:02 PM Matthias Pohl
> > >  wrote:
> > >
> > >> What was the reason you decided to go for 0 as the fallback value
> > instead
> > >> of null? Wouldn't that be a more reasonable value for error cases?
> > >>
> > >> On Mon, Jul 24, 2023 at 12:51 PM Chesnay Schepler  >
> > >> wrote:
> > >>
> > >>> There are a number of cases where the REST API can return infinity or
> > >>> NaN for certain double fields.
> > >>>
> > >>> This is problematic because the JSON spec does not allow such values,
> > >>> and tooling working against that spec may run into issues when
> > >>> encountering such a value.
> > >>>
> > >>> Specifically we've seen this become an issue in clients generated
> from
> > >>> the OpenAPI spec.
> > >>>
> > >>>
> > >>>
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425797
> >
> >
>

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Jark Wu]

Congratulations, Yong Fang!

Best,
Jark

On Mon, 24 Jul 2023 at 22:11, Wencong Liu  wrote:

> Congratulations!
>
> Best,
> Wencong Liu
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> 在 2023-07-24 11:03:30，"Paul Lam"  写道：
> >Congrats, Shammon!
> >
> >Best,
> >Paul Lam
> >
> >> 2023年7月24日 10:56，Jingsong Li  写道：
> >>
> >> Shammon
> >
>

Re: [DISCUSS][2.0] FLIP-352: Use camelCast for all REST API fields/parameters

[Xintong Song]

I think adding the convention to our code-style guidelines [1] makes sense.
In addition, I wonder if this can be enforced with some code-style checking
tools.

What impact (if any) will there be on existing users?

Virtually every tooling against the REST API will break. Users that
> generated a client from the OpenAPI spec should only have to regenerate the
> client.



> If we are changing behavior how will we phase out the older behavior?

It is theoretically possible to duplicate all fields/parameters in the 1.x
> APIs, but it would double the serialization overhead and invite a new
> classes of bugs (== both fields being in the request / consistency errors
> between supposedly identical fields).
>

In this case, I wonder if it makes sense to bump the REST API version to
V2. That should allow us to gradually phase out the old APIs without the
above mentioned problems.

@Jing
You can find how many REST API fields / parameters using hyphens in the
OpenAPI yaml [2].

Best,

Xintong


[1]
https://flink.apache.org/how-to-contribute/code-style-and-quality-preamble/
[2]
https://github.com/apache/flink/blob/master/docs/static/generated/rest_v1_dispatcher.yml

On Mon, Jul 24, 2023 at 11:56 PM Jing Ge  wrote:

> camel case is the standard code convention in Java since day one[1]. We
> don't need to describe it again for Flink. Legacy hyphens might be coded by
> contributors with other language backgrounds. If we really care about it,
> it might make sense to add the reference.
>
> It would be great if the FLIP could show one hyphen example, so that we
> will know what the issue exactly is and how many classes are involved in.
>
> Best regards,
> Jing
>
> [1]
>
> https://www.oracle.com/java/technologies/javase/codeconventions-namingconventions.html
>
> On Mon, Jul 24, 2023 at 9:09 PM Matthias Pohl
>  wrote:
>
> > Do we have conventions on how the REST API should be formatted written
> down
> > somewhere?
> > Checking our coding conventions [1] didn't reveal anything.
> >
> > We could refer to a style guide like [2] to cover this topic. That would
> > also include the camel-cased JSON fields.
> >
> > [1]
> >
> https://flink.apache.org/how-to-contribute/code-style-and-quality-preamble/
> > [2]
> >
> >
> https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format
> >
> > On Mon, Jul 24, 2023 at 1:03 PM Chesnay Schepler 
> > wrote:
> >
> > > Make the REST API more consistent and easier to work with from the UI.
> > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425799
> > >
> >
>

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Zhu Zhu]

+1 (binding)

Thanks,
Zhu

Xintong Song  于2023年7月25日周二 09:46写道：
>
> +1 (binding)
>
> Best,
>
> Xintong
>
>
>
> On Mon, Jul 24, 2023 at 11:58 PM Jing Ge  wrote:
>
> > +1
> >
> > On Mon, Jul 24, 2023 at 11:43 PM Chesnay Schepler 
> > wrote:
> >
> > > I update the endpoint in the FLIP.
> > >
> > > On 24/07/2023 14:28, Matthias Pohl wrote:
> > > > I should have mentioned it in the discussion thread but I missed going
> > > over
> > > > that ML thread earlier: We might want to update the FLIP to refer to
> > the
> > > > actual endpoint /jobs/:jobid/rescaling (AFAIU) with the corresponding
> > > cause
> > > > being FLINK-12312 [1].
> > > >
> > > > But that's just a minor thing.
> > > >
> > > > +1 (binding)
> > > >
> > > > [1] https://issues.apache.org/jira/browse/FLINK-12312
> > > >
> > > > On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf 
> > > wrote:
> > > >
> > > >> +1 (binding)
> > > >>
> > > >> Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> > > >> martijnvis...@apache.org>:
> > > >>
> > > >>> +1 (binding)
> > > >>>
> > > >>> On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler  > >
> > > >>> wrote:
> > > >>>
> > >  Hello,
> > > 
> > >  I'd like to start a vote on FLIP-340.
> > > 
> > >  Discussion thread:
> > >  https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
> > >  FLIP:
> > > 
> > > 
> > > >>
> > >
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
> > >  Regards,
> > >  Chesnay
> > > 
> > > >>
> > > >> --
> > > >> https://twitter.com/snntrable
> > > >> https://github.com/knaufk
> > > >>
> > >
> > >
> >

Re: [VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

[Xintong Song]

+1 (binding)

Best,

Xintong



On Mon, Jul 24, 2023 at 11:26 PM Jing Ge  wrote:

> +1(binding)
>
> On Mon, Jul 24, 2023 at 8:55 PM Matthias Pohl
>  wrote:
>
> > +1 (binding)
> >
> > On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf 
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> > > martijnvis...@apache.org>:
> > >
> > > > +1 (binding)
> > > >
> > > > On Mon, Jul 24, 2023 at 1:08 PM Chesnay Schepler  >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I'd like to start a vote on FLIP-336.
> > > > >
> > > > > Discussion thread:
> > > > > https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
> > > > > FLIP:
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789
> > > > >
> > > > > Regards,
> > > > > Chesnay
> > > > >
> > > >
> > >
> > >
> > > --
> > > https://twitter.com/snntrable
> > > https://github.com/knaufk
> > >
> >
>

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Xintong Song]

+1 (binding)

Best,

Xintong



On Mon, Jul 24, 2023 at 11:58 PM Jing Ge  wrote:

> +1
>
> On Mon, Jul 24, 2023 at 11:43 PM Chesnay Schepler 
> wrote:
>
> > I update the endpoint in the FLIP.
> >
> > On 24/07/2023 14:28, Matthias Pohl wrote:
> > > I should have mentioned it in the discussion thread but I missed going
> > over
> > > that ML thread earlier: We might want to update the FLIP to refer to
> the
> > > actual endpoint /jobs/:jobid/rescaling (AFAIU) with the corresponding
> > cause
> > > being FLINK-12312 [1].
> > >
> > > But that's just a minor thing.
> > >
> > > +1 (binding)
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-12312
> > >
> > > On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf 
> > wrote:
> > >
> > >> +1 (binding)
> > >>
> > >> Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> > >> martijnvis...@apache.org>:
> > >>
> > >>> +1 (binding)
> > >>>
> > >>> On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler  >
> > >>> wrote:
> > >>>
> >  Hello,
> > 
> >  I'd like to start a vote on FLIP-340.
> > 
> >  Discussion thread:
> >  https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
> >  FLIP:
> > 
> > 
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
> >  Regards,
> >  Chesnay
> > 
> > >>
> > >> --
> > >> https://twitter.com/snntrable
> > >> https://github.com/knaufk
> > >>
> >
> >
>

[jira] [Created] (FLINK-32664) TableSourceJsonPlanTest.testReuseSourceWithoutProjectionPushDown is failing

[Sergey Nuyanzin (Jira)]

Sergey Nuyanzin created FLINK-32664:
---

 Summary: 
TableSourceJsonPlanTest.testReuseSourceWithoutProjectionPushDown is failing
 Key: FLINK-32664
 URL: https://issues.apache.org/jira/browse/FLINK-32664
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Planner
Affects Versions: 1.18.0
Reporter: Sergey Nuyanzin
Assignee: Sergey Nuyanzin


Blocker since it's failing on every build and reproduced locally
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51661=logs=0c940707-2659-5648-cbe6-a1ad63045f0a=075c2716-8010-5565-fe08-3c4bb45824a4=11529



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-32663) RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList fails on AZP

[Sergey Nuyanzin (Jira)]

Sergey Nuyanzin created FLINK-32663:
---

 Summary: 
RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList fails on 
AZP
 Key: FLINK-32663
 URL: https://issues.apache.org/jira/browse/FLINK-32663
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Coordination
Affects Versions: 1.18.0
Reporter: Sergey Nuyanzin


This build 
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51501=logs=8fd9202e-fd17-5b26-353c-ac1ff76c8f28=ea7cf968-e585-52cb-e0fc-f48de023a7ca=8665
fails as
{noformat}
Jul 21 01:24:54 01:24:54.146 [ERROR] 
RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList  Time 
elapsed: 1.485 s  <<< FAILURE!
Jul 21 01:24:54 java.lang.AssertionError: expected:<530> but was:<30>
Jul 21 01:24:54 at org.junit.Assert.fail(Assert.java:89)
Jul 21 01:24:54 at org.junit.Assert.failNotEquals(Assert.java:835)
Jul 21 01:24:54 at org.junit.Assert.assertEquals(Assert.java:647)
Jul 21 01:24:54 at org.junit.Assert.assertEquals(Assert.java:633)
Jul 21 01:24:54 at 
org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingPartitionedOperatorState(RescalingITCase.java:621)
Jul 21 01:24:54 at 
org.apache.flink.test.checkpointing.RescalingITCase.testSavepointRescalingInPartitionedOperatorStateList(RescalingITCase.java:508)
Jul 21 01:24:54 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
Jul 21 01:24:54 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jul 21 01:24:54 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:4
...
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-32662) JobMasterTest.testRetrievingCheckpointStats fails with NPE on AZP

[Sergey Nuyanzin (Jira)]

Sergey Nuyanzin created FLINK-32662:
---

 Summary: JobMasterTest.testRetrievingCheckpointStats fails with 
NPE on AZP
 Key: FLINK-32662
 URL: https://issues.apache.org/jira/browse/FLINK-32662
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Coordination
Affects Versions: 1.18.0
Reporter: Sergey Nuyanzin


This build 
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51452=logs=0e7be18f-84f2-53f0-a32d-4a5e4a174679=7c1d86e3-35bd-5fd5-3b7c-30c126a78702=8654
fails with NPE as
{noformat}
Jul 20 01:01:33 01:01:33.491 [ERROR] 
org.apache.flink.runtime.jobmaster.JobMasterTest.testRetrievingCheckpointStats  
Time elapsed: 0.036 s  <<< ERROR!
Jul 20 01:01:33 java.lang.NullPointerException
Jul 20 01:01:33 at 
org.apache.flink.runtime.jobmaster.JobMasterTest.testRetrievingCheckpointStats(JobMasterTest.java:2132)
Jul 20 01:01:33 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
Jul 20 01:01:33 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jul 20 01:01:33 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jul 20 01:01:33 at java.lang.reflect.Method.invoke(Method.java:498)
Jul 20 01:01:33 at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
Jul 20 01:01:33 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
...
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-32661) OperationRelatedITCase.testOperationRelatedApis fails on AZP

[Sergey Nuyanzin (Jira)]

Sergey Nuyanzin created FLINK-32661:
---

 Summary: OperationRelatedITCase.testOperationRelatedApis fails on 
AZP
 Key: FLINK-32661
 URL: https://issues.apache.org/jira/browse/FLINK-32661
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Gateway
Affects Versions: 1.18.0
Reporter: Sergey Nuyanzin


This build 
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=51452=logs=a9db68b9-a7e0-54b6-0f98-010e0aff39e2=cdd32e0b-6047-565b-c58f-14054472f1be=12114
fails as 
{noformat}
Jul 20 04:23:49 org.opentest4j.AssertionFailedError: 
Jul 20 04:23:49 
Jul 20 04:23:49 Expecting actual's toString() to return:
Jul 20 04:23:49   "PENDING"
Jul 20 04:23:49 but was:
Jul 20 04:23:49   "RUNNING"
Jul 20 04:23:49 at 
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
Jul 20 04:23:49 at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
Jul 20 04:23:49 at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
Jul 20 04:23:49 at 
org.apache.flink.table.gateway.rest.OperationRelatedITCase.testOperationRelatedApis(OperationRelatedITCase.java:91)
Jul 20 04:23:49 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
Jul 20 04:23:49 at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Jul 20 04:23:49 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Jul 20 04:23:49 at java.lang.reflect.Method.invoke(Method.java:498)
Jul 20 04:23:49 at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
Jul 20 04:23:49 at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:21
...
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[DISCUSS] Proposing an LTS Release for the 1.x Line

[Alexander Fedulov]

Hello everyone,

Recently, there were a lot of discussions about the deprecation of various
APIs for the upcoming 2.0 release. It appears there are two main motivations
with opposing directions, causing these discussions to remain unsettled. On
one hand, there's a desire to finally trim a wide range of legacy APIs, some
lingering around since the beginning of the 1.x release line (as far back as
2016). On the other hand, there is a commitment to uphold our guarantees to
the users, ensuring a smooth transition.

I believe we could reconcile these two motivations. My proposition is to
designate the final release of the 1.x timeline as a Long-Term Support (LTS)
release. By doing so, we would:

1. Enable more efficient cleanup and be liberated to introduce more breaking
   changes, paving the way for greater innovation in the 2.0 release.
2. Sustain a positive user experience by granting enough time for the
changes
   introduced in 2.0 to stabilize, allowing users to confidently transition
   their production code to the new release.

I look forward to hearing your thoughts on this proposal.

Best Regards,
Alex

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

[David Anderson]

I'm delighted to see interest in developing support for
processing-time temporal joins.

The proposed implementation seems rather complex, and I'm not
convinced this complexity is justified/necessary. I'd like to outline
a simpler alternative that I think would satisfy the key objectives.

Key ideas:

1. Limit support to the HybridSource (or a derivative thereof). (E.g.,
I'm guessing the MySQL CDC Source could be reworked to be a hybrid
source.)
2. Have this HybridSource wait to begin emitting watermarks until it
has handled all events from the bounded sources. (I'm not sure how the
HybridSource handles this now; if this is an incompatible change, we
can find a way to deal with that.)
3. Instruct users to use an ingestion time watermarking strategy for
their unbounded source (the source the HybridSource handles last) if
they want to do something like a processing time temporal join.

One objection to this is the limitation of only supporting the
HybridSource -- what about cases where the user has a single source,
e.g., a Kafka topic? I'm suggesting the user would divide their
build-side stream into two parts -- a bounded component that is fully
ingested by the hybrid source before watermarking begins, followed by
an unbounded component.

I think this alternative handles use cases like processing-time
temporal join rather nicely, without requiring any changes to
watermarks or the core runtime.

David

On Thu, Jun 29, 2023 at 1:39 AM Martijn Visser  wrote:
>
> Hi Dong and Xuannan,
>
> I'm excited to see this FLIP. I think support for processing-time
> temporal joins is something that the Flink users will greatly benefit
> off. I specifically want to call-out that it's great to see the use
> cases that this enables. From a technical implementation perspective,
> I defer to the opinion of others with expertise on this topic.
>
> Best regards,
>
> Martijn
>
> On Sun, Jun 25, 2023 at 9:03 AM Xuannan Su  wrote:
> >
> > Hi all,
> >
> > Dong(cc'ed) and I are opening this thread to discuss our proposal to
> > enhance the watermark to properly support processing-time temporal
> > join, which has been documented in FLIP-326 [1].
> >
> > We want to support the use case where the records from the probe side
> > of the processing-time temporal join need to wait until the build side
> > finishes the snapshot phrase by enhancing the expressiveness of the
> > Watermark. Additionally, these changes lay the groundwork for
> > simplifying the DataStream APIs, eliminating the need for users to
> > explicitly differentiate between event-time and processing-time,
> > resulting in a more intuitive user experience.
> >
> > Please refer to the FLIP document for more details about the proposed
> > design and implementation. We welcome any feedback and opinions on
> > this proposal.
> >
> > Best regards,
> >
> > Dong and Xuannan
> >
> > [1] 
> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-326%3A+Enhance+Watermark+to+Support+Processing-Time+Temporal+Join

Re: Support read ROWKIND metadata by ROW_KIND() function

[Feng Jin]

Hi casel,

Thank you for initiating the discussion about RowKind meta. I believe that
in certain scenarios, it is necessary to expose RowKind. We also have
similar situations internally:

In simple terms, we need to be able to control the behavior of RowKind in
both Source and Sink:
- When reading data from the Source, filter out unnecessary delete records.
- When writing data to the Sink, filter out unnecessary delete records.


Firstly, I don't think it is appropriate to expose RowKind at the Flink SQL
framework level as it may lead to ambiguity in certain semantic scenarios.
Therefore, I am more inclined towards perceiving specific information at
the format layer.

Here is my preliminary proposal:
Introduce a proxy format where we can freely control which RowKinds are
needed and which ones are not. At the same time, we can also expose RowKind.

1. For your scenario one, using a proxy format allows us to do this:

```

CREATE TABLE kafka_source (
f1 VARCHAR,
f2 VARCHAR
) with (
'connector' = 'kafka',
'format' = 'proxy',
'proxy.format' = 'canal-json',
'proxy.filter' = 'DELETE,UPDATE_BEFORE'
);


CREATE TABLE kafka_sink (
f1 VARCHAR,
f2 VARCHAR
) with (
'connector' = 'kafka',
'format' = 'json'
);


INSERT INTO kafka_sink select * from kafka_source;

```

By using a proxy, we can filter out the unwanted RowKind data types and
transform the original RetractStream into an AppendOnly Stream.


2. For scenario 2, we can  also use the proxy-format to achieve this.

```

CREATE TABLE kafka_source (
f1 VARCHAR,
f2 VARCHAR
) with (
'connector' = 'kafka',
'format' = 'canal-json'
);


CREATE TABLE kafka_sink (
f1 VARCHAR,
f2 VARCHAR
) with (
'connector' = 'kafka',
'format' = 'proxy',
'proxy.format' = 'json',
'proxy.filter' = 'DELETE,UPDATE_BEFORE'
);


INSERT INTO kafka_sink select * from kafka_source;

```

Through the proxy format, the original appendOnly sink is enabled to
support sink retract stream.


This is my preliminary idea. There are probably many more details to
consider, but the overall concept is to use proxy format to implement some
logic that we want to achieve without affecting the original format.


Best,
Feng


On Wed, Jul 19, 2023 at 10:06 PM casel.chen  wrote:

> CDC format like debezium-json and canal-json support read ROWKIND metadata.
>
>
> 1. Our first scenario is syncing data of operational tables into our
> streaming warehouse.
> All operational data in mysql should NOT be physically deleted, so we use
> "is_deleted" column to do logical delete, and there should NOT be any
> delete operations happen on our streaming warehouse.
> But as data grows up quickly we need to delete old data such as half year
> ago in operational table to keep table size manageable and ensure the query
> performance not to be decreased. These deleted records for maintain purpose
> should be not synced into our streaming warehouse. So we have to filter our
> them in our flink sql jobs. But currently it is not convenient to do
> ROWKIND filtering. That is why I ask flink support read ROWKIND metadata by
> ROW_KIND() function. Then we can use the following flink sql to do
> filtering. For example:
>
> create table customer_source (
>   id BIGINT PRIMARY KEY NOT ENFORCED,
>   name STRING,
>   region STRING
>) with (
>   'connector' = 'kafka',
>   'format' = 'canal-json',
>   ...
>);
>
>
>create table customer_sink (
>   id BIGINT PRIMARY KEY NOT ENFORCED,
>   name STRING,
>   region STRING
>) with (
>   'connector' = 'paimon'
>   ...
>);
>
>
>   INSERT INTO customer_sink SELECT * FROM customer_source WHERE ROW_KIND()
> <> '-D';
>
>
> 2. Out secondary scenario is we need sink aggregation result into MQ which
> does NOT support retract data. Although flink provide upsert kafka
> connector, but unfortunetly our sink system is NOT kafka, so we have to
> write customized connector like upsert-kafka again. If flink sql support
> filter data by ROWKIND, we don't need write any more upsert-xxx connector.
> For example,
>
>create table customer_source (
>   id BIGINT PRIMARY KEY NOT ENFORCED,
>   name STRING,
>   region STRING
>) with (
>   'connector' = 'kafka',
>   'format' = 'canal-json',
>   ...
>);
>
>
>create table customer_agg_sink (
>   region STRING,
>   cust_count BIGINT
>) with (
>   'connector' = 'MQ',
>   'format' = 'json',
>   ...
>);
>
>
>INSERT INTO customer_agg_sink SELECT * FROM (SELECT region, count(1) as
> cust_count  from customer_source group by region) t WHERE ROW_KIND() <>
> '-U' AND ROW_KIND() <> '-D';
>
>
> How do you think? Looking forward to your feedback, thanks!

Re: Scaling Flink Jobs without Restarting Job

[Talat Uyarer via dev]

Hi Chen,

Thanks for the reply. Yes we have a very low interval. But our jobs usually
have 10-20 independent Kafka topics. When we perform scaling we stop
reading from all. Do you think we can implement partial failover to
calculate Kafka Partition assignments? Or rather than sending an
interrupted signal. Is there any better way to stop Task after it finished
its work to prevent data duplication ?



On Sun, Jul 23, 2023 at 10:44 PM Chen Zhanghao 
wrote:

> Hi Talat,
>
> In reactive mode, rescaling is performed by a whole-graph failover, which
> is already less costly compared to a full job restart where all containers
> need to be requested again. For simple stateless jobs, this usually won't
> take long (a few seconds), you can measure how long it takes for all tasks
> returning to RUNNING status during a rescaling. Fluctuating traffic might
> be caused by re-consuming some data when recovering from a previous
> checkpoint. In this case, reducing the checkpoint interval will help.
>
> As regards to partial failover for rescaling, it might be challenging.
> Rescaling stateless job will still involve redistribution of Kafka
> partitions (for Kafka sources for example) and requires some coordination
> works.
>
> Best,
> Zhanghao Chen
> --
> *发件人:* Talat Uyarer via dev 
> *发送时间:* 2023年7月23日 15:28
> *收件人:* dev 
> *主题:* Scaling Flink Jobs without Restarting Job
>
> HI,
>
> We are using Flink with Adaptive Scheduler(Reactive Mode) on Kubernetes
> with Standalone deployment Application mode for our streaming
> infrastructure. Our autoscaler is scaling up or down our jobs. However,
> each scale action causes a job restart.
>
> Our customers complain about fluctuating traffic that we are sending. Is
> there any way to reschedule tasks and calculate graphs without restarting
> the whole job ? Or Reduce restart time ?
>
> Job is set max parallelism 2x of maxWorker and we use GCS for checkpointing
> storage. I know rescaling stateful jobs requires keygroups to be
> redistributed. But we have stateless jobs also Such as reading from Kafka
> and extracting data and writing a sink. If you can provide some entry
> points we can start implementation support for those jobs.
>
> Thanks
>

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

[Jing Ge]

We might consider using 0 as null for values that never return 0. But null
is still not equal to 0 and it will be very difficult to let every
contributor in this community follow this rule, especially for future
unknown APIs, which means there will be some cases that still need null.
Personally, I would choose accuracy over convenience and consistency over
convenience. Therefore, null is recommended.

Best regards,
Jing

On Mon, Jul 24, 2023 at 11:48 PM Chesnay Schepler 
wrote:

> The downside to null is that it again forces users to handle this case
> themselves.
>
> The reality is that there is no good default value.
>
> Ideally we fix all cases where we return such values, such that the
> fallback to 0 isn't even used.
> Arguably the same could be said for null, but I'd think that 0 is less
> of a surprise.
>
> On 24/07/2023 17:21, Gyula Fóra wrote:
> > I agree that it's a bit strange to have 0 as a fallback value because it
> > can also naturally occur for many metrics.
> > If we want to omit the value null would be probably better as Matthias
> > suggested.
> >
> > Gyula
> >
> > On Mon, Jul 24, 2023 at 4:02 PM Matthias Pohl
> >  wrote:
> >
> >> What was the reason you decided to go for 0 as the fallback value
> instead
> >> of null? Wouldn't that be a more reasonable value for error cases?
> >>
> >> On Mon, Jul 24, 2023 at 12:51 PM Chesnay Schepler 
> >> wrote:
> >>
> >>> There are a number of cases where the REST API can return infinity or
> >>> NaN for certain double fields.
> >>>
> >>> This is problematic because the JSON spec does not allow such values,
> >>> and tooling working against that spec may run into issues when
> >>> encountering such a value.
> >>>
> >>> Specifically we've seen this become an issue in clients generated from
> >>> the OpenAPI spec.
> >>>
> >>>
> >>>
> >>>
> >>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425797
>
>

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Jing Ge]

+1

On Mon, Jul 24, 2023 at 11:43 PM Chesnay Schepler 
wrote:

> I update the endpoint in the FLIP.
>
> On 24/07/2023 14:28, Matthias Pohl wrote:
> > I should have mentioned it in the discussion thread but I missed going
> over
> > that ML thread earlier: We might want to update the FLIP to refer to the
> > actual endpoint /jobs/:jobid/rescaling (AFAIU) with the corresponding
> cause
> > being FLINK-12312 [1].
> >
> > But that's just a minor thing.
> >
> > +1 (binding)
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-12312
> >
> > On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf 
> wrote:
> >
> >> +1 (binding)
> >>
> >> Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> >> martijnvis...@apache.org>:
> >>
> >>> +1 (binding)
> >>>
> >>> On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler 
> >>> wrote:
> >>>
>  Hello,
> 
>  I'd like to start a vote on FLIP-340.
> 
>  Discussion thread:
>  https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
>  FLIP:
> 
> 
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
>  Regards,
>  Chesnay
> 
> >>
> >> --
> >> https://twitter.com/snntrable
> >> https://github.com/knaufk
> >>
>
>

Re: [DISCUSS][2.0] FLIP-352: Use camelCast for all REST API fields/parameters

[Jing Ge]

camel case is the standard code convention in Java since day one[1]. We
don't need to describe it again for Flink. Legacy hyphens might be coded by
contributors with other language backgrounds. If we really care about it,
it might make sense to add the reference.

It would be great if the FLIP could show one hyphen example, so that we
will know what the issue exactly is and how many classes are involved in.

Best regards,
Jing

[1]
https://www.oracle.com/java/technologies/javase/codeconventions-namingconventions.html

On Mon, Jul 24, 2023 at 9:09 PM Matthias Pohl
 wrote:

> Do we have conventions on how the REST API should be formatted written down
> somewhere?
> Checking our coding conventions [1] didn't reveal anything.
>
> We could refer to a style guide like [2] to cover this topic. That would
> also include the camel-cased JSON fields.
>
> [1]
> https://flink.apache.org/how-to-contribute/code-style-and-quality-preamble/
> [2]
>
> https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format
>
> On Mon, Jul 24, 2023 at 1:03 PM Chesnay Schepler 
> wrote:
>
> > Make the REST API more consistent and easier to work with from the UI.
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425799
> >
>

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

[Chesnay Schepler]

The downside to null is that it again forces users to handle this case 
themselves.


The reality is that there is no good default value.

Ideally we fix all cases where we return such values, such that the 
fallback to 0 isn't even used.
Arguably the same could be said for null, but I'd think that 0 is less 
of a surprise.


On 24/07/2023 17:21, Gyula Fóra wrote:

I agree that it's a bit strange to have 0 as a fallback value because it
can also naturally occur for many metrics.
If we want to omit the value null would be probably better as Matthias
suggested.

Gyula

On Mon, Jul 24, 2023 at 4:02 PM Matthias Pohl
 wrote:


What was the reason you decided to go for 0 as the fallback value instead
of null? Wouldn't that be a more reasonable value for error cases?

On Mon, Jul 24, 2023 at 12:51 PM Chesnay Schepler 
wrote:


There are a number of cases where the REST API can return infinity or
NaN for certain double fields.

This is problematic because the JSON spec does not allow such values,
and tooling working against that spec may run into issues when
encountering such a value.

Specifically we've seen this become an issue in clients generated from
the OpenAPI spec.





https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425797

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Chesnay Schepler]

I update the endpoint in the FLIP.

On 24/07/2023 14:28, Matthias Pohl wrote:

I should have mentioned it in the discussion thread but I missed going over
that ML thread earlier: We might want to update the FLIP to refer to the
actual endpoint /jobs/:jobid/rescaling (AFAIU) with the corresponding cause
being FLINK-12312 [1].

But that's just a minor thing.

+1 (binding)

[1] https://issues.apache.org/jira/browse/FLINK-12312

On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf  wrote:


+1 (binding)

Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
martijnvis...@apache.org>:


+1 (binding)

On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler 
wrote:


Hello,

I'd like to start a vote on FLIP-340.

Discussion thread:
https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
FLIP:



https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint

Regards,
Chesnay



--
https://twitter.com/snntrable
https://github.com/knaufk

Re: Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

[Jing Ge]

agree, since we want to try our best to deprecate APIs in 1.18, it makes
sense.


Best regards,
Jing

On Mon, Jul 24, 2023 at 12:11 PM Wencong Liu  wrote:

> Hi Jing and Matthias,
>
>
> I believe it is reasonable to examine all classes that implement the
> IOReadableWritable
> interface and summarize their actual usage. However, due to time
> constraints, I suggest
> we minimize the scope of this FLIP to focus on the Path class. As for
> other components
> that implement IOReadableWritable, we can make an effort to investigate
> them
> in the future. WDYT?
>
>
> Best regards,
> Wencong Liu
>
>
>
>
>
>
>
>
>
>
>
> At 2023-07-22 00:46:45, "Jing Ge"  wrote:
> >Hi Wencong,
> >
> >Thanks for the clarification. I got your point. It makes sense.
> >
> >Wrt IOReadableWritable, the suggestion was to check all classes that
> >implemented it, e.g. BlockInfo, Value, Configuration, etc. Not limited to
> >the Path.
> >
> >Best regards,
> >Jing
> >
> >On Fri, Jul 21, 2023 at 4:31 PM Wencong Liu  wrote:
> >
> >> Hello Jing,
> >>
> >>
> >> Thanks for your reply. The URI field should be final and the
> >> Path will be immutable.The static method deserializeFromDataInputView
> >> will create a new Path object instead of replacing the URI field
> >> in a existed Path Object.
> >>
> >>
> >> For the crossing multiple modules issue, I've explained it in the reply
> >> to Matthias.
> >>
> >>
> >> Best regards,
> >>
> >>
> >> Wencong Liu
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> At 2023-07-21 18:05:26, "Jing Ge"  wrote:
> >> >Hi Wencong,
> >> >
> >> >Just out of curiosity, will the newly introduced
> >> >deserializeFromDataInputView() method make the Path mutable again?
> >> >
> >> >What Matthias suggested makes sense, although the extension might make
> >> this
> >> >FLIP cross multiple modules.
> >> >
> >> >Best regards,
> >> >Jing
> >> >
> >> >On Fri, Jul 21, 2023 at 10:23 AM Matthias Pohl
> >> > wrote:
> >> >
> >> >> There's a kind-of-related issue FLINK-4758 [1] that proposes removing
> >> the
> >> >> IOReadableWritable interface from more classes. It was briefly
> >> mentioned in
> >> >> the must-have work items discussion [2].
> >> >>
> >> >> I'm not too sure about the usage of IOReadableWritable: ...whether it
> >> would
> >> >> go away with the removal of the DataSet API in general (the Jira
> issue
> >> has
> >> >> DataSet as a component), anyway.
> >> >>
> >> >> Otherwise, might it make sense to extend the scope of this FLIP?
> >> >>
> >> >> [1] https://issues.apache.org/jira/browse/FLINK-4758
> >> >> [2] https://lists.apache.org/thread/gf0h4gh3xfsj78cpdsxsnj70nhzcmv9r
> >> >>
> >> >> On Fri, Jul 21, 2023 at 6:04 AM Xintong Song 
> >> >> wrote:
> >> >>
> >> >> > +1
> >> >> >
> >> >> > Best,
> >> >> >
> >> >> > Xintong
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Fri, Jul 21, 2023 at 10:54 AM Wencong Liu  >
> >> >> wrote:
> >> >> >
> >> >> > > Hi devs,
> >> >> > >
> >> >> > > I would like to start a discussion on FLIP-347: Remove
> >> >> IOReadableWritable
> >> >> > > serialization in Path [1].
> >> >> > >
> >> >> > >
> >> >> > > The Path class is currently mutable to support IOReadableWritable
> >> >> > > serialization. However, many parts
> >> >> > > of the code assume that the Path is immutable. By making the Path
> >> class
> >> >> > > immutable, we can ensure
> >> >> > > that paths are stored correctly without the possibility of
> mutation
> >> and
> >> >> > > eliminate the occurrence of subtle errors.
> >> >> > > As such I propose to modify the Path class to no longer implement
> >> the
> >> >> > > IOReadableWritable interface.
> >> >> > > Looking forward to your feedback.
> >> >> > > [1]
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-347%3A+Remove+IOReadableWritable+serialization+in+Path
> >> >> > > Best regards,
> >> >> > >
> >> >> > >
> >> >> > > Wencong Liu
> >> >> >
> >> >>
> >>
>

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

[Jing Ge]

+1

On Mon, Jul 24, 2023 at 5:08 PM Timo Walther  wrote:

> +1
>
> Regards,
> Timo
>
>
> On 24.07.23 04:00, liu ron wrote:
> > +1
> >
> > Best,
> > Ron
> >
> > Lincoln Lee  于2023年7月21日周五 16:09写道：
> >
> >> +1
> >>
> >> Best,
> >> Lincoln Lee
> >>
> >>
> >> Leonard Xu  于2023年7月21日周五 16:07写道：
> >>
> >>> +1
> >>>
> >>> Best,
> >>> Leonard
> >>>
>  On Jul 21, 2023, at 4:02 PM, yuxia 
> >> wrote:
> 
>  +1（binging）
> 
>  Best regards,
>  Yuxia
> 
>  - 原始邮件 -
>  发件人: "Jane Chan" 
>  收件人: "dev" 
>  发送时间: 星期五, 2023年 7 月 21日 下午 3:41:11
>  主题: [VOTE] FLIP-346: Deprecate ManagedTable related APIs
> 
>  Hi developers,
> 
>  Thanks for all the feedback on FLIP-346: Deprecate ManagedTable
> related
>  APIs[1].
>  Based on the discussion[2], we have reached a consensus, so I would
> >> like
> >>> to
>  start a vote.
> 
>  The vote will last for at least 72 hours unless there is an objection
> >> or
>  insufficient votes.
> 
>  [1]
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-346%3A+Deprecate+ManagedTable+related+APIs
>  [2] https://lists.apache.org/thread/5dvqyhqp5fbtm54944xohts71modwd99
> 
>  Best,
>  Jane
> >>>
> >>>
> >>
> >
>
>

Re: [VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

[Jing Ge]

+1(binding)

On Mon, Jul 24, 2023 at 8:55 PM Matthias Pohl
 wrote:

> +1 (binding)
>
> On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf 
> wrote:
>
> > +1 (binding)
> >
> > Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> > martijnvis...@apache.org>:
> >
> > > +1 (binding)
> > >
> > > On Mon, Jul 24, 2023 at 1:08 PM Chesnay Schepler 
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I'd like to start a vote on FLIP-336.
> > > >
> > > > Discussion thread:
> > > > https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
> > > > FLIP:
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789
> > > >
> > > > Regards,
> > > > Chesnay
> > > >
> > >
> >
> >
> > --
> > https://twitter.com/snntrable
> > https://github.com/knaufk
> >
>

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

[Gyula Fóra]

I agree that it's a bit strange to have 0 as a fallback value because it
can also naturally occur for many metrics.
If we want to omit the value null would be probably better as Matthias
suggested.

Gyula

On Mon, Jul 24, 2023 at 4:02 PM Matthias Pohl
 wrote:

> What was the reason you decided to go for 0 as the fallback value instead
> of null? Wouldn't that be a more reasonable value for error cases?
>
> On Mon, Jul 24, 2023 at 12:51 PM Chesnay Schepler 
> wrote:
>
> > There are a number of cases where the REST API can return infinity or
> > NaN for certain double fields.
> >
> > This is problematic because the JSON spec does not allow such values,
> > and tooling working against that spec may run into issues when
> > encountering such a value.
> >
> > Specifically we've seen this become an issue in clients generated from
> > the OpenAPI spec.
> >
> >
> >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425797
> >
>

[RESULT][VOTE] FLIP-346: Deprecate ManagedTable related APIs

[Jane Chan]

Hi devs,

FLIP-346 [1] has been accepted and voted through this thread [2].

There are five approving votes, four of which are binding:

   - Leonard Xu (binding)
   - Lincoln Lee (binding)
   - Liu Ron (non-binding)
   - Timo Walther (binding)
   - Yuxia Luo (binding)

And there is no disapproval.

Thanks for participating and providing valuable feedback.

[1]
https://cwiki.apache.org/confluence/display/FLINK/FLIP-346%3A+Deprecate+ManagedTable+related+APIs
[2] https://lists.apache.org/thread/7fy34vhbgjc5r2488dnf61bjo8yzfjx2

Best,
Jane

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Wencong Liu]

Congratulations!

Best,
Wencong Liu















在 2023-07-24 11:03:30，"Paul Lam"  写道：
>Congrats, Shammon!
>
>Best,
>Paul Lam
>
>> 2023年7月24日 10:56，Jingsong Li  写道：
>> 
>> Shammon
>

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

[Matthias Pohl]

What was the reason you decided to go for 0 as the fallback value instead
of null? Wouldn't that be a more reasonable value for error cases?

On Mon, Jul 24, 2023 at 12:51 PM Chesnay Schepler 
wrote:

> There are a number of cases where the REST API can return infinity or
> NaN for certain double fields.
>
> This is problematic because the JSON spec does not allow such values,
> and tooling working against that spec may run into issues when
> encountering such a value.
>
> Specifically we've seen this become an issue in clients generated from
> the OpenAPI spec.
>
>
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425797
>

Re: [DISCUSS][2.0] FLIP-352: Use camelCast for all REST API fields/parameters

[Matthias Pohl]

Do we have conventions on how the REST API should be formatted written down
somewhere?
Checking our coding conventions [1] didn't reveal anything.

We could refer to a style guide like [2] to cover this topic. That would
also include the camel-cased JSON fields.

[1]
https://flink.apache.org/how-to-contribute/code-style-and-quality-preamble/
[2]
https://google.github.io/styleguide/jsoncstyleguide.xml?showone=Property_Name_Format#Property_Name_Format

On Mon, Jul 24, 2023 at 1:03 PM Chesnay Schepler  wrote:

> Make the REST API more consistent and easier to work with from the UI.
>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425799
>

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Lincoln Lee]

Congratulations, Yong Fang

Best,
Lincoln Lee


Kelu Tao  于2023年7月24日周一 20:45写道：

> Congratulations, Shammon!
>
>
> On 2023/07/24 02:56:47 Jingsong Li wrote:
> > Hi, everyone
> >
> > On behalf of the PMC, I'm very happy to announce Yong Fang (Shammon)
> > (zjur...@gmail.com) as a new Flink Committer.
> >
> > Yong is an old flinker, he has been contributing to Flink since 2017.
> >
> > He actively participated in dev discussions and answered many
> > questions on the user mailing list.
> >
> > He contributed the JDBC Driver for Flink SQL. And continue to
> > contribute to Flink's lineage management and other features in the
> > future.
> >
> > Please join me in congratulating Yong Fang for becoming a Flink
> Committer!
> >
> > Best,
> > Jingsong Lee (on behalf of the Flink PMC)
> >
>

Re: [VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

[Matthias Pohl]

+1 (binding)

On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf  wrote:

> +1 (binding)
>
> Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> martijnvis...@apache.org>:
>
> > +1 (binding)
> >
> > On Mon, Jul 24, 2023 at 1:08 PM Chesnay Schepler 
> > wrote:
> >
> > > Hello,
> > >
> > > I'd like to start a vote on FLIP-336.
> > >
> > > Discussion thread:
> > > https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
> > > FLIP:
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789
> > >
> > > Regards,
> > > Chesnay
> > >
> >
>
>
> --
> https://twitter.com/snntrable
> https://github.com/knaufk
>

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Kelu Tao]

Congratulations, Shammon!


On 2023/07/24 02:56:47 Jingsong Li wrote:
> Hi, everyone
> 
> On behalf of the PMC, I'm very happy to announce Yong Fang (Shammon)
> (zjur...@gmail.com) as a new Flink Committer.
> 
> Yong is an old flinker, he has been contributing to Flink since 2017.
> 
> He actively participated in dev discussions and answered many
> questions on the user mailing list.
> 
> He contributed the JDBC Driver for Flink SQL. And continue to
> contribute to Flink's lineage management and other features in the
> future.
> 
> Please join me in congratulating Yong Fang for becoming a Flink Committer!
> 
> Best,
> Jingsong Lee (on behalf of the Flink PMC)
> 

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Matthias Pohl]

I should have mentioned it in the discussion thread but I missed going over
that ML thread earlier: We might want to update the FLIP to refer to the
actual endpoint /jobs/:jobid/rescaling (AFAIU) with the corresponding cause
being FLINK-12312 [1].

But that's just a minor thing.

+1 (binding)

[1] https://issues.apache.org/jira/browse/FLINK-12312

On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf  wrote:

> +1 (binding)
>
> Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
> martijnvis...@apache.org>:
>
> > +1 (binding)
> >
> > On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler 
> > wrote:
> >
> > > Hello,
> > >
> > > I'd like to start a vote on FLIP-340.
> > >
> > > Discussion thread:
> > > https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
> > > FLIP:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
> > >
> > > Regards,
> > > Chesnay
> > >
> >
>
>
> --
> https://twitter.com/snntrable
> https://github.com/knaufk
>

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Konstantin Knauf]

+1 (binding)

Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
martijnvis...@apache.org>:

> +1 (binding)
>
> On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler 
> wrote:
>
> > Hello,
> >
> > I'd like to start a vote on FLIP-340.
> >
> > Discussion thread:
> > https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
> > FLIP:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
> >
> > Regards,
> > Chesnay
> >
>


-- 
https://twitter.com/snntrable
https://github.com/knaufk

Re: [VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

[Konstantin Knauf]

+1 (binding)

Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser <
martijnvis...@apache.org>:

> +1 (binding)
>
> On Mon, Jul 24, 2023 at 1:08 PM Chesnay Schepler 
> wrote:
>
> > Hello,
> >
> > I'd like to start a vote on FLIP-336.
> >
> > Discussion thread:
> > https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
> > FLIP:
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789
> >
> > Regards,
> > Chesnay
> >
>


-- 
https://twitter.com/snntrable
https://github.com/knaufk

Re: [VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

[Martijn Visser]

+1 (binding)

On Mon, Jul 24, 2023 at 1:08 PM Chesnay Schepler  wrote:

> Hello,
>
> I'd like to start a vote on FLIP-336.
>
> Discussion thread:
> https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
> FLIP:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789
>
> Regards,
> Chesnay
>

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Martijn Visser]

+1 (binding)

On Mon, Jul 24, 2023 at 1:10 PM Chesnay Schepler  wrote:

> Hello,
>
> I'd like to start a vote on FLIP-340.
>
> Discussion thread:
> https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
> FLIP:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint
>
> Regards,
> Chesnay
>

Re:Re: [DISCUSS][2.0] FLIP-344: Remove parameter in RichFunction#open

[Wencong Liu]

Hi Timo,


Thanks for you reply. I think adding an empty OpenContext to keep the signature 
is
reasonable. I'll modify the FLIP at a later time.


Best,
Wencong Liu

















At 2023-07-24 17:11:44, "Timo Walther"  wrote:
>+1
>
>But instead we should add a OpenContext there to keep the signature 
>stable but still be able to add parameters.
>
>Regards,
>Timo
>
>On 21.07.23 12:24, Jing Ge wrote:
>> +1
>> 
>> On Fri, Jul 21, 2023 at 10:22 AM Yuxin Tan  wrote:
>> 
>>> +1
>>>
>>> Best,
>>> Yuxin
>>>
>>>
>>> Xintong Song  于2023年7月21日周五 12:04写道：
>>>
 +1

 Best,

 Xintong



 On Fri, Jul 21, 2023 at 10:52 AM Wencong Liu 
>>> wrote:

> Hi devs,
>
> I would like to start a discussion on FLIP-344: Remove parameter in
> RichFunction#open [1].
>
> The open() method in RichFunction requires a Configuration instance as
>>> an
> argument,
> which is always passed as a new instance without any configuration
> parameters in
> AbstractUdfStreamOperator#open. Thus, it is unnecessary to include this
> parameter
> in the open() method.
> As such I propose to remove the Configuration field from
> RichFunction#open(Configuration parameters).
> Looking forward to your feedback.
> [1]
>

>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425231
> Best regards,
>
>
> Wencong Liu

>>>
>> 

[jira] [Created] (FLINK-32660) Support external file systems in FileCatalogStore

[Ferenc Csaky (Jira)]

Ferenc Csaky created FLINK-32660:


 Summary: Support external file systems in FileCatalogStore
 Key: FLINK-32660
 URL: https://issues.apache.org/jira/browse/FLINK-32660
 Project: Flink
  Issue Type: Sub-task
Reporter: Ferenc Csaky
 Fix For: 1.18.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-32659) DB connection may leak if exception is thrown in JdbcOutputFormat#close

[Aitozi (Jira)]

Aitozi created FLINK-32659:
--

 Summary: DB connection may leak if exception is thrown in 
JdbcOutputFormat#close
 Key: FLINK-32659
 URL: https://issues.apache.org/jira/browse/FLINK-32659
 Project: Flink
  Issue Type: Improvement
  Components: Connectors / JDBC
Reporter: Aitozi






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-32658) State should not be silently removed when ignore-unclaimed-state is false

[Rui Fan (Jira)]

Rui Fan created FLINK-32658:
---

 Summary: State should not be silently removed when 
ignore-unclaimed-state is false
 Key: FLINK-32658
 URL: https://issues.apache.org/jira/browse/FLINK-32658
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Checkpointing
Affects Versions: 1.17.1, 1.18.0
Reporter: Rui Fan
Assignee: Rui Fan


When ignore-unclaimed-state is false and the old state is removed, flink should 
throw exception.

It's similar with removing an stateful operator.

This case occurs not only when the user removes state, but also when the 
operator is replaced. 

For example: upgrade FlinkKafkaConsumer to KafkaSource. All logical are not 
changed, so the operator id isn't changed. The KafkaSource cannot resume from 
the state of FlinkKafkaConsumer. However, flink job can start, and the state is 
silently discarded.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[VOTE][2.0] FLIP-340: Remove rescale REST endpoint

[Chesnay Schepler]

Hello,

I'd like to start a vote on FLIP-340.

Discussion thread: 
https://lists.apache.org/thread/zkslk0qzttwgs8j3s951rht3v1tsyqqk
FLIP: 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint


Regards,
Chesnay

[VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

[Chesnay Schepler]

Hello,

I'd like to start a vote on FLIP-336.

Discussion thread: 
https://lists.apache.org/thread/ms3sk0p21n7q2oq0fjtq43koqj2pmwv4
FLIP: 
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263424789


Regards,
Chesnay

[DISCUSS][2.0] FLIP-352: Use camelCast for all REST API fields/parameters

[Chesnay Schepler]

Make the REST API more consistent and easier to work with from the UI.

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425799

[DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

[Chesnay Schepler]

There are a number of cases where the REST API can return infinity or 
NaN for certain double fields.


This is problematic because the JSON spec does not allow such values, 
and tooling working against that spec may run into issues when 
encountering such a value.


Specifically we've seen this become an issue in clients generated from 
the OpenAPI spec.




https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425797

[DISCUSS][2.0] FLIP-350: Remove query parameters from Jar handlers

[Chesnay Schepler]

The jar handlers currently accept a variety of parameters both as query 
parameters and via the request body.


While it is primarily a problem for sending program args as query 
parameters, because it's a nightmare with escaping parameters, the 
remaining parameters should follow suite for consistency.


https://cwiki.apache.org/confluence/display/FLINK/FLIP-350%3A+Remove+query+parameters+from+Jar+handlers 

[DISCUSS][2.0] FLIP-349: Move RocksDB statebackend classes to o.a.f.state.rocksdb package

[Chesnay Schepler]

To properly reflect the state of the rocksdb statebackend I propose to 
move all classes in the state-backend-rocksdb module under the classes 
to o.a.f.state.rocksdb package.



https://cwiki.apache.org/confluence/display/FLINK/FLIP-349%3A+Move+RocksDB+statebackend+classes+to+o.a.f.state.rocksdb+package 

[jira] [Created] (FLINK-32657) Revert upgrading ExecNode versions for StateMetadata

[Timo Walther (Jira)]

Timo Walther created FLINK-32657:


 Summary: Revert upgrading ExecNode versions for StateMetadata
 Key: FLINK-32657
 URL: https://issues.apache.org/jira/browse/FLINK-32657
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / Planner
Reporter: Timo Walther
Assignee: Jane Chan


In theory, introducing a new attribute in ExecNode requires upgrading the 
version of ExecNodeMetadata. However, since this is currently an experimental 
feature, the attributes that need to be serialized for the final exec node are 
still being iterated, and upgrading the version will make the serialization 
scheme of the lower version become immutable (unless a patch is applied to the 
old version), and the testing framework is not perfect either. Therefore, 
upgrading the ExecNode version is not necessary if state compatibility can be 
maintained at the implementation level.

It should be okay to roll back ExecNodeMetadata to version 1 because 
compatibility handling is enabled at the code level.

Long-term we need a larger testing framework, per-Flink and per-ExecNode 
version that validates all attributes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS][2.0] FLIP-344: Remove parameter in RichFunction#open

[Timo Walther]

+1

But instead we should add a OpenContext there to keep the signature 
stable but still be able to add parameters.


Regards,
Timo

On 21.07.23 12:24, Jing Ge wrote:

+1

On Fri, Jul 21, 2023 at 10:22 AM Yuxin Tan  wrote:


+1

Best,
Yuxin


Xintong Song  于2023年7月21日周五 12:04写道：


+1

Best,

Xintong



On Fri, Jul 21, 2023 at 10:52 AM Wencong Liu 

wrote:



Hi devs,

I would like to start a discussion on FLIP-344: Remove parameter in
RichFunction#open [1].

The open() method in RichFunction requires a Configuration instance as

an

argument,
which is always passed as a new instance without any configuration
parameters in
AbstractUdfStreamOperator#open. Thus, it is unnecessary to include this
parameter
in the open() method.
As such I propose to remove the Configuration field from
RichFunction#open(Configuration parameters).
Looking forward to your feedback.
[1]




https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=263425231

Best regards,


Wencong Liu

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

[Timo Walther]

+1

Regards,
Timo


On 24.07.23 04:00, liu ron wrote:

+1

Best,
Ron

Lincoln Lee  于2023年7月21日周五 16:09写道：


+1

Best,
Lincoln Lee


Leonard Xu  于2023年7月21日周五 16:07写道：


+1

Best,
Leonard


On Jul 21, 2023, at 4:02 PM, yuxia 

wrote:


+1（binging）

Best regards,
Yuxia

- 原始邮件 -
发件人: "Jane Chan" 
收件人: "dev" 
发送时间: 星期五, 2023年 7 月 21日 下午 3:41:11
主题: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

Hi developers,

Thanks for all the feedback on FLIP-346: Deprecate ManagedTable related
APIs[1].
Based on the discussion[2], we have reached a consensus, so I would

like

to

start a vote.

The vote will last for at least 72 hours unless there is an objection

or

insufficient votes.

[1]




https://cwiki.apache.org/confluence/display/FLINK/FLIP-346%3A+Deprecate+ManagedTable+related+APIs

[2] https://lists.apache.org/thread/5dvqyhqp5fbtm54944xohts71modwd99

Best,
Jane

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Yuepeng Pan]

Congrats, Yong Fang!

Best,
Yuepeng Pan



在 2023-07-24 16:51:08，"Dan Zou"  写道：
>Congrats, Yong Fang!
>
>Best,
>Dan Zou   
>
>
>
>
>
>> 2023年7月24日 15:41，Matt Wang  写道：
>> 
>> Yong Fang
>

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Dan Zou]

Congrats, Yong Fang!

Best,
Dan Zou   





> 2023年7月24日 15:41，Matt Wang  写道：
> 
> Yong Fang

[jira] [Created] (FLINK-32656) Deprecate ManagedTable related APIs

[Jane Chan (Jira)]

Jane Chan created FLINK-32656:
-

 Summary: Deprecate ManagedTable related APIs
 Key: FLINK-32656
 URL: https://issues.apache.org/jira/browse/FLINK-32656
 Project: Flink
  Issue Type: Sub-task
  Components: Table SQL / API
Affects Versions: 1.18.0
Reporter: Jane Chan
 Fix For: 1.18.0


Please refer to [FLIP-346: Deprecate ManagedTable related 
APIs|https://cwiki.apache.org/confluence/display/FLINK/FLIP-346%3A+Deprecate+ManagedTable+related+APIs]
 for more details.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Rui Fan]

Congratulations, Yong Fang

Best,
Rui Fan

On Mon, Jul 24, 2023 at 3:42 PM Matt Wang  wrote:

> Congratulations, Yong Fang
>
>
> --
>
> Best,
> Matt Wang
>
>
>  Replied Message 
> | From | Feng Jin |
> | Date | 07/24/2023 15:27 |
> | To |  |
> | Cc | Shammon FY |
> | Subject | Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang |
> Congratulations, Yong Fang
>
> Best,
> Feng
>
>
> On Mon, Jul 24, 2023 at 3:12 PM Leonard Xu  wrote:
>
> Congratulations, Yong Fang
>
> Best,
> Leonard
>
> On Jul 24, 2023, at 2:01 PM, yuxia  wrote:
>
> Congrats, Shammon!
>
> Best regards,
> Yuxia
>
> - 原始邮件 -
> 发件人: "Benchao Li" 
> 收件人: "dev" 
> 抄送: "Shammon FY" 
> 发送时间: 星期一, 2023年 7 月 24日 下午 1:23:55
> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
>
> Congratulations, Yong! Well deserved!
>
> Yangze Guo  于2023年7月24日周一 12:16写道：
>
> Congrats, Yong!
>
> Best,
> Yangze Guo
>
> On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng 
> wrote:
>
> Congratulations, Yong!
>
> Best,
> Xiangyu
>
> liu ron  于2023年7月24日周一 11:48写道：
>
> Congratulations,
>
> Best,
> Ron
>
> Qingsheng Ren  于2023年7月24日周一 11:18写道：
>
> Congratulations and welcome aboard, Yong!
>
> Best,
> Qingsheng
>
> On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao <
> zhanghao.c...@outlook.com>
> wrote:
>
> Congrats, Shammon!
>
> Best,
> Zhanghao Chen
> 
> 发件人: Weihua Hu 
> 发送时间: 2023年7月24日 11:11
> 收件人: dev@flink.apache.org 
> 抄送: Shammon FY 
> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
>
> Congratulations!
>
> Best,
> Weihua
>
>
> On Mon, Jul 24, 2023 at 11:04 AM Paul Lam 
> wrote:
>
> Congrats, Shammon!
>
> Best,
> Paul Lam
>
> 2023年7月24日 10:56，Jingsong Li  写道：
>
> Shammon
>
>
>
>
>
>
>
>
> --
>
> Best,
> Benchao Li
>
>
>

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Matt Wang]

Congratulations, Yong Fang


--

Best,
Matt Wang


 Replied Message 
| From | Feng Jin |
| Date | 07/24/2023 15:27 |
| To |  |
| Cc | Shammon FY |
| Subject | Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang |
Congratulations, Yong Fang

Best,
Feng


On Mon, Jul 24, 2023 at 3:12 PM Leonard Xu  wrote:

Congratulations, Yong Fang

Best,
Leonard

On Jul 24, 2023, at 2:01 PM, yuxia  wrote:

Congrats, Shammon!

Best regards,
Yuxia

- 原始邮件 -
发件人: "Benchao Li" 
收件人: "dev" 
抄送: "Shammon FY" 
发送时间: 星期一, 2023年 7 月 24日 下午 1:23:55
主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

Congratulations, Yong! Well deserved!

Yangze Guo  于2023年7月24日周一 12:16写道：

Congrats, Yong!

Best,
Yangze Guo

On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng 
wrote:

Congratulations, Yong!

Best,
Xiangyu

liu ron  于2023年7月24日周一 11:48写道：

Congratulations,

Best,
Ron

Qingsheng Ren  于2023年7月24日周一 11:18写道：

Congratulations and welcome aboard, Yong!

Best,
Qingsheng

On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao <
zhanghao.c...@outlook.com>
wrote:

Congrats, Shammon!

Best,
Zhanghao Chen

发件人: Weihua Hu 
发送时间: 2023年7月24日 11:11
收件人: dev@flink.apache.org 
抄送: Shammon FY 
主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

Congratulations!

Best,
Weihua


On Mon, Jul 24, 2023 at 11:04 AM Paul Lam 
wrote:

Congrats, Shammon!

Best,
Paul Lam

2023年7月24日 10:56，Jingsong Li  写道：

Shammon








--

Best,
Benchao Li

[jira] [Created] (FLINK-32655) RecreateOnResetOperatorCoordinator did not forward notifyCheckpointAborted to the real OperatorCoordinator

[Ming Li (Jira)]

Ming Li created FLINK-32655:
---

 Summary: RecreateOnResetOperatorCoordinator did not forward 
notifyCheckpointAborted to the real OperatorCoordinator
 Key: FLINK-32655
 URL: https://issues.apache.org/jira/browse/FLINK-32655
 Project: Flink
  Issue Type: Bug
  Components: Runtime / Checkpointing
Affects Versions: 1.17.1
Reporter: Ming Li


{{[RecreateOnResetOperatorCoordinator|https://github.com/apache/flink/blob/master/flink-runtime/src/main/java/org/apache/flink/runtime/operators/coordination/RecreateOnResetOperatorCoordinator.java#L115]}}
 does not override {{{}notifyCheckpointAborted{}}}, which causes the 
{{SplitEnumerator}} in {{SourceCoordinator}} can not receive the checkpoint 
abort message.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-32654) Deprecate ExecutionConfig#canEqual(obj)

[Zhu Zhu (Jira)]

Zhu Zhu created FLINK-32654:
---

 Summary: Deprecate ExecutionConfig#canEqual(obj)
 Key: FLINK-32654
 URL: https://issues.apache.org/jira/browse/FLINK-32654
 Project: Flink
  Issue Type: Technical Debt
  Components: Runtime / Configuration
 Environment: ExecutionConfig#canEqual(obj) checks whether the object 
is an instance of ExecutionConfig.
It is not intended to be used by users but was used for internal comparison. 
Unfortunately, is was exposed as `@Public` because ExecutionConfig is `@Public`.
We should deprecate it so that we can remove it in Flink 2.0.
Reporter: Zhu Zhu
 Fix For: 1.18.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Feng Jin]

Congratulations, Yong Fang

Best,
Feng


On Mon, Jul 24, 2023 at 3:12 PM Leonard Xu  wrote:

> Congratulations, Yong Fang
>
> Best,
> Leonard
>
> > On Jul 24, 2023, at 2:01 PM, yuxia  wrote:
> >
> > Congrats, Shammon!
> >
> > Best regards,
> > Yuxia
> >
> > - 原始邮件 -
> > 发件人: "Benchao Li" 
> > 收件人: "dev" 
> > 抄送: "Shammon FY" 
> > 发送时间: 星期一, 2023年 7 月 24日 下午 1:23:55
> > 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
> >
> > Congratulations, Yong! Well deserved!
> >
> > Yangze Guo  于2023年7月24日周一 12:16写道：
> >
> >> Congrats, Yong!
> >>
> >> Best,
> >> Yangze Guo
> >>
> >> On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng 
> >> wrote:
> >>>
> >>> Congratulations, Yong!
> >>>
> >>> Best,
> >>> Xiangyu
> >>>
> >>> liu ron  于2023年7月24日周一 11:48写道：
> >>>
>  Congratulations,
> 
>  Best,
>  Ron
> 
>  Qingsheng Ren  于2023年7月24日周一 11:18写道：
> 
> > Congratulations and welcome aboard, Yong!
> >
> > Best,
> > Qingsheng
> >
> > On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao <
>  zhanghao.c...@outlook.com>
> > wrote:
> >
> >> Congrats, Shammon!
> >>
> >> Best,
> >> Zhanghao Chen
> >> 
> >> 发件人: Weihua Hu 
> >> 发送时间: 2023年7月24日 11:11
> >> 收件人: dev@flink.apache.org 
> >> 抄送: Shammon FY 
> >> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
> >>
> >> Congratulations!
> >>
> >> Best,
> >> Weihua
> >>
> >>
> >> On Mon, Jul 24, 2023 at 11:04 AM Paul Lam 
>  wrote:
> >>
> >>> Congrats, Shammon!
> >>>
> >>> Best,
> >>> Paul Lam
> >>>
>  2023年7月24日 10:56，Jingsong Li  写道：
> 
>  Shammon
> >>>
> >>>
> >>
> >
> 
> >>
> >
> >
> > --
> >
> > Best,
> > Benchao Li
>
>

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[Leonard Xu]

Congratulations, Yong Fang

Best,
Leonard

> On Jul 24, 2023, at 2:01 PM, yuxia  wrote:
> 
> Congrats, Shammon!
> 
> Best regards,
> Yuxia
> 
> - 原始邮件 -
> 发件人: "Benchao Li" 
> 收件人: "dev" 
> 抄送: "Shammon FY" 
> 发送时间: 星期一, 2023年 7 月 24日 下午 1:23:55
> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
> 
> Congratulations, Yong! Well deserved!
> 
> Yangze Guo  于2023年7月24日周一 12:16写道：
> 
>> Congrats, Yong!
>> 
>> Best,
>> Yangze Guo
>> 
>> On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng 
>> wrote:
>>> 
>>> Congratulations, Yong!
>>> 
>>> Best,
>>> Xiangyu
>>> 
>>> liu ron  于2023年7月24日周一 11:48写道：
>>> 
 Congratulations,
 
 Best,
 Ron
 
 Qingsheng Ren  于2023年7月24日周一 11:18写道：
 
> Congratulations and welcome aboard, Yong!
> 
> Best,
> Qingsheng
> 
> On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao <
 zhanghao.c...@outlook.com>
> wrote:
> 
>> Congrats, Shammon!
>> 
>> Best,
>> Zhanghao Chen
>> 
>> 发件人: Weihua Hu 
>> 发送时间: 2023年7月24日 11:11
>> 收件人: dev@flink.apache.org 
>> 抄送: Shammon FY 
>> 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
>> 
>> Congratulations!
>> 
>> Best,
>> Weihua
>> 
>> 
>> On Mon, Jul 24, 2023 at 11:04 AM Paul Lam 
 wrote:
>> 
>>> Congrats, Shammon!
>>> 
>>> Best,
>>> Paul Lam
>>> 
 2023年7月24日 10:56，Jingsong Li  写道：
 
 Shammon
>>> 
>>> 
>> 
> 
 
>> 
> 
> 
> -- 
> 
> Best,
> Benchao Li

Check points are discarded with reason NULL

[Y SREEKARA BHARGAVA REDDY]

Hi Team,

While running flink streaming. i  got following exception,

Did you any one faced the issue?
Check points are discarded with *reason is* NULL.

org.apache.flink.util.FlinkRuntimeException: Exceeded checkpoint tolerable
failure threshold.

at org.apache.flink.runtime.checkpoint.CheckpointFailureManager
.handleTaskLevelCheckpointException(CheckpointFailureManager.java:87)

at org.apache.flink.runtime.checkpoint.CheckpointCoordinator
.failPendingCheckpointDueToTaskFailure(CheckpointCoordinator.java:1467)

at org.apache.flink.runtime.checkpoint.CheckpointCoordinator
.discardCheckpoint(CheckpointCoordinator.java:1377)

at org.apache.flink.runtime.checkpoint.CheckpointCoordinator
.receiveDeclineMessage(CheckpointCoordinator.java:719)

at org.apache.flink.runtime.scheduler.SchedulerBase
.lambda$declineCheckpoint$5(SchedulerBase.java:807)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
511)

at java.util.concurrent.FutureTask.run(FutureTask.java:266)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
.access$201(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask
.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor
.java:1149)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:624)

at java.lang.Thread.run(Thread.java:748)


Please let me know, hpow to fix the issue.

Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

[yuxia]

Congrats, Shammon!

Best regards,
Yuxia

- 原始邮件 -
发件人: "Benchao Li" 
收件人: "dev" 
抄送: "Shammon FY" 
发送时间: 星期一, 2023年 7 月 24日 下午 1:23:55
主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

Congratulations, Yong! Well deserved!

Yangze Guo  于2023年7月24日周一 12:16写道：

> Congrats, Yong!
>
> Best,
> Yangze Guo
>
> On Mon, Jul 24, 2023 at 12:02 PM xiangyu feng 
> wrote:
> >
> > Congratulations, Yong!
> >
> > Best,
> > Xiangyu
> >
> > liu ron  于2023年7月24日周一 11:48写道：
> >
> > > Congratulations,
> > >
> > > Best,
> > > Ron
> > >
> > > Qingsheng Ren  于2023年7月24日周一 11:18写道：
> > >
> > > > Congratulations and welcome aboard, Yong!
> > > >
> > > > Best,
> > > > Qingsheng
> > > >
> > > > On Mon, Jul 24, 2023 at 11:14 AM Chen Zhanghao <
> > > zhanghao.c...@outlook.com>
> > > > wrote:
> > > >
> > > > > Congrats, Shammon!
> > > > >
> > > > > Best,
> > > > > Zhanghao Chen
> > > > > 
> > > > > 发件人: Weihua Hu 
> > > > > 发送时间: 2023年7月24日 11:11
> > > > > 收件人: dev@flink.apache.org 
> > > > > 抄送: Shammon FY 
> > > > > 主题: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang
> > > > >
> > > > > Congratulations!
> > > > >
> > > > > Best,
> > > > > Weihua
> > > > >
> > > > >
> > > > > On Mon, Jul 24, 2023 at 11:04 AM Paul Lam 
> > > wrote:
> > > > >
> > > > > > Congrats, Shammon!
> > > > > >
> > > > > > Best,
> > > > > > Paul Lam
> > > > > >
> > > > > > > 2023年7月24日 10:56，Jingsong Li  写道：
> > > > > > >
> > > > > > > Shammon
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>


-- 

Best,
Benchao Li