Re: [VOTE] FLIP-314: Support Customized Job Lineage Listener

2023-09-25 Thread Jing Ge
+1(binding) Thanks! Best Regards, Jing On Sun, Sep 24, 2023 at 10:30 PM Shammon FY wrote: > Hi devs, > > Thanks for all the feedback on FLIP-314: Support Customized Job Lineage > Listener [1] in thread [2]. > > I would like to start a vote for it. The vote will be opened for at least > 72 hours

Re: [Discuss] FLIP-362: Support minimum resource limitation

2023-09-25 Thread Jing Ge
he execution of batch jobs. > > WDYT? > > Best, > Yangze Guo > > On Thu, Sep 21, 2023 at 4:49 AM Jing Ge > wrote: > > > > Hi Xiangyu, > > > > Thanks for driving it! There is one thing I am not really sure if I > > understa

Re: [DISCUSS] FLIP-367: Support Setting Parallelism for Table/SQL Sources

2023-09-25 Thread Jing Ge
Hi Zhanghao, Thanks for driving the FLIP. This is a nice feature users are looking for. >From users' perspective, would you like to add explicit description about any potential(or none) compatibility issues if users want to use this new feature and start existing jobs with savepoints or checkpoin

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-25 Thread Jing Ge
ea2cbb8a10c705a25a4ff6deab60711 > [3] > https://docs.google.com/spreadsheets/d/1V0-duzNTgu7H6R7kioF-TAPhlqWl7Co6Q9ikTBuaULo/edit?usp=sharing > > On Sun, Sep 24, 2023 at 11:29 AM ConradJam wrote: > > > > +1 for testing with Java 17 > > > > Jing Ge 于2023年9月24日周日 09:40写道:

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-23 Thread Jing Ge
a pipeline with Java 17 later since > it is supported in 1.18[1]. > > > Best regards, > Zakelly > > [1] > https://github.com/apache/flink/commit/9c1318ca7fa5b2e7b11827068ad1288483aaa464#diff-8310c97396d60e96766a936ca8680f1e2971ef486cfc2bc55ec9ca5a5333c47fR53 > > On

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-22 Thread Jing Ge
gt; attention to it. > > > Thanks. > > > > > > > > > Best, > > > Zakelly > > > > > > On Thu, Sep 21, 2023 at 10:08 AM Yuxin Tan > wrote: > > > > > > > > Hi, Jing, Qingsheng, > > > > > > > > Thanks a lot. >

Re: [VOTE] FLIP-327: Support switching from batch to stream mode to improve throughput when processing backlog data

2023-09-22 Thread Jing Ge
+1(binding) Thanks! Best regards, Jing On Fri, Sep 22, 2023 at 8:08 AM Dong Lin wrote: > Hi all, > > We would like to start the vote for FLIP-327: Support switching from batch > to stream mode to improve throughput when processing backlog data [1]. This > FLIP was discussed in this thread [2].

Re: [Discuss] FLIP-366: Support standard YAML for FLINK configuration

2023-09-21 Thread Jing Ge
Hi Junrui, +1 for following the standard. Thanks for your effort! Best regards, Jing On Thu, Sep 21, 2023 at 5:09 AM Junrui Lee wrote: > Hi Jane, > > Thank you for your valuable feedback and suggestions. > I agree with your point about differentiating between "flink-config.yaml" > and "flink-c

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

2023-09-21 Thread Jing Ge
t; Zakelly > > [1] https://github.com/apache/flink/pull/3380#issuecomment-286807853 > [2] https://github.com/apache/flink/pull/3380#issuecomment-286932133 > [3] https://github.com/apache/flink/pull/3380#issuecomment-281631160 > > On Thu, Sep 21, 2023 at 1:27 AM Jing Ge > wrote

Re: [Discuss] FLIP-362: Support minimum resource limitation

2023-09-20 Thread Jing Ge
Hi Xiangyu, Thanks for driving it! There is one thing I am not really sure if I understand you correctly. According to the FLIP: "The minimum resource limitation will be implemented in the DefaultResourceAllocationStrategy of FineGrainedSlotManager. Each time when SlotManager needs to reconcile

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

2023-09-20 Thread Jing Ge
sorry, typo: It is a known "anti-pattern" instead of "ant-pattern" Best regards, Jing On Wed, Sep 20, 2023 at 7:23 PM Jing Ge wrote: > Hi Zakelly, > > Thanks for driving this topic. From good software engineering's > perspective, I have different though

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

2023-09-20 Thread Jing Ge
Hi Zakelly, Thanks for driving this topic. From good software engineering's perspective, I have different thoughts: 1. The idea to get rid of all checked Exceptions and replace them with unchecked Exceptions is a known ant-pattern: "Generally speaking, do not throw a RuntimeException or create a

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-20 Thread Jing Ge
d to the final commit of > > this release. I will try to get this result first. > > > > > > Best, > > Zakelly > > > > On Tue, Sep 19, 2023 at 4:55 PM Jing Ge > > wrote: > > > > > > Hi > > > > > > Thanks Zakel

Re: [DISCUSS] Backport fix to release 1.18

2023-09-20 Thread Jing Ge
Hi folks, Please ignore this email. It seems the dev mail server has some issues that emails have been sent but will be received by dev@flink.apache.org with a big delay at [1]. Participants who have been sent to or cc directly should get the email instantly. Let's consolidate the discussion in th

Re: [VOTE] FLIP-312: Prometheus Sink Connector

2023-09-20 Thread Jing Ge
+1(binding) Thanks! Best regards, Jing On Wed, Sep 20, 2023 at 3:20 PM Martijn Visser wrote: > +1 (binding) > > Thanks for driving this. Cheers, M > > On Mon, Sep 18, 2023 at 1:51 PM Lorenzo Nicora > wrote: > > > > Hi All, > > > > Thanks for the feedback on FLIP-312: Prometheus Sink Connector

Re: [DISCUSS]clean up the savepoints compatibility table

2023-09-20 Thread Jing Ge
Flink version to the latest, so that > they are aware of potential issues that could run into. > > Best regards, > > Martijn > > On Thu, Sep 14, 2023 at 10:22 PM Jing Ge > wrote: > > > Hi, > > > > According the community update policy[1] for old

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-20 Thread Jing Ge
of >> this release. I will try to get this result first. >> >> >> Best, >> Zakelly >> >> On Tue, Sep 19, 2023 at 4:55 PM Jing Ge >> wrote: >> > >> > Hi >> > >> > Thanks Zakelly and Yuan for your effort and update.

Re: [VOTE] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-19 Thread Jing Ge
.apache.org/confluence/display/FLINK/FLIP-363%3A+Unify+the+Representation+of+TaskManager+Location+in+REST+API+and+Web+UI > [2] https://lists.apache.org/thread/sls1196mmk25w8nm2qf585254nbjr9hd > > Best, > Zhanghao Chen > > 发件人: Chen Zhanghao > 发送时间

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-19 Thread Jing Ge
ke endpoint can help us to distinguish it from the existing concept > of "location". +1 for using the term endpoint and introducing > TaskManagerLocation.getEndpoint(). > > Best, > Yangze Guo > > On Mon, Sep 18, 2023 at 11:52 PM Jing Ge > wrote: > > > > H

Re: [ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-19 Thread Jing Ge
e the pipeline is fully ready, we will proceed > > with the performance test for release 1.18.0. > > > > Please let me know if you have any concerns. Thank you all for your > > patience. > > > > Best, > > Zakelly > > > > On Mon, Sep 18, 2023 at 6:57 PM Jing

Re: [VOTE] FLIP-361: Improve GC Metrics

2023-09-18 Thread Jing Ge
looks great, thanks! +1(binding) Best regards, Jing On Mon, Sep 18, 2023 at 8:11 AM Yuepeng Pan wrote: > +1 (non-binding). > > > Best, > Yuepeng. > > > > > > 在 2023-09-16 12:52:09,"Weihua Hu" 写道: > >+1 (binding) > > > >Best, > >Weihua > > > > > >On Fri, Sep 15, 2023 at 4:28 PM Yangze Guo wrot

Re: [VOTE] FLIP-307: Flink Connector Redshift

2023-09-18 Thread Jing Ge
+1(binding). Thanks! Best regards, Jing On Mon, Sep 18, 2023 at 5:26 PM Danny Cranmer wrote: > Thanks for driving this Samrat! > > +1 (binding) > > Thanks, > Danny > > On Mon, Sep 18, 2023 at 4:17 AM Samrat Deb wrote: > > > Hi All, > > > > Thanks for all the feedback on FLIP-307: Flink Connect

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-18 Thread Jing Ge
ion" is already used widely in the codebase, e.g. > TaskManagerLocation and JobExceptions-related classes. > > WDYT? > > Best, > Zhanghao Chen > > 发件人: Jing Ge > 发送时间: 2023年9月13日 4:52 > 收件人: dev@flink.apache.org > 主题: Re: [DISCUSS] FLIP

[ANNOUNCE] Release 1.18.0, release candidate #0

2023-09-18 Thread Jing Ge
Hi everyone, The RC0 for Apache Flink 1.18.0 has been created. This RC is currently for preview only to facilitate the integrated testing since the benchmark tests are not available yet[1] and the release announcement is still under review. The RC1 will be released after all benchmarks tests are p

Re: [DISCUSS]clean up the savepoints compatibility table

2023-09-14 Thread Jing Ge
which aligns with the community's support policy. Best regards, Jing [1] https://flink.apache.org/downloads/#update-policy-for-old-releases On Thu, Sep 14, 2023 at 1:03 PM Jing Ge wrote: > Hi folks, > > The compatibility table[1] contains old Flink versions. Does it make sense >

Re: [DISSCUSS] Kubernetes Operator Flink Version Support Policy

2023-09-14 Thread Jing Ge
Thanks for the information! On Thu, Sep 14, 2023 at 9:07 PM Gyula Fóra wrote: > https://flink.apache.org/downloads/#update-policy-for-old-releases > > On Thu, Sep 14, 2023 at 8:47 PM Jing Ge > wrote: > > > +1 Thanks! I have an off-track question: where could we find the

Re: [DISSCUSS] Kubernetes Operator Flink Version Support Policy

2023-09-14 Thread Jing Ge
+1 Thanks! I have an off-track question: where could we find the reference that the community only supports the last 2 minor releases? Thanks! Best Regards, Jing On Thu, Sep 14, 2023 at 3:13 PM Ahmed Hamdy wrote: > Makes sense, > Thanks for the clarification. > Best Regards > Ahmed Hamdy > > >

[DISCUSS]clean up the savepoints compatibility table

2023-09-14 Thread Jing Ge
Hi folks, The compatibility table[1] contains old Flink versions. Does it make sense to clean up and remove some very old versions? My proposal would be: option 1: keeping less than the last 10 versions in this table so that in most cases reader does not need to scroll to the right to read hidden

Re: [VOTE] FLIP-357: Deprecate Iteration API of DataStream

2023-09-14 Thread Jing Ge
+1(binding) Best regards, Jing On Thu, Sep 14, 2023 at 7:31 AM Dong Lin wrote: > Thanks Wencong for the FLIP. > > +1 (binding) > > On Thu, Sep 14, 2023 at 12:36 PM Wencong Liu wrote: > > > Hi dev, > > > > > > I'd like to start a vote on FLIP-357. > > > > > > Discussion thread: > > https://list

Re: Flink and Flink shaded dependency

2023-09-13 Thread Jing Ge
tail/CVE-2022-39135 > [2] https://nvd.nist.gov/vuln/detail/CVE-2023-2976 > [3] https://github.com/google/guava/issues/2575 > [4] > > https://github.com/google/guava/commit/b3719763b9ca777b94554d7ea2d2b92e27ac6164 > > On Wed, Sep 13, 2023 at 6:26 PM Jing Ge > wrote: > > &g

Re: Flink and Flink shaded dependency

2023-09-13 Thread Jing Ge
her major version > usually > > leads to package rename > > since the major version number is a part of the package name... > > Not 100% sure however this could be the reason for some potential > breaking > > changes (we also faced that with connectors while the last fl

Re: Flink and Flink shaded dependency

2023-09-13 Thread Jing Ge
k-shaded > 16.2, since that version doesn't exist. > > Best regards, > > Martijn > > On Wed, Sep 13, 2023 at 1:48 PM Jing Ge > wrote: > > > Hi Martijn, > > > > Thanks for your reply with details. Appreciate it. > > > > > > > Fli

Re: Flink and Flink shaded dependency

2023-09-13 Thread Jing Ge
elease cycle, so that there's enough time to stabilize Flink. All in all, we shouldn't upgrade flink-shaded in Flink 1.17. > > Best regards, > > Martijn > > On Tue, Sep 12, 2023 at 7:26 PM Jing Ge > wrote: > > > Hi Dev, > > > > Currently Flin

Re: [DISCUSS] FLIP-363: Unify the Representation of TaskManager Location in REST API and Web UI

2023-09-12 Thread Jing Ge
Hi Zhanghao, Thanks for bringing this to our attention. It is a good proposal to improve data consistency. Speaking of naming conventions of choosing location over host, how about "endpoint" with the following thoughts: 1. endpoint is a more professional word than location in the network context

Flink and Flink shaded dependency

2023-09-12 Thread Jing Ge
Hi Dev, Currently Flink 1.17 is using flink-shaded 16.1 and Flink 1.18 is using flink-shaded 17.0. Do we need to consider any compatibility rules between them? E.g. is there any concern to upgrade flink-shaded from 16.1 to 17.x for Flink 1.17? Or there are some implicit dependency rules between th

Re: [DISCUSS] Flink annotation strategy/consensus

2023-09-12 Thread Jing Ge
cated. Instead, an immediate refactor should be done to remove the > "deprecated" internal methods / classes / interfaces, and migrate the code > to its successor. Otherwise, technical debts will build up. > > Thanks, > > Jiangjie (Becket) Qin > > > >

Re: [DISCUSS] Flink annotation strategy/consensus

2023-09-11 Thread Jing Ge
public > > > API will not evolve and will be removed according to the deprecation > > > process. > > > > > > 4. The internal methods / classes / interfaces SHOULD NOT be marked as > > > deprecated. Instead, an immediate refactor should be done to

Re: [VOTE] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-09-10 Thread Jing Ge
+1(binding) Best Regards, Jing On Sun, Sep 10, 2023 at 10:17 AM Dong Lin wrote: > Thanks Allison for proposing the FLIP. > > +1 (binding) > > On Fri, Sep 8, 2023 at 4:21 AM Allison Chang > > wrote: > > > Hi everyone, > > > > Would like to start the VOTE for FLIP-323< > > > https://cwiki.apache

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-09-10 Thread Jing Ge
Hi folks, Thanks for the informative discussion. @Allison @Becket currently the FLIP only focuses on Yarn, but after reading all your discussions, if I am not mistaken, both Yarn and Kubernetes clusters should be supported. Does it make sense to update the FLIP accordingly? Best regards, Jing O

Re: [DISCUSS] [FLINK-32873] Add a config to allow disabling Query hints

2023-09-09 Thread Jing Ge
Hi, Thanks for bringing this to our attention. At the first glance, it looks reasonable to offer a new configuration to enable/disable SQL hints globally. However, IMHO, it is not the right timing to do it now, because we should not only think as platform providers but also as end users(the number

[DISCUSS] Flink annotation strategy/consensus

2023-09-08 Thread Jing Ge
Hi devs, While I was joining the flink-avro enhancement and cleanup discussion driven by Becket[1], I realized that there are some issues with the current Flink API annotation usage in the source code. As far as I am concerned, Flink wants to control the access/visibility of APIs across modules a

Re: [NOTICE] Experimental Java 17 support now available on master

2023-09-07 Thread Jing Ge
he.org/confluence/display/FLINK/FLIP-317%3A+Upgrade+Kryo+from+2.24.0+to+5.5.0 > [2] https://flink.apache.org/roadmap/ > [3] https://issues.apache.org/jira/browse/FLINK-15736 > > On Wed, Sep 6, 2023 at 11:26 PM Jing Ge wrote: > >> Hi folks, >> >> Sorry to come b

Re: [NOTICE] Experimental Java 17 support now available on master

2023-09-06 Thread Jing Ge
compatibility scenarios. I also created this > simple application https://github.com/kurtostfeld/flink-kryo-upgrade-demo > to create state with Flink 1.17 and test the upgrade process. From what I > can see it works, but this would definitely need more testing from people > other than

[jira] [Created] (FLINK-33052) codespeed server is down

2023-09-06 Thread Jing Ge (Jira)
Jing Ge created FLINK-33052: --- Summary: codespeed server is down Key: FLINK-33052 URL: https://issues.apache.org/jira/browse/FLINK-33052 Project: Flink Issue Type: Bug Components: Test

Re: [DISCUSS] FLIP-358: flink-avro enhancement and cleanup

2023-09-06 Thread Jing Ge
it, while introducing a > new method. This is a bad practice adding to technical debts. Instead, a > proper refactor should be done immediately in the same patch to just remove > that private method and migrate all the usage to the new method. > > Thanks, > > Jiangjie (Becket)

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-09-06 Thread Jing Ge
time T and the last source of > the HybridSource to read from time T. > > Best, > Xuannan > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-309%3A+Support+using+larger+checkpointing+interval+when+source+is+processing+backlog > > > On Mon, Sep

[jira] [Created] (FLINK-33037) Bump Guava to 32.1.2-jre

2023-09-05 Thread Jing Ge (Jira)
Jing Ge created FLINK-33037: --- Summary: Bump Guava to 32.1.2-jre Key: FLINK-33037 URL: https://issues.apache.org/jira/browse/FLINK-33037 Project: Flink Issue Type: Improvement Reporter

Re: [DISCUSS] Drop python 3.7 support in 1.19

2023-09-04 Thread Jing Ge
+1 @Dian should we add support of python 3.11 Best regards, Jing On Mon, Sep 4, 2023 at 3:39 PM Gabor Somogyi wrote: > Thanks for all the responses! > > Based on the suggestions I've created the following jiras and started to > work on them: > * https://issues.apache.org/jira/browse/FLINK-3302

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-09-04 Thread Jing Ge
time, the job is not > processing backlog data. On the other hand, when event times are far from > processing time, causing watermarks to also be distant, if the lag > surpasses the defined threshold, the job is considered processing backlog > data. > > Best, > Xuannan > >

Re: Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

2023-09-01 Thread Jing Ge
The placement of the Iteration API in the repository is a topic that has > multiple > potential solutions. WDYT? > > > Best, > Wencong > > > > > > > > > > > > At 2023-09-01 17:59:34, "Jing Ge" wrote: > >Hi Wencong, > > >

Re: [DISCUSS] FLIP-357: Deprecate Iteration API of DataStream

2023-09-01 Thread Jing Ge
Hi Wencong, Thanks for the proposal! "The Iteration API in DataStream is planned be deprecated in Flink 1.19 and then finally removed in Flink 2.0. For the users that rely on the Iteration API in DataStream, they will have to migrate to Flink ML." - Does it make sense to migrate the iteration mod

Re: Azure pipelines CI view permissions

2023-08-31 Thread Jing Ge
Hi Ryan, It seems we are facing some issues on Azure side[1]. In the meantime, as a workaround, you can go to the "Runs" tab and find your build via filter. Best regards, Jing [1] https://developercommunity.visualstudio.com/t/Pipeline-runs-are-not-visible-for-non-ad/10455034 On Thu, Aug 31, 202

Re: [DISCUSS] FLIP-358: flink-avro enhancement and cleanup

2023-08-31 Thread Jing Ge
Hi Becket, It is a very useful proposal, thanks for driving it. +1. I'd like to ask some questions to make sure I understand your thoughts correctly: 1. "For the batch cases, currently the BulkFormat for DataStream is missing" - true, and there is another option to leverage StreamFormatAdapter[1]

Re: [VOTE] FLIP-348: Make expanding behavior of virtual metadata columns configurable

2023-08-31 Thread Jing Ge
+1(binding) On Thu, Aug 31, 2023 at 11:22 AM Sergey Nuyanzin wrote: > +1 (binding) > > On Thu, Aug 31, 2023 at 9:28 AM Benchao Li wrote: > > > +1 (binding) > > > > Martijn Visser 于2023年8月31日周四 15:24写道: > > > > > > +1 (binding) > > > > > > On Thu, Aug 31, 2023 at 9:09 AM Timo Walther > wrote:

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-30 Thread Jing Ge
org/confluence/display/FLINK/FLIP-33%3A+Standardize+Connector+Metrics > > > > On Aug 28, 2023, at 01:04, Jing Ge wrote: > > > > Hi Xuannan, > > > > Thanks for the proposal. +1 for me. > > > > There is one tiny thing that I am not sure if

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-27 Thread Jing Ge
Hi Xuannan, Thanks for the proposal. +1 for me. There is one tiny thing that I am not sure if I understand it correctly. Since there will be many different WatermarkStrategies and different WatermarkGenerators. Could you please update the FLIP and add the description of how the watermark lag is c

Re: [Re-DISCUSS] FLIP-202: Introduce ClickHouse Connector

2023-08-23 Thread Jing Ge
Hi Conrad, Thanks for driving it! +1 for starting a new round of discussion based on the input you wrote in the comment. Look forward to the new design. Best regards, Jing On Thu, Aug 24, 2023 at 8:00 AM ConradJam wrote: > Hi Community > > I want to re-initiate the discussion related to FLIP-2

Re: Flink 1.17.2 planned?

2023-08-23 Thread Jing Ge
also be patched by us). > > > > Kind regards, > > Christian > > > > *Von: *Jing Ge via user > *Datum: *Dienstag, 22. August 2023 um 11:40 > *An: *liu ron > *Cc: *u...@flink.apache.org > *Betreff: *Re: Flink 1.17.2 planned? > > This email has reached Mapp via

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Jing Ge
yes please On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Jing, > > Thanks for the effort and update! > > It means the PR of flink-1.19 can be merged to master branch, right? > > Best, > Rui > > On Wed, Aug 23, 2023 at 9:29 PM

[ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Jing Ge
Hi devs, The release-1.18 branch has been forked out from the master branch, with commit ID cfa4a9c35563fd8a5973ec2f35251190e365be14. The version on the master branch has been upgraded to 1.19-SNAPSHOT. From now on, for PRs that should be presented in 1.18.0, please make sure: Merge the PR into bot

Re: [DISCUSS] Update Flink Roadmap

2023-08-23 Thread Jing Ge
Beyond a > > >>> > >> SQL Stream/Batch Processing Engine" and "Large-Scale State Jobs" > > >>> > sections. > > >>> > >> > > >>> > >> Best, > > >>> > >> Jark > > >>

Fwd: [Discussion] Slack Channel

2023-08-23 Thread Jing Ge
Hi devs, Thanks Giannis for your suggestion. It seems that the last email wasn't sent to the dev ML. It is also an interesting topic for devs and user-zh. Best regards, Jing -- Forwarded message - From: Giannis Polyzos Date: Tue, Aug 22, 2023 at 11:11 AM Subject: [Discussion] Sl

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-22 Thread Jing Ge
>>>>> which is still a little complex > > >>>>>> for users and devs to understand. > > >>>>>> > > >>>>>> Best, > > >>>>>> Jark > > >>>>>> > > >>>>>

[RELEASE][TEST] Flink 1.18 Release Tests need more contributors

2023-08-22 Thread Jing Ge
Hi devs, The community is focusing on the 1.18 release. There is an umbrella ticket[1] which contains all testing tasks that should be done before we can create new release candidates. Currently, there are still some open tasks. We would like to ask the community for more volunteers to join us and

[jira] [Created] (FLINK-32906) Release Testing: Verify FLINK-30025

2023-08-21 Thread Jing Ge (Jira)
Jing Ge created FLINK-32906: --- Summary: Release Testing: Verify FLINK-30025 Key: FLINK-32906 URL: https://issues.apache.org/jira/browse/FLINK-32906 Project: Flink Issue Type: Sub-task

Re: Plans for Schema Evolution in Table API

2023-08-17 Thread Jing Ge
Hi Ashish, Table API has many restrictions. Even if the source schema has been changed in a compatible manner from the database's perspective, the added column should not be used in the downstream operators, e.g. join. In case schema evolution is required in your business, you might want to consid

Re: [DISCUSS] How about adding OLAP to Flink Roadmap?

2023-08-09 Thread Jing Ge
Hi Shammon, Hi Xiangyu, Thanks for bringing this to our attention. I can see this is a great proposal born from real business scenarios. +1 for it. People have been keen to use one platform to cover all their data production and consumption requirements. Flink did a great job for the production,

Re: [ANNOUNCE] New Apache Flink Committer - Hangxiang Yu

2023-08-08 Thread Jing Ge
Congrats, Hangxiang! Best regards, Jing On Tue, Aug 8, 2023 at 3:04 PM Yangze Guo wrote: > Congrats, Hangxiang! > > Best, > Yangze Guo > > On Tue, Aug 8, 2023 at 11:28 AM yh z wrote: > > > > Congratulations, Hangxiang ! > > > > > > Best, > > Yunhong Zheng (Swuferhong) > > > > yuxia 于2023年8月8日

Re: [ANNOUNCE] New Apache Flink Committer - Yanfei Lei

2023-08-08 Thread Jing Ge
Congrats, YanFei! Best regards, Jing On Tue, Aug 8, 2023 at 3:04 PM Yangze Guo wrote: > Congrats, Yanfei! > > Best, > Yangze Guo > > On Tue, Aug 8, 2023 at 9:20 AM yuxia wrote: > > > > Congratulations, Yanfei! > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "ron9 liu" > > 收件

Re: [DISCUSS] addition of Flink Script Operators

2023-08-04 Thread Jing Ge
Hi Kevin, Thanks for your proposal. It is quite interesting. We had a similar solution back to the Apache Storm era many years ago. The job deployment with operators itself could be considered as physical distributed task containers. There will be a management tool, like a control plane, to deploy

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

2023-08-03 Thread Jing Ge
Congrats! Matthias! Best regards, Jing On Fri, Aug 4, 2023 at 12:09 PM Yangze Guo wrote: > Congrats, Matthias! > > Best, > Yangze Guo > > On Fri, Aug 4, 2023 at 11:44 AM Qingsheng Ren wrote: > > > > Congratulations, Matthias! This is absolutely well deserved. > > > > Best, > > Qingsheng > > >

Re: [ANNOUNCE] New Apache Flink Committer - Hong Teoh

2023-08-03 Thread Jing Ge
congrats! Hong! Best regards, Jing On Fri, Aug 4, 2023 at 11:48 AM Qingsheng Ren wrote: > Congratulations and welcome aboard, Hong! > > Best, > Qingsheng > > On Fri, Aug 4, 2023 at 11:04 AM Matt Wang wrote: > > > Congratulations, Hong! > > > > > > -- > > > > Best, > > Matt Wang > > > > > > ---

Re: [ANNOUNCE] New Apache Flink Committer - Weihua Hu

2023-08-03 Thread Jing Ge
congrats! Weihua! Best regards, Jing On Fri, Aug 4, 2023 at 12:20 PM Matt Wang wrote: > Congratulations Weihua ~ > > > -- > > Best, > Matt Wang > > > Replied Message > | From | Rui Fan<1996fan...@gmail.com> | > | Date | 08/4/2023 11:29 | > | To | | > | Cc | Weihua Hu | > | Subject |

Re: [VOTE] FLIP-333: Redesign Apache Flink website

2023-08-03 Thread Jing Ge
+1, thanks for driving it! Best regards, Jing On Thu, Aug 3, 2023 at 4:49 AM Mohan, Deepthi wrote: > Hi, > > Thank you all for your feedback on FLIP-333. I’d like to start a vote. > > Discussion thread: > https://lists.apache.org/thread/z9j0rqt61ftgbkr37gzwbjg0n4fl1hsf > FLIP: > https://cwiki.a

Re: [ANNOUNCE] Flink 1.18 feature freeze

2023-08-01 Thread Jing Ge
Hi, Thanks for keeping us in the loop. +1 for merging these PRs. @Alex Please feel free to contact us for CR and PR merges. @Matthias the Pekko migration is an important task, since it is related to CVE issues. Thanks for bringing it to our attention. Best regards, Jing On Tue, Aug 1, 2023 a

Re: [DISCUSS] Add missing visibility annotation for Table APIs

2023-07-31 Thread Jing Ge
from these three modules. > > > > > > Another point is that if we allow classes without visibility > annotations, > > > Flink developers may easily forget to mark some APIs that should have > > been > > > marked as `@PublicEvolving` during development. This ph

Re: [DISCUSS] Add missing visibility annotation for Table APIs

2023-07-29 Thread Jing Ge
l and cannot promise the same compatibility > guarantee as public APIs). > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-248%3A+Introduce+dynamic+partition+pruning#:~:text=PublicEvolving%0Apublic%20class-,DynamicFilteringEvent,-implements%20SourceEvent%20%7B > > Best,

Re: [DISCUSS] Flink and Externalized connectors leads to block and circular dependency problems

2023-07-28 Thread Jing Ge
Hi Ran Tao, What is the current status? @Dian There were many options. Which one is the most feasible one you prefer? Best regards, Jing On Fri, Jul 7, 2023 at 2:37 PM Mason Chen wrote: > Hi all, > > I also agree with what's been said above. > > +1, I think the Table API delegation is a good s

Re: [DISCUSS] FLIP-326: Enhance Watermark to Support Processing-Time Temporal Join

2023-07-27 Thread Jing Ge
Hi Dong, Thanks for your proposal. It is a very interesting feature and also a complex one. Especially the "shotgun surgery"[1] of handling the useProcessingTime logic. While reading the FLIP, I was wondering if it possible to leverage the Visitor design pattern(not exactly use the pattern directl

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

2023-07-27 Thread Jing Ge
all major releases, rather than only discuss it for the 2.0 version > > bump. > > > > > > > > > I guess we should also make it explicit that we only support one LTS > > > version to reduce the number of supported versions to 3 (x.y, x.[y-1], > > &

Re: [VOTE] Release 2.0 must-have work items

2023-07-26 Thread Jing Ge
thanks everyone for the discussion. > > The back-and-force reflects the importance and complexity around this > > topic. Glad to see we finally reached consensus. > > > > Best, > > > > Xintong > > > > > > > > On Wed, Jul 26, 2023 at 12:4

Re: [VOTE][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-26 Thread Jing Ge
+1(binding). Theoretically, I agree with Matthias. But in the practices, afaic, those touched classes are low level APIs. We'd better have enough time to evaluate them with care. More time might be needed than we could have before the 1.18 release. A follow-up ticket might solve the concern. WDYT?

[DISCUSS] The adapted @deprecated usage

2023-07-26 Thread Jing Ge
Hi, There were many discussions wrt Source-/Sink-Function's deprecation. Especially after FLIP-321 has been newly introduced. One reason, which causes those discussions to happen back and forth, is that we have two different understandings of how and when to use @deprecated. This is the focus of

Re: [VOTE][2.0] FLIP-343: Remove parameter in WindowAssigner#getDefaultTrigger()

2023-07-26 Thread Jing Ge
+1 (binding) Please don't forget to update the release version to 2.0 in the FLIP. Best regards, Jing On Wed, Jul 26, 2023 at 5:49 PM Matthias Pohl wrote: > +1 (binding) > > On Wed, Jul 26, 2023 at 10:11 AM Yuxin Tan wrote: > > > +1 (non-binding) > > > > Best, > > Yuxin > > > > > > Xintong So

Re: [VOTE][2.0] FLIP-344: Remove parameter in RichFunction#open

2023-07-26 Thread Jing Ge
+1 (binding) On Wed, Jul 26, 2023 at 4:21 PM weijie guo wrote: > +1 (binding) > > Best regards, > > Weijie > > > Yuxin Tan 于2023年7月26日周三 16:11写道: > > > +1 (non-binding) > > > > Best, > > Yuxin > > > > > > Xintong Song 于2023年7月26日周三 16:09写道: > > > > > +1 (binding) > > > > > > Best, > > > > > >

Re: [VOTE] Release 2.0 must-have work items - Round 2

2023-07-26 Thread Jing Ge
+1 (non-binding), glad to see we are now on the same page. Thank you all. Best regards, Jing On Wed, Jul 26, 2023 at 5:18 PM Yun Tang wrote: > +1 (non-binding), thanks @xintong for driving this work. > > > Best > Yun Tang > > From: Zhu Zhu > Sent: Wednesday, Ju

Re: [DISCUSS] Add missing visibility annotation for Table APIs

2023-07-25 Thread Jing Ge
Hi Jane, Thanks for your effort of walking through all classes and compiling the sheet. It is quite helpful. Just out of curiosity, do we really need to mark some many classes as @Internal? What is the exactly different between a public class with no annotation and with the @Internal? Best regard

Re: [VOTE] Release 2.0 must-have work items

2023-07-25 Thread Jing Ge
Thanks Leonard for driving it. We are now on the same page. Best regards, Jing On Tue, Jul 25, 2023 at 9:19 PM Leonard Xu wrote: > We’ve detailed offline discussions with @Alexander and @Jingsong, about > “Remove SourceFunction” item, we’ve reached a consensus as following: > > 1. Deprecate Sou

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

2023-07-25 Thread Jing Ge
e are not aware of right now. > > Best, > > Konstantin > > Am Di., 25. Juli 2023 um 12:48 Uhr schrieb Jing Ge > : > > > Hi all, > > > > Overall, it is a good idea to provide the LTS release, but I'd like to > > reference a concrete case as an examp

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-07-25 Thread Jing Ge
Hi Timo, Thanks for your proposal. It is a very pragmatic feature. Among all options in the FLIP, option 3 is one I prefer too and I'd like to ask some questions to understand your thoughts. 1. I did some research on pseudo columns, just out of curiosity, do you know why most SQL systems do not n

Re: Re: [ANNOUNCE] New Apache Flink Committer - Yong Fang

2023-07-25 Thread Jing Ge
Congrats, Yong Fang! Best regards, Jing On Tue, Jul 25, 2023 at 7:35 PM Yu Li wrote: > Congrats, Yong! > > Best Regards, > Yu > > > On Tue, 25 Jul 2023 at 18:03, Sergey Nuyanzin wrote: > > > Congratulations, Yong Fang! > > > > On Tue, Jul 25, 2023 at 7:53 AM ConradJam wrote: > > > > > Congrat

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

2023-07-25 Thread Jing Ge
Hi all, Overall, it is a good idea to provide the LTS release, but I'd like to reference a concrete case as an example to understand what restrictions the LTS should have. Hypothetically, Source-/Sink- Function have been deprecated in 1.x LTS and removed in 2.0 and the issues[1] are not solved in

Re: [DISCUSS][2.0] FLIP-349: Move RocksDB statebackend classes to o.a.f.state.rocksdb package

2023-07-25 Thread Jing Ge
make sense. Best regards, Jing On Tue, Jul 25, 2023 at 4:29 PM Stefan Richter wrote: > > +1 > > > > > On 24. Jul 2023, at 12:25, Chesnay Schepler wrote: > > > > To properly reflect the state of the rocksdb statebackend I propose to > move all classes in the state-backend-rocksdb module under t

Re: [DISCUSS][2.0] FLIP-351: REST API normalizes +/-Inf / NaN to 0

2023-07-24 Thread Jing Ge
We might consider using 0 as null for values that never return 0. But null is still not equal to 0 and it will be very difficult to let every contributor in this community follow this rule, especially for future unknown APIs, which means there will be some cases that still need null. Personally, I

Re: [VOTE][2.0] FLIP-340: Remove rescale REST endpoint

2023-07-24 Thread Jing Ge
+1 On Mon, Jul 24, 2023 at 11:43 PM Chesnay Schepler wrote: > I update the endpoint in the FLIP. > > On 24/07/2023 14:28, Matthias Pohl wrote: > > I should have mentioned it in the discussion thread but I missed going > over > > that ML thread earlier: We might want to update the FLIP to refer t

Re: [DISCUSS][2.0] FLIP-352: Use camelCast for all REST API fields/parameters

2023-07-24 Thread Jing Ge
camel case is the standard code convention in Java since day one[1]. We don't need to describe it again for Flink. Legacy hyphens might be coded by contributors with other language backgrounds. If we really care about it, it might make sense to add the reference. It would be great if the FLIP coul

Re: Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-24 Thread Jing Ge
WDYT? > > > Best regards, > Wencong Liu > > > > > > > > > > > > At 2023-07-22 00:46:45, "Jing Ge" wrote: > >Hi Wencong, > > > >Thanks for the clarification. I got your point. It makes sense. > > > >Wrt IORead

Re: [VOTE] FLIP-346: Deprecate ManagedTable related APIs

2023-07-24 Thread Jing Ge
+1 On Mon, Jul 24, 2023 at 5:08 PM Timo Walther wrote: > +1 > > Regards, > Timo > > > On 24.07.23 04:00, liu ron wrote: > > +1 > > > > Best, > > Ron > > > > Lincoln Lee 于2023年7月21日周五 16:09写道: > > > >> +1 > >> > >> Best, > >> Lincoln Lee > >> > >> > >> Leonard Xu 于2023年7月21日周五 16:07写道: > >> > >

Re: [VOTE][2.0] FLIP-336: Remove "now" timestamp field from REST responses

2023-07-24 Thread Jing Ge
+1(binding) On Mon, Jul 24, 2023 at 8:55 PM Matthias Pohl wrote: > +1 (binding) > > On Mon, Jul 24, 2023 at 2:24 PM Konstantin Knauf > wrote: > > > +1 (binding) > > > > Am Mo., 24. Juli 2023 um 14:15 Uhr schrieb Martijn Visser < > > martijnvis...@apache.org>: > > > > > +1 (binding) > > > > > >

Re: Re: [DISCUSS][2.0] FLIP-347: Remove IOReadableWritable serialization in Path

2023-07-21 Thread Jing Ge
rossing multiple modules issue, I've explained it in the reply > to Matthias. > > > Best regards, > > > Wencong Liu > > > > > > > > > > > > > > > > > > > > At 2023-07-21 18:05:26, "Jing Ge" wrote:

<    1   2   3   4   5   6   7   8   >