Re: [DISCUSS] FLIP-285: Refactoring the leader election code base in Flink

2023-01-24 Thread Yang Wang
Having the *start()* in *LeaderContender* interface and bringing back the *LeaderElection* with some new methods make sense to me. I have no more concerns now. >- *LeaderContender*: The LeaderContender is integrated as usual except >that it accesses the LeaderElection instead of the

[ANNOUNCE] Share your Streaming Stories with us at Current 2023

2023-01-24 Thread Israel Ekpo
Do you have a great data streaming story to share? We want to hear from you! Speaking at Current 2023 is a great way to connect with hundreds of your peers, become more involved in the data streaming community, and have a public platform for you to share your story of the future of streaming and

[jira] [Created] (FLINK-30782) Use https for schemaLocations

2023-01-24 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-30782: --- Summary: Use https for schemaLocations Key: FLINK-30782 URL: https://issues.apache.org/jira/browse/FLINK-30782 Project: Flink Issue Type: Technical

Re: Re: Stateful Functions with Flink 1.15 and onwards

2023-01-24 Thread Artun Duman
Hi, I am also curious about the current state of development for Stateful Functions. Has the community moved to a new solution for remote functions that I need to be aware of? Is anyone actively using this project? Or is the lack of new development because this is a thin layer/SDK on top of Flink

[jira] [Created] (FLINK-30781) Add completed checkpoint check to cluster health check

2023-01-24 Thread Gabor Somogyi (Jira)
Gabor Somogyi created FLINK-30781: - Summary: Add completed checkpoint check to cluster health check Key: FLINK-30781 URL: https://issues.apache.org/jira/browse/FLINK-30781 Project: Flink

Re: [VOTE] Release flink-connector-hbase v3.0.0, release candidate #1

2023-01-24 Thread Ferenc Csaky
Hi! According to the last comment on the e2e test PR [1], it would be okay to release the connector in its current format and deliver a new/rewritten test later. Every other issue should be resolved. [1] https://github.com/apache/flink-connector-hbase/pull/5 --- Original Message ---

Re: [DISCUSS] Externalize AWS formats to flink-connector-aws

2023-01-24 Thread Teoh, Hong
Hi all, Thanks for the feedback. @Martijn Yes that is an interesting question. I agree with Danny that we should have a future discussion about the S3 filesystem, and we might treat it differently given it’s a "core" feature. It would naturally come together with the flink-s3-fs-hadoop and

Re: [DISCUSS] Externalize AWS formats to flink-connector-aws

2023-01-24 Thread Martijn Visser
+1 Op di 24 jan. 2023 om 15:44 schreef Danny Cranmer : > Hello Hong, > > Thanks for starting this discussion. +1 from me, the GSR formats are still > technically "AWS connectors" so it makes sense to move them. Especially > since they have a common dependency on flink-connector-aws-base and we

Re: [DISCUSS] Externalize AWS formats to flink-connector-aws

2023-01-24 Thread Danny Cranmer
Hello Hong, Thanks for starting this discussion. +1 from me, the GSR formats are still technically "AWS connectors" so it makes sense to move them. Especially since they have a common dependency on flink-connector-aws-base and we are targeting this repository for the Native Glue SQL integration

[jira] [Created] (FLINK-30780) Develop ElasticCatalog to connect Flink with Elastic tables and ecosystem

2023-01-24 Thread Jira
Grzegorz Kołakowski created FLINK-30780: --- Summary: Develop ElasticCatalog to connect Flink with Elastic tables and ecosystem Key: FLINK-30780 URL: https://issues.apache.org/jira/browse/FLINK-30780

[jira] [Created] (FLINK-30779) Bomp josdk version to 4.3.2

2023-01-24 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-30779: -- Summary: Bomp josdk version to 4.3.2 Key: FLINK-30779 URL: https://issues.apache.org/jira/browse/FLINK-30779 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Externalize AWS formats to flink-connector-aws

2023-01-24 Thread Martijn Visser
Hi Hong, I think it's a good idea, but it does make up for another interesting question since the AWS SDK is also used in flink-s3-fs-base. I'm assuming that will keep for now or do you also have plans for that in the future? Best regards, Martijn Op di 24 jan. 2023 om 12:52 schreef Jing Ge :

Re: [DISCUSS] Externalize AWS formats to flink-connector-aws

2023-01-24 Thread Jing Ge
Hi, +1, I think, for those formats that are only used by one or few specific connectors, it is a good idea to externalize them. Best regards, Jing On Tue, Jan 24, 2023 at 9:44 AM Ahmed Hamdy wrote: > Hi Hong > +1 for the proposal. Additionally I believe it serves the same value for > the

RE: Re: Stateful Functions with Flink 1.15 and onwards

2023-01-24 Thread Dheeraj Balakavi
Hello! Is there any update on this thread? I’m happy to help in any way I can too. On 2023/01/19 16:04:53 Galen Warren wrote: > I thought I'd check in again here, is there any update? If there's anything > I can do to help with releasing statefun 3.3, just let me know. Thanks. > > On Mon, Nov

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-01-24 Thread Jing Ge
Hi Martijn, Thanks for sharing more information to help us have a clear big picture. 1) I agree with you. We should graduate the SinkV2 API asap. I have also tried many times in the past and there were always some issues that postponed the graduation and they made sense. One rule to check if an

[jira] [Created] (FLINK-30778) Add Configuration into FunctionContext

2023-01-24 Thread Alexander Smirnov (Jira)
Alexander Smirnov created FLINK-30778: - Summary: Add Configuration into FunctionContext Key: FLINK-30778 URL: https://issues.apache.org/jira/browse/FLINK-30778 Project: Flink Issue Type:

Re: [DISCUSS] Externalize AWS formats to flink-connector-aws

2023-01-24 Thread Ahmed Hamdy
Hi Hong +1 for the proposal. Additionally I believe it serves the same value for the schema classes as it does for connectors. Best Regards, Ahmed Hamdy On Tue, 24 Jan 2023 at 07:51, Teoh, Hong wrote: > Hi all, > > As part of the effort to externalize connectors, I would like to propose > that