[jira] [Created] (FLINK-35348) Implement materialized table refresh rest api

2024-05-13 Thread dalongliu (Jira)
dalongliu created FLINK-35348: - Summary: Implement materialized table refresh rest api Key: FLINK-35348 URL: https://issues.apache.org/jira/browse/FLINK-35348 Project: Flink Issue Type:

[jira] [Created] (FLINK-35347) Implement InMemory workflow scheduler service and plugin to support materialized table

2024-05-13 Thread dalongliu (Jira)
dalongliu created FLINK-35347: - Summary: Implement InMemory workflow scheduler service and plugin to support materialized table Key: FLINK-35347 URL: https://issues.apache.org/jira/browse/FLINK-35347

[jira] [Created] (FLINK-35346) Introduce pluggable workflow scheduler interface for materialized table

2024-05-13 Thread dalongliu (Jira)
dalongliu created FLINK-35346: - Summary: Introduce pluggable workflow scheduler interface for materialized table Key: FLINK-35346 URL: https://issues.apache.org/jira/browse/FLINK-35346 Project: Flink

[jira] [Created] (FLINK-35345) FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table

2024-05-13 Thread dalongliu (Jira)
dalongliu created FLINK-35345: - Summary: FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table Key: FLINK-35345 URL: https://issues.apache.org/jira/browse/FLINK-35345 Project:

[RESULT][VOTE] FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table

2024-05-13 Thread Ron Liu
Hi, Dev I'm happy to announce that FLIP-448: Introduce Pluggable Workflow Scheduler Interface for Materialized Table[1] has been accepted with 8 approving votes (4 binding) [2]. - Xuyang - Feng Jin - Lincoln Lee(binding) - Jark Wu(binding) - Ron Liu(binding) - Shengkai Fang(binding) - Keith Lee

[jira] [Created] (FLINK-35344) Move same code from multiple subclasses to JdbcSourceChunkSplitter

2024-05-13 Thread Hongshun Wang (Jira)
Hongshun Wang created FLINK-35344: - Summary: Move same code from multiple subclasses to JdbcSourceChunkSplitter Key: FLINK-35344 URL: https://issues.apache.org/jira/browse/FLINK-35344 Project: Flink

[jira] [Created] (FLINK-35343) NullPointerException in SourceReaderBase

2024-05-13 Thread zyh (Jira)
zyh created FLINK-35343: --- Summary: NullPointerException in SourceReaderBase Key: FLINK-35343 URL: https://issues.apache.org/jira/browse/FLINK-35343 Project: Flink Issue Type: Bug Components:

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-13 Thread Xuannan Su
Hi all, Thank you for all the comments and suggestions! If there are no further comments, I'd like to close the discussion and start the voting in two days. Best regards, Xuannan On Mon, May 13, 2024 at 3:10 PM Jeyhun Karimov wrote: > > Hi Xuannan, > > Thanks a lot for the update. The FLIP

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-13 Thread Alexey Leonov-Vendrovskiy
Thanks Jim. > 1. For the testing, I'd call the tests "execution" tests rather than > "restore" tests. For streaming execution, restore tests have the compiled > plan and intermediate state; the tests verify that those can work together > and continue processing. Agree that we don't need to

Re: [DISCUSSION] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-05-13 Thread Jim Hughes
Hi Alexey, After some thought, I have a question about deprecations: 3. Can we remove old versions of batch operators eventually? Or do we need to keep them forever like we would for streaming operators? Cheers, Jim On Thu, May 9, 2024 at 11:29 AM Jim Hughes wrote: > Hi Alexey, > >

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread Ahmed Hamdy
Thanks David, I have replied to your question in the discussion thread. Best Regards Ahmed Hamdy On Mon, 13 May 2024 at 16:21, David Radley wrote: > Hi, > I raised a question on the discussion thread, around retriable errors, as > a possible alternative, > Kind regards, David. > > >

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-13 Thread Ahmed Hamdy
Hi David, yes there error classification was initially left to sink implementers to handle while we provided utilities to classify[1] and bubble up[2] fatal exceptions to avoid retrying them. Additionally some sink implementations provide an option to short circuit the failures by exposing a

Re: Alignment with FlinkSQL Standards | Feedback requested

2024-05-13 Thread Jim Hughes
Hi Kanchi, One can already directly query Kafka topics using Flink SQL. The Flink Kafka connector allows one to use Kafka to back a Flink table. As a concrete example, this blog post should explain things:

[jira] [Created] (FLINK-35342) MaterializedTableStatementITCase test can check for wrong status

2024-05-13 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35342: --- Summary: MaterializedTableStatementITCase test can check for wrong status Key: FLINK-35342 URL: https://issues.apache.org/jira/browse/FLINK-35342 Project: Flink

[jira] [Created] (FLINK-35341) Retraction stop working with Clock dependent function in Filter

2024-05-13 Thread Lim Qing Wei (Jira)
Lim Qing Wei created FLINK-35341: Summary: Retraction stop working with Clock dependent function in Filter Key: FLINK-35341 URL: https://issues.apache.org/jira/browse/FLINK-35341 Project: Flink

RE: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread David Radley
Hi, I raised a question on the discussion thread, around retriable errors, as a possible alternative, Kind regards, David. From: Aleksandr Pilipenko Date: Monday, 13 May 2024 at 16:07 To: dev@flink.apache.org Subject: [EXTERNAL] Re: [VOTE] FLIP-451: Introduce timeout configuration to

RE: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-13 Thread David Radley
Hi, I wonder if the way that the async request fails could be a retriable or non-retriable error, so it would retry only for retriable (transient) errors (like IOExceptions) . I see some talk on the internet around retriable SQL errors. If this was the case then we may need configuration to

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread Aleksandr Pilipenko
Thanks for driving this! +1 (non-binding) Thanks, Aleksandr On Mon, 13 May 2024 at 14:08, wrote: > Thanks Ahmed! > > +1 non binding > On May 13, 2024 at 12:40 +0200, Jeyhun Karimov , > wrote: > > Thanks for driving this Ahmed. > > > > +1 (non-binding) > > > > Regards, > > Jeyhun > > > > On

[jira] [Created] (FLINK-35340) Flink Kubernetes Operator CRDs not applying correctly

2024-05-13 Thread Neil Prasad (Jira)
Neil Prasad created FLINK-35340: --- Summary: Flink Kubernetes Operator CRDs not applying correctly Key: FLINK-35340 URL: https://issues.apache.org/jira/browse/FLINK-35340 Project: Flink Issue

[jira] [Created] (FLINK-35339) Compilation timeout while building flink-dist

2024-05-13 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35339: --- Summary: Compilation timeout while building flink-dist Key: FLINK-35339 URL: https://issues.apache.org/jira/browse/FLINK-35339 Project: Flink Issue Type: Bug

Re: [jira] [Updated] (FLINK-35336) SQL failed to restore from savepoint after change in default-parallelism

2024-05-13 Thread Martijn Visser
Hi Keith Lee, Yes, a FLIP will be required. Best regards, Martijn On Mon, May 13, 2024 at 2:51 PM Keith Lee wrote: > Thank you Martijn for confirming and switching the Jira to New Feature. > > I intend to explore approaches on implementing the feature to allow for: > > 1. Configurations that

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread lorenzo . affetti
Thanks Ahmed! +1 non binding On May 13, 2024 at 12:40 +0200, Jeyhun Karimov , wrote: > Thanks for driving this Ahmed. > > +1 (non-binding) > > Regards, > Jeyhun > > On Mon, May 13, 2024 at 12:37 PM Muhammet Orazov > wrote: > > > Thanks Ahmed, +1 (non-binding) > > > > Best, > > Muhammet > > > >

Re: [jira] [Updated] (FLINK-35336) SQL failed to restore from savepoint after change in default-parallelism

2024-05-13 Thread Keith Lee
Thank you Martijn for confirming and switching the Jira to New Feature. I intend to explore approaches on implementing the feature to allow for: 1. Configurations that will make Flink SQL job restore robust to parallelism changes 2. Configurations that will allow best effort Flink SQL job

[jira] [Created] (FLINK-35338) Enable FS Plugins as non-root

2024-05-13 Thread Rasmus Thygesen (Jira)
Rasmus Thygesen created FLINK-35338: --- Summary: Enable FS Plugins as non-root Key: FLINK-35338 URL: https://issues.apache.org/jira/browse/FLINK-35338 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-35337) Keep up with the latest version of tikv client

2024-05-13 Thread ouyangwulin (Jira)
ouyangwulin created FLINK-35337: --- Summary: Keep up with the latest version of tikv client Key: FLINK-35337 URL: https://issues.apache.org/jira/browse/FLINK-35337 Project: Flink Issue Type:

Re: [jira] [Created] (FLINK-35336) SQL failed to restore from savepoint after change in default-parallelismFlink/Flink:User

2024-05-13 Thread Keith Lee
Hi all, I ran into issue where unmodified SQL failed to restore from savepoint after change in default-parallelism. This seems to be quite a big limitation it prevents Flink SQL user from scaling freely. Would appreciate if anyone can dis/confirm that this is a bug before I start disambiguating

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread Jeyhun Karimov
Thanks for driving this Ahmed. +1 (non-binding) Regards, Jeyhun On Mon, May 13, 2024 at 12:37 PM Muhammet Orazov wrote: > Thanks Ahmed, +1 (non-binding) > > Best, > Muhammet > > On 2024-05-13 09:50, Ahmed Hamdy wrote: > > Hi all, > > > > Thanks for the feedback on the discussion thread[1], I

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread Muhammet Orazov
Thanks Ahmed, +1 (non-binding) Best, Muhammet On 2024-05-13 09:50, Ahmed Hamdy wrote: Hi all, Thanks for the feedback on the discussion thread[1], I would like to start a vote on FLIP-451[2]: Introduce timeout configuration to AsyncSink The vote will be open for at least 72 hours unless

[jira] [Created] (FLINK-35336) SQL failed to restore from savepoint after change in default-parallelismFlink/Flink:User

2024-05-13 Thread Keith Lee (Jira)
Keith Lee created FLINK-35336: - Summary: SQL failed to restore from savepoint after change in default-parallelismFlink/Flink:User Key: FLINK-35336 URL: https://issues.apache.org/jira/browse/FLINK-35336

Re: Access to Wiki for raising a Flip request

2024-05-13 Thread Jeyhun Karimov
Hi Vikas, Thanks for your request. According to the new updates from ASF infra, new users can't signup and access Confluence to create FLIPs. As initiated by Martijn and discussed by the community [1], the current approach for the new users is to create a Google Doc and make that view-only, and

RE: Discussion: Condition field in the CR status

2024-05-13 Thread David Radley
Hi Lajith, This idea for a Flip is a good addition, which I support. As discussed: These conditions populate a status field in the K8s UI – Openshift in our case. Currently the status does not contain any information. With the conditions present, the status will be populated with meaningful

[DISCUSS] FLIP-XXX: Improve JDBC connector extensibility for Table API

2024-05-13 Thread lorenzo . affetti
Hello dev! I want to share a draft of my FLIP to refactor the JDBC connector to improve its extensibility [1]. The goal is to allow implementers to write new connectors on top of the JDBC one for Table API with clean and maintainable code. Any feedback from the community is more and welcome.

[jira] [Created] (FLINK-35335) StateCheckpointedITCase failed fatally with 127 exit code

2024-05-13 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35335: --- Summary: StateCheckpointedITCase failed fatally with 127 exit code Key: FLINK-35335 URL: https://issues.apache.org/jira/browse/FLINK-35335 Project: Flink

Access to Wiki for raising a Flip request

2024-05-13 Thread Vikas Garg
Hello, Can you grant me the access on Confluence? I have to raise a FLIP on the site for Vertica Flink connector. Vikas Garg General Engineering | General Engineering Mobile: (+91) 9811716552 LinkedIn: https://www.linkedin.com/company/opentext Twitter:

Re: [DISCUSS] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

2024-05-13 Thread Ahmed Hamdy
Thanks for the clarification. +1 to starting the vote. Best Regards Ahmed Hamdy On Mon, 13 May 2024 at 09:10, Martijn Visser wrote: > Hi Ahmed, > > There's no reason to refrain from releases for Flink 1.* versions: these > connector implementations are still supported in the Flink 1.* series.

[VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-13 Thread Ahmed Hamdy
Hi all, Thanks for the feedback on the discussion thread[1], I would like to start a vote on FLIP-451[2]: Introduce timeout configuration to AsyncSink The vote will be open for at least 72 hours unless there is an objection or insufficient votes.

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-13 Thread Muhammet Orazov
Great, thanks for clarifying! Best, Muhammet On 2024-05-06 13:40, Ahmed Hamdy wrote: Hi Muhammet, Thanks for the feedback. Could you please add more here why it is harder? Would the `completeExceptionally` method be related to it? Maybe you can add usage example for it also. this is

Re: Discussion: Condition field in the CR status

2024-05-13 Thread Lajith Koova
Thanks for the feedback Jeyhun. Regarding Will the usage of Conditions be enabled by default? Or will there be any disadvantages for Flink users . Yes the conditions will be enabled by default for the CR. You are right , when there are multiple conditions of the same type, approach is we will

Re: Discussion: Condition field in the CR status

2024-05-13 Thread Lajith Koova
Thanks for the feedback Gyula. I have updated a couple of samples with proposed conditions. Thanks Lajith K On Fri, May 3, 2024 at 12:24 PM Gyula Fóra wrote: > Hi Lajith! > > Can you please include some examples in the document to help reviewers? > Just some examples with the status and the

[jira] [Created] (FLINK-35334) Code generation: init method exceeds 64 KB when there is a long array field with Table API

2024-05-13 Thread Simon Lee (Jira)
Simon Lee created FLINK-35334: - Summary: Code generation: init method exceeds 64 KB when there is a long array field with Table API Key: FLINK-35334 URL: https://issues.apache.org/jira/browse/FLINK-35334

[jira] [Created] (FLINK-35333) JdbcXaSinkTestBase fails in weekly Flink JDBC Connector tests

2024-05-13 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-35333: -- Summary: JdbcXaSinkTestBase fails in weekly Flink JDBC Connector tests Key: FLINK-35333 URL: https://issues.apache.org/jira/browse/FLINK-35333 Project: Flink

Re: [DISCUSS] FLIP-453: Promote Unified Sink API V2 to Public and Deprecate SinkFunction

2024-05-13 Thread Martijn Visser
Hi Ahmed, There's no reason to refrain from releases for Flink 1.* versions: these connector implementations are still supported in the Flink 1.* series. Best regards, Martijn On Sun, May 12, 2024 at 5:55 PM Ahmed Hamdy wrote: > Thanks Martijn > I believe you missed my question, > > Should

[jira] [Created] (FLINK-35332) Manually setting rest.bind-address does not work for Flink running on Hadoop Yarn dual network environment

2024-05-13 Thread Yao Zhang (Jira)
Yao Zhang created FLINK-35332: - Summary: Manually setting rest.bind-address does not work for Flink running on Hadoop Yarn dual network environment Key: FLINK-35332 URL:

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

2024-05-13 Thread Jeyhun Karimov
Hi Xuannan, Thanks a lot for the update. The FLIP looks good to me. +1 for it. Regards, Jeyhun On Mon, May 13, 2024 at 4:45 AM Xuannan Su wrote: > Hi Jeyhun, > > Thanks for the comment! > > Yes, we intended to remove the StreamPiplineOptions in 2.0. I updated > the FLIP to include the