Re: [VOTE] FLIP-378: Support Avro timestamp with local timezone

2023-11-15 Thread Jark Wu
+1 (binding) Best, Jark On Thu, 16 Nov 2023 at 12:41, Leonard Xu wrote: > +1(binding) > > Best, > Leonard > > > 2023年11月16日 下午12:13,Mingliang Liu 写道: > > > > +1 (non-binding) > > > > On Wed, Nov 15, 2023 at 3:38 PM Peter Huang > > wrote: > > > >> Hi Devs, > >> > >> I'd like to start a vote

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-11-15 Thread Jiabao Sun
Thanks Becket, I still believe that adding a configuration at the source level to disable filter pushdown is needed. This demand exists in spark as well[1]. In Spark, most sources that support filter pushdown provide their own corresponding configuration options to enable or disable filter

Re: [DISCUSS] FLIP-391: Deprecate RuntimeContext#getExecutionConfig

2023-11-15 Thread Rui Fan
Sorry, I read something wrong when I read FLIP. It's fine for me. Best, Rui On Thu, Nov 16, 2023 at 2:32 PM Junrui Lee wrote: > Hi Rui, > > Thank you for your feedback. Regarding your question, I want to let you > know that we have included the alternative getter method "@PublicEvolving > Map

Re: [DISCUSS] FLIP-391: Deprecate RuntimeContext#getExecutionConfig

2023-11-15 Thread Junrui Lee
Hi Rui, Thank you for your feedback. Regarding your question, I want to let you know that we have included the alternative getter method "@PublicEvolving Map getGlobalJobParameters()" in this FLIP. And this method will provide users with global job parameters. I would appreciate it if you could

Re: [VOTE] Release 1.17.2, release candidate #1

2023-11-15 Thread Rui Fan
+1 (non-binding) - Verified signatures - Reviewed the flink-web PR, left a couple of comments - The source archives do not contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - bin/start-cluster.sh with java8, it works fine and no any unexpected LOG - Ran

Re: [DISCUSS] FLIP-391: Deprecate RuntimeContext#getExecutionConfig

2023-11-15 Thread Rui Fan
Thanks Junrui for driving this proposal! Overall looks good to me! I have a question: How do the flink users to getGlobalJobParameters if RuntimeContext#getExecutionConfig is removed? Is there any alternative api? Or is this feature removed? Maybe this is a noob question and I may have missed

Re: [VOTE] FLIP-378: Support Avro timestamp with local timezone

2023-11-15 Thread Leonard Xu
+1(binding) Best, Leonard > 2023年11月16日 下午12:13,Mingliang Liu 写道: > > +1 (non-binding) > > On Wed, Nov 15, 2023 at 3:38 PM Peter Huang > wrote: > >> Hi Devs, >> >> I'd like to start a vote on FLIP-378: Support Avro timestamp with local >> timezone which has been discussed in this thread

Re: [DISCUSS] FLIP-379: Dynamic source parallelism inference for batch jobs

2023-11-15 Thread Leonard Xu
Thanks Xia for the detailed reply. >> `How user disable the parallelism inference if they want to use fixed source >> parallelism?` >> `Could you explain the priority the static parallelism set from table layer >> and the proposed dynamic source parallelism?` > > From the user's perspective,

Re: [VOTE] FLIP-378: Support Avro timestamp with local timezone

2023-11-15 Thread Mingliang Liu
+1 (non-binding) On Wed, Nov 15, 2023 at 3:38 PM Peter Huang wrote: > Hi Devs, > > I'd like to start a vote on FLIP-378: Support Avro timestamp with local > timezone which has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objection or >

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-15 Thread Rui Fan
Hi Zhu and Matthias: > 3. failure counting > Flink currently will try to recognize concurrent failures and group them > together, which can be seen in the web UI. So how about to align the > failure counting with the concurrent failures computing? This can make it > more consistent and easier for

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-15 Thread Rui Fan
Hi Zhu, Jing and Mingliang: Thanks for your feedback about consider exponential-delay as the default restart-strategy, and updating the default values of exponential-delay as well. I have started a discussion on user, user-zh and dev mail list about it[1]. [1]

[jira] [Created] (FLINK-33566) HBase sql-connector needs overwrite the rowKey

2023-11-15 Thread JankoWilliam (Jira)
JankoWilliam created FLINK-33566: Summary: HBase sql-connector needs overwrite the rowKey Key: FLINK-33566 URL: https://issues.apache.org/jira/browse/FLINK-33566 Project: Flink Issue Type:

[DISCUSS] Change the default restart-strategy to exponential-delay

2023-11-15 Thread Rui Fan
Hi dear flink users and devs: FLIP-364[1] intends to make some improvements to restart-strategy and discuss updating some of the default values of exponential-delay, and whether exponential-delay can be used as the default restart-strategy. After discussing at dev mail list[2], we hope to collect

Re: [DISCUSS] FLIP-391: Deprecate RuntimeContext#getExecutionConfig

2023-11-15 Thread Zhu Zhu
Thanks Junrui for creating the FLIP and kicking off this discussion. Exposing a mutable ExecutionConfig which is even shared by multiple operators is truly a defect which can result in weird results. +1 Thanks, Zhu Junrui Lee 于2023年11月15日周三 16:53写道: > Hi all, > > I'd like to start a

[jira] [Created] (FLINK-33565) The concurrentExceptions doesn't work

2023-11-15 Thread Rui Fan (Jira)
Rui Fan created FLINK-33565: --- Summary: The concurrentExceptions doesn't work Key: FLINK-33565 URL: https://issues.apache.org/jira/browse/FLINK-33565 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-33564) Implement restore tests for GroupWindowAggregate node

2023-11-15 Thread Bonnie Varghese (Jira)
Bonnie Varghese created FLINK-33564: --- Summary: Implement restore tests for GroupWindowAggregate node Key: FLINK-33564 URL: https://issues.apache.org/jira/browse/FLINK-33564 Project: Flink

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-11-15 Thread Becket Qin
Hi Jiabao, While we can always fix the formality of the config, a more fundamental issue here is whether this configuration is common enough. Personally I am still not convinced it is. Remember we don't have a common implementation for SupportsFilterPushdown itself. Why does a potential behavior

[VOTE] FLIP-378: Support Avro timestamp with local timezone

2023-11-15 Thread Peter Huang
Hi Devs, I'd like to start a vote on FLIP-378: Support Avro timestamp with local timezone which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/x/Hgt1E [2]

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-15 Thread Mingliang Liu
Thanks for sharing your data points. Among a few thousand jobs (from the smallest 1 task manager and the largest 300+ task managers), I presume most of them use the default. However, the default values we have been using were not broadly discussed but instead based on a priori knowledge as we

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-15 Thread Jing Ge
Hi Piotr, Sorry for the late reply and thanks for the proposal, it looks awesome! In the discussion, you pointed out that it is difficult to build true distributed traces. afaiu from FLIP-384 and FLIP-385, the upcoming OpenTelemetry based TraceReporter will use the same Span implementation and

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-11-15 Thread Jiabao Sun
Thanks Becket for the feedback, Regarding concerns about common configurations, I think we can introduce FiltersApplier to unify the behavior of various connectors. public static class FiltersApplier { private final ReadableConfig config; private final Function, Result> action;

[VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #2

2023-11-15 Thread Danny Cranmer
Hi everyone, Please review and vote on the release candidate #2 for the version 3.0.2 of flink-connector-gcp-pubsub, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-11-15 Thread Becket Qin
Hi Jiabao, Yes, I still have concerns. The FLIP violates the following two principles regarding configuration: 1.* A config of a class should never negate the semantic of a decorative interface implemented by that class. * A decorative interface is a public contract with other components, while

[VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

2023-11-15 Thread Gyula Fóra
Hi Everyone, Please review and vote on the release candidate #1 for the version 1.7.0 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of

Re: [DISCUSS] FLIP-393: Make QueryOperations SQL serializable

2023-11-15 Thread Benchao Li
+1 for the idea of choosing SQL as the serialization format for QueryOperation, thanks for Dawid for driving this FLIP. Regarding the implementation, I didn't see the proposal for how to translate QueryOperation to SQL yet, am I missing something? Or the FLIP is still in preliminary state, you

[jira] [Created] (FLINK-33563) Implement type inference for Agg functions

2023-11-15 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-33563: Summary: Implement type inference for Agg functions Key: FLINK-33563 URL: https://issues.apache.org/jira/browse/FLINK-33563 Project: Flink Issue

[DISCUSS] FLIP-393: Make QueryOperations SQL serializable

2023-11-15 Thread Dawid Wysakowicz
Hi, I would like to propose a follow-up improvement to some of the work that has been done over the years to the Table API. I posted the proposed changes in the FLIP-393. I'd like to get to know what others think of choosing SQL as the serialization format for QueryOperations. Regards, Dawid

[jira] [Created] (FLINK-33562) Externalize RabbitMQ Python connector code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33562: -- Summary: Externalize RabbitMQ Python connector code Key: FLINK-33562 URL: https://issues.apache.org/jira/browse/FLINK-33562 Project: Flink Issue Type:

[jira] [Created] (FLINK-33561) Externalize Pulsar Python connector code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33561: -- Summary: Externalize Pulsar Python connector code Key: FLINK-33561 URL: https://issues.apache.org/jira/browse/FLINK-33561 Project: Flink Issue Type:

[jira] [Created] (FLINK-33560) Externalize Kinesis Python connector code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33560: -- Summary: Externalize Kinesis Python connector code Key: FLINK-33560 URL: https://issues.apache.org/jira/browse/FLINK-33560 Project: Flink Issue Type:

[jira] [Created] (FLINK-33559) Externalize Kafka Python connector code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33559: -- Summary: Externalize Kafka Python connector code Key: FLINK-33559 URL: https://issues.apache.org/jira/browse/FLINK-33559 Project: Flink Issue Type:

[jira] [Created] (FLINK-33558) Externalize Elasticsearch Python connector code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33558: -- Summary: Externalize Elasticsearch Python connector code Key: FLINK-33558 URL: https://issues.apache.org/jira/browse/FLINK-33558 Project: Flink Issue

[jira] [Created] (FLINK-33557) Externalize Cassandra Python connector code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33557: -- Summary: Externalize Cassandra Python connector code Key: FLINK-33557 URL: https://issues.apache.org/jira/browse/FLINK-33557 Project: Flink Issue Type:

[jira] [Created] (FLINK-33556) Test infrastructure for externalized python code

2023-11-15 Thread Jira
Márton Balassi created FLINK-33556: -- Summary: Test infrastructure for externalized python code Key: FLINK-33556 URL: https://issues.apache.org/jira/browse/FLINK-33556 Project: Flink Issue

[jira] [Created] (FLINK-33555) LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory:

2023-11-15 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33555: - Summary: LocalRecoveryITCase.testRecoverLocallyFromProcessCrashWithWorkingDirectory: Key: FLINK-33555 URL: https://issues.apache.org/jira/browse/FLINK-33555

[jira] [Created] (FLINK-33554) SemiJoinITCase.testRewriteScalarQueryWithCorrelation3 failed due to heartbeat timeout

2023-11-15 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33554: - Summary: SemiJoinITCase.testRewriteScalarQueryWithCorrelation3 failed due to heartbeat timeout Key: FLINK-33554 URL: https://issues.apache.org/jira/browse/FLINK-33554

[jira] [Created] (FLINK-33553) FileUtilsTest.testDeleteDirectoryConcurrently failed (no progress/output)

2023-11-15 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33553: - Summary: FileUtilsTest.testDeleteDirectoryConcurrently failed (no progress/output) Key: FLINK-33553 URL: https://issues.apache.org/jira/browse/FLINK-33553 Project:

[jira] [Created] (FLINK-33552) Job (and its data) is lost

2023-11-15 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33552: - Summary: Job (and its data) is lost Key: FLINK-33552 URL: https://issues.apache.org/jira/browse/FLINK-33552 Project: Flink Issue Type: Sub-task

[SUMMARY] Flink 1.19 Release Sync 11/14/2023

2023-11-15 Thread Lincoln Lee
Hi devs and users, Yesterday was the first release sync of Flink 1.19, I’d like to share the summary: - Sync meeting We switched back to google meet because there's some account limitation for zoom on some region and the google meet is available when creator is not online. The meeting will

[jira] [Created] (FLINK-33551) HiveTemporalJoinITCase.testProcTimeTemporalJoinHiveTable failed due to

2023-11-15 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33551: - Summary: HiveTemporalJoinITCase.testProcTimeTemporalJoinHiveTable failed due to Key: FLINK-33551 URL: https://issues.apache.org/jira/browse/FLINK-33551 Project:

[jira] [Created] (FLINK-33550) "DataSet allround end-to-end test" failed due to certificate error

2023-11-15 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33550: - Summary: "DataSet allround end-to-end test" failed due to certificate error Key: FLINK-33550 URL: https://issues.apache.org/jira/browse/FLINK-33550 Project: Flink

[jira] [Created] (FLINK-33549) Exception "Factory does not implement interface YieldingOperatorFactory" thrown in batch mode

2023-11-15 Thread luoyuxia (Jira)
luoyuxia created FLINK-33549: Summary: Exception "Factory does not implement interface YieldingOperatorFactory" thrown in batch mode Key: FLINK-33549 URL: https://issues.apache.org/jira/browse/FLINK-33549

[DISCUSS] FLIP-391: Deprecate RuntimeContext#getExecutionConfig

2023-11-15 Thread Junrui Lee
Hi all, I'd like to start a discussion of FLIP-391: Deprecate RuntimeContext#getExecutionConfig[1]. Currently, the FLINK RuntimeContext is important for connecting user functions to the underlying runtime details. It provides users with necessary runtime information during job execution.

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-15 Thread Rui Fan
Thanks to Hangxiang and Feng's feedback! Best, Rui On Wed, Nov 15, 2023 at 3:37 PM Feng Jin wrote: > Hi, Rui > > Thanks for driving this. over LGTM. > > I agree that redirecting stdout to LOG at first as it is not the default > behavior . > However, if there are other scenarios in the future,