[jira] [Created] (FLINK-33543) Move Java 17-specific code into its own module to please Intellij

2023-11-14 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33543: - Summary: Move Java 17-specific code into its own module to please Intellij Key: FLINK-33543 URL: https://issues.apache.org/jira/browse/FLINK-33543 Project: Flink

[jira] [Created] (FLINK-33542) Update HBase connector tests to JUnit5

2023-11-14 Thread Ferenc Csaky (Jira)
Ferenc Csaky created FLINK-33542: Summary: Update HBase connector tests to JUnit5 Key: FLINK-33542 URL: https://issues.apache.org/jira/browse/FLINK-33542 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-14 Thread Rui Fan
Thanks a lot Zhu and Jing for the comments! Regarding concurrent failures mentioned by zhu, I am not familiar with it before and need some time to get familiar with it. So I will reply to them later. I will give Jing an answer first: > NIT: @Rui it would be great if you could point out the

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

2023-11-14 Thread Hongshun Wang
Hi Becket, I agree with you and try to modify this Flip[1], which include these changes: 1. Mark constructor of SingleThreadMultiplexSourceReaderBase as @Depricated 2. Mark constructor of SourceReaderBase as *@Depricated* and provide a new constructor without

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-14 Thread Piotr Nowojski
Hi All, Thanks for the answers! Unless there are some objections or suggestions, I will open a voting thread later this week. > My original thought was to show how much time a sampled record is processed > within each operator in stream processing. By saying 'sampled', I mean we > won't

[DISCUSS] REST API behaviour when user main method throws error

2023-11-14 Thread Danny Cranmer
Hey all, We run Flink clusters in session mode; we upload the user jar and then invoke "/jars/:jarid/run" [1] REST API endpoint. We have noticed a discrepancy in the run endpoint and were hoping to get some feedback from the community before proposing a FLIP or Jira to fix it. Some problem

[jira] [Created] (FLINK-33541) RAND_INTEGER can't be existed in a IF statement

2023-11-14 Thread Guojun Li (Jira)
Guojun Li created FLINK-33541: - Summary: RAND_INTEGER can't be existed in a IF statement Key: FLINK-33541 URL: https://issues.apache.org/jira/browse/FLINK-33541 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-33540) Test CI on arm

2023-11-14 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33540: - Summary: Test CI on arm Key: FLINK-33540 URL: https://issues.apache.org/jira/browse/FLINK-33540 Project: Flink Issue Type: Sub-task Reporter:

Re: [DISCUSS] FLIP-378: Support Avro timestamp with local timezone

2023-11-14 Thread Jing Ge
+1 Thanks for the proposal! Best regards, Jing On Tue, Nov 14, 2023 at 8:46 AM Mingliang Liu wrote: > +1 on this effort. > > I opened a JIRA sometime ago on this topic [1] and Jagadesh submitted a PR > [2] for review. In the comments, James proposed a different approach from > the PR. I don't

[jira] [Created] (FLINK-33544) Flink documentation fails to build

2023-11-14 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-33544: -- Summary: Flink documentation fails to build Key: FLINK-33544 URL: https://issues.apache.org/jira/browse/FLINK-33544 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-378: Support Avro timestamp with local timezone

2023-11-14 Thread Peter Huang
Thanks for all of the feedback so far. I will update the FLIP accordingly, and start a vote thread later today. On Tue, Nov 14, 2023 at 6:47 AM Jing Ge wrote: > +1 Thanks for the proposal! > > Best regards, > Jing > > On Tue, Nov 14, 2023 at 8:46 AM Mingliang Liu wrote: > >> +1 on this effort.

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Hangxiang Yu
Hi, Rui. As I mentioned before: the user didn't really want > to log into taskmanager.out, it just happened by accident. > So, if users change the System.out to LOG.info, it still happen. Thanks for the feedback. I saw some users may debug operator correctness using print() or System.out which

[jira] [Created] (FLINK-33547) Primitive SQL array type after upgrading to Flink 1.18.0

2023-11-14 Thread Xingcan Cui (Jira)
Xingcan Cui created FLINK-33547: --- Summary: Primitive SQL array type after upgrading to Flink 1.18.0 Key: FLINK-33547 URL: https://issues.apache.org/jira/browse/FLINK-33547 Project: Flink Issue

Re: [VOTE] Release flink-shaded 16.2, release candidate #1

2023-11-14 Thread Xintong Song
+1 (binding) - verified checksum and signature - verified license and notice file - verified source archive does not contain any binary - reviewed the release notes and web pr Best, Xintong On Mon, Nov 13, 2023 at 4:21 PM Yuxin Tan wrote: > Thanks weijie for driving the new release! > > +1

[jira] [Created] (FLINK-33546) Implement type inference for order(asc/desc) function

2023-11-14 Thread Jacky Lau (Jira)
Jacky Lau created FLINK-33546: - Summary: Implement type inference for order(asc/desc) function Key: FLINK-33546 URL: https://issues.apache.org/jira/browse/FLINK-33546 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-389: Annotate SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving

2023-11-14 Thread Becket Qin
Hi Hongshun, > > > However, it will be tricky because SplitFetcherManager includes extends SourceSplit>, while FutureCompletingBlockingQueue includes . > This means that SplitFetcherManager would have to be modified to SplitT extends SourceSplit>, which would affect the compatibility of the >

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-14 Thread Mingliang Liu
Thanks Rui for driving this. I just call out that making exponential-delay the default is a good change. At Netflix, we have enabled this as the default restart strategy 2 quarters ago and it has been working well. Keeping it restarting indefinitely by default makes sense to me. On Mon, Oct 16,

[jira] [Created] (FLINK-33545) KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching

2023-11-14 Thread Kevin Tseng (Jira)
Kevin Tseng created FLINK-33545: --- Summary: KafkaSink implementation can cause dataloss during broker issue when not using EXACTLY_ONCE if there's any batching Key: FLINK-33545 URL:

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-14 Thread Jing Ge
awesome! @Rui Thanks for your effort! Appreciate it! Best regards, Jing On Tue, Nov 14, 2023 at 1:32 PM Rui Fan <1996fan...@gmail.com> wrote: > Thanks a lot Zhu and Jing for the comments! > > Regarding concurrent failures mentioned by zhu, I am not familiar with it > before > and need some time

Re: [DISCUSS] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources

2023-11-14 Thread Jiabao Sun
Hi Becket, The purpose of introducing this configuration is that not all filter pushdowns can improve overall performance. If the filter can hit the external index, then pushdown is definitely worth it, as it can not only improve query time but also decrease network overhead. However, for

[RESULT][VOTE] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects

2023-11-14 Thread Junrui Lee
Hi all, The voting time of FLIP-381[1]: Deprecate configuration getters/setters that return/set complex Java objects has passed. I'm closing the vote now. There were 12 +1 votes, 6 of which are binding: Rui Fan (binding) Weijie Guo (binding) Yangze Guo (binding) Lijie Wang (binding) Jing Ge

[jira] [Created] (FLINK-33548) Replaces taskManager/jobmanager.resource with resources conforming to ResourceRequirements

2023-11-14 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-33548: -- Summary: Replaces taskManager/jobmanager.resource with resources conforming to ResourceRequirements Key: FLINK-33548 URL: https://issues.apache.org/jira/browse/FLINK-33548

Re: [DISCUSS] FLIP-364: Improve the restart-strategy

2023-11-14 Thread Rui Fan
Hi Mingliang: Thanks you for the feedback here! Glad to hear Netflix have made exponential-delay as the default restart strategy. Our production(Shopee) also makes exponential-delay as the default since May 2021, and the current number of flink jobs far exceeds tens of thousands. These jobs work

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Rui Fan
Hi Hangxiang, Thanks for your feedback! > I saw some users may debug operator correctness using print() or System.out > which may bring lots of info. > It's fine if files are separate, otherwise these will make taskMnanager.log > roll quickly. > But I think it's fine if we don't use it as

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Hangxiang Yu
Hi, Rui. Overall LGTM now. Thanks for driving this again! On Wed, Nov 15, 2023 at 2:07 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Hangxiang, > > Thanks for your feedback! > > > I saw some users may debug operator correctness using print() or > System.out > > which may bring lots of info. > >

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Feng Jin
Hi, Rui Thanks for driving this. over LGTM. I agree that redirecting stdout to LOG at first as it is not the default behavior . However, if there are other scenarios in the future, we can consider adding additional parameters to redirect stdout to a separate file. Best, Feng On Wed, Nov 15,