Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

2023-01-04 Thread yh z
Hi Benchao, Thanks for your reply. Since our existing test results are based on multiple performance optimization points on the TPC-DS benchmark[1][2], we haven't separately tested the performance improvement brought by new bushy join reorder rule. I will complete this test recently and update

[jira] [Created] (FLINK-30569) File Format can not change with data file exists

2023-01-04 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-30569: Summary: File Format can not change with data file exists Key: FLINK-30569 URL: https://issues.apache.org/jira/browse/FLINK-30569 Project: Flink Issue Type:

[jira] [Created] (FLINK-30568) Add benchmark for PolyNomialExpansion, Normalizer, Binarizer, Interaction, MaxAbsScaler, VectorSlicer, ElementWiseProduct and Featurehasher

2023-01-04 Thread weibo zhao (Jira)
weibo zhao created FLINK-30568: -- Summary: Add benchmark for PolyNomialExpansion, Normalizer, Binarizer, Interaction, MaxAbsScaler, VectorSlicer, ElementWiseProduct and Featurehasher Key: FLINK-30568 URL:

[jira] [Created] (FLINK-30567) Wrong insert overwrite behavior when the table contains uppercase character with Hive dialect

2023-01-04 Thread luoyuxia (Jira)
luoyuxia created FLINK-30567: Summary: Wrong insert overwrite behavior when the table contains uppercase character with Hive dialect Key: FLINK-30567 URL: https://issues.apache.org/jira/browse/FLINK-30567

[jira] [Created] (FLINK-30566) Add benchmark configurations for agglomerativeclustering, hashingtf, idf, kbinsdiscretizer, linearregression, linearsvc, logisticregression, ngram, regextokenizer, toke

2023-01-04 Thread Zhipeng Zhang (Jira)
Zhipeng Zhang created FLINK-30566: - Summary: Add benchmark configurations for agglomerativeclustering, hashingtf, idf, kbinsdiscretizer, linearregression, linearsvc, logisticregression, ngram, regextokenizer, tokenizer and vectorindexer

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-04 Thread Jark Wu
Hi Adam, Thanks for sharing this interesting project. I think it definitely is valuable for users for better speed. I am one of the maintainers of flink-cdc-connector project. The project offers a “oracle-cdc” connector which uses Debezium (depends on LogMiner) as the CDC library. From the

[jira] [Created] (FLINK-30565) Flink-parquet free for flink-table-store-format

2023-01-04 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-30565: Summary: Flink-parquet free for flink-table-store-format Key: FLINK-30565 URL: https://issues.apache.org/jira/browse/FLINK-30565 Project: Flink Issue Type:

Re: [VOTE] FLIP-281: Sink Supports Speculative Execution For Batch Job

2023-01-04 Thread Biao Liu
Hi Martijn, Sure, thanks for the reminder about the holiday period. Looking forward to your feedback! Thanks, Biao /'bɪ.aʊ/ On Thu, 5 Jan 2023 at 03:07, Martijn Visser wrote: > Hi Biao, > > To be honest, I haven't read the FLIP yet since this is still a holiday > period in Europe. I would

Re: [DISCUSS] Release Flink 1.16.1

2023-01-04 Thread Dong Lin
Hi Martijn, Thank you for bringing this up! I am also in favor of releasing 1.16.1. Regards, Dong On Fri, Dec 16, 2022 at 2:53 AM Martijn Visser wrote: > Hi everyone, > > I would like to open a discussion about releasing Flink 1.16.1. We've > released Flink 1.16 at the end of October, but we

[jira] [Created] (FLINK-30564) Select from a new table with Kafka LogStore crashes with UnknownTopicOrPartitionException

2023-01-04 Thread Alex Sorokoumov (Jira)
Alex Sorokoumov created FLINK-30564: --- Summary: Select from a new table with Kafka LogStore crashes with UnknownTopicOrPartitionException Key: FLINK-30564 URL: https://issues.apache.org/jira/browse/FLINK-30564

Re: CDC from Oracle database reading directly logs - integration with OpenLogReplicator

2023-01-04 Thread Adam Leszczyński
H Márton, Thank you very much for your answer. The point with Kafka makes sense. It offers huge bag of potential connectors that could be used. But … not everybody wants or needs Kafka. This brings additional architectural complication and delays, which might not be acceptable by everybody.

[VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

2023-01-04 Thread Hao t Chang
I did the following: Ran OLM bundle CI test suite for Kubernetes. Generated and Deployed OLM bundle. Created standalone/session jobs. All Look good. Thanks for managing the release! -- Best, Ted Chang | Software Engineer | htch...@us.ibm.com

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-04 Thread Martijn Visser
Hi Hang, I haven't had time to read the FLIP yet since this is still a holiday period in Europe. I would like to read it in the next few days. Can you keep the vote open a little longer? Best regards, Martijn On Wed, Jan 4, 2023 at 2:01 PM Dong Lin wrote: > Thanks for proposing the FLIP! > >

Re: [VOTE] FLIP-281: Sink Supports Speculative Execution For Batch Job

2023-01-04 Thread Martijn Visser
Hi Biao, To be honest, I haven't read the FLIP yet since this is still a holiday period in Europe. I would like to read it in the next few days. Can you keep the vote open a little longer? Best regards, Martijn On Wed, Jan 4, 2023 at 1:31 PM Biao Liu wrote: > Hi everyone, > > Thanks for all

[jira] [Created] (FLINK-30563) Update training exercises to use Flink 1.16

2023-01-04 Thread David Anderson (Jira)
David Anderson created FLINK-30563: -- Summary: Update training exercises to use Flink 1.16 Key: FLINK-30563 URL: https://issues.apache.org/jira/browse/FLINK-30563 Project: Flink Issue Type:

Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

2023-01-04 Thread John Roesler
Hi Wencong, Thanks for the proposal! I agree that we should fix this disconnect between the metric and what is actually happening in those source connectors. My instincts agree with Dong's. Adding a configuration option in order to tune the relationship between a superclass and a subclass

Re: [VOTE]FLIP-266: Simplify network memory configurations for TaskManager

2023-01-04 Thread Lijie Wang
+1 (binding) Best, Lijie 17610775726 <17610775...@163.com> 于2023年1月4日周三 13:03写道: > > > +1 (no binding) > > > Best > JasonLee > > > Replied Message > | From | Yuxin Tan | > | Date | 01/3/2023 17:56 | > | To | | > | Subject | [VOTE]FLIP-266: Simplify network memory configurations for >

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

2023-01-04 Thread Geng Biao
Thanks Gyula for the release! +1(non-binding). Successfully verified the following: - Checksums and gpg signatures of the tar files and check licenses in source code - No binaries in source release - Build from source, build image from source without errors - Helm Repo works, Helm install works

[jira] [Created] (FLINK-30562) Patterns are not emitted with parallelism >1 since 1.15.x+

2023-01-04 Thread Thomas Wozniakowski (Jira)
Thomas Wozniakowski created FLINK-30562: --- Summary: Patterns are not emitted with parallelism >1 since 1.15.x+ Key: FLINK-30562 URL: https://issues.apache.org/jira/browse/FLINK-30562 Project:

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.3.1, release candidate #1

2023-01-04 Thread Gyula Fóra
+1 (binding) - Verified hashes, signatures, notice files and source release. - Verified and tested Helm chart/repo - Ran stateful examples with complex upgrade scenarios involving failures, verified all works as expected and logs look good. Cheers, Gyula On Wed, Jan 4, 2023 at 12:49 AM Jim

Re: [VOTE] FLIP-274: Introduce metric group for OperatorCoordinator

2023-01-04 Thread Dong Lin
Thanks for proposing the FLIP! +1 (binding) Regards, Dong On Wed, Jan 4, 2023 at 10:08 AM Hang Ruan wrote: > Hi all, > > Thanks for all the feedback so far. > Based on the discussion[1], we have come to a consensus, so I would like to > start a vote on FLIP-274: Introduce metric group for >

Re: [DISCUSS] Allow source readers extending SourceReaderBase to override numRecordsIn report logic

2023-01-04 Thread Dong Lin
Hi Wencong, Thanks for kicking off the discussion! I think it would be nice to address this problem. Is the config supposed to be publicly visible only to source connector developers but not to end users? It might be a bit unusual to have a subclass use a config to disable a public feature in

[jira] [Created] (FLINK-30561) ChangelogStreamHandleReaderWithCache cause FileNotFoundException

2023-01-04 Thread Feifan Wang (Jira)
Feifan Wang created FLINK-30561: --- Summary: ChangelogStreamHandleReaderWithCache cause FileNotFoundException Key: FLINK-30561 URL: https://issues.apache.org/jira/browse/FLINK-30561 Project: Flink

Re: [DISCUSS] Release Flink 1.16.1

2023-01-04 Thread Lincoln Lee
Hi Martijn, Both FLINK-28988 and FLINK-29849 have been picked into 1.16 branch and the 'Release Note' content of the jira has been updated. Best, Lincoln Lee Zhu Zhu 于2022年12月23日周五 10:51写道: > Hi Martjin, > > Thank you for bringing this up! +1 to release 1.16.1. > > There's a critical problem

[VOTE] FLIP-281: Sink Supports Speculative Execution For Batch Job

2023-01-04 Thread Biao Liu
Hi everyone, Thanks for all the feedback! Based on the discussion[1], we seem to have a consensus. So I'd like to start a vote on FLIP-281: Sink Supports Speculative Execution For Batch Job[2]. The vote will last for 72 hours, unless there is an objection or insufficient votes. [1]

[jira] [Created] (FLINK-30560) Add more description of 'Overwriting a Partition' to doc 'Writing Tables'

2023-01-04 Thread yuzelin (Jira)
yuzelin created FLINK-30560: --- Summary: Add more description of 'Overwriting a Partition' to doc 'Writing Tables' Key: FLINK-30560 URL: https://issues.apache.org/jira/browse/FLINK-30560 Project: Flink

Re: [DISCUSS] Adding a option for planner to decide which join reorder rule to choose

2023-01-04 Thread Benchao Li
Hi Yunhong, Thanks for the updating. And introducing the new bushy join reorder algorithm would be great. And I also agree with the newly added config option "table.optimizer.bushy-join-reorder-threshold" and 12 as the default value. > As for optimization > latency, this is the problem to be

[jira] [Created] (FLINK-30559) May get wrong result for `if` expression if it's string data type

2023-01-04 Thread luoyuxia (Jira)
luoyuxia created FLINK-30559: Summary: May get wrong result for `if` expression if it's string data type Key: FLINK-30559 URL: https://issues.apache.org/jira/browse/FLINK-30559 Project: Flink

Re: [DISCUSS] FLIP-283: Use adaptive batch scheduler as default scheduler for batch jobs

2023-01-04 Thread Xintong Song
Thanks for the proposal. Another potential benefit I see in this FLIP is that it may reduce the complexity and maintenance overhead of the scheduler. During developing hybrid shuffle, we had to re-implement some similar logic to make both default and adaptive batch schedulers support the new

[jira] [Created] (FLINK-30558) The metric 'numRestarts' reported in SchedulerBase will be overridden by metric 'fullRestarts'

2023-01-04 Thread xingbe (Jira)
xingbe created FLINK-30558: -- Summary: The metric 'numRestarts' reported in SchedulerBase will be overridden by metric 'fullRestarts' Key: FLINK-30558 URL: https://issues.apache.org/jira/browse/FLINK-30558

[jira] [Created] (FLINK-30557) Remove flink-connector-aws-kinesis-streams from Flink master branch

2023-01-04 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-30557: - Summary: Remove flink-connector-aws-kinesis-streams from Flink master branch Key: FLINK-30557 URL: https://issues.apache.org/jira/browse/FLINK-30557 Project: Flink

[jira] [Created] (FLINK-30556) Improve the logic for enumerating splits for Hive source to avoid potential OOM

2023-01-04 Thread luoyuxia (Jira)
luoyuxia created FLINK-30556: Summary: Improve the logic for enumerating splits for Hive source to avoid potential OOM Key: FLINK-30556 URL: https://issues.apache.org/jira/browse/FLINK-30556 Project: