Re: [DISCUSS] FLIP-248: Introduce dynamic partition pruning

2022-07-25 Thread godfrey he
Thanks for all the inputs, I have updated the document and POC code. Best, Godfrey Yun Gao 于2022年7月26日周二 11:11写道: > > Hi, > > Thanks all for all the valuable discussion on this FLIP, +1 for implementing > dynamic partition pruning / dynamic filtering pushdown since it is a key > optimization

[jira] [Created] (FLINK-28685) Optimize CatalogManager getTable implementation

2022-07-25 Thread Paul Lin (Jira)
Paul Lin created FLINK-28685: Summary: Optimize CatalogManager getTable implementation Key: FLINK-28685 URL: https://issues.apache.org/jira/browse/FLINK-28685 Project: Flink Issue Type:

[jira] [Created] (FLINK-28684) NullPointerException at OneHotEncoder.GenerateModelDataOperator.snapshot

2022-07-25 Thread Zhipeng Zhang (Jira)
Zhipeng Zhang created FLINK-28684: - Summary: NullPointerException at OneHotEncoder.GenerateModelDataOperator.snapshot Key: FLINK-28684 URL: https://issues.apache.org/jira/browse/FLINK-28684 Project:

[jira] [Created] (FLINK-28683) Modify changelog-file to changelog-producer

2022-07-25 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28683: Summary: Modify changelog-file to changelog-producer Key: FLINK-28683 URL: https://issues.apache.org/jira/browse/FLINK-28683 Project: Flink Issue Type:

[jira] [Created] (FLINK-28682) support join hint in flink batch

2022-07-25 Thread xuyang (Jira)
xuyang created FLINK-28682: -- Summary: support join hint in flink batch Key: FLINK-28682 URL: https://issues.apache.org/jira/browse/FLINK-28682 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-28681) The number of windows allocated by sliding windows is inaccurate, when the window length is irregular

2022-07-25 Thread nyingping (Jira)
nyingping created FLINK-28681: - Summary: The number of windows allocated by sliding windows is inaccurate, when the window length is irregular Key: FLINK-28681 URL: https://issues.apache.org/jira/browse/FLINK-28681

Re: [DISCUSS] FLIP-248: Introduce dynamic partition pruning

2022-07-25 Thread Yun Gao
Hi, Thanks all for all the valuable discussion on this FLIP, +1 for implementing dynamic partition pruning / dynamic filtering pushdown since it is a key optimization to improve the performance on batch processing. Also due to introducing the speculative execution for the batch processing, we

[jira] [Created] (FLINK-28680) No space left on device on Azure e2e_2_ci tests

2022-07-25 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-28680: Summary: No space left on device on Azure e2e_2_ci tests Key: FLINK-28680 URL: https://issues.apache.org/jira/browse/FLINK-28680 Project: Flink Issue Type:

Re: [VOTE] FLIP-247 Bulk fetch of table and column statistics for given partitions

2022-07-25 Thread godfrey he
+1 Best, Godfrey Jark Wu 于2022年7月25日周一 17:23写道: > > +1 (binding) > > Best, > Jark > > On Mon, 25 Jul 2022 at 15:10, Jing Ge wrote: > > > Hi all, > > > > Many thanks for all your feedback. Based on the discussion in [1], I'd like > > to start a vote on FLIP-247 [2]. > > > > The vote will last

[jira] [Created] (FLINK-28679) HiveServer2Endpoint compile failed

2022-07-25 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-28679: Summary: HiveServer2Endpoint compile failed Key: FLINK-28679 URL: https://issues.apache.org/jira/browse/FLINK-28679 Project: Flink Issue Type: Bug

Re: BOUNDED VS UNBOUNDED

2022-07-25 Thread Israel Ekpo
Hello , Let me attempt to clarify You have two categories of terminology: data sources and data processing runtimes. For data sources you have BOUNDED and UNBOUNDED For processing runtime you have BATCH and STREAMING When you are referring to data sources, BOUNDED data sets have a fixed

[jira] [Created] (FLINK-28678) Support SQL hint for Filter, SetOp, Sort, Window, Values

2022-07-25 Thread xuyang (Jira)
xuyang created FLINK-28678: -- Summary: Support SQL hint for Filter, SetOp, Sort, Window, Values Key: FLINK-28678 URL: https://issues.apache.org/jira/browse/FLINK-28678 Project: Flink Issue Type: New

BOUNDED VS UNBOUNDED

2022-07-25 Thread 하지양
Hello Again! I am very beginners in flink. it is not important issue. so you can skip. but, I have question. noting relation one. I confuse in some streaming oneday some people want to not finished protocol. I want to explain "Dedekind Cut" is it unbounded streaming? can be? or

[jira] [Created] (FLINK-28677) Primary key should not be null when creating Spark table

2022-07-25 Thread Jane Chan (Jira)
Jane Chan created FLINK-28677: - Summary: Primary key should not be null when creating Spark table Key: FLINK-28677 URL: https://issues.apache.org/jira/browse/FLINK-28677 Project: Flink Issue

[jira] [Created] (FLINK-28676) Update copyright year to 2014-2022 in NOTICE files

2022-07-25 Thread Benchao Li (Jira)
Benchao Li created FLINK-28676: -- Summary: Update copyright year to 2014-2022 in NOTICE files Key: FLINK-28676 URL: https://issues.apache.org/jira/browse/FLINK-28676 Project: Flink Issue Type:

[jira] [Created] (FLINK-28675) AvroDeserializationSchema should eagerly validate that class is SpecificRecord

2022-07-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-28675: Summary: AvroDeserializationSchema should eagerly validate that class is SpecificRecord Key: FLINK-28675 URL: https://issues.apache.org/jira/browse/FLINK-28675

[jira] [Created] (FLINK-28674) EqualiserCodeGenerator generates wrong equaliser for Timestamp fields in BinaryRowData

2022-07-25 Thread Weike Dong (Jira)
Weike Dong created FLINK-28674: -- Summary: EqualiserCodeGenerator generates wrong equaliser for Timestamp fields in BinaryRowData Key: FLINK-28674 URL: https://issues.apache.org/jira/browse/FLINK-28674

[jira] [Created] (FLINK-28673) Migrate Flink ML to Flink 1.15.1

2022-07-25 Thread Yunfeng Zhou (Jira)
Yunfeng Zhou created FLINK-28673: Summary: Migrate Flink ML to Flink 1.15.1 Key: FLINK-28673 URL: https://issues.apache.org/jira/browse/FLINK-28673 Project: Flink Issue Type: Improvement

[GitHub] [flink-connector-redis] sazzad16 commented on pull request #2: [FLINK-15571][connector][WIP] Redis Stream connector for Flink

2022-07-25 Thread GitBox
sazzad16 commented on PR #2: URL: https://github.com/apache/flink-connector-redis/pull/2#issuecomment-1193897653 Hi @knaufk, Thanks for the pointers and the feedback! @MartijnVisser has also sent me more read material including

[jira] [Created] (FLINK-28672) properties.allow.auto.create.topics无效

2022-07-25 Thread songwenbin (Jira)
songwenbin created FLINK-28672: -- Summary: properties.allow.auto.create.topics无效 Key: FLINK-28672 URL: https://issues.apache.org/jira/browse/FLINK-28672 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-28671) Update Spark3 to create/drop table in Compatibility Matrix

2022-07-25 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28671: Summary: Update Spark3 to create/drop table in Compatibility Matrix Key: FLINK-28671 URL: https://issues.apache.org/jira/browse/FLINK-28671 Project: Flink

Re: Re: [VOTE] FLIP-252: Amazon DynamoDB Sink Connector

2022-07-25 Thread Danny Cranmer
Hi everyone, The voting time for FLIP-252: Amazon DynamoDB Sink Connector [1] has passed. I'm closing the vote now. There were 6 +1 votes, 5 of which are binding: - Tzu-Li (Gordon) Tai (binding) - Konstantin Knauf (binding) - Robert Metzger (binding) - Martijn Visser (binding) - Becket Qin

[jira] [Created] (FLINK-28670) Documentation of spark2 is wrong

2022-07-25 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28670: Summary: Documentation of spark2 is wrong Key: FLINK-28670 URL: https://issues.apache.org/jira/browse/FLINK-28670 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-28669) FileStoreExpireTest.testExpireWithTime failed

2022-07-25 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28669: Summary: FileStoreExpireTest.testExpireWithTime failed Key: FLINK-28669 URL: https://issues.apache.org/jira/browse/FLINK-28669 Project: Flink Issue Type:

[jira] [Created] (FLINK-28668) Fail to create partitioned table without primary key for Spark

2022-07-25 Thread Jane Chan (Jira)
Jane Chan created FLINK-28668: - Summary: Fail to create partitioned table without primary key for Spark Key: FLINK-28668 URL: https://issues.apache.org/jira/browse/FLINK-28668 Project: Flink

Re: [VOTE] FLIP-247 Bulk fetch of table and column statistics for given partitions

2022-07-25 Thread Jark Wu
+1 (binding) Best, Jark On Mon, 25 Jul 2022 at 15:10, Jing Ge wrote: > Hi all, > > Many thanks for all your feedback. Based on the discussion in [1], I'd like > to start a vote on FLIP-247 [2]. > > The vote will last for at least 72 hours unless there is an objection or > insufficient votes. >

[jira] [Created] (FLINK-28667) "evictor" operations of "window_all" are supported in Python datastream API

2022-07-25 Thread zhangjingcun (Jira)
zhangjingcun created FLINK-28667: Summary: "evictor" operations of "window_all" are supported in Python datastream API Key: FLINK-28667 URL: https://issues.apache.org/jira/browse/FLINK-28667 Project:

[jira] [Created] (FLINK-28666) Alter spark table's primary key should throw exception

2022-07-25 Thread Jane Chan (Jira)
Jane Chan created FLINK-28666: - Summary: Alter spark table's primary key should throw exception Key: FLINK-28666 URL: https://issues.apache.org/jira/browse/FLINK-28666 Project: Flink Issue Type:

[jira] [Created] (FLINK-28665) flink-table-store-benchmark-jar-with-dependencies.jar NOTICE file is wrong

2022-07-25 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-28665: Summary: flink-table-store-benchmark-jar-with-dependencies.jar NOTICE file is wrong Key: FLINK-28665 URL: https://issues.apache.org/jira/browse/FLINK-28665 Project:

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-07-25 Thread Piotr Nowojski
Thanks for the update Sebastian :) Best, Piotrek pon., 25 lip 2022 o 08:12 Sebastian Mattheis napisał(a): > Hi everybody, > > I discussed last week the semantics and an implementation stragegy of the > configuration parameter with Piotr and did the implementation and some > tests this weekend.

[VOTE] FLIP-247 Bulk fetch of table and column statistics for given partitions

2022-07-25 Thread Jing Ge
Hi all, Many thanks for all your feedback. Based on the discussion in [1], I'd like to start a vote on FLIP-247 [2]. The vote will last for at least 72 hours unless there is an objection or insufficient votes. Best regards, Jing [1]

Re: [VOTE] FLIP-250: Support Customized Kubernetes Schedulers Proposal

2022-07-25 Thread Martijn Visser
+1 (binding) Op ma 25 jul. 2022 om 04:56 schreef Yang Wang : > +1 (binding) > > Best, > Yang > > bo zhaobo 于2022年7月25日周一 09:38写道: > > > Hi all, > > > > Thank you very much for all feedback after the discussion in [2][3]. > > Now I'd like to proceed with the vote for FLIP-250 [1], as no more > >

Re: Re: [VOTE] FLIP-252: Amazon DynamoDB Sink Connector

2022-07-25 Thread Martijn Visser
+1 (binding) Op ma 25 jul. 2022 om 06:01 schreef Becket Qin : > +1 > > On Mon, Jul 25, 2022 at 9:22 AM Grant L (Grant) > wrote: > > > +1 > > > > On 2022/07/21 18:27:52 Robert Metzger wrote: > > > +1 > > > > > > On Wed, Jul 20, 2022 at 10:48 PM Konstantin Knauf > > wrote: > > > > > > > +1.

[jira] [Created] (FLINK-28664) Support AvroWriters in PyFlink

2022-07-25 Thread Juntao Hu (Jira)
Juntao Hu created FLINK-28664: - Summary: Support AvroWriters in PyFlink Key: FLINK-28664 URL: https://issues.apache.org/jira/browse/FLINK-28664 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-28663) Allow multiple downstream consumer job vertices sharing the same intermediate dataset at scheduler side

2022-07-25 Thread Yingjie Cao (Jira)
Yingjie Cao created FLINK-28663: --- Summary: Allow multiple downstream consumer job vertices sharing the same intermediate dataset at scheduler side Key: FLINK-28663 URL:

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-07-25 Thread Sebastian Mattheis
Hi everybody, I discussed last week the semantics and an implementation stragegy of the configuration parameter with Piotr and did the implementation and some tests this weekend. A short summary of what I discussed and recapped with Piotr: - The configuration parameter allows (and tolerates)