Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-15 Thread Congxian Qiu
Congratulations, Zhu Zhu! Best, Congxian godfrey he 于2020年10月16日周五 上午11:03写道: > Congratulations, Zhu > > Godfrey, > Best > > 刘 芃成 于2020年10月10日周六 下午1:03写道: > > > Congratulations, Zhu! > > > > Best, > > Pengcheng > > > > 在 2020/10/10 上午11:14,“Kurt Young” 写入: > > > > Congratulations, Zhu

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-15 Thread godfrey he
Congratulations, Zhu Godfrey, Best 刘 芃成 于2020年10月10日周六 下午1:03写道: > Congratulations, Zhu! > > Best, > Pengcheng > > 在 2020/10/10 上午11:14,“Kurt Young” 写入: > > Congratulations, Zhu Zhu! > > Best, > Kurt > > > On Sat, Oct 10, 2020 at 11:03 AM Yang Wang > wrote: > > >

Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread Jark Wu
+1 On Fri, 16 Oct 2020 at 10:27, admin <17626017...@163.com> wrote: > +1 > > > 2020年10月16日 上午10:05,Danny Chan 写道: > > > > +1, nice job ! > > > > Best, > > Danny Chan > > 在 2020年10月15日 +0800 PM8:08,Jingsong Li ,写道: > >> Hi all, > >> > >> I would like to start the vote for FLIP-146 [1], which is

Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread admin
+1 > 2020年10月16日 上午10:05,Danny Chan 写道: > > +1, nice job ! > > Best, > Danny Chan > 在 2020年10月15日 +0800 PM8:08,Jingsong Li ,写道: >> Hi all, >> >> I would like to start the vote for FLIP-146 [1], which is discussed and >> reached consensus in the discussion thread [2]. The vote will be open

[jira] [Created] (FLINK-19669) PipelinedRegionSchedulingStrategy#init ResultPartitionType blocking check should use isBlocking method

2020-10-15 Thread Xuannan Su (Jira)
Xuannan Su created FLINK-19669: -- Summary: PipelinedRegionSchedulingStrategy#init ResultPartitionType blocking check should use isBlocking method Key: FLINK-19669 URL:

[jira] [Created] (FLINK-19668) Fix the plan regression by Calcite upgrade to 1.26

2020-10-15 Thread Danny Chen (Jira)
Danny Chen created FLINK-19668: -- Summary: Fix the plan regression by Calcite upgrade to 1.26 Key: FLINK-19668 URL: https://issues.apache.org/jira/browse/FLINK-19668 Project: Flink Issue Type:

Re: [VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread Danny Chan
+1, nice job ! Best, Danny Chan 在 2020年10月15日 +0800 PM8:08,Jingsong Li ,写道: > Hi all, > > I would like to start the vote for FLIP-146 [1], which is discussed and > reached consensus in the discussion thread [2]. The vote will be open until > 20th Oct. (72h, exclude weekends), unless there is an

[jira] [Created] (FLINK-19667) Add integration with AWS Glue

2020-10-15 Thread Mohit Paliwal (Jira)
Mohit Paliwal created FLINK-19667: - Summary: Add integration with AWS Glue Key: FLINK-19667 URL: https://issues.apache.org/jira/browse/FLINK-19667 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-19666) CLONE - [Kinesis][EFO] Add AWS SDK v2.x dependency and KinesisProxyV2

2020-10-15 Thread Mohit Paliwal (Jira)
Mohit Paliwal created FLINK-19666: - Summary: CLONE - [Kinesis][EFO] Add AWS SDK v2.x dependency and KinesisProxyV2 Key: FLINK-19666 URL: https://issues.apache.org/jira/browse/FLINK-19666 Project:

Re: [DISCUSS] Remove flink-connector-filesystem module.

2020-10-15 Thread Kostas Kloudas
@Arvid Heise I also do not remember exactly what were all the problems. The fact that we added some more bulk formats to the streaming file sink definitely reduced the non-supported features. In addition, the latest discussion I found on the topic was [1] and the conclusion of that discussion

[jira] [Created] (FLINK-19665) AlternatingCheckpointBarrierHandlerTest.testMetricsAlternation

2020-10-15 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19665: -- Summary: AlternatingCheckpointBarrierHandlerTest.testMetricsAlternation Key: FLINK-19665 URL: https://issues.apache.org/jira/browse/FLINK-19665 Project: Flink

[jira] [Created] (FLINK-19664) Upload logs before end to end tests time out

2020-10-15 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19664: -- Summary: Upload logs before end to end tests time out Key: FLINK-19664 URL: https://issues.apache.org/jira/browse/FLINK-19664 Project: Flink Issue Type:

[jira] [Created] (FLINK-19663) I would like to unsubscribe

2020-10-15 Thread Suryanarayana Murthy Maganti (Jira)
Suryanarayana Murthy Maganti created FLINK-19663: Summary: I would like to unsubscribe Key: FLINK-19663 URL: https://issues.apache.org/jira/browse/FLINK-19663 Project: Flink

[jira] [Created] (FLINK-19662) Create or extend REST endpoint to expose memory configuration

2020-10-15 Thread Matthias (Jira)
Matthias created FLINK-19662: Summary: Create or extend REST endpoint to expose memory configuration Key: FLINK-19662 URL: https://issues.apache.org/jira/browse/FLINK-19662 Project: Flink Issue

[jira] [Created] (FLINK-19661) Implement changes required to resolve FLIP-104

2020-10-15 Thread Matthias (Jira)
Matthias created FLINK-19661: Summary: Implement changes required to resolve FLIP-104 Key: FLINK-19661 URL: https://issues.apache.org/jira/browse/FLINK-19661 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2020-10-15 Thread Yingjie Cao
Hi Till, Thanks for your reply and comments. You are right, the proposed sort-merge based shuffle is an extension of the existing blocking shuffle and does not change any default behavior of Flink. As for the performance, according to our previous experience, sort-merge based implementation can

[jira] [Created] (FLINK-19660) Translate page 'table_environment.zh.md' into Chinese

2020-10-15 Thread hailong wang (Jira)
hailong wang created FLINK-19660: Summary: Translate page 'table_environment.zh.md' into Chinese Key: FLINK-19660 URL: https://issues.apache.org/jira/browse/FLINK-19660 Project: Flink Issue

[jira] [Created] (FLINK-19659) Array type supports equals and not_equals operator when element types are different but castable

2020-10-15 Thread hailong wang (Jira)
hailong wang created FLINK-19659: Summary: Array type supports equals and not_equals operator when element types are different but castable Key: FLINK-19659 URL: https://issues.apache.org/jira/browse/FLINK-19659

[jira] [Created] (FLINK-19658) Local recovery and sticky scheduling end-to-end test hangs with "Expected to find info here."

2020-10-15 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19658: -- Summary: Local recovery and sticky scheduling end-to-end test hangs with "Expected to find info here." Key: FLINK-19658 URL: https://issues.apache.org/jira/browse/FLINK-19658

[jira] [Created] (FLINK-19657) YARNSessionCapacitySchedulerITCase.checkForProhibitedLogContents: netty failed with java.io.IOException: Broken pipe

2020-10-15 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19657: -- Summary: YARNSessionCapacitySchedulerITCase.checkForProhibitedLogContents: netty failed with java.io.IOException: Broken pipe Key: FLINK-19657 URL:

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-15 Thread Danny Chan
Hi, Timo ~ > We are not forced by the standard to do it as stated in the `One SQL to Rule it all` paper No, slide to the SQL standard is always better, i think this is a common routine of our Flink SQL now, without a standard, everyone can give a preference and the discussion would easily go

Re: Re: [VOTE] Add Translation Specification for Stateful Functions

2020-10-15 Thread Congxian Qiu
Hi all Thanks everyone for the voting. The voting time for "Add Translation Specification for Stateful Functions" has passed, I'm closing the vote now. There were 7 votes, 4 of which are binding: - Yu Li (binding) - Jark Wu (binding) - Xintong Song (binding) - Smile - Dian

[VOTE] FLIP-146: Improve new TableSource and TableSink interfaces

2020-10-15 Thread Jingsong Li
Hi all, I would like to start the vote for FLIP-146 [1], which is discussed and reached consensus in the discussion thread [2]. The vote will be open until 20th Oct. (72h, exclude weekends), unless there is an objection or not enough votes. [1]

[jira] [Created] (FLINK-19656) Automatically replace delimiter in metric name components

2020-10-15 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-19656: Summary: Automatically replace delimiter in metric name components Key: FLINK-19656 URL: https://issues.apache.org/jira/browse/FLINK-19656 Project: Flink

[jira] [Created] (FLINK-19655) NPE when using blink planner and TemporalTableFunction after setting IdleStateRetentionTime

2020-10-15 Thread seunjjs (Jira)
seunjjs created FLINK-19655: --- Summary: NPE when using blink planner and TemporalTableFunction after setting IdleStateRetentionTime Key: FLINK-19655 URL: https://issues.apache.org/jira/browse/FLINK-19655

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-15 Thread Jark Wu
Hi Timo, I still believe a tumbling window assigner doesn't need to be partitioned first, it just behaves like a map function. The window_start and window_end is calculated from the timestamp on input row, not depending on the partitioned data. I'm very confused about what PARTITION BY fields

Re: Need help in creating Flink Streaming s3 Job for multiple path reader one by one

2020-10-15 Thread Chesnay Schepler
hmm...I don't see an easy way. You may have to replicated StreamExecutionEnvironment#createFileInput and create a custom ContinuousFileMonitoringFunction that ignores missing files in it's run() method. Alternatively, use some library to check the existence of the S3 directories before

Re: Need help in creating Flink Streaming s3 Job for multiple path reader one by one

2020-10-15 Thread Satyaa Dixit
Hi Chesnay/Team, Thanks, we got the fix for our problem but got stuck with the below issue, request your support. How to catch FileNotFoundException during runtime,if any directory is missing in s3 as part of the below source code to avoid job failure. s3PathList.stream().map(directory ->

Re: [DISCUSS] FLIP-145: Support SQL windowing table-valued function

2020-10-15 Thread Timo Walther
Hi Jark, I think we should discuss the SET vs ROW semantics again. The FLIP itself is not wrong but I find it a bit verbose to define a window via TUMBLE + grouping by window_start and window_end. This is the Beam approach and exposes how Flink window assigners work internally but do we want

Re: [DISCUSS] Support KeyedSortedMapState in DataStream API

2020-10-15 Thread Sean Z
Hi Jark, Thanks for the reply and sharing thoughts. Yes, negative long will make things complicated. We had the exact same issue when implementing our own sortedMapState. This could be solved by some special pre-defined serializers, and it looks like that's what blink did [1] as you mentioned.

Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to Flink

2020-10-15 Thread Till Rohrmann
Hi Yingjie, thanks for proposing the sort-merge based blocking shuffle. I like the proposal and it does not seem to change the internals of Flink. Instead it is an extension of existing interfaces which makes it a non-invasive addition. Do you have any numbers comparing the performance of the

Re: [DISCUSS][docker] Adopt Jemalloc as default memory allocator for debian based Flink docker image

2020-10-15 Thread Yang Wang
Thanks Yun Tang for starting this discussion. I think this is very important when deploy Flink with container environment in production. I just have quick question. Could we have both memory allocator(e.g. glibc, jemalloc) in the Flink official image and enable a specific one by setting ENV?