Re: Regarding access to Jira

Hi, Only Flink committers can assign [1] I just took a look and assigned to you. [1] https://flink.apache.org/contributing/contribute-code.html#create-jira-ticket-and-reach-consensus Best, Jingsong Lee -- From:Sivaprasanna Send

Re: 开发相关问题咨询Development related problems consultation

Hi, welcome, For user side, u...@flink.apache.org is for English. user...@flink.apache.org is for Chinese. dev@flink.apache.org is for development related discussions, so please not send to it. Best, Jingsong Lee -- From:王博迪

Re: Mongo Connector

Hi vijay: I developed an append stream sink for Mongo internally, which writes data in batches according to configureable batch size, and also provides asynchronous flush. But only insert not update or upsert. It's a good job. It's been working very well for a long time. (Throughput depends

Re: [VOTE] FLIP-63: Rework table partition support

Thanks for the votes! We have 3 binding +1's from Kurt, Jark, Bowen. Have 3 non-binding +1's from xuefu, terry, rui. Thank you for the vote. Since there are no disapproving votes and the voting time has passed, I will see this FLIP as approved to be adopted into Apache Flink. Best, Jingsong Lee

Re: [VOTE] FLIP-66: Support Time Attribute in SQL DDL (#2)

+1 non-binding (nit: Add a road map?) Best, Jingsong Lee -- From:Kurt Young Send Time:2019年9月26日(星期四) 20:44 To:dev Subject:Re: [VOTE] FLIP-66: Support Time Attribute in SQL DDL (#2) +1 Best, Kurt On Thu, Sep 26, 2019 at 11:52

Re: [DISCUSS] FLIP-63: Rework table partition support

APIs that I just left, it LGTM. Not sure why, but the voting thread in gmail shows in the same thread as the discussion's. After addressing all the comments, could you start a new, separate thread to let other people be aware of it? Thanks, Bowen On Mon, Sep 23, 2019 at 1:25 AM JingsongLee

Re: [DISCUSS] FLIP-63: Rework table partition support

other people be aware of it? Thanks, Bowen On Mon, Sep 23, 2019 at 1:25 AM JingsongLee wrote: > Thanks for your discussion on google document. > Comments addressed and added FileSystem connector chapter, and introduce > code prototype for file system connector to unify flink fi

Re: [DISCUSS] FLIP-63: Rework table partition support

, Sep 12, 2019 at 12:43 PM Biao Liu wrote: Hi Jingsong, Thanks for explaining. It looks cool! Thanks, Biao /'bɪ.aʊ/ On Wed, 11 Sep 2019 at 11:37, JingsongLee wrote: > Hi biao, thanks for your feedbacks: > > Actually, the runtime source partition of runtime is similar to split,

Re: [DISCUSS] Add Bucket File System Connector

Hi jun: Sorry for the late reply, I share my thoughts on StreamingFileSink in FLIP-63 [1] and I don't recommend using StreamingFileSink to support partitioning in Table. 1.The bucket concept and SQL's bucket concept are in serious conflict.[2] 2.In table, we need support single-partition

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

I know Hive and Spark can shadow built-in functions by temporary function. Mysql, Oracle, Sql server can not shadow. User can use full names to access functions instead of shadowing. So I think it is a completely new thing, and the direct way to deal with new things is to add new grammar. So, +1

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

019 at 5:14 PM JingsongLee wrote: > After some development and thinking, I have a general understanding. > +1 to registering a source/sink does not fit into the SQL world. > I am OK to have a deprecated registerTemporarySource/Sink to compatible > with old ways

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

logical table schema, we would need a possibility of defining time attributes and interpreting/converting a changelog. This should be done by the framework with information from the DDL/connect() and not be defined in every table source. Regards, Timo On 09.09.19 14:16, JingsongLee wrote

[VOTE] FLIP-63: Rework table partition support

Hi Flink devs, I would like to start the voting for FLIP-63 Rework table partition support. FLIP wiki: https://cwiki.apache.org/confluence/display/FLINK/FLIP-63%3A+Rework+table+partition+support Discussion thread:

Re: [DISCUSS] FLIP-63: Rework table partition support

to be a fairly big efforts, so I suggest we can settle down the design doc ASAP and start vote process. Best, Kurt On Thu, Sep 12, 2019 at 12:43 PM Biao Liu wrote: Hi Jingsong, Thanks for explaining. It looks cool! Thanks, Biao /'bɪ.aʊ/ On Wed, 11 Sep 2019 at 11:37, JingsongLee wrote: >

Re: [DISCUSS] FLIP-63: Rework table partition support

-- From:Biao Liu Send Time:2019年9月10日(星期二) 14:57 To:dev ; JingsongLee Subject:Re: [DISCUSS] FLIP-63: Rework table partition support Hi Jingsong, Thank you for bringing this discussion. Since I don't have much experience of Flink table/SQL, I'll ask some questions from runtime or engine

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

guarantee > > the semantic and use the accumulators to implement distributed barrier or > > lock to solve the distributed access problem. > > > > Best, > > Shimin > > > > JingsongLee 于2019年9月9日周一 下午1:33写道： > > > >> Thanks jark and dian: > &g

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

Congrats, Kostas! Well deserved. Best, Jingsong Lee -- From:Kostas Kloudas Send Time:2019年9月9日(星期一) 15:50 To:dev ; Yun Gao Subject:Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC Thanks a lot everyone for the warm welcome!

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

stance any way. Having string parsing logic in the code is a little dangerous as it duplicates part of the DDL/DML parsing, and they can easily get out of sync. Thanks, Xuefu On Fri, Sep 6, 2019 at 1:57 PM JingsongLee wrote: Thanks dawid, +1 for this approach. One concern is the removal of regi

Re: [DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

> Big fan of this idea. We faced the same problem and resolved by adding > a > >>> distributed lock. It would be nice to have this feature in JobMaster, > >> which > >>> can replace the lock. > >>> > >>> Best, > >>> Shimin >

Re: [DISCUSS] FLIP-64: Support for Temporary Objects in Table module

Thanks dawid, +1 for this approach. One concern is the removal of registerTableSink & registerTableSource in TableEnvironment. It has two alternatives: 1.the properties approach (DDL, descriptor). 2.from/toDataStream. #1 can only be properties, not java states, and some Connectors are

[DISCUSS] FLIP-63: Rework table partition support

Hi everyone, thank you for your comments. Mail name was updated and streaming-related concepts were added. We would like to start a discussion thread on "FLIP-63: Rework table partition support"(Design doc: [1]), where we describe how to partition support in flink and how to integrate to hive

[DISCUSS] Support notifyOnMaster for notifyCheckpointComplete

Hi devs: I try to implement streaming file sink for table[1] like StreamingFileSink. If the underlying is a HiveFormat, or a format that updates visibility through a metaStore, I have to update the metaStore in the notifyCheckpointComplete, but this operation occurs on the task side, which

Re: Re: [DISCUSS] Support JSON functions in Flink SQL

+1 Nice document. I think it is easier to do after expression reworking[1]. By the way, which planner do you want to start? [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design Best, Jingsong Lee

Re: [VOTE] FLIP-62: Set default restart delay for FixedDelay- and FailureRateRestartStrategy to 1s

+1 (non-binding) default 0 is really not user production friendly. Best, Jingsong Lee -- From:Zhu Zhu Send Time:2019年9月4日(星期三) 17:13 To:dev Subject:Re: [VOTE] FLIP-62: Set default restart delay for FixedDelay- and

FLIP-63: Rework table partition support

Hi everyone, We would like to start a discussion thread on "FLIP-63: Rework table partition support"(Design doc: [1]), where we describe how to partition support in flink and how to integrate to hive partition. This FLIP addresses: - Introduce whole story about partition support. -

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

Thank you for your wonderful points. I like timo's proposal to enrich built-in functions to flexible function modules (For example, the financial model is useful to bank system). But I agree with bowen, I don't think hive functions deserves be a function module. I think all function modules

Re: [DISCUSS] FLIP-57 - Rework FunctionCatalog

Thanks Bowen: +1 for this. And +1 to Kurt's suggestion. My other points are: 1.Hive built-in functions is an intermediate solution. So we should not introduce interfaces to influence the framework. To make Flink itself more powerful, we should implement the functions we need to add.

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

+1 non-binding Best, Jingsong Lee -- From:Dawid Wysakowicz Send Time:2019年8月27日(星期二) 13:55 To:dev ; Timo Walther Subject:Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration +1 to the FLIP Also I think we should mention

Re: [ANNOUNCE] Apache Flink 1.9.0 released

Congratulations~~~ Thanks gordon and everyone~ Best, Jingsong Lee -- From:Oytun Tez Send Time:2019年8月22日(星期四) 14:06 To:Tzu-Li (Gordon) Tai Cc:dev ; user ; announce Subject:Re: [ANNOUNCE] Apache Flink 1.9.0 released

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

Hi stephan: About option 2: if additional threads not cleanly shut down before we can exit the task: In the current case of memory reuse, it has freed up the memory it uses. If this memory is used by other tasks and asynchronous threads of exited task may still be writing, there will be

Re: [VOTE] FLIP-51: Rework of the Expression Design

> > Best, > > Dawid > > On 16/08/2019 10:31, Jark Wu wrote: >> +1 from my side. >> >> Thanks Jingsong for driving this. >> >> Best, >> Jark >> >> On Thu, 15 Aug 2019 at 22:09, Timo Walther wrote: >> >>> +1 for this.

Re: [DISCUSS] FLIP-54: Evolve ConfigOption and Configuration

+1 to this, thanks Timo and Dawid for the design. This allows the currently cluttered configuration of various modules to be unified. This is also first step of one of the keys to making new unified TableEnvironment available for production. Previously, we did encounter complex configurations,

[VOTE] FLIP-51: Rework of the Expression Design

Hi Flink devs, I would like to start the voting for FLIP-51 Rework of the Expression Design. FLIP wiki: https://cwiki.apache.org/confluence/display/FLINK/FLIP-51%3A+Rework+of+the+Expression+Design Discussion thread:

Re: [DISCUSS] FLIP-51: Rework of the Expression Design

t; > We might need to discuss some of the behavior of particular functions > but this should not affect the actual FLIP-51. > > Regards, > Timo > > > Am 13.08.19 um 12:55 schrieb JingsongLee: > > Hi everyone, > > > > We would like to start a discussion thread

Re: [DISCUSS] FLIP-51: Rework of the Expression Design

some of the behavior of particular functions > but this should not affect the actual FLIP-51. > > Regards, > Timo > > > Am 13.08.19 um 12:55 schrieb JingsongLee: > > Hi everyone, > > > > We would like to start a discussion thread on "FLIP-51: Rework of

Re: [VOTE] Flink Project Bylaws

+1 (non-binding) Thanks Becket. I've learned a lot from current bylaws. Best, Jingsong Lee -- From:Yu Li Send Time:2019年8月13日(星期二) 17:48 To:dev Subject:Re: [VOTE] Flink Project Bylaws +1 (non-binding) Thanks for the efforts

[DISCUSS] FLIP-51: Rework of the Expression Design

Hi everyone, We would like to start a discussion thread on "FLIP-51: Rework of the Expression Design"(Design doc: [1], FLIP: [2]), where we describe how to improve the new java Expressions to work with type inference and convert expression to the calcite RexNode. This is a follow-up plan for

Re: [ANNOUNCE] Hequn becomes a Flink committer

Congrats Hequn! Best, Jingsong Lee -- From:Biao Liu Send Time:2019年8月7日(星期三) 12:05 To:Zhu Zhu Cc:Zili Chen ; Jeff Zhang ; Paul Lam ; jincheng sun ; dev ; user Subject:Re: [ANNOUNCE] Hequn becomes a Flink committer Congrats

Re: [DISCUSS][CODE STYLE] Usage of Java Optional

Hi, First, Optional is just a wrapper, just like boxed value. So as long as it's not a field level operation, I think it is OK to performance. I think guava optional has a good summary to the uses. [1] > As a method return type, as an alternative to returning null to indicate that no value was

Re: [DISCUSS] Removing the flink-mapr-fs module

+1 for removing it. We never run mvn clean test success in China with mapr-fs... Best, Jingsong Lee -- From:Biao Liu Send Time:2019年7月29日(星期一) 12:05 To:dev Subject:Re: [DISCUSS] Removing the flink-mapr-fs module +1 for removing

Re: flink-mapr-fs failed in travis

9 07:57, Jark Wu wrote: >>> It seems that it is introduced by this commit: >>> https://github.com/apache/flink/commit/5c36c650e6520d92191ce2da33f7dcae774319f6 >>> >>> >>> Hi @Chesnay Schepler , do we need to add >>> "-

Re: flink-mapr-fs failed in travis

troduced by this commit: >>> https://github.com/apache/flink/commit/5c36c650e6520d92191ce2da33f7dcae774319f6 >>> >>> >>> Hi @Chesnay Schepler , do we need to add >>> "-Punsafe-mapr-repo" to the ".travis.yml"? >&

Re: Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project

Congratulations Zhijiang! Best, Jingsong Lee -- From:wenlong.lwl Send Time:2019年7月23日(星期二) 17:34 To:dev Subject:Re: Re: [ANNOUNCE] Zhijiang Wang has been added as a committer to the Flink project Congrats, Zhijiang! On Tue, 23

Re: [DISCUSS] Support temporary tables in SQL API

And I think we should recommend user to use catalog api to createTable and createFunction,(I guess most scenarios do not use temporary objects) in this way, it is good to option #3 Best, JingsongLee -- From:JingsongLee Send Time

Re: [DISCUSS] Support temporary tables in SQL API

Thanks Dawid and other people. +1 for using option #3 for 1.9.0 and go with option #1 in 1.10.0. Regarding Xuefu's concern, I don't know how necessary it is for each catalog to deal with tmpView. I think Catalog is different from DB, we can have single concept for tmpView, that make user

Re: [ANNOUNCE] Kete Young is now part of the Flink PMC

Congratulations Kurt! Best, Jingsong Lee -- From:Robert Metzger Send Time:2019年7月23日(星期二) 17:24 To:dev Subject:[ANNOUNCE] Kete Young is now part of the Flink PMC Hi all, On behalf of the Flink PMC, I'm happy to announce that

flink-mapr-fs failed in travis

Hi everyone: flink-mapr-fs failed in travis, and I retried many times, and also failed. Anyone has idea about this? 01:32:54.755 [ERROR] Failed to execute goal on project flink-mapr-fs: Could not resolve dependencies for project org.apache.flink:flink-mapr-fs:jar:1.10-SNAPSHOT: Failed to

Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

Congratulations Becket! Best, Jingsong Lee -- From:Congxian Qiu Send Time:2019年7月18日(星期四) 16:09 To:dev@flink.apache.org Subject:Re: [ANNOUNCE] Jiangjie (Becket) Qin has been added as a committer to the Flink project

Re: blink planner issue

Hi caizhi and kaka and jark: I think we should keep current equals in RowType and StructuredType. @Timo WaltherWhat do you think? Do you think we can use asSerializableString? Best, JingsongLee -- From:kaka chen Send Time:2019年7月

Re: No SchemaValidator When switch to blink table planner.

Hi kaka: Yeah, there are still some problems, I create a JIRA[1] to trace it, I think we should solve it before 1.9 release. [1] https://issues.apache.org/jira/browse/FLINK-13285 Best, JingsongLee -- From:kaka chen Send Time

Re: [ANNOUNCE] Rong Rong becomes a Flink committer

Congratulations Rong. Rong Rong has done a lot of nice work in the past time to the flink community. Best, JingsongLee -- From:Rong Rong Send Time:2019年7月12日(星期五) 08:09 To:Hao Sun Cc:Xuefu Z ; dev ; Flink ML Subject:Re

Re: [VOTE] Migrate to sponsored Travis account

+1 for the migration Best, JingsongLee -- From:Jark Wu Send Time:2019年7月5日(星期五) 10:35 To:dev Cc:private ; Bowen Li Subject:Re: [VOTE] Migrate to sponsored Travis account +1 for the migration and great thanks to Chesnay

Re: [ANNOUNCE] Apache Flink 1.8.1 released

Thanks jincheng for your great job. Best, JingsongLee -- From:Congxian Qiu Send Time:2019年7月3日(星期三) 14:35 To:dev@flink.apache.org Cc:Dian Fu ; jincheng sun ; Hequn Cheng ; user ; announce Subject:Re: [ANNOUNCE] Apache Flink

Re: [DISCUSS] Vectorization Support in Flink

ory management? Best, JingsongLee -- From:Fan Liya Send Time:2019年7月2日(星期二) 16:31 To:dev ; Ji Liu Subject:Re: [DISCUSS] Vectorization Support in Flink @Ji Liu, thanks a lot for your feedback. This work must be perfor

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Jincheng, Congratulations! Best, JingsongLee -- From:Dawid Wysakowicz Send Time:2019年6月25日(星期二) 14:21 To:dev Subject:Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC Congratulations! On 25/06/2019 08:06, Xingcan Cui

Re: [DISCUSS] Adopting a Code Style and Quality Guide

, JingsongLee -- From:Piotr Nowojski Send Time:2019年6月13日(星期四) 16:03 To:dev Subject:Re: [DISCUSS] Adopting a Code Style and Quality Guide +1 for it and general content and thank everybody that was involved in creating & writing this

Re: [DISCUSS] Clean up and reorganize the JIRA components

Big +1 to add benchmark component. 1.Many of our code changes now require benchmark. Having a benchmark component makes it much easier for us to align. 2.Running benchmark regularly can also prevent performance degradation caused by our code. Best, JingsongLee

Re: Using guava ImmutableMap Serializer from magro/kyro-serializers

conf.registerTypeWithKryoSerializer(denseImmutableTable.columnMap().getClass(), serializer); conf.registerTypeWithKryoSerializer(((Map)denseImmutableTable.columnMap().get(1)).getClass(), serializer); } Best, JingsongLee -- From:vijikarthi Se

Re: [ANNOUNCE] New Flink committer Shaoxuan Wang

Congrats! Best, Jingsong Lee --From:Andrew Psaltis Time:2017 Jun 22 (Thu) 11:06To:dev Subject:Re: [ANNOUNCE] New Flink committer Shaoxuan Wang Congrats Shaoxuan! On Wed, Jun 21, 2017