Re: [DISCUSS] Adopting a Code Style and Quality Guide

I think it makes sense to also start individual [DISCUSS] threads about extensions and follow-ups. Various suggestions came up, partly as comments in the doc, some as questions in other threads. Examples: - use of null in return types versus Optional versus @Nullable/@Nonnull - initialization

Re: [DISCUSS] Adopting a Code Style and Quality Guide

Thanks for the pointer David. I was not aware of this tool and I have no experience with its practical impact. For example I would be curious what the effect of this is for existing PRs, merge conflicts, etc. If you want, feel free to start another discuss thread on the introduction of such a

[jira] [Created] (FLINK-12964) add commented-out defaults to sql client yaml file to make it easier for users to adopt

Bowen Li created FLINK-12964: Summary: add commented-out defaults to sql client yaml file to make it easier for users to adopt Key: FLINK-12964 URL: https://issues.apache.org/jira/browse/FLINK-12964

[DISCUSS] solve unstable build capacity problem on TravisCI

Hi devs, I've been experiencing the pain resulting from lack of stable build capacity on Travis for Flink PRs [1]. Specifically, I noticed often that no build in the queue is making any progress for hours, and suddenly 5 or 6 builds kick off all together after the long pause. I'm at PST (UTC-08)

[jira] [Created] (FLINK-12966) finalize package name of Hive table source/sink

Bowen Li created FLINK-12966: Summary: finalize package name of Hive table source/sink Key: FLINK-12966 URL: https://issues.apache.org/jira/browse/FLINK-12966 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-12965) unify catalog view implementations

Bowen Li created FLINK-12965: Summary: unify catalog view implementations Key: FLINK-12965 URL: https://issues.apache.org/jira/browse/FLINK-12965 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] Releasing Flink 1.8.1

Due to no new blocker jumps up, I am preparing the RC1 now ... jincheng sun 于2019年6月21日周五 下午4:55写道： > Hi All, > > The last blocker(FLINK-12863) of 1.8.1 release have been fixed! > But also welcome report any issues you think is a blocker! > I will also do the final check, if no new problems are

Re:[ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations! Best, Haibo At 2019-06-24 23:08:54, "Robert Metzger" wrote: >Hi all, > >On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now >part of the Apache Flink Project Management Committee (PMC). > >Jincheng has been a committer since July 2017. He has been very

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

Hi Bowen, Thanks for bringing this. We also suffered from the long build time. I agree that we should focus on solving build capacity problem in the thread. My observation is there is only one build is running, all the others (other PRs, master) are pending. The pricing plan[1] of travis shows

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congrats Jincheng! Well deserved! On Tue, Jun 25, 2019 at 9:56 AM LakeShen wrote: > Congratulations! Jincheng Sun > > Best, > LakeShen > > Robert Metzger 于2019年6月24日周一 下午11:09写道： > > > Hi all, > > > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is > now > > part of the

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Best, Kurt On Tue, Jun 25, 2019 at 9:56 AM LakeShen wrote: > Congratulations! Jincheng Sun > > Best, > LakeShen > > Robert Metzger 于2019年6月24日周一 下午11:09写道： > > > Hi all, > > > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is > now > > part of

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations! Jincheng Sun Best, LakeShen Robert Metzger 于2019年6月24日周一 下午11:09写道： > Hi all, > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now > part of the Apache Flink Project Management Committee (PMC). > > Jincheng has been a committer since July 2017. He has

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! qianjin Xu 于2019年6月25日周二 上午10:25写道： > Congratulations Jincheng! > > > Best > > Forward > > > > > Robert Metzger 于2019年6月24日周一 下午11:09写道： > > > Hi all, > > > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is > now > > part of the Apache Flink

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi Simon, Good question! For event time semantics, we reuse the window operator can keep the correct behavior which is the same as the current window operator. The window operator will trigger based on the watermark. About your example, the window of three partitions will trigger normally. For

[jira] [Created] (FLINK-12967) Change the input selection switching in StreamTwoInputSelectableProcessor#checkFinished to invoking the nextSelection method of the stream operator

Haibo Sun created FLINK-12967: - Summary: Change the input selection switching in StreamTwoInputSelectableProcessor#checkFinished to invoking the nextSelection method of the stream operator Key: FLINK-12967 URL:

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Well deserved! On Tue, Jun 25, 2019 at 10:10 AM Kurt Young wrote: > Congratulations Jincheng! > > Best, > Kurt > > > On Tue, Jun 25, 2019 at 9:56 AM LakeShen > wrote: > > > Congratulations! Jincheng Sun > > > > Best, > > LakeShen > > > > Robert Metzger 于2019年6月24日周一

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations! Jincheng well deserved! BR, Ryan Kurt Young 于2019年6月25日周二 上午10:10写道： > Congratulations Jincheng! > > Best, > Kurt > > > On Tue, Jun 25, 2019 at 9:56 AM LakeShen > wrote: > > > Congratulations! Jincheng Sun > > > > Best, > > LakeShen > > > > Robert Metzger 于2019年6月24日周一

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Best Forward Robert Metzger 于2019年6月24日周一 下午11:09写道： > Hi all, > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now > part of the Apache Flink Project Management Committee (PMC). > > Jincheng has been a committer since July 2017. He has

the contributor permission

hi, I've been using Flink for more than a year. I want to contribute to Apache Flink. Would you please give me the contributor permission? My JIRA ID is sadfddd gygz...@gmail.com

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Biao Liu 于2019年6月25日周二 上午10:29写道： > Congratulations Jincheng! > > > qianjin Xu 于2019年6月25日周二 上午10:25写道： > > > Congratulations Jincheng! > > > > > > Best > > > > Forward > > > > > > > > > > Robert Metzger 于2019年6月24日周一 下午11:09写道： > > > > > Hi all, > > > > > > On

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

Hi Folks, Zeppelin meet this kind of issue before, we solve it by delegating each one's PR build to his travis account (Everyone can have 5 free slot for travis build). Apache account travis build is only triggered when PR is merged. Kurt Young 于2019年6月25日周二 上午10:16写道： > (Forgot to cc

[jira] [Created] (FLINK-12968) Add a casting utility

Timo Walther created FLINK-12968: Summary: Add a casting utility Key: FLINK-12968 URL: https://issues.apache.org/jira/browse/FLINK-12968 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

https://travis-ci.org/apache/flink/builds/549681530 This build request has been sitting at **HEAD of the queue** since I first saw it at PST 10:30am (not sure how long it's been there before 10:30am). It's PST 4:12pm now and it hasn't started yet. On Mon, Jun 24, 2019 at 2:48 PM Bowen Li wrote:

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Well deserved! On Tue, 25 Jun 2019 at 01:38, Bowen Li wrote: > Congratulations! > > On Mon, Jun 24, 2019 at 9:53 AM Yun Tang wrote: > > > Congratulations Jincheng! > > > > Best > > Yun Tang > > > > From: Yu Li > > Sent: Tuesday, June

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Best, Zhijiang -- From:Kurt Young Send Time:2019年6月25日(星期二) 10:10 To:dev Subject:Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC Congratulations Jincheng! Best, Kurt On Tue, Jun 25, 2019 at

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi Vino Thanks for your reply. It seems feasible if a StreamStatus.IDLE was send to downstream, Still two questions. 1. Do we need to add a method to allow users control when to send StreamStatus.IDLE to downsteram in this case? 2. If a partial data comes after your IDLE status to

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

long and sometimes unstable build is definitely a pain point. I suspect the build failure here in flink-connector-kafka is not related to my change. but there is no easy re-run the build on travis UI. Google search showed a trick of close-and-open the PR will trigger rebuild. but that could add

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Jark Wu 于2019年6月25日周二 上午9:28写道： > Congratulations Jincheng! Well deserved! > > On Tue, 25 Jun 2019 at 01:38, Bowen Li wrote: > > > Congratulations! > > > > On Mon, Jun 24, 2019 at 9:53 AM Yun Tang wrote: > > > > > Congratulations Jincheng! > > > > > > Best > > > Yun

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Well deserved! Best, Congxian Jark Wu 于2019年6月25日周二 上午9:28写道： > Congratulations Jincheng! Well deserved! > > On Tue, 25 Jun 2019 at 01:38, Bowen Li wrote: > > > Congratulations! > > > > On Mon, Jun 24, 2019 at 9:53 AM Yun Tang wrote: > > > > > Congratulations

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

Hi Bowen, Thanks for bringing this up. We actually have discussed about this, and I think Till and George have already spend sometime investigating it. I have cced both of them, and maybe they can share their findings. Best, Kurt On Tue, Jun 25, 2019 at 10:08 AM Jark Wu wrote: > Hi Bowen, >

Re: [DISCUSS] solve unstable build capacity problem on TravisCI

(Forgot to cc George) Best, Kurt On Tue, Jun 25, 2019 at 10:16 AM Kurt Young wrote: > Hi Bowen, > > Thanks for bringing this up. We actually have discussed about this, and I > think Till and George have > already spend sometime investigating it. I have cced both of them, and > maybe they can

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations! Best, Yun -- From:Robert Metzger Send Time:2019 Jun. 24 (Mon.) 23:16 To:dev Subject:[ANNOUNCE] Jincheng Sun is now part of the Flink PMC Hi all, On behalf of the Flink PMC, I'm happy to announce that Jincheng

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! > 在 2019年6月24日，下午11:08，Robert Metzger 写道： > > Hi all, > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now > part of the Apache Flink Project Management Committee (PMC). > > Jincheng has been a committer since July 2017. He has been very

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi Simon, IMO, we do not need special processing for your example scenarios, Flink suggests users extracting watermarks in source function. Generally, the IDLE is temporary status, when the data coming, it will send ACTIVE status to the downstream and the processing will continue. Keep in mind,

Re: Something wrong with travis?

Unfortunately, I met this problem again just now https://api.travis-ci.org/v3/job/549534496/log.txt (the build overview https://travis-ci.org/apache/flink/builds/549534489). For those non-committers, including me, we have to close-reopen the PR or push another commit to re-trigger the PR

Re: [DISCUSS] Flink client api enhancement for downstream project

Hi all, After a closer look on our client apis, I can see there are two major issues to consistency and integration, namely different deployment of job cluster which couples job graph creation and cluster deployment, and submission via CliFrontend confusing control flow of job graph compilation

Re: [DISCUSS] Ground Source and Sink Concepts in Flink SQL

Thanks for working on this great design document Jark. I think having well-defined terminilogy and semantics around tables, changelogs, table sources/sinks, and DDL should have been done much earlier. I will take a closer look at the concepts and give feedback soon. I think having those

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi vino, Sorry I don't see the consensus about reusing window operator and keep the API design of localKeyBy. But I think we should definitely more thoughts about this topic. I also try to loop in Stephan for this discussion. Best, Kurt On Mon, Jun 24, 2019 at 3:26 PM vino yang wrote: > Hi

[jira] [Created] (FLINK-12958) Integrate AsyncWaitOperator with mailbox

Stefan Richter created FLINK-12958: -- Summary: Integrate AsyncWaitOperator with mailbox Key: FLINK-12958 URL: https://issues.apache.org/jira/browse/FLINK-12958 Project: Flink Issue Type:

[jira] [Created] (FLINK-12956) Add a upsert table sink for JDBC

Jingsong Lee created FLINK-12956: Summary: Add a upsert table sink for JDBC Key: FLINK-12956 URL: https://issues.apache.org/jira/browse/FLINK-12956 Project: Flink Issue Type: New Feature

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

Hi community, I have tested two most widely used mailbox website mail.qq.com and mail.163.com(belongs to Netease). qq.mail works fine, good job @vino yang ! 163.mail almost works fine. I have received all the response mails, although the last one is marked as spam. I received the new mail of

Re: [DISCUSS] Ground Source and Sink Concepts in Flink SQL

Thanks Timo, I think it's fine to target it for Flink 1.10. Looking forward for your feedback. On Mon, 24 Jun 2019 at 15:07, Timo Walther wrote: > Thanks for working on this great design document Jark. I think having > well-defined terminilogy and semantics around tables, changelogs, table >

Re: [DISCUSS] Flink client api enhancement for downstream project

Hi, Tison, Thanks for your comments. Overall I agree with you that it is difficult for down stream project to integrate with flink and we need to refactor the current flink client api. And I agree that CliFrontend should only parsing command line arguments and then pass them to

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi Kurt, You did not give more further different opinions, so I thought you have agreed with the design after we promised to support two kinds of implementation. In API level, we have answered your question about pass an AggregateFunction to do the aggregation. No matter introduce localKeyBy API

Rolling policy when using StreamingFileSink for bulk-encoded output

Dear Flink community: We have a use case where StreamingFileSink is used for persisting bulk-encoded data to AWS s3. In our case, the data sources consist of hybrid types of events, for which each type is

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi all, I am happy we have a wonderful discussion and received many valuable opinions in the last few days. Now, let me try to summarize what we have reached consensus about the changes in the design. - provide a unified abstraction to support two kinds of implementation; - reuse

Re: Rolling policy when using StreamingFileSink for bulk-encoded output

Hi Ying, Thanks for using the StreamingFileSink. The reason why the StreamingFileSink only supports OnCheckpointRollingPolicy with bulk formats has to do with the fact that currently Flink relies on the Hadoop writer for Parquet. Bulk formats keep important details about how they write the

Re: [DISCUSS] Connectors and NULL handling

Hi Aljoscha, Thanks for raising the issue. It seems there are two issues here: 1) The null value handling, and 2) The error handling. For null value handling, my understanding is the following: - Null values could have a realistic meaning in some systems. So Flink needs to support them. - By

[jira] [Created] (FLINK-12957) Fix thrift and protobuf dependency examples in documentation

Nico Kruber created FLINK-12957: --- Summary: Fix thrift and protobuf dependency examples in documentation Key: FLINK-12957 URL: https://issues.apache.org/jira/browse/FLINK-12957 Project: Flink

Re: [DISCUSS] Flink client api enhancement for downstream project

That's exactly what I suggested a long time ago: the Flink REST client should not require any Flink dependency, only http library to call the REST services to submit and monitor a job. What I suggested also in [1] was to have a way to automatically suggest the user (via a UI) the available main

[jira] [Created] (FLINK-12959) Use BoundedInput and InputSelectable in blink

Jingsong Lee created FLINK-12959: Summary: Use BoundedInput and InputSelectable in blink Key: FLINK-12959 URL: https://issues.apache.org/jira/browse/FLINK-12959 Project: Flink Issue Type:

[jira] [Created] (FLINK-12960) Move ResultPartitionDeploymentDescriptor#releasedOnConsumption to PartitionDescriptor#releasedOnConsumption

Andrey Zagrebin created FLINK-12960: --- Summary: Move ResultPartitionDeploymentDescriptor#releasedOnConsumption to PartitionDescriptor#releasedOnConsumption Key: FLINK-12960 URL:

[jira] [Created] (FLINK-12961) StreamExecutionEnvironment supports executing job with StreamGraph

Biao Liu created FLINK-12961: Summary: StreamExecutionEnvironment supports executing job with StreamGraph Key: FLINK-12961 URL: https://issues.apache.org/jira/browse/FLINK-12961 Project: Flink

[jira] [Created] (FLINK-12962) Allows pyflink to be pip installed

Dian Fu created FLINK-12962: --- Summary: Allows pyflink to be pip installed Key: FLINK-12962 URL: https://issues.apache.org/jira/browse/FLINK-12962 Project: Flink Issue Type: Sub-task

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi vino, I think there are several things still need discussion. a) We all agree that we should first go with a unified abstraction, but the abstraction is not reflected by the FLIP. If your answer is "locakKeyBy" API, then I would ask how do we combine with `AggregateFunction`, and how do we do

Re: [DISCUSS] Ground Source and Sink Concepts in Flink SQL

Hi Jark, Impressive document! I have gone over the document quickly and left some comments. I will have a detailed look later. Below are two main thoughts from my side: 1. In the TableSource interface, can we move the getBoundedness() method into the underneath Source? This brings some benefits

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi vino, One thing to add, for a), I think use one or two examples like how to do local aggregation on a sliding window, and how do we do local aggregation on an unbounded aggregate, will do a lot help. Best, Kurt On Mon, Jun 24, 2019 at 6:06 PM Kurt Young wrote: > Hi vino, > > I think

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Best, tison. Hequn Cheng 于2019年6月24日周一 下午11:48写道： > Congratulations Jincheng! > > Best, Hequn > > On Mon, Jun 24, 2019 at 11:43 PM Dian Fu wrote: > > > Congratulations Jincheng! > > > > On Mon, Jun 24, 2019 at 11:09 PM Robert Metzger > > wrote: > > > > > Hi all, >

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Well deserved! Best Regards, Yu On Tue, 25 Jun 2019 at 00:01, Zili Chen wrote: > Congratulations Jincheng! > > Best, > tison. > > > Hequn Cheng 于2019年6月24日周一 下午11:48写道： > > > Congratulations Jincheng! > > > > Best, Hequn > > > > On Mon, Jun 24, 2019 at 11:43 PM Dian

Re: [DISCUSS] FLIP-44: Support Local Aggregation in Flink

Hi vino Thanks for proposal. For Local Aggregation I have a question about doing this in window aggregation. As we know , window aggregation like sliding window should based on Time trigger, and there may exists a problem in event time if we do local aggregation. For example if I want to do a

Re: [DISCUSS] A more restrictive JIRA workflow

Hey all, I would like to drive this discussion to an end soon. I've just merged the updated contribution guide to the Flink website: https://flink.apache.org/contributing/contribute-code.html I will now ask Apache IINFRA to change the permissions in our Jira. Here's the updated TODO list: 1. I

[ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Hi all, On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now part of the Apache Flink Project Management Committee (PMC). Jincheng has been a committer since July 2017. He has been very active on Flink's Table API / SQL component, as well as helping with releases.

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! On Mon, Jun 24, 2019 at 11:09 PM Robert Metzger wrote: > Hi all, > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is now > part of the Apache Flink Project Management Committee (PMC). > > Jincheng has been a committer since July 2017. He has been

[jira] [Created] (FLINK-12963) Add savepoint writer for bootstrapping new savepoints

Seth Wiesman created FLINK-12963: Summary: Add savepoint writer for bootstrapping new savepoints Key: FLINK-12963 URL: https://issues.apache.org/jira/browse/FLINK-12963 Project: Flink Issue

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Best, Hequn On Mon, Jun 24, 2019 at 11:43 PM Dian Fu wrote: > Congratulations Jincheng! > > On Mon, Jun 24, 2019 at 11:09 PM Robert Metzger > wrote: > > > Hi all, > > > > On behalf of the Flink PMC, I'm happy to announce that Jincheng Sun is > now > > part of the

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations Jincheng! Best Yun Tang From: Yu Li Sent: Tuesday, June 25, 2019 0:07 To: dev Subject: Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC Congratulations Jincheng! Well deserved! Best Regards, Yu On Tue, 25 Jun 2019 at 00:01, Zili Chen

Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC

Congratulations! On Mon, Jun 24, 2019 at 9:53 AM Yun Tang wrote: > Congratulations Jincheng! > > Best > Yun Tang > > From: Yu Li > Sent: Tuesday, June 25, 2019 0:07 > To: dev > Subject: Re: [ANNOUNCE] Jincheng Sun is now part of the Flink PMC > >

Re: [ANNOUNCE] Feature freeze for Apache Flink 1.9.0 release

Hi Gordon, Thanks for driving this effort. Xuefu responded to the discussion thread [1] and I want to bring that to our attention here: Hive integration depends on a few features that are actively developed. If the completion of those features don't leave enough time for us to integrate, then