Re: [Feature]Returning RuntimeException to REST client while job submission

2019-01-24 Thread Lavkesh Lahngir
Hello, I created a Jira. Can somebody review it, please? Or suggest if this is useful? https://issues.apache.org/jira/browse/FLINK-11423 Thank you. ~Lavkesh On Thu, Jan 24, 2019 at 11:40 AM Lavkesh Lahngir wrote: > Hi, > It's not fixed in the master. I compiled and ran it yesterday. > I am not

Re: [ANNOUNCE] Contributing Alibaba's Blink

2019-01-24 Thread jincheng sun
Thanks Stephan, The entire plan makes sense to me. Regarding the branch name, how about using "blink-flink-1.5" wich meant that the branche is based on flink-1.5. If the name is "blink-1.5", some users will think that this is the version number of the internal Blink of alibaba, and will not

Re: [DISCUSS] Towards a leaner flink-dist

2019-01-24 Thread Jark Wu
+1 for the leaner distribution and improve the "Download" page. On Fri, 25 Jan 2019 at 01:54, Bowen Li wrote: > +1 for leaner distribution and a better 'download' webpage. > > +1 for a full distribution if we can automate it besides supporting the > leaner one. If we support both, I'd image

Re: [DISCUSS] Towards a leaner flink-dist

2019-01-24 Thread jincheng sun
Hi Chesnay, Thank you for the proposal. And i like it very much. +1 for the leaner distribution. About improve the "Download" page, I think we can add the connectors download link in the "Optional components" section which @Timo Walther mentioned above. Regards, Jincheng Chesnay Schepler

[jira] [Created] (FLINK-11432) YarnFlinkResourceManagerTest test case timeout

2019-01-24 Thread vinoyang (JIRA)
vinoyang created FLINK-11432: Summary: YarnFlinkResourceManagerTest test case timeout Key: FLINK-11432 URL: https://issues.apache.org/jira/browse/FLINK-11432 Project: Flink Issue Type: Test

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Guowei Ma
+1 This not only helps Chinese users but also helps the community to collect more feedback and scenarios. > 在 2019年1月25日,上午2:29,Zhang, Xuefu 写道: > > +1 on the idea. This will certainly help promote Flink in China industries. > On a side note, it would be great if anyone in the list can help

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Jark Wu
a big +1 to this! An user-zh mailing list will help more Chinese speaking users and there will be more people willing to help answer questions who want to contribute for Flink. It's exciting to see the community is embracing Chinese speaking users. Best, Jark On Fri, 25 Jan 2019 at 11:59,

Re: [DISCUSS] Shall we make SpillableSubpartition repeatedly readable to support fine grained recovery

2019-01-24 Thread zhijiang
Hi Bo, Your mentioned problems can be summaried into two issues: 1. Failover strategy should consider whether the upstream produced partition is still available when the downstream fails. If the produced partition is available, then only downstream region needs to restarted, otherwise the

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Hequn Cheng
+1 This would be very helpful! On Fri, Jan 25, 2019 at 8:15 AM Guowei Ma wrote: > +1 > > This not only helps Chinese users but also helps the community to collect > more feedback and scenarios. > > > > 在 2019年1月25日,上午2:29,Zhang, Xuefu 写道: > > > > +1 on the idea. This will certainly help

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread shengjk1
+1, a good idea thanks On 01/25/2019 09:26,Hequn Cheng wrote: +1 This would be very helpful! On Fri, Jan 25, 2019 at 8:15 AM Guowei Ma wrote: +1 This not only helps Chinese users but also helps the community to collect more feedback and scenarios. 在 2019年1月25日,上午2:29,Zhang, Xuefu 写道:

Re: [DISCUSS] FLIP-32: Restructure flink-table for future contributions

2019-01-24 Thread jincheng sun
Hi Timo, Thanks a lot for bring up the FLIP-32 discussion and the very detailed implementation plan document ! Restructure `flink-table` is an important part of flink merge blink, looking forward to the JIRAs which will be opened ! Cheers, Jincheng Timo Walther 于2019年1月24日周四 下午9:06写道: > Hi

Re: [DISCUSS] Shall we make SpillableSubpartition repeatedly readable to support fine grained recovery

2019-01-24 Thread Guowei Ma
Thanks to zhijiang for a detailed explanation. I would do some supplements Blink has indeed solved this particular problem. This problem can be identified in Blink and the upstream will be restarted by Blink thanks zhijiang 于2019年1月25日周五 下午12:04写道: > Hi Bo, > > Your mentioned problems can be

[jira] [Created] (FLINK-11424) org.apache.flink.metrics.datadog.DatadogHttpReporter#report remove will throw exception

2019-01-24 Thread lining (JIRA)
lining created FLINK-11424: -- Summary: org.apache.flink.metrics.datadog.DatadogHttpReporter#report remove will throw exception Key: FLINK-11424 URL: https://issues.apache.org/jira/browse/FLINK-11424

Re: Flink CEP : Doesn't generate output

2019-01-24 Thread Chesnay Schepler
Can you provide us a self-contained reproducing example? (preferably as elementary as possible) On 22.01.2019 18:58, dhanuka ranasinghe wrote: Hi All, I have used Flink CEP to filter some events and generate some alerts based on certain conditions. But unfortunately doesn't print any

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2019-01-24 Thread Piotr Nowojski
Hi Biao! This discussion was stalled because of preparations for the open sourcing & merging Blink. I think before creating the tickets we should split this discussion into topics/areas outlined by Stephan and create Flips for that. I think there is no chance for this to be completed in couple

[jira] [Created] (FLINK-11427) Protobuf parquet writer implementation

2019-01-24 Thread Guang Hu (JIRA)
Guang Hu created FLINK-11427: Summary: Protobuf parquet writer implementation Key: FLINK-11427 URL: https://issues.apache.org/jira/browse/FLINK-11427 Project: Flink Issue Type: Improvement

Re: [ANNOUNCE] Contributing Alibaba's Blink

2019-01-24 Thread Timo Walther
Regarding the content of a `blink-1.5` branch, is it possible to rebase the big Blink commit on top of the current master or the last Flink release? I don't mean a full rebase here, but just forking the branch from current Flink, and putting the Blink content into the repository, and commit

[jira] [Created] (FLINK-11426) FlinkKafkaProducerBase test case failed

2019-01-24 Thread vinoyang (JIRA)
vinoyang created FLINK-11426: Summary: FlinkKafkaProducerBase test case failed Key: FLINK-11426 URL: https://issues.apache.org/jira/browse/FLINK-11426 Project: Flink Issue Type: Test

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Tzu-Li (Gordon) Tai
Hi Robert, Thanks a lot for starting this discussion! +1 to a user-zh@flink.a.o mailing list (you mentioned -zh in the title, but -cn in the opening email content. I think -zh would be better as we are establishing the tool for general Chinese-speaking users). All dev@ discussions / JIRAs should

[jira] [Created] (FLINK-11425) Support of “Hash Teams” in Hybrid Hash Join

2019-01-24 Thread LiuJi (JIRA)
LiuJi created FLINK-11425: - Summary: Support of “Hash Teams” in Hybrid Hash Join Key: FLINK-11425 URL: https://issues.apache.org/jira/browse/FLINK-11425 Project: Flink Issue Type: New Feature

Re: [ANNOUNCE] Contributing Alibaba's Blink

2019-01-24 Thread Kurt Young
Sure, i will do the rebase before pushing the branch. Timo Walther 于2019年1月24日 周四18:20写道: > Regarding the content of a `blink-1.5` branch, is it possible to rebase > the big Blink commit on top of the current master or the last Flink > release? > > I don't mean a full rebase here, but just

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2019-01-24 Thread Stephan Ewen
Before creating any JIRA issues, we need to converge a bit further on the design. There are too many unsolved questions in the above summary. I would try and come up with a next version of the interface proposal in the coming days and use that as the base to continue the discussion. Whether this

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Robert Metzger
Thanks for your response. You are right, I'm proposing "user...@flink.apache.org" as the mailing list's name! On Thu, Jan 24, 2019 at 12:37 PM Tzu-Li (Gordon) Tai wrote: > Hi Robert, > > Thanks a lot for starting this discussion! > > +1 to a user-zh@flink.a.o mailing list (you mentioned -zh in

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Kurt Young
Big +1 on this, it will indeed help Chinese speaking users a lot. fudian.fd 于2019年1月24日 周四20:18写道: > +1. I noticed that many folks from China are requesting the JIRA > permission in the past year. It reflects that more and more developers from > China are using Flink. A Chinese oriented mailing

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread fudian.fd
+1. I noticed that many folks from China are requesting the JIRA permission in the past year. It reflects that more and more developers from China are using Flink. A Chinese oriented mailing list will definitely be helpful for the growth of Flink in China. > 在 2019年1月24日,下午7:42,Stephan Ewen

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Piotr Nowojski
+1, good idea, especially with that many Chinese speaking contributors, committers & users :) Piotrek > On 24 Jan 2019, at 13:20, Kurt Young wrote: > > Big +1 on this, it will indeed help Chinese speaking users a lot. > > fudian.fd 于2019年1月24日 周四20:18写道: > >> +1. I noticed that many folks

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Jeff Zhang
+1 Piotr Nowojski 于2019年1月24日周四 下午8:38写道: > +1, good idea, especially with that many Chinese speaking contributors, > committers & users :) > > Piotrek > > > On 24 Jan 2019, at 13:20, Kurt Young wrote: > > > > Big +1 on this, it will indeed help Chinese speaking users a lot. > > > > fudian.fd

[DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Robert Metzger
Hey all, I would like to create a new user support mailing list called " user...@flink.apache.org" to cater the Chinese-speaking Flink community. Why? In the last year 24% of the traffic on flink.apache.org came from the US, 22% from China. In the last three months, China is at 30%, the US at

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Stephan Ewen
+1, a very nice idea On Thu, Jan 24, 2019 at 12:41 PM Robert Metzger wrote: > Thanks for your response. > > You are right, I'm proposing "user...@flink.apache.org" as the mailing > list's name! > > On Thu, Jan 24, 2019 at 12:37 PM Tzu-Li (Gordon) Tai > wrote: > > > Hi Robert, > > > > Thanks a

[DISCUSS] FLIP-32: Restructure flink-table for future contributions

2019-01-24 Thread Timo Walther
Hi everyone, as Stephan already announced on the mailing list [1], the Flink community will receive a big code contribution from Alibaba. The flink-table module is one of the biggest parts that will receive many new features and major architectural improvements. Instead of waiting until the

[jira] [Created] (FLINK-11428) BufferFileWriterFileSegmentReaderTest#testWriteRead failed on Travis

2019-01-24 Thread Congxian Qiu (JIRA)
Congxian Qiu created FLINK-11428: Summary: BufferFileWriterFileSegmentReaderTest#testWriteRead failed on Travis Key: FLINK-11428 URL: https://issues.apache.org/jira/browse/FLINK-11428 Project: Flink

[jira] [Created] (FLINK-11430) Incorrect Akka timeout syntax

2019-01-24 Thread Sina Madani (JIRA)
Sina Madani created FLINK-11430: --- Summary: Incorrect Akka timeout syntax Key: FLINK-11430 URL: https://issues.apache.org/jira/browse/FLINK-11430 Project: Flink Issue Type: Task

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Fabian Hueske
Thanks Robert! I think this is a very good idea. +1 Fabian Am Do., 24. Jan. 2019 um 14:09 Uhr schrieb Jeff Zhang : > +1 > > Piotr Nowojski 于2019年1月24日周四 下午8:38写道: > > > +1, good idea, especially with that many Chinese speaking contributors, > > committers & users :) > > > > Piotrek > > > > >

Re: [DISCUSS] Shall we make SpillableSubpartition repeatedly readable to support fine grained recovery

2019-01-24 Thread Piotr Nowojski
Hi, I’m not sure how much effort we will be willing to invest in the existing batch stack. We are currently focusing on the support of bounded DataStreams (already done in Blink and will be merged to Flink soon) and unifing batch & stream under DataStream API. Piotrek > On 23 Jan 2019, at

[jira] [Created] (FLINK-11429) Flink fails to authenticate s3a with core-site.xml

2019-01-24 Thread Mario Georgiev (JIRA)
Mario Georgiev created FLINK-11429: --- Summary: Flink fails to authenticate s3a with core-site.xml Key: FLINK-11429 URL: https://issues.apache.org/jira/browse/FLINK-11429 Project: Flink

Re: [DISCUSS] Shall we make SpillableSubpartition repeatedly readable to support fine grained recovery

2019-01-24 Thread Stephan Ewen
The SpillableSubpartition can also be used during the execution of bounded DataStreams programs. I think this is largely independent from deprecating the DataSet API. I am wondering if this particular issue is one that has been addressed in the Blink code already (we are looking to merge much of

flink runtime incompatibility with java 9 or above due to akka version

2019-01-24 Thread Matthieu Bonneviot
Hi i am working on migrating my flink cluster to 1.7.1 with java 11. I am facing a runtime error in the taskmanager: 2019-01-24 14:43:37,014 ERROR akka.remote.Remoting - class [B cannot be cast to class [C ([B and [C are in module java.base of loader

[jira] [Created] (FLINK-11431) Akka dependency not compatible with java 9 or above

2019-01-24 Thread Matthieu Bonneviot (JIRA)
Matthieu Bonneviot created FLINK-11431: -- Summary: Akka dependency not compatible with java 9 or above Key: FLINK-11431 URL: https://issues.apache.org/jira/browse/FLINK-11431 Project: Flink

Re: flink runtime incompatibility with java 9 or above due to akka version

2019-01-24 Thread Gary Yao
Hi Matthieu, Flink is not Java 9 compatible at the moment [1]. Although Flink can be compiled with Java 9, not all tests are currently passing. For a list of modules that have failing tests, see [2]. The problem that you found persists in Akka versions prior to 2.5.6 [3][4]. Best, Gary [1]

Re: flink runtime incompatibility with java 9 or above due to akka version

2019-01-24 Thread Chesnay Schepler
Flink only runs on java 8. There were efforts to add support for java 9 in 1.8 but in turned into a massive rabbit hole with rather wide implications, like the akka version upgrade that you just found. On 24.01.2019 17:04, Matthieu Bonneviot wrote: Hi i am working on migrating my flink

Re: [DISCUSS] Towards a leaner flink-dist

2019-01-24 Thread Bowen Li
+1 for leaner distribution and a better 'download' webpage. +1 for a full distribution if we can automate it besides supporting the leaner one. If we support both, I'd image release managers should be able to package two distributions with a single change of parameter instead of manually package

Re: Side Outputs for late arriving records

2019-01-24 Thread Fabian Hueske
Hi Ramya, This would be a great feature, but unfortunately is not support (yet) by Flink SQL. Currently, all late records are dropped. A workaround is to ingest the stream as a DataStream, have a custom operator that routes all late records to a side output, and registering the DataStream

Re: [ANNOUNCE] Contributing Alibaba's Blink

2019-01-24 Thread Bowen Li
Exciting to see this happening! Wrt doc, have we done a diff which can show us how much differences are between Flink's and Blink's documentation (flink/docs)? For example, how many pages and how much percentage of each page is different? How many new pages (for new features) does Blink have?If

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Bowen Li
+1, a great starting point! It'll be nice if we can later come up with some way to bridge the knowledge between two email channels. On Thu, Jan 24, 2019 at 5:32 AM Fabian Hueske wrote: > Thanks Robert! > I think this is a very good idea. > +1 > > Fabian > > Am Do., 24. Jan. 2019 um 14:09 Uhr

Re: [DISCUSS] Start a user...@flink.apache.org mailing list for the Chinese-speaking community?

2019-01-24 Thread Zhang, Xuefu
+1 on the idea. This will certainly help promote Flink in China industries. On a side note, it would be great if anyone in the list can help source ideas, bug reports, and feature requests to dev@ list and/or JIRAs so as to gain broader attention. Thanks, Xuefu