Re: [DISCUSS] Flink 1.6 features

2018-07-05 Thread Vishal Santoshi
I am not sure whether this is in any roadmap and as someone suggested wishes are free...Tensorflow on flink though ambitious should be a big win. I am not sure how operator isolation for a hybrid GPU/CPU would be achieved and how repetitive execution could be natively supported by flink but it

Re: [DISCUSS] Flink 1.6 features

2018-07-04 Thread Stephan Ewen
Hi all! A late follow-up with some thoughts: In principle, all these are good suggestions and are on the roadmap. We are trying to make the release "by time", meaning for it at a certain date (roughly in two weeks) and take what features are ready into the release. Looking at the status of the

Re: [DISCUSS] Flink 1.6 features

2018-06-24 Thread zhangminglei
Hi, Community By the way, there is a very important feature I think it should be. Currently, the BucketingSink does not support when a bucket is ready for user use. This situation will be very obvious when flink work with offline end. We called that real time/offline integration in business.

Re: [DISCUSS] Flink 1.6 features

2018-06-18 Thread zhangminglei
> > >>>> Atomic cancel-with-savepoint: Agreed, this is important. Making this > >>>> work with all sources needs a bit more work. We should have this in the > >>>> roadmap. > >>>> > >>>> Elastic Bloomfilters: This s

Re: [DISCUSS] Flink 1.6 features

2018-06-18 Thread sagar loke
> work with all sources needs a bit more work. We should have this in the > roadmap. > >>>> > >>>> Elastic Bloomfilters: This seems like an interesting new feature - > the above suggested feature set was more about addressing some longer > standing issues

Re: [DISCUSS] Flink 1.6 features

2018-06-17 Thread zhangminglei
uld have this in the roadmap. >>>> >>>> Elastic Bloomfilters: This seems like an interesting new feature - the >>>> above suggested feature set was more about addressing some longer standing >>>> issues/requests. However, nothing should prevent contribut

Re: [DISCUSS] Flink 1.6 features

2018-06-17 Thread Will Du
vent contributors to work on >>> that. >>> >>> Best, >>> Stephan >>> >>> >>> On Wed, Jun 6, 2018 at 6:23 AM, Yan Zhou [FDS Science] >> <mailto:yz...@coupang.com>> wrote: >>> +1 on https://issues.apache.org/

Re: [DISCUSS] Flink 1.6 features

2018-06-17 Thread zhangminglei
ps://issues.apache.org/jira/browse/FLINK-5479> >> issues.apache.org <http://issues.apache.org/> >> Reported in ML: >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-topic-partition-skewness-causes-watermark-not-being-emitted-td11008.html >&

Re: [DISCUSS] Flink 1.6 features

2018-06-17 Thread zhangminglei
pache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-topic-partition-skewness-causes-watermark-not-being-emitted-td11008.html > > <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-topic-partition-skewness-causes-watermark-not-being-emitted-td110

Re: [DISCUSS] Flink 1.6 features

2018-06-16 Thread sagar loke
gt;> [FLINK-5479] Per-partition watermarks in ... >>>> <https://issues.apache.org/jira/browse/FLINK-5479> >>>> issues.apache.org >>>> Reported in ML: >>>> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Kafka-topic-partition-ske

Re: [DISCUSS] Flink 1.6 features

2018-06-16 Thread Elias Levy
Reported in ML: http://apache-flink-user-maili >>> ng-list-archive.2336050.n4.nabble.com/Kafka-topic-partition- >>> skewness-causes-watermark-not-being-emitted-td11008.html It's normally >>> not a common case to have Kafka partitions not producing any data, but >>> it'll p

Re: [DISCUSS] Flink 1.6 features

2018-06-09 Thread sihua zhou
good to handle this as well. I ... | From: Rico Bergmann Sent: Tuesday, June 5, 2018 9:12:00 PM To: Hao Sun Cc:d...@flink.apache.org; user Subject: Re: [DISCUSS] Flink 1.6 features +1 on K8s integration Am 06.06.2018 um 00:01 schrieb Hao Sun : adding my vote to K8S Job mode

Re: [DISCUSS] Flink 1.6 features

2018-06-08 Thread Elias Levy
lly >> not a common case to have Kafka partitions not producing any data, but >> it'll probably be good to handle this as well. I ... >> >> -- >> *From:* Rico Bergmann >> *Sent:* Tuesday, June 5, 2018 9:12:00 PM >> *To:* Hao S

Re: [DISCUSS] Flink 1.6 features

2018-06-08 Thread Stephan Ewen
8 9:12:00 PM > *To:* Hao Sun > *Cc:* d...@flink.apache.org; user > *Subject:* Re: [DISCUSS] Flink 1.6 features > > +1 on K8s integration > > > > Am 06.06.2018 um 00:01 schrieb Hao Sun : > > adding my vote to K8S Job mode, maybe it is this? > > Smoothen the

Re: [DISCUSS] Flink 1.6 features

2018-06-05 Thread Yan Zhou [FDS Science]
Cc: d...@flink.apache.org; user Subject: Re: [DISCUSS] Flink 1.6 features +1 on K8s integration Am 06.06.2018 um 00:01 schrieb Hao Sun mailto:ha...@zendesk.com>>: adding my vote to K8S Job mode, maybe it is this? > Smoothen the integration in Container environment, like "Flink as a Lib

Re: [DISCUSS] Flink 1.6 features

2018-06-05 Thread Rico Bergmann
y Shum >>> >>> Stephan Ewen ---06/04/2018 02:21:47 AM---Hi Flink Community! The release of >>> Apache Flink 1.5 has happened (yay!) - so it is a good time >>> >>> From: Stephan Ewen >>> To: d...@flink.apache.org, user >>> Dat

Re: [DISCUSS] Flink 1.6 features

2018-06-05 Thread Hao Sun
has happened (yay!) - so it is a good time >> >> From: Stephan Ewen >> To: d...@flink.apache.org, user >> Date: 06/04/2018 02:21 AM >> Subject: [DISCUSS] Flink 1.6 features >> -- >> >> >> >> Hi Flink Community! &

Re: [DISCUSS] Flink 1.6 features

2018-06-05 Thread Ben Yan
he > Flink 1.5 has happened (yay!) - so it is a good time > > From: Stephan Ewen > To: d...@flink.apache.org, user > Date: 06/04/2018 02:21 AM > Subject: [DISCUSS] Flink 1.6 features > -- > > > > Hi Flink Community! > > The release of Apache

Re: [DISCUSS] Flink 1.6 features

2018-06-04 Thread Che Lui Shum
, Shirley Shum From: Stephan Ewen To: d...@flink.apache.org, user Date: 06/04/2018 02:21 AM Subject:[DISCUSS] Flink 1.6 features Hi Flink Community! The release of Apache Flink 1.5 has happened (yay!) - so it is a good time to start talking about what to do for release 1.6

Re: [DISCUSS] Flink 1.6 features

2018-06-04 Thread Till Rohrmann
Before removing the legacy code, I would still wait a bit and see what the user feedback is. The legacy mode is a good safety net against severe deployment regressions. Thus, it should be a very conscious decision to remove the code. As far as I know, there is currently nobody actively working on

Re: [DISCUSS] Flink 1.6 features

2018-06-04 Thread Antoine Philippot
Hi Stephen, Is it planned to consider this ticket https://issues.apache.org/jira/browse/FLINK-7883 about an atomic cancel-with-savepoint ? It is my main concern about Flink and I have to maintain a fork myself as we can't afford dupplicate events due to reprocess of messages between a savepoint

Re:[DISCUSS] Flink 1.6 features

2018-06-04 Thread sihua zhou
Hi Stephan, could you please also consider the "Elastic Filter " feature discussioned in thread http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/PROPOSAL-Introduce-Elastic-Bloom-Filter-For-Flink-td22430.html ? Best, Sihua On 06/4/2018 17:21,Stephan Ewen wrote: Hi Flink

Re: [DISCUSS] Flink 1.6 features

2018-06-04 Thread Chesnay Schepler
Will we remove the legacy mode for 1.6? I can see value in keeping it for now so that legacy issues are still visible on master, but at the same time removing this code would reduce a lot of complexity and ambiguity in the codebase... On 04.06.2018 11:21, Stephan Ewen wrote: Hi Flink

[DISCUSS] Flink 1.6 features

2018-06-04 Thread Stephan Ewen
Hi Flink Community! The release of Apache Flink 1.5 has happened (yay!) - so it is a good time to start talking about what to do for release 1.6. *== Suggested release timeline ==* I would propose to release around *end of July* (that is 8-9 weeks from now). The rational behind that: There was