Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

Thanks for the update! Just merged to 1.2.1 also: [FLINK-5962] [checkpoints] Remove scheduled cancel-task from timer queue to prevent memory leaks The remaining issue list looks good, but I would say that (5) is optional. It is not a critical production bug. On Wed, Mar 15, 2017 at 5:38 PM,

Re: Flink as a Service (FaaS)

Hi jinkui, I haven't go down to that deep yet. Sounds like you have better idea what needs to be in place. Can you try to come up with a doc and may be draw some diagram so we can discuss from there? My original intention is to discuss general function gap of running lots of micro services(like

[jira] [Created] (FLINK-6065) Make TransportClient for ES5 pluggable

Robert Metzger created FLINK-6065: - Summary: Make TransportClient for ES5 pluggable Key: FLINK-6065 URL: https://issues.apache.org/jira/browse/FLINK-6065 Project: Flink Issue Type:

Re: [DISCUSS] Project build time and possible restructuring

@Robert - I think once we know that a separate git repo works well, and that it actually solves problems, I see no reason to not create a connectors repository later. The infrastructure changes should be identical for two or more repositories. On Wed, Mar 15, 2017 at 5:22 PM, Till Rohrmann

Re: [Discuss] Retraction for Flink Streaming

Hi Shaoxuan, thanks a lot for this proposal! Support for retractions is a super nice and important feature and will enable many more use cases for the Table API / SQL. I'm really excited to see this happening. I made a first pass over your proposal and added a few comments. I'll do another pass

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

Thanks a lot for the updates so far everyone! From the discussion so far, the below is the still unfixed pending issues for 1.1.5 / 1.2.1 release. Since there’s only one backport for 1.1.5 left, I think having an RC for 1.1.5 near the end of this week / early next week is very promising, as

Re: [DISCUSS] Project build time and possible restructuring

"flink-core" means the main repository, not the "flink-core" module. When doing a release, we need to build the flink main code first, because the flink-libraries depend on that. Once the "flink-libraries" are build, we need to run the main build again (at least the flink-dist module), so that it

Re: [DISCUSS] Project build time and possible restructuring

I'm ok with point 3. Concerning point 8: Why do we have to build flink-core twice after having it built as a dependency for flink-libraries? This seems wrong to me. Cheers, Till On Wed, Mar 15, 2017 at 4:23 PM, Robert Metzger wrote: > Thank you. Running on AWS is a good

Re: [DISCUSS] Project build time and possible restructuring

Thank you. Running on AWS is a good idea! Let me know if you (or anybody else) wants to help me with the infrastructure work! Any help is much appreciated (as I've said before, I don't really have time for doing this, but it has to be done :) ) I'm against creating two new repositories. I fear

[jira] [Created] (FLINK-6064) wrong BlobServer hostname in TaskExecutor

Nico Kruber created FLINK-6064: -- Summary: wrong BlobServer hostname in TaskExecutor Key: FLINK-6064 URL: https://issues.apache.org/jira/browse/FLINK-6064 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-17 Side Inputs

Hi, thanks for you input! :-) Regarding 1) I don't see the benefit of integrating windowing into the side-input logic. Windowing can happen upstream and whenever that emits new data then operator will notice because there is new input. Having windowing inside the side-input of an operator as well

Re: [DISCUSS] Project build time and possible restructuring

Robert, appreciate your kickstarting this task. We should compare the verification time with and without the listed modules. I’ll try to run this by tomorrow on AWS and on Travis. Should we maintain separate repos for flink-contrib and flink-libraries? Are you intending that we move

[jira] [Created] (FLINK-6063) HA Configuration doesn't work with Flink 1.2

Razvan created FLINK-6063: - Summary: HA Configuration doesn't work with Flink 1.2 Key: FLINK-6063 URL: https://issues.apache.org/jira/browse/FLINK-6063 Project: Flink Issue Type: Bug

Re: [POLL] Who still uses Java 7 with Flink ?

I've put it also on our Twitter account: https://twitter.com/ApacheFlink/status/842015062667755521 On Wed, Mar 15, 2017 at 2:19 PM, Martin Neumann wrote: > I think this easier done in a straw poll than in an email conversation. > I created one at:

Re: [DISCUSS] Project build time and possible restructuring

Thank you for looking into this Till. I think we then have to split the repositories. My main motivation for doing this is that it seems to be the only feasible way of scaling the community to allow more committers working on the libraries. I'll take care of getting things started. As the next

Re: [POLL] Who still uses Java 7 with Flink ?

I think this easier done in a straw poll than in an email conversation. I created one at: http://www.strawpoll.me/12535073 (Note that you have multiple choices.) Though I prefer Java 8 most of the time I have to work on Java 7. A lot of the infrastructure I work on still runs Java 7, one of the

Re: [DISCUSS] Project build time and possible restructuring

In theory we could have a merging bot which solves the problem of the "commit window". Once the PR passes all tests and has enough +1s, the bot could do the merging and, thus, it effectively linearizes the merge process. I think the second point is actually a disadvantage because there is not

[jira] [Created] (FLINK-6062) Re-read a Kafka topic at the reception of an event

Julien Preisner created FLINK-6062: -- Summary: Re-read a Kafka topic at the reception of an event Key: FLINK-6062 URL: https://issues.apache.org/jira/browse/FLINK-6062 Project: Flink Issue

[POLL] Who still uses Java 7 with Flink ?

Hi all! I would like to get a feeling how much Java 7 is still being used among Flink users. At some point, it would be great to drop Java 7 support and make use of Java 8's new features, but first we would need to get a feeling how much Java 7 is still used. Would be happy if users on Java 7

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

Can we fix this issue in the 1.2.1: Flink-python tests cost too long time https://issues.apache.org/jira/browse/FLINK-5650 > 在 2017年3月15日，下午6:29，Vladislav Pernin 写道： > > I just tested in in my reproducer. It works.

Re: [jira] [Created] (FLINK-6061) NPE on TypeSerializer.serialize with a RocksDBStateBackend calling state.entries in the open() function

What would be the better / less dirty way of expiring a MapState or more generally being able to access entries of a keyed state at some interval ? The currentKey has to be set in the AbstractRocksDBState. Incrementing a counter in the rich function and at some modulo, doing the job does not

[jira] [Created] (FLINK-6061) NPE on TypeSerializer.serialize with a RocksDBStateBackend calling state.entries in the open() function

Vladislav Pernin created FLINK-6061: --- Summary: NPE on TypeSerializer.serialize with a RocksDBStateBackend calling state.entries in the open() function Key: FLINK-6061 URL:

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

I just tested in in my reproducer. It works. 2017-03-15 11:22 GMT+01:00 Aljoscha Krettek : > I did in fact just open a PR for > > https://issues.apache.org/jira/browse/FLINK-6001 > > NPE on TumblingEventTimeWindows with ContinuousEventTimeTrigger and > > allowedLateness > >

Re: Question: RemoteStreamEnvironment submit in detach mode.

I would also really like to have this feature in. I'll try and see if I can find some time to get the PR into mergeable shape. On Wed, Mar 15, 2017, at 09:11, Liangfei Su wrote: > Thanks for the information, Evgeny. > > It looks no fixed version for these feature which i think it quite common >

Re: [DISCUSS] Release Flink 1.1.5 / Flink 1.2.1

I did in fact just open a PR for > https://issues.apache.org/jira/browse/FLINK-6001 > NPE on TumblingEventTimeWindows with ContinuousEventTimeTrigger and > allowedLateness On Tue, Mar 14, 2017, at 18:20, Vladislav Pernin wrote: > Hi, > > I would also include the following (not yet resolved)

[jira] [Created] (FLINK-6059) Reject DataSet and DataStream without RowTypeInformation

Fabian Hueske created FLINK-6059: Summary: Reject DataSet and DataStream without RowTypeInformation Key: FLINK-6059 URL: https://issues.apache.org/jira/browse/FLINK-6059 Project: Flink

回复：[jira] [Created] (FLINK-6057) Better default needed for num network buffers

Hi Luke,        Yes, it is really not very friendly for users to set the number of network buffers. And it may cause OOM exception if not set the proper amount for large scale job. The proper amount should be calculated automatically by framework based on the number of input and output channels

[jira] [Created] (FLINK-6057) Better default needed for num network buffers

Luke Hutchison created FLINK-6057: - Summary: Better default needed for num network buffers Key: FLINK-6057 URL: https://issues.apache.org/jira/browse/FLINK-6057 Project: Flink Issue Type:

Re: Question: RemoteStreamEnvironment submit in detach mode.

Thanks for the information, Evgeny. It looks no fixed version for these feature which i think it quite common requirement. Do we have any planned release to include this fix? Thanks Ralph On Wed, Mar 15, 2017 at 3:49 PM, Evgeny Kincharov wrote: > Hi, Ralph. > >

Re: [DISCUSS] FLIP-17 Side Inputs

Hi, Aljoscha, I just go through your prototype. I like the design of the SideInputReader which can make it flexible to determine when we can get the side input. I agree that side inputs are API sugar on the top of the three components(n-ary inputs, broadcast state and input buffering), following