Re: Release notes 1.1.0?

Hi! In the above example the keySelector would run once before and once inside the window operator. In that sense, the version below is a better way to do it. You can also create windows of 50 or max 100 ms by writing your own trigger. Have a look at the count trigger. You can augment it by

Re: Release notes 1.1.0?

Hi Stephan Thanks for the explanation! We will stick to 1.0.3 to keep our code clean. In the workaround case, how does key selector instantiated? One instance per window operator? By the way is there a way to create a hybrid window of count and time, like 50 items or max process time 100ms?

Re: Release notes 1.1.0?

Hi Andrew! Here is the reason for what is happening with your job: You have used some sort of undocumented and unofficial corner case behavior of Flink 1.0.0, namely, using parallel windowAll(). Initially, windowAll() was supposed to not be parallel, but the system did not prevent to set a

Re: Release notes 1.1.0?

Hi Aljoscha We are not using state backend explicitly, recovery and state backend are pointed to file path. See attached json file Thanks for the help. Best regards Andrew > On 10 Aug 2016, at 11:38, Aljoscha Krettek wrote: > > Oh, are you by any chance specifying a

Re: Release notes 1.1.0?

Oh, are you by any chance specifying a custom state backend for your job? For example, RocksDBStateBackend. Cheers, Aljoscha On Wed, 10 Aug 2016 at 11:17 Aljoscha Krettek wrote: > Hi, > could you maybe send us the output of "env.getExecutionPlan()". This would > help us

Re: Release notes 1.1.0?

Hi, could you maybe send us the output of "env.getExecutionPlan()". This would help us better understand which operators are used exactly. (You can of course remove any security sensitive stuff.) Cheers, Aljoscha On Tue, 9 Aug 2016 at 15:30 Andrew Ge Wu wrote: > Oh

Re: Release notes 1.1.0?

Oh sorry missed that part, no, Im not explicitly set that. > On 09 Aug 2016, at 15:29, Aljoscha Krettek wrote: > > Hi, > are you setting a StreamTimeCharacteristic, i.e. > env.setStreamTimeCharacteristic? > > Cheers, > Aljoscha > > On Tue, 9 Aug 2016 at 14:52 Andrew Ge

Re: Release notes 1.1.0?

Hi, are you setting a StreamTimeCharacteristic, i.e. env.setStreamTimeCharacteristic? Cheers, Aljoscha On Tue, 9 Aug 2016 at 14:52 Andrew Ge Wu wrote: > Hi Aljoscha > > > Plan attached, there are split streams and union operations around, but > here is how windows are

Re: Release notes 1.1.0?

Hi, could you maybe post how exactly you specify the window? Also, did you set a "stream time characteristic", for example EventTime? That could help us pinpoint the problem. Cheers, Aljoscha On Tue, 9 Aug 2016 at 12:42 Andrew Ge Wu wrote: > I rolled back to 1.0.3 > If

Re: Release notes 1.1.0?

I rolled back to 1.0.3 If I understand this correctly, the peak when topology starts is because it is trying to fill all the buffers, but I can not see that in 1.1.0. > On 09 Aug 2016, at 12:10, Robert Metzger wrote: > > Which source are you using? > > On Tue, Aug 9,

Re: Release notes 1.1.0?

Which source are you using? On Tue, Aug 9, 2016 at 11:50 AM, Andrew Ge Wu wrote: > Hi Robert > > > Thanks for the quick reply, I guess I’m one of the early birds. > Yes, it is much slower, I’m not sure why, I copied slaves, masters, > log4j.properties and flink-conf.yaml

Re: Release notes 1.1.0?

Hi Robert Thanks for the quick reply, I guess I’m one of the early birds. Yes, it is much slower, I’m not sure why, I copied slaves, masters, log4j.properties and flink-conf.yaml directly from 1.0.3 I have parallelization 1 on my sources, I can increase that to achieve the same speed, but I’m

Re: Release notes 1.1.0?

Hi Andrew, here is the release announcement, with a list of all changes: http://flink.apache.org/news/2016/08/08/release-1.1.0.html, http://flink.apache.org/blog/release_1.1.0-changelog.html What does the chart say? Are the results different? is Flink faster or slower now? Regards, Robert On