dynamic streams and patterns

Hey everyone, I'm quite new to Apache Flink. I'm trying to build a system with Flink and wanted to hear your opinion and whether the proposed architecture is even possible with Flink. The environment for the system will be a microservice architecture handling messaging via async events. I want

AW: dynamic streams and patterns

te easy to deploy, and a simple testing job is implemented in a few hours, I would suggest to do some experiments to see how Flink behaves in the given environment. Regards, Robert On Mon, Jul 11, 2016 at 9:39 AM, Claudia Wegmann mailto:c.wegm...@kasasi.de>> wrote: Hey everyone, I’m qui

WG: dynamic streams and patterns

to be recompiled? Do I overlook something important? Thx for some input. Best, Claudia Von: Claudia Wegmann Gesendet: Freitag, 15. Juli 2016 12:58 An: 'user@flink.apache.org' Betreff: AW: dynamic streams and patterns Hey Robert, thanks for the valuable input. I have some follow-up

partial savepoints/combining savepoints

Hey everyone, I've got some questions regarding savepoints in Flink. I have the following situation: There is a microservice that reads data from Kafka topics, creates Flink streams from this data and does different computations/pattern matching workloads. If the overall workload for this serv

AW: partial savepoints/combining savepoints

r job. However, you would only be able to scale within a single Flink job and not across Flink jobs. Cheers, Till On Mon, Aug 1, 2016 at 9:49 PM, Claudia Wegmann mailto:c.wegm...@kasasi.de>> wrote: Hey everyone, I’ve got some questions regarding savepoints in Flink. I have the following

max aggregator dosen't work as expected

Hey, I have some questions to aggregate functions such as max or min. Take the following example: //create Stream with event time where Data contains an ID, a timestamp and a temperature value DataStream oneStream = env.fromElements( new Data(123, new Date(116, 8,8,11,11,11), 5),

AW: max aggregator dosen't work as expected

OK, found my mistake reagarding question 2.). I key by the id value and gave all the data sets different values there. So of course all 4 data sets are printed. Sorry :) But question 1.) still remains. Von: Claudia Wegmann [mailto:c.wegm...@kasasi.de] Gesendet: Montag, 8. August 2016 14:27 An

AW: max aggregator dosen't work as expected

ple with the max value at the specified position, the other fields of the tuple/pojo are undefined - maxBy() returns a tuple with the max value at the specified position (the other fields are retained) On Mon, Aug 8, 2016 at 2:55 PM, Claudia Wegmann mailto:c.wegm...@kasasi.de>> wrote

AW: partial savepoints/combining savepoints

rios how they appear when using docker images, for example. I could imagine that this could be helpful for your use case as well. Cheers, Till On Mon, Aug 1, 2016 at 10:40 PM, Claudia Wegmann mailto:c.wegm...@kasasi.de>> wrote: Hi Till, thanks for the quick reply. Too bad, I thought I w

AW: window-like use case

Hey, I'm no expert at all, but for me this sounds like a use case for Complex Event Processing (CEP). I don't know if you're aware of Flinks CEP Library [1, 2]? Maybe that solves your problem of multiple firings. But best to wait for the experts to answer your questions on handling state and fi

AW: What is the best way to load/add patterns dynamically (at runtime) with Flink?

Hey, I face the same problem and decided to go with your third solution. I use Groovy as the scripting language, which has access to Java classes and therefore also to Flink constructs like Time.seconds(10). See below for an example of a pattern definition with Groovy: private static Binding b