Ingesting data from an API

2018-11-11 Thread Aarti Gupta
e ingested. The same object instance is shared with the API and the Flink Execution environment, however, the output of the API does not get ingested into the Flink DataStream. Is this the right pattern to use, or is Kafka the recommended way of streaming data into Flink ? --Aarti -- Aarti Gu

Re: Grok and Flink

2018-08-30 Thread Aarti Gupta
> java-grok > > With that you should be able to get rid of the full Logstash piece and use > only the Grok part. > > Another solution, for example if you have logs/events in CEF Format, you > can just use 'split' in the flatmap function for example. > > Hope will help. > > Rega

Grok and Flink

2018-08-30 Thread Aarti Gupta
-- Aarti Gupta <https://www.linkedin.com/company/qualys> Director, Engineering, Correlation aagu...@qualys.com T Qualys, Inc. – Blog <https://qualys.com/blog> | Community <https://community.qualys.com> | Twitter <https://twitter.com/qualys> <https://www.qualys.com/email-banner>

Re: Dynamic Rule Evaluation in Flink

2018-07-09 Thread Aarti Gupta
on to >> evaluate all these streams against your defined rules. >> >> d) For Storing your aggregations and rules you can build your cache layer >> and pass as a argument >> to the constructor of that flatmap. >> >> >> >> >> >> >> >> >>

Re: Dynamic Rule Evaluation in Flink

2018-07-05 Thread Aarti Gupta
>> evaluate all these streams against your defined rules. >> >> d) For Storing your aggregations and rules you can build your cache layer >> and pass as a argument >> to the constructor of that flatmap. >> >> >> >> >> >> >> >&g

Re: Dynamic Rule Evaluation in Flink

2018-07-05 Thread Aarti Gupta
+Ken. --Aarti On Thu, Jul 5, 2018 at 6:48 PM, Aarti Gupta wrote: > Thanks everyone, will take a look. > > --Aarti > > On Thu, Jul 5, 2018 at 6:37 PM, Fabian Hueske wrote: > >> Hi, >> >> > Flink doesn't support connecting multiple streams with heterogene

Dynamic Rule Evaluation in Flink

2018-07-02 Thread Aarti Gupta
t, how do we evaluate rules that require pattern detection across events (Match rule if A happens, followed by B). 3. How do you dynamically define a window function. --Aarti -- Aarti Gupta <https://www.linkedin.com/company/qualys> Director, Engineering, Correlation aagu...@