Aggregation based on Timestamp

Hi We have use case where we have thousands of Telegraf agents sending data to kafka( some of them are sending 10s interval, 15s interval and 30s interval). We would like to aggregate the incoming data to 1 minuter interval based on the hostname as key before we write into influxdb. Is it

Flink streaming Python

Hi I have asked the same question back in Jan 2016 and checking again with community to see if there is any update or plan for supporting streaming Flink in python.

Amazon Athena

Anyone used used Amazon Athena with Apache Flink? I have use case where I want to write streaming data ( which is in Avro format) from kafka to s3 by converting into parquet format and update S3 location with daily partitions on Athena table. Any guidance is appreciated.

HTTP listener source

Hi As anyone implemented HTTP listener in flink source which acts has a Rest API to receive JSON payload via Post method and writes to Kafka or kinesis or any sink sources. Any guidance or sample snippet will be appreciated.

Re: Elasticsearch Http Connector

Thanks Philipp, I am also started looking at Jest client. Did you use it with Flink? is possible for you to share the project so that i can reuse it? On Tue, Oct 25, 2016 at 11:54 AM, Philipp Bussche wrote: > Hi there, > not me (which I guess is not what you wanted

Elasticsearch Http Connector

Friends Any one using new Elasticsearch RestClient( https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html) with Flink?

Upserts with Flink-elasticsearch

Is it possible to do Upsert with existing flink-elasticsearch connector today?

protobuf messages from Kafka to elasticsearch using flink

Friends, Can someone guide me or share an example on how to consume protobuf message from kafka and index into Elasticsearch using flink?

Re: flink kafka scala error

r of your job. > > Cheers, > Till > ​ > > On Wed, Jan 6, 2016 at 5:54 AM, Madhukar Thota <madhukar.th...@gmail.com> > wrote: > >> Hi >> >> I am seeing the following error when i am trying to run the jar in Flink >> Cluster. I am not sure what

flink kafka scala error

Hi I am seeing the following error when i am trying to run the jar in Flink Cluster. I am not sure what dependency is missing. /opt/DataHUB/flink-0.10.1/bin/flink run datahub-heka-1.0-SNAPSHOT.jar flink.properties java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class

Re: Flink-Elasticsearch connector support for elasticsearch 2.0

week. > > Best, > Max > > On Fri, Dec 4, 2015 at 4:16 PM, Madhukar Thota <madhukar.th...@gmail.com> > wrote: > > i have created working connector for Elasticsearch 2.0 based on > > elasticsearch-flink connector. I am using it right now but i want > offic

Running Flink in Cloudfoundry Environment

Hi Is it possible to run Flink in Cloudfoundry Environment? if yes, How can we achive this? Any help is appreciated. Thanks in Advance. Thanks, Madhu

Re: Kafka + confluent schema registry Avro parsing

gt; > FlinkKafkaConsumer consumer = new FlinkKafkaConsumer("myTopic"), decoder, > props, offsetStore, fetcherType); > > > Let me know if that works for you. > > Best regards, > Max > > On Thu, Nov 19, 2015 at 9:51 AM, Madhukar Thota <madhukar.th...@gmail.c

Re: Kafka + confluent schema registry Avro parsing

nagers in the cluster. > > > > I think you need to "lazily" initialize the decoder, in the first > invocation > > of "deserialize()". That should do it. > > > > Stephan > > > > > > On Thu, Nov 19, 2015 at 12:10 PM, Madhukar Thota &