Kafka + confluent schema registry Avro parsing

Hi I am very new to Avro. Currently I am using confluent Kafka version and I am able to write an Avro message to Kafka by storing schema in schema registry. Now I need to consume those messages using Flink Kafka Consumer and having a hard time to deserialize the messages. I am looking for an exam

Re: Kafka + confluent schema registry Avro parsing

fkaConsumer consumer = new FlinkKafkaConsumer("myTopic"), decoder, > props, offsetStore, fetcherType); > > > Let me know if that works for you. > > Best regards, > Max > > On Thu, Nov 19, 2015 at 9:51 AM, Madhukar Thota > wrote: > >> Hi >> >

Re: Kafka + confluent schema registry Avro parsing

need to "lazily" initialize the decoder, in the first > invocation > > of "deserialize()". That should do it. > > > > Stephan > > > > > > On Thu, Nov 19, 2015 at 12:10 PM, Madhukar Thota < > madhukar.th...@gmail.com> > > wrote: > &

Running Flink in Cloudfoundry Environment

Hi Is it possible to run Flink in Cloudfoundry Environment? if yes, How can we achive this? Any help is appreciated. Thanks in Advance. Thanks, Madhu

Flink-Elasticsearch connector support for elasticsearch 2.0

is current elasticsearch-flink connector support elasticsearch 2.x version? -Madhu

Re: Flink-Elasticsearch connector support for elasticsearch 2.0

quot;, tags); test.put("measurement", measure); test.put("@timestamp", dateFormat.format(new Date(time))); return Requests.indexRequest() .index("metrics") .type("test") .source(new Gson().toJson(test).toLowerCas

Re: Flink-Elasticsearch connector support for elasticsearch 2.0

t, > Max > > On Fri, Dec 4, 2015 at 4:16 PM, Madhukar Thota > wrote: > > i have created working connector for Elasticsearch 2.0 based on > > elasticsearch-flink connector. I am using it right now but i want > official > > connector fr

flink kafka scala error

Hi I am seeing the following error when i am trying to run the jar in Flink Cluster. I am not sure what dependency is missing. /opt/DataHUB/flink-0.10.1/bin/flink run datahub-heka-1.0-SNAPSHOT.jar flink.properties java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class

Re: flink kafka scala error

ers, > Till > ​ > > On Wed, Jan 6, 2016 at 5:54 AM, Madhukar Thota > wrote: > >> Hi >> >> I am seeing the following error when i am trying to run the jar in Flink >> Cluster. I am not sure what dependency is missing. >> >> /op

Flink streaming Python

Hi Is streaming supported in Flink-Python API? If so, can you point me to the documentation?

protobuf messages from Kafka to elasticsearch using flink

Friends, Can someone guide me or share an example on how to consume protobuf message from kafka and index into Elasticsearch using flink?

Re: protobuf messages from Kafka to elasticsearch using flink

d-applications-with-apache-flink-elasticsearch-and-kibana > > Hope this helps, > Fabian > > 2016-03-09 2:52 GMT+01:00 Madhukar Thota : > >> Friends, >> >> Can someone guide me or share an example on how to consume protobuf >> message from kafka and index into Elasticsearch using flink? >> > >

Upserts with Flink-elasticsearch

Is it possible to do Upsert with existing flink-elasticsearch connector today?

Elasticsearch Http Connector

Friends Any one using new Elasticsearch RestClient( https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html) with Flink?

Re: Elasticsearch Http Connector

Thanks Philipp, I am also started looking at Jest client. Did you use it with Flink? is possible for you to share the project so that i can reuse it? On Tue, Oct 25, 2016 at 11:54 AM, Philipp Bussche wrote: > Hi there, > not me (which I guess is not what you wanted to hear) but I had to > impl

HTTP listener source

Hi As anyone implemented HTTP listener in flink source which acts has a Rest API to receive JSON payload via Post method and writes to Kafka or kinesis or any sink sources. Any guidance or sample snippet will be appreciated.

Amazon Athena

Anyone used used Amazon Athena with Apache Flink? I have use case where I want to write streaming data ( which is in Avro format) from kafka to s3 by converting into parquet format and update S3 location with daily partitions on Athena table. Any guidance is appreciated.

Flink streaming Python

Hi I have asked the same question back in Jan 2016 and checking again with community to see if there is any update or plan for supporting streaming Flink in python.

Aggregation based on Timestamp

Hi We have use case where we have thousands of Telegraf agents sending data to kafka( some of them are sending 10s interval, 15s interval and 30s interval). We would like to aggregate the incoming data to 1 minuter interval based on the hostname as key before we write into influxdb. Is it possible

ARIMA Model

is there a ARIMA(Autoregressive integrated moving average ) module available in Flink machine learning?