Reg. custom sinks in Flink

2016-12-09 Thread Meghashyam Sandeep V
Hi there, I have a flink streaming app where my source is Kafka and a custom sink to Cassandra(I can't use standard C* sink that comes with flink as I have customized auth to C*). I'm currently have the following: messageStream .rebalance() .map( s-> { return mapper.readValu

Re: Reg. custom sinks in Flink

2016-12-09 Thread Meghashyam Sandeep V
n > the position in or content in the stream. It's a necessary helper object, > yes, but you don't need Flink to checkpoint it. > > You can still use the sinks provided with flink-connector-cassandra and > customize the cluster building by passing your own ClusterBuilder into th

Re: Reg. custom sinks in Flink

2016-12-12 Thread Meghashyam Sandeep V
ndra sink again. > > Cheers, > Till > ​ > > On Fri, Dec 9, 2016 at 8:05 PM, Meghashyam Sandeep V < > vr1meghash...@gmail.com> wrote: > >> Thanks a lot for the quick reply Shannon. >> >> 1. I will create a class that extends SinkFunction and write my >&

Support for Auto scaling

2017-02-01 Thread Meghashyam Sandeep V
Hi Guys, I currently run flink 1.1.4 streaming jobs in EMR in AWS with yarn. I understand that EMR supports auto scaling but Flink doesn't. Is there a plan for this support in 1.2. Thanks, Sandeep