Re: can flink do streaming from data sources other than Kafka?

2017-09-07 Thread Elias Levy
If you want to ensure you see all changes to a Cassandra table, you need to make use of the Change Data Capture feature. For that, you'll need code running on the Cassandra nodes to read the commit log segments from the Cassandra CDC

Re: can flink do streaming from data sources other than Kafka?

2017-09-07 Thread kant kodali
Yes I can indeed create them but I wonder if that is even possible? I haven't see any framework doing this as of today. Flink has something called AsyncDataStream? and I wonder if this can be leveraged to create a Stream out of Cassandra source? Thanks! On Thu, Sep 7, 2017 at 1:16 AM, Tzu-Li

Re: can flink do streaming from data sources other than Kafka?

2017-09-07 Thread Tzu-Li (Gordon) Tai
Ah, I see. I’m not aware of any existing work / JIRAs on streaming sources for Cassandra or HBase, only sinks. If you are interested in one, could you open JIRAs for them? On 7 September 2017 at 4:11:05 PM, kant kodali (kanth...@gmail.com) wrote: Hi Gordon, Thanks for the response, I did go

Re: can flink do streaming from data sources other than Kafka?

2017-09-07 Thread kant kodali
Hi Gordon, Thanks for the response, I did go over the links for sources and sinks prior to posting my question. Maybe, I didn't get my question across correctly so let me rephrase it. Can I get data out of data stores like Cassandra, Hbase in a streaming manner? coz, currently more or less all

Re: can flink do streaming from data sources other than Kafka?

2017-09-07 Thread Tzu-Li (Gordon) Tai
Hi! I am wondering if Flink can do streaming from data sources other than Kafka. For example can Flink do streaming from a database like Cassandra, HBase, MongoDb to sinks like says Elastic search or Kafka. Yes, Flink currently supports various connectors for different sources and sinks. For

can flink do streaming from data sources other than Kafka?

2017-09-07 Thread kant kodali
Hi All, I am wondering if Flink can do streaming from data sources other than Kafka. For example can Flink do streaming from a database like Cassandra, HBase, MongoDb to sinks like says Elastic search or Kafka. Also for out of core stateful streaming. Is RocksDB the only option? Can I use some