Re: Does receiver based approach lose any data in case of a leader/broker loss in Spark Streaming?

2015-11-24 Thread Cody Koeninger
wrong with your kafka cluster and fix it, regardless of what consumer you're using. On Tue, Nov 24, 2015 at 10:55 PM, SRK wrote: > Hi, > > Does receiver based approach lose any data in case of a leader/broker loss > in Spark Streaming? We currently use Kafka Direct for Spark

Does receiver based approach lose any data in case of a leader/broker loss in Spark Streaming?

2015-11-24 Thread SRK
Hi, Does receiver based approach lose any data in case of a leader/broker loss in Spark Streaming? We currently use Kafka Direct for Spark Streaming and it seems to be failing out when there is a leader loss and we can't really guarantee that there won't be any leader loss due rebalan