Kafka is designed to only allow reads from leaders. You need to fix this at the kafka level not the spark level.
On Fri, Jan 6, 2017 at 7:33 AM, Raghu Vadapalli <raghuvadapa...@aol.com> wrote: > > My spark 2.0 + kafka 0.8 streaming job fails with error partition leaderset > exception. When I check the kafka topic the partition, it is indeed in error > with Leader = -1 and empty ISR. I did lot of google and all of them point > to either restarting or deleting the topic. To do any of those two in > production system while other topics are in heavy use is next to impossible. > Now my question, is there way to force spark to read from leaderless > partition accepting some dataloss or inconsistency ? Or force the immediate > sync followed by election ( a kafka users group question but I am pushing my > luck:) here ) > > Topic: vzecsapplog Partition: 8 Leader: -1 Replicas: 5,4 Isr: > > -- > Regards, > Raghu > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
