Hi,

My spark streaming application is pulling data from Kafka.To prevent data
loss I have implemented WAL and enable checkpointing.On killing my
application and restarting it I am able to prevent data loss now but
however I am getting duplicate messages.

Is it because the application got killed before it was able checkpoint the
current processing state??
If yes how to tackle the duplicate messages?

Thanks,
Dinesh

Reply via email to