Thanks Cody for your help. Actually i found out it was a issue on our
network. After doing a ping from spark node to kafka node i found there
were dup packages. After rebooting the kafka node everything went back to
normal!
Thanks for your help!
Roman
El jue., 8 de octubre de 2015 17:13, Cody Koen
It sounds like you moved the job from one environment to another?
This may sound silly, but make sure (eg using lsof) the brokers the job is
connecting to are actually the ones you expect.
As far as the checkpoint goes, the log output should indicate whether the
job is restoring from checkpoint.
I'm running Spark Streaming using Kafka Direct stream, expecting
exactly-once semantics using checkpoints (which are stored onto HDFS).
My Job is really simple, it opens 6 Kafka streams (6 topics with 4 parts
each) and stores every row to ElasticSearch using ES-Spark integration.
This job was work
