Just after receiving the data from kafka, you can do a
dstream.count().print() to see spark and kafka is not the problem, after
that next step would be to identify where is the problem, you can do the
same count and print on each of the dstreams that you are creating (by
transforming), and finally, the indexing into elastic search peace, where
you need to make sure the connection is not a problem. May be you can
create a simple RDD of Map("id" -> "1", "Something" -> "Something") and
call the .saveToES to make sure it is getting written into ES.Thanks Best Regards On Thu, May 28, 2015 at 7:03 PM, boci <boci.b...@gmail.com> wrote: > Hi guys, > > I using spark streaming with kafka... In local machine (start as java > application without using spark-submit) it's work, connect to kafka and do > the job (*). I tried to put into spark docker container (hadoop 2.6, spark > 1.3.1, try spark submit wil local[5] and yarn-client too ) but I'm out of > success... > > No error on the console (the application started), I see something > received from kafka but the result is not written out to elasticsearch... > > Where can I start the debug? I see in the spark console two job, both > 0/1... > > Thanks > > ---------------------------------------------------------------------------------------------------------------------------------- > Skype: boci13, Hangout: boci.b...@gmail.com >
