Exception while running a spark job

[Balachandar R.A.]

Hello users,

In one of my usecase, I create a RDD of a input file stored in HDFS that
uses custom input format. I use NewHadoopRDD for this purpose.

 var hRDD = new NewHadoopRDD(sc, classOf[RandomAccessInputFormat],

        classOf[IntWritable],

        classOf[BytesWritable],

        job.getConfiguration()

        )


Now, when I perform an operation on this rdd, i get the exception below


com.fasterxml.jackson.databind.JsonMappingException: Could not find creator
property with name 'id' (in class org.apache.spark.rdd.RDDOperationScope)
at [Source: {"id":"0","name":"collect"}; line: 1, column: 1]


I am using zeppelin 0.5.5 with spark 1.5.1 (hadoop 2.6)


The spark error output contains the below exception


java.io.IOException: Premature EOF from inputStream
at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:201)


The same code, when run in spark-shell , throw the above exception but I am
able to print the output (partial may be). I was expecting similar
behaviour in zeppelin too, that is, see the exception but collects the
output generated till this point.


Any clue to fix this will help


thanks and regards

Bala