I don't think using Void class is the right choice - it is not even a Writable.
BTW in the future, capture text output instead of image. Thanks On Fri, Jul 31, 2015 at 12:35 PM, Umesh Kacha <umesh.ka...@gmail.com> wrote: > Hi Ted thanks My key is always Void because my custom format file is non > splittable so key is Void and values is MyRecordWritable which extends > Hadoop Writable. I am sharing my log as snap please dont mind as I cant > paste code outside. > > Regards, > Umesh > > On Sat, Aug 1, 2015 at 12:59 AM, Ted Yu <yuzhih...@gmail.com> wrote: > >> Looking closer at the code you posted, the error likely was caused by the >> 3rd parameter: Void.class >> >> It is supposed to be the class of key. >> >> FYI >> >> On Fri, Jul 31, 2015 at 11:24 AM, unk1102 <umesh.ka...@gmail.com> wrote: >> >>> Hi I am having my own Hadoop custom InputFormat which I need to use in >>> creating DataFrame. I tried to do the following >>> >>> JavaPairRDD<Void,MyRecordWritable> myFormatAsPairRdd = >>> >>> jsc.hadoopFile("hdfs://tmp/data/myformat.xyz",MyInputFormat.class,Void.class,MyRecordWritable.class); >>> JavaRDD<MyRecordWritable> myformatRdd = myFormatAsPairRdd.values(); >>> DataFrame myFormatAsDataframe = >>> sqlContext.createDataFrame(myformatRdd,MyFormatSchema.class); >>> myFormatAsDataframe.show(); >>> >>> Above code does not work and throws exception saying >>> java.lang.IllegalArgumentException object is not an instance of declaring >>> class >>> >>> My custom Hadoop InputFormat works very well with Hive,MapReduce etc How >>> do >>> I make it work with Spark please guide I am new to Spark. Thank in >>> advance. >>> >>> >>> >>> >>> -- >>> View this message in context: >>> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-create-Spark-DataFrame-using-custom-Hadoop-InputFormat-tp24101.html >>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>> For additional commands, e-mail: user-h...@spark.apache.org >>> >>> >> >