Re: issue with spark and bson input

2014-08-06 Thread Dmitriy Selivanov
Finally I made it work. The trick was in "asSubclass" method: val mongoRDD = sc.newAPIHadoopFile("file:///root/jobs/dump/input.bson", classOf[BSONFileInputFormat].asSubclass(classOf[org.apache.hadoop.mapreduce.lib.input.FileInputFormat[Object, BSONObject]]), classOf[Object], classOf[BSONObject], co

issue with spark and bson input

2014-08-05 Thread Dmitriy Selivanov
Hello, I have issue when try to use bson file as spark input. I use mongo-hadoop-connector 1.3.0 and spark 1.0.0: val sparkConf = new SparkConf() val sc = new SparkContext(sparkConf) val config = new Configuration() config.set("mongo.job.input.format", "com.mongodb.hadoop.BSONFileIn