We have RDD.saveAsTextFile and RDD.saveAsObjectFile for saving the output
to any location specified. The params to be provided are:
>path of storage location
>no. of partitions
For giving an hdfs path we use the following format:
"/user///"
On Tue, Nov 11, 2014 at 6:28 PM, Naveen Kumar Pokala <
n
One approach would be to use SaveAsNewAPIHadoop file and specify
jsonOutputFormat.
Another simple one would be like:
val rdd = sc.parallelize(1 to 100)
val json = rdd.map(x => {
val m: Map[String, Int] = Map("id" -> x)
new JSONObject(m) })
json.saveAsTextFile("output")
Thanks
Best
Hi,
I am spark 1.1.0. I need a help regarding saving rdd in a JSON file?
How to do that? And how to mentions hdfs path in the program.
-Naveen
