Hi Is it possible to write custom RDD in java?
Requirement is - I am having a list of Sqlserver tables need to be dumped in HDFS. So I have a List<String> tables = {dbname.tablename,dbname.tablename2......}; then JavaRDD<String> rdd = javasparkcontext.parllelise(tables); JavaRDDString> tablecontent = rdd.map(new Function<String,Iterable<String>>){fetch table and return populate iterable} tablecontent.storeAsTextFile("hffs path"); In rdd.map(new Function<String,>). I cannot keep complete table content in memory , so I want to creat my own RDD to handle it. Thanks Shushant