Have a look at MultipleOutputs in the hadoop API. Spark can read and write to
arbitrary hadoop formats.
> On Apr 4, 2014, at 6:01 AM, dmpour23 <dmpou...@gmail.com> wrote:
>
> Hi all,
> Say I have an input file which I would like to partition using
> HashPartitioner k times.
>
> Calling rdd.saveAsTextFile(""hdfs://"); will save k files as part-00000
> part-0000k
> Is there a way to save each partition in specific folders?
>
> i.e. src
> part0/part-00000
> part1/part-00001
> part1/part-0000k
>
> thanks
> Dimitri
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/how-to-save-RDD-partitions-in-different-folders-tp3754.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
