Re: saving rdd to multiple files named by the key

[Nicholas Chammas]

There is also SPARK-3533 ,
which proposes to add a convenience method for this.
​

On Mon Jan 26 2015 at 10:38:56 PM Aniket Bhatnagar <
aniket.bhatna...@gmail.com> wrote:

> This might be helpful:
> http://stackoverflow.com/questions/23995040/write-to-multiple-outputs-by-key-spark-one-spark-job
>
> On Tue Jan 27 2015 at 07:45:18 Sharon Rapoport  wrote:
>
>> Hi,
>>
>> I have an rdd of [k,v] pairs. I want to save each [v] to a file named [k].
>> I got them by combining many [k,v] by [k]. I could then save to file by
>> partitions, but that still doesn't allow me to choose the name, and leaves
>> me stuck with foo/part-...
>>
>> Any tips?
>>
>> Thanks,
>> Sharon
>>
>

Re: saving rdd to multiple files named by the key

[Aniket Bhatnagar]

This might be helpful:
http://stackoverflow.com/questions/23995040/write-to-multiple-outputs-by-key-spark-one-spark-job

On Tue Jan 27 2015 at 07:45:18 Sharon Rapoport  wrote:

> Hi,
>
> I have an rdd of [k,v] pairs. I want to save each [v] to a file named [k].
> I got them by combining many [k,v] by [k]. I could then save to file by
> partitions, but that still doesn't allow me to choose the name, and leaves
> me stuck with foo/part-...
>
> Any tips?
>
> Thanks,
> Sharon
>

saving rdd to multiple files named by the key

[Sharon Rapoport]

Hi,

I have an rdd of [k,v] pairs. I want to save each [v] to a file named [k].
I got them by combining many [k,v] by [k]. I could then save to file by
partitions, but that still doesn't allow me to choose the name, and leaves
me stuck with foo/part-...

Any tips?

Thanks,
Sharon