Thanks Aniket! It is working now.
Anny
On Mon, Jan 19, 2015 at 5:56 PM, Aniket Bhatnagar
aniket.bhatna...@gmail.com wrote:
When you repartiton, ordering can get lost. You would need to sort after
repartitioning.
Aniket
On Tue, Jan 20, 2015, 7:08 AM anny9699 anny9...@gmail.com wrote:
Hi,
I am using Spark on AWS and want to write the output to S3. It is a
relatively small file and I don't want them to output as multiple parts. So
I use
result.repartition(1).saveAsTextFile(s3://...)
However as long as I am using the saveAsTextFile method, the output doesn't
keep the original
When you repartiton, ordering can get lost. You would need to sort after
repartitioning.
Aniket
On Tue, Jan 20, 2015, 7:08 AM anny9699 anny9...@gmail.com wrote:
Hi,
I am using Spark on AWS and want to write the output to S3. It is a
relatively small file and I don't want them to output as
