Issues with partitionBy: FetchFailed

2014-09-21 Thread Julien Carme
Hello, I am facing an issue with partitionBy, it is not clear whether it is a problem with my code or with my spark setup. I am using Spark 1.1, standalone, and my other spark projects work fine. So I have to repartition a relatively large file (about 70 million lines). Here is a minimal version

RE: Issues with partitionBy: FetchFailed

2014-09-21 Thread Shao, Saisai
, you can try setting a large value of this configuration. Thanks Jerry From: Julien Carme [mailto:julien.ca...@gmail.com] Sent: Sunday, September 21, 2014 7:43 PM To: user@spark.apache.org Subject: Issues with partitionBy: FetchFailed Hello, I am facing an issue with partitionBy

Re: Issues with partitionBy: FetchFailed

2014-09-21 Thread David Rowe
*Subject:* Issues with partitionBy: FetchFailed Hello, I am facing an issue with partitionBy, it is not clear whether it is a problem with my code or with my spark setup. I am using Spark 1.1, standalone, and my other spark projects work fine. So I have to repartition a relatively large