Re: [Spark SQL 1.3.1] data frame saveAsTable returns exception

Hi, is this fixed in master? Grega On Thu, May 14, 2015 at 7:50 PM, Michael Armbrust mich...@databricks.com wrote: End of the month is the target: https://cwiki.apache.org/confluence/display/SPARK/Wiki+Homepage On Thu, May 14, 2015 at 3:45 AM, Ishwardeep Singh

Shuffle size difference - operations on RDD vs. operations on SchemaRDD

Hi, I am seeing different shuffle write sizes when using SchemaRDD (versus normal RDD). I'm doing the following: case class DomainObj(a: String, b: String, c: String, d: String) val logs: RDD[String] = sc.textFile(...) val filtered: RDD[String] = logs.filter(...) val myDomainObjects:

java.io.IOException: Failed to save output of task

$TaskRunner.run(Executor.scala:178) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724) Grega -- [image: Inline image 1] *Grega

Re: java.io.IOException: Failed to save output of task

I have since resolved the issue. The problem was that multiple rdds were trying to write to the same s3 bucket. Grega -- [image: Inline image 1] *Grega Kešpret* Analytics engineer Celtra — Rich Media Mobile Advertising celtra.com http://www.celtra.com/ | @celtramobilehttp://www.twitter.com