Re: Writing file

2018-08-01 Thread mattbuttow
Thank you cloud0fan. That's really helpful. -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Writing file

2018-07-31 Thread Wenchen Fan
It depends on how you deploy Spark. The writer just writes data to your specified path(HDFS or local path), but the writer is run on executors. If you deploy Spark with the local mode, i.e. executor and driver are together, then you will see the output file on the driver node. If you deploy Spark