You can't, since that's how it's designed to work. Batches are saved in different "files", which are really directories containing partitions, as is common in Hadoop. You can move them later, or just read them where they are.
On Thu, Apr 16, 2015 at 6:32 PM, Vadim Bichutskiy <vadim.bichuts...@gmail.com> wrote: > I am using Spark Streaming where during each micro-batch I output data to S3 > using > saveAsTextFile. Right now each batch of data is put into its own directory > containing > 2 objects, "_SUCCESS" and "part-00000." > > How do I output each batch into a common directory? > > Thanks, > Vadim > ᐧ --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
