As I wrote before, the result of my pipeline is binary objects, which I want to write directly as raw bytes, and not serializing them again.
Is it possible? On Sat, Jul 25, 2015 at 11:28 AM Akhil Das <ak...@sigmoidanalytics.com> wrote: > Its been added from spark 1.1.0 i guess > https://issues.apache.org/jira/browse/SPARK-1161 > > Thanks > Best Regards > > On Sat, Jul 25, 2015 at 12:06 AM, Oren Shpigel <o...@yowza3d.com> wrote: > >> Sorry, I didn't mention I'm using the Python API, which doesn't have the >> saveAsObjectFiles method. >> Is there any alternative from Python? >> And also, I want to write the raw bytes of my object into files on disk, >> and not using some Serialization format to be read back into Spark. >> >> Is it possible? >> Any alternatives for that? >> >> Thanks, >> Oren >> >> On Thu, Jul 23, 2015 at 8:04 PM Akhil Das <ak...@sigmoidanalytics.com> >> wrote: >> >>> You can look into .saveAsObjectFiles >>> >>> Thanks >>> Best Regards >>> >>> On Thu, Jul 23, 2015 at 8:44 PM, Oren Shpigel <o...@yowza3d.com> wrote: >>> >>>> Hi, >>>> I use Spark to read binary files using SparkContext.binaryFiles(), and >>>> then >>>> do some calculations, processing, and manipulations to get new objects >>>> (also >>>> binary). >>>> The next thing I want to do is write the results back to binary files on >>>> disk. >>>> >>>> Is there any equivalence like saveAsTextFile just for binary files? >>>> Is there any other way to save the results to be used outside Spark? >>>> >>>> Thanks, >>>> Oren >>>> >>>> >>>> >>>> -- >>>> View this message in context: >>>> http://apache-spark-user-list.1001560.n3.nabble.com/Writing-binary-files-in-Spark-tp23970.html >>>> Sent from the Apache Spark User List mailing list archive at Nabble.com. >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >>>> For additional commands, e-mail: user-h...@spark.apache.org >>>> >>>> >
