Re: Serialize a DataFrame with Vector values into text/csv file

Try to_json on the vector column. That should do it. -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Re: Serialize a DataFrame with Vector values into text/csv file

Hi, I was hoping that there is a casting vector into String method (instead of writing my UDF), so that it can then be serialized it into csv/text file. Best regards, Mina On Tue, Feb 20, 2018 at 6:52 PM, vermanurag wrote: > If your dataframe has columns types

Re: Serialize a DataFrame with Vector values into text/csv file

Hi Snehasish, Unfortunately, none of the solutions worked. Regards, Mina On Tue, Feb 20, 2018 at 5:12 PM, SNEHASISH DUTTA wrote: > Hi Mina, > > Even text won't work you may try this df.coalesce(1).write.option("h >

Re: Serialize a DataFrame with Vector values into text/csv file

If your dataframe has columns types like vector then you cannot save as csv/ text as there are no direct equivalent supported by flat formats like csv/ text. You may need to convert the column type appropriately (eg. convert the incompatible column to StringType before saving the output as csv.

Re: Serialize a DataFrame with Vector values into text/csv file

Hi Mina, Even text won't work you may try this df.coalesce(1).write.option("h eader","true").mode("overwrite").save("output",format=text) Else convert to an rdd and use saveAsTextFile Regards, Snehasish On Wed, Feb 21, 2018 at 3:38 AM, SNEHASISH DUTTA wrote: > Hi

Re: Serialize a DataFrame with Vector values into text/csv file

Hi Mina, This might work then df.coalesce(1).write.option("header","true").mode("overwrite ").text("output") Regards, Snehasish On Wed, Feb 21, 2018 at 3:21 AM, Mina Aslani wrote: > Hi Snehasish, > > Using df.coalesce(1).write.option("header","true").mode("overwrite >

Re: Serialize a DataFrame with Vector values into text/csv file

Hi Snehasish, Using df.coalesce(1).write.option("header","true").mode("overwrite ").csv("output") throws java.lang.UnsupportedOperationException: CSV data source does not support struct<...> data type. Regards, Mina On Tue, Feb 20, 2018 at 4:36 PM, SNEHASISH DUTTA

Re: Serialize a DataFrame with Vector values into text/csv file

Hi Mina, This might help df.coalesce(1).write.option("header","true").mode("overwrite").csv("output") Regards, Snehasish On Wed, Feb 21, 2018 at 1:53 AM, Mina Aslani wrote: > Hi, > > I would like to serialize a dataframe with vector values into a text/csv > in pyspark. >