Re: Complex transformation on a dataframe column

2015-10-17 Thread Raghavendra Pandey
f { (s: String) => s.split(":").mkString("-")} val outputDF = df.withColumn("FormattedAddress", formatAddress(df("Address"))) -Raghav On Thu, Oct 15, 2015 at 10:34 PM, Hao Wang wrote: > Hi, > > I have searched around but could not find a sa

Complex transformation on a dataframe column

2015-10-15 Thread Hao Wang
Hi, I have searched around but could not find a satisfying answer to this question: what is the best way to do a complex transformation on a dataframe column? For example, I have a dataframe with the following schema and a function that has pretty complex logic to format addresses. I would