Hi, Thanks Ted.
It means that it's not only possible to rename a column using withColumnRenamed, but also replace the content of a column (in one shot) using withColumn with an existing column name. I can live with that :) Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Thu, Mar 31, 2016 at 5:14 PM, Ted Yu <[email protected]> wrote: > Looks like this is result of the following check: > > val shouldReplace = output.exists(f => resolver(f.name, colName)) > if (shouldReplace) { > > where existing column, text, was replaced. > > On Thu, Mar 31, 2016 at 12:08 PM, Jacek Laskowski <[email protected]> wrote: >> >> Hi, >> >> Just ran into the following. Is this a bug? >> >> scala> df.join(nums, df("id") === nums("id")).withColumn("TEXT2", >> lit(5)).show >> +---+-------+---+-----+-----+ >> | id| text| id| text|TEXT2| >> +---+-------+---+-----+-----+ >> | 0| hello| 0| two| 5| >> | 1|swiecie| 1|three| 5| >> +---+-------+---+-----+-----+ >> >> >> scala> df.join(nums, df("id") === nums("id")).withColumn("TEXT", >> lit(5)).show >> +---+----+---+----+ >> | id|TEXT| id|TEXT| >> +---+----+---+----+ >> | 0| 5| 0| 5| >> | 1| 5| 1| 5| >> +---+----+---+----+ >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> https://medium.com/@jaceklaskowski/ >> Mastering Apache Spark http://bit.ly/mastering-apache-spark >> Follow me at https://twitter.com/jaceklaskowski >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
