Re: Casting nested columns and updated nested struct fields.

2018-11-23 Thread Colin Williams
Looks like it's been reported already. It's too bad it's been a year but should be released into spark 3: https://issues.apache.org/jira/browse/SPARK-22231 On Fri, Nov 23, 2018 at 8:42 AM Colin Williams wrote: > > Seems like it's worthy of filing a bug against withColumn > > On Wed, Nov 21, 2018,

Re: Casting nested columns and updated nested struct fields.

2018-11-23 Thread Colin Williams
Seems like it's worthy of filing a bug against withColumn On Wed, Nov 21, 2018, 6:25 PM Colin Williams < colin.williams.seat...@gmail.com wrote: > Hello, > > I'm currently trying to update the schema for a dataframe with nested > columns. I would either like to update the schema itself or cast

Casting nested columns and updated nested struct fields.

2018-11-21 Thread Colin Williams
Hello, I'm currently trying to update the schema for a dataframe with nested columns. I would either like to update the schema itself or cast the column without having to explicitly select all the columns just to cast one. In regards to updating the schema it looks like I would probably need to