Depending on your use case, 'df.withColumn("my_existing_or_new_col",
lit(0l))' could work?On Fri, Nov 18, 2016 at 11:18 AM, Kristoffer Sjögren <sto...@gmail.com> wrote: > Thanks for your answer. I have been searching the API for doing that > but I could not find how to do it? > > Could you give me a code snippet? > > On Fri, Nov 18, 2016 at 8:03 PM, Mendelson, Assaf > <assaf.mendel...@rsa.com> wrote: > > You can always add the columns to old dataframes giving them null (or > some literal) as a preprocessing. > > > > -----Original Message----- > > From: Kristoffer Sjögren [mailto:sto...@gmail.com] > > Sent: Friday, November 18, 2016 4:32 PM > > To: user > > Subject: DataFrame select non-existing column > > > > Hi > > > > We have evolved a DataFrame by adding a few columns but cannot write > select statements on these columns for older data that doesn't have them > since they fail with a AnalysisException with message "No such struct > field". > > > > We also tried dropping columns but this doesn't work for nested columns. > > > > Any non-hacky ways to get around this? > > > > Cheers, > > -Kristoffer > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
