I have a pretty complex nested structure with several levels. So in order to
create it I use SQLContext.createDataFrame method and provide specific Rows
with specific StrucTypes, both of which I build myself.
To build a Row I iterate over my values and literally build a Row.
List row = n
You can use struct function of org.apache.spark.sql.function class to
combine two columns to create struct column.
Sth like.
val nestedCol = struct(df("d"), df("e"))
df.select(df(a), df(b), df(c), nestedCol)
On Aug 7, 2015 3:14 PM, "Rishabh Bhardwaj" wrote:
> I am doing it by creating a new data
I am doing it by creating a new data frame out of the fields to be nested
and then join with the original DF.
Looking for some optimized solution here.
On Fri, Aug 7, 2015 at 2:06 PM, Rishabh Bhardwaj wrote:
> Hi all,
>
> I want to have some nesting structure from the existing columns of
> the d
Hi all,
I want to have some nesting structure from the existing columns of
the dataframe.
For that,,I am trying to transform a DF in the following way,but couldn't
do it.
scala> df.printSchema
root
|-- a: string (nullable = true)
|-- b: string (nullable = true)
|-- c: string (nullable = true)
