Re: Renaming nested columns in dataframe

2016-05-16 Thread Alexander Chermenin
Hello. I think you can use something like this:  df.select(    struct(        column("site.site_id").as("id"),        column("site.site_name").as("name"),        column("site.site_domain").as("domain"),        column("site.site_cat").as("cat"),        struct(            column("site.publisher.site_pub_id").as("id"),            column("site.publisher.site_pub_name").as("name")        ).as("publisher")    ).as("site")) Best regards, Alexander Chermenin.Web: http://chermenin.ruMail: a...@chermenin.ru   16.05.2016, 14:58, "Prashant Bhardwaj" :Hi How can I rename nested columns in dataframe through scala API? Like following schema|-- site: struct (nullable = false)|    |-- site_id: string (nullable = true)|    |-- site_name: string (nullable = true)|    |-- site_domain: string (nullable = true)|    |-- site_cat: array (nullable = true)|    |    |-- element: string (containsNull = false)|    |-- publisher: struct (nullable = false)|    |    |-- site_pub_id: string (nullable = true)|    |    |-- site_pub_name: string (nullable = true) I want to change it to  |-- site: struct (nullable = false)|    |-- id: string (nullable = true)|    |-- name: string (nullable = true)|    |-- domain: string (nullable = true)|    |-- cat: array (nullable = true)|    |    |-- element: string (containsNull = false)|    |-- publisher: struct (nullable = false)|    |    |-- id: string (nullable = true)|    |    |-- name: string (nullable = true)  RegardsPrashant

Renaming nested columns in dataframe

2016-05-16 Thread Prashant Bhardwaj
Hi

How can I rename nested columns in dataframe through scala API? Like
following schema

> |-- site: struct (nullable = false)
>
> ||-- site_id: string (nullable = true)
>
> ||-- site_name: string (nullable = true)
>
> ||-- site_domain: string (nullable = true)
>
> ||-- site_cat: array (nullable = true)
>
> |||-- element: string (containsNull = false)
>
> ||-- publisher: struct (nullable = false)
>
> |||-- site_pub_id: string (nullable = true)
>
> |||-- site_pub_name: string (nullable = true)
>

I want to change it to

> |-- site: struct (nullable = false)
>
> ||-- id: string (nullable = true)
>
> ||-- name: string (nullable = true)
>
> ||-- domain: string (nullable = true)
>
> ||-- cat: array (nullable = true)
>
> |||-- element: string (containsNull = false)
>
> ||-- publisher: struct (nullable = false)
>
> |||-- id: string (nullable = true)
>
> |||-- name: string (nullable = true)
>
>
Regards
Prashant