Hi,
Interesting, but I personally would opt for withColumn since it'd be
less to type (and also be consistent with ticks (')) as follows:
df.withColumn(explode('myArray) as 'arrayItem)
(Spark SQL made my SQL developer's life so easy these days :))
Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark 2.0 https://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski
On Tue, Nov 22, 2016 at 11:58 PM, Michael Armbrust
<mich...@databricks.com> wrote:
> Both collect_list and explode are available in the function library.
>
> The following is an example of using it:
> df.select($"*", explode($"myArray") as 'arrayItem)
>
> On Tue, Nov 22, 2016 at 2:42 PM, Mahender Sarangam
> <mahender.bigd...@outlook.com> wrote:
>>
>> Hi,
>>
>> We are converting our hive logic which is using lateral view and explode
>> functions. Is there any builtin function in scala for performing lateral
>> view explore.
>>
>>
>> Below is our query in Hive. temparray is temp table with c0 and c1
>> columns
>>
>> SELECT id, CONCAT_WS(',', collect_list(LineID)) as LineiD
>> FROM (SELECT cast(LineID as STRING) as LineiD, cast(id as STRING) as id
>> FROM temparray LATERAL VIEW explode(`_c1`) adTable AS id) T
>> GROUP BY id;
>>
>>
>> Can any one provide pointer for string functions available in scala. We
>> would like perform operations like Collect_List, get starting index of
>> matching string.
>>
>>
>>
>> Nanu
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscr...@spark.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
