Please see if this works
-- aggregate array into map of element of count
SELECT aggregate(array(1,2,3,4,5),
map('cnt',0),
(acc,x) -> map('cnt', acc.cnt+1)) as array_count
thanks
Vijay
On 2023/05/05 19:32:04 Yong Zhang wrote:
> Hi, This is on Spark 3.1 environment.
>
> For some reason, I can
Hi, Mich:
Thanks for your reply, but maybe I didn't make my question clear.
I am looking for a solution to compute the count of each element in an array,
without "exploding" the array, and output a Map structure as a column.
For example, for an array as ('a', 'b', 'a'), I want to output a
Hi, Mich:
Thanks for your reply, but maybe I didn't make my question clear.
I am looking for a solution to compute the count of each element in an array,
without "exploding" the array, and output a Map structure as a column.
For example, for an array as ('a', 'b', 'a'), I want to output a
you can create DF from your SQL RS and work with that in Python the way you
want
## you don't need all these
import findspark
findspark.init()
from pyspark.sql import SparkSession
from pyspark import SparkContext
from pyspark.sql import SQLContext
from pyspark.sql.functions import udf, col,
