I have data that I pull in using a sql context and then I convert to an rdd.
The problem is that the type in the rdd is [Any, Iterable[Any]]
And I need to have the type RDD[Array[String]] -- convert the Iterable to an
Array.
Here’s more detail:
val zdata = sqlContext.read.parquet("s3://.. parquet").select('Pk, explode('Pg)
as "P").select($"Pk", $"P.A.n")
val r1data = zdata.rdd
val r2data = r1data.map(t => (t(0),t(1))).groupByKey()
and at this point r2data’s type is [Any, Iterable[Any]]
robert
________________________________
This message (including any attachments) contains confidential and/or
privileged information. It is intended for a specific individual and purpose
and is protected by law. If you are not the intended recipient, please notify
the sender immediately and delete this message. Any disclosure, copying, or
distribution of this message, or the taking of any action based on it, is
strictly prohibited.
