Hi,
could not agree more with Molotch :)
Regards,
Gourav Sengupta
On Thu, May 27, 2021 at 7:08 PM Molotch wrote:
> You can specify the line separator to make spark split your records into
> separate rows.
>
> df = spark.read.option("lineSep","^^^").text("path")
>
> Then you need to df.select(
You can specify the line separator to make spark split your records into
separate rows.
df = spark.read.option("lineSep","^^^").text("path")
Then you need to df.select(split("value", "***").as("arrayColumn")) the
column into an array and map over it with getItem to create a column for
each proper
