You can specify the line separator to make spark split your records into
separate rows.
df = spark.read.option("lineSep","^^^").text("path")
Then you need to df.select(split("value", "***").as("arrayColumn")) the
column into an array and map over it with getItem to create a column for
each property.
df.select((0 until 8).map(i => $"arrayColumn".getItem(i).as(s"col$i")): _* )
Then you should have a DataFrame with each record on a row and each property
in a column.
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
---------------------------------------------------------------------
To unsubscribe e-mail: [email protected]
