Hello, I wonder if there is a way (preferably efficient) in Spark to reshape hive table and save it to parquet.
Here is a minimal example, input hive table: col1 col2 col3 1 2 3 4 5 6 output parquet: col1 newcol2 1 [2 3] 4 [5 6] p.s. The real input hive table has ~1000 columns. Thank you, Anton