Bjørn Jørgensen created SPARK-36950:
---------------------------------------
Summary: Normalize semi-structured data into a flat table.
Key: SPARK-36950
URL: https://issues.apache.org/jira/browse/SPARK-36950
Project: Spark
Issue Type: Wish
Components: PySpark
Affects Versions: 3.3.0
Reporter: Bjørn Jørgensen
Hi, in pandas there is this json_normalize that flat out nested data.
https://github.com/pandas-dev/pandas/blob/v1.3.3/pandas/io/json/_normalize.py#L112-L353
I have opened a request for this function at koalas. Now there are more people
that will have some function over to pyspark.
https://github.com/databricks/koalas/issues/2162
This is also a function that geopandas are using. In the meantime I have found
a gist that has code that flattens out the whole dataframe.
https://gist.github.com/nmukerje/e65cde41be85470e4b8dfd9a2d6aed50
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
