[
https://issues.apache.org/jira/browse/SPARK-36950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634097#comment-17634097
]
Hyukjin Kwon commented on SPARK-36950:
--------------------------------------
[~bjornjorgensen] are you interested in submitting a PR?
> Normalize semi-structured data into a flat table.
> -------------------------------------------------
>
> Key: SPARK-36950
> URL: https://issues.apache.org/jira/browse/SPARK-36950
> Project: Spark
> Issue Type: Improvement
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: Bjørn Jørgensen
> Priority: Major
>
> Hi, in pandas there is this json_normalize that flat out nested data.
> https://github.com/pandas-dev/pandas/blob/v1.3.3/pandas/io/json/_normalize.py#L112-L353
>
> I have opened a request for this function at koalas. Now there are more
> people that will have some function over to pyspark.
> https://github.com/databricks/koalas/issues/2162
> This is also a function that geopandas are using. In the meantime I have
> found a gist that has code that flattens out the whole dataframe.
> https://gist.github.com/nmukerje/e65cde41be85470e4b8dfd9a2d6aed50
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
