kunalgoyal98 commented on PR #38285:
URL: https://github.com/apache/spark/pull/38285#issuecomment-1617642625
@HyukjinKwon
For our application, we save the spark data types in its json format. We
parse this json back to create the spark data types. In scala spark, we use
this code to parse the json -
```
import org.apache.spark.sql.types.DataType
val schema =
"""
|{
| "type": "struct",
| "fields": [
| {
| "name": "c1",
| "type": "string",
| "nullable": true
| }
| ]
|}
|""".stripMargin
val dt = DataType.fromJson(schema)
```
We want to do parse the json in python too. The only way I could find is
this - `StructType.fromJson`. However, this call fails if there is no metadata
field in the json unlike the above scala `DataType.fromJson` call.
This is the problem I wanted to solve and this PR solves it. I am open to
any other suggestions too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
