dejankrak-db commented on PR #59305: URL: https://github.com/apache/spark/pull/59305#issuecomment-6272885025
Thanks @uros-b. 1. Fixed in 2a392898d26: `ExprUtils.evalTypeExpr` now rejects FILE with `UNSUPPORTED_FILE_TYPE` when the type is disabled, so `from_json`/`from_xml`/`from_csv`, the variant overload and `WrapUDT` cannot take FILE through a JSON schema string. `DataType.fromJson` itself stays ungated. I check only FILE there rather than calling `TypeUtils.failUnsupportedDataType`, since that would also start rejecting TIME, nanosecond timestamps and geospatial types on this path, which is outside this PR. 2. Yes, this is intended and recorded in the description under the user-facing change. Parquet support for FILE is a later part of the SPIP, and the type stays disabled by default until it lands. 3. Agreed. Conversions between the modes belong to the casts part of the SPIP, which will be its own sub-task of SPARK-59132. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
