HyukjinKwon commented on PR #59316:
URL: https://github.com/apache/spark/pull/59316#issuecomment-6104075571
Thanks, I checked 8e1c55b. Struct-field maps go through `convertField`
again, `convertComplicatedType` only sends `MapType(_: StringType, _, _)` to
`convertMap` (the new `format("xml").load()` test keeps `MAP<INT, STRING>` a
malformed record), the prefixed-attribute and `valueTag` tests use mixed
content, and the `convertMap` scaladoc states the flag condition. One
correction is on me: my earlier description of the 4.3 baseline was too narrow.
In branch-4.3, `from_xml` and `DataFrameReader` schemas never reached the XML
parser with CHAR/VARCHAR types, because `failIfHasCharVarchar` rejected them
with `UNSUPPORTED_CHAR_OR_VARCHAR_AS_STRING`, or turned them into STRING under
`spark.sql.legacy.charVarcharAsString`. So the new 'were malformed records'
sentence in the migration bullet should say that instead. While there, the
flag-off sentence could also mention that an over-length CHAR key raises
`EXCEED_LIMIT_LENGTH`.
<!-- SPARK_DEV_REVIEW_REPLY
{"feedback_id":"conversation:6100927504","thread_id":"conversation:6100927504","verdict_sha256":"45db306fc5581d80f68906ea6735835c7c310f24af9c6c0988939c02b1cbfd32"}
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
