HyukjinKwon commented on code in PR #59188:
URL: https://github.com/apache/spark/pull/59188#discussion_r4157794530
##########
sql/core/src/test/scala/org/apache/spark/sql/CharVarcharTestSuite.scala:
##########
@@ -2607,37 +2605,12 @@ class BasicCharVarcharTestSuite extends
SharedSparkSession {
checkAnswer(
sql("""SELECT length(CAST('{"a":1}' AS CHAR(20)))"""),
Row(20))
- checkAnswer(
- sql("""SELECT schema_of_json(CAST('{"a":1}' AS CHAR(20)))"""),
Review Comment:
**Nit (P3):** The `schema_of_json` / `schema_of_xml` CHAR checks removed
here (and the `COLLATE SR_AI` trio below) don't depend on trimming. JSON and
XML ignore trailing whitespace, so `schema_of_json(CAST('{"a":1}' AS
CHAR(20)))` should still give `STRUCT<a: BIGINT>`. They are also the only
remaining coverage for the CHAR acceptance SPARK-59274 added to `SchemaOfJson`
and `SchemaOfXml`: each has its own `!child.dataType.isInstanceOf[StringType]`
check (`jsonExpressions.scala:2107`, `xmlExpressions.scala:212`), separate from
`SchemaOfCsv`'s. With them gone, the surviving `schema_of_csv` CHAR tests
wouldn't catch a regression of the JSON/XML checks back to `!= StringType`.
Could you keep the JSON/XML (and collated) cases? Update the collated
`schema_of_csv` one to the new untrimmed `STRUCT<_c0: INT, _c1: STRING, _c2:
STRING>` rather than dropping it. The `from_xml` CHAR(30) case would also be
cheap to keep.
**Verification:**
- **Regression:** Reverting SchemaOfJson or SchemaOfXml input checking to
exact StringType equality makes a restored CHAR-document assertion fail.
- **Behavior:** The restored and updated CHAR-document assertions pass under
standard semantics with the PR's no-trim behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
