srielau commented on code in PR #59516:
URL: https://github.com/apache/spark/pull/59516#discussion_r4316127472
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -2758,10 +2750,22 @@ private[sql] object QueryExecutionErrors extends
QueryErrorsBase with ExecutionE
new SparkRuntimeException(
errorClass = "UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY",
messageParameters = Map(
- "key" -> toSQLValue(key, StringType),
+ "key" -> toSQLCharVarcharMapKey(key),
"dataType" -> toSQLType(dataType)))
}
+ // A JSON object name or XML name used as a map key is often user data (ids,
emails) and can
+ // be very long (Jackson allows names of up to 50,000 characters), so cap
what we echo.
+ private def toSQLCharVarcharMapKey(key: UTF8String): String = {
Review Comment:
Master already has `CharVarcharUtils.truncateMapKeyForError` (SPARK-60242 /
Avro). Please rebase onto that and call it from JSON + XML instead of adding
`toSQLCharVarcharMapKey`. Avro already does
`toSQLValue(CharVarcharUtils.truncateMapKeyForError(key), StringType)` -- match
that and drop the private helper so we do not keep two 128-char echo caps.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -2758,10 +2750,22 @@ private[sql] object QueryExecutionErrors extends
QueryErrorsBase with ExecutionE
new SparkRuntimeException(
errorClass = "UNSUPPORTED_XML_CHAR_VARCHAR_MAP_KEY",
messageParameters = Map(
- "key" -> toSQLValue(key, StringType),
+ "key" -> toSQLCharVarcharMapKey(key),
"dataType" -> toSQLType(dataType)))
}
+ // A JSON object name or XML name used as a map key is often user data (ids,
emails) and can
+ // be very long (Jackson allows names of up to 50,000 characters), so cap
what we echo.
Review Comment:
Please do not cite Jackson's name limit on a path shared with XML (and
Avro). `CharVarcharUtils.truncateMapKeyForError`'s doc is already
format-agnostic -- lean on that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
