llphxd commented on PR #55820: URL: https://github.com/apache/spark/pull/55820#issuecomment-4430357180
One possible question is why this new option is needed when `spark.sql.legacy.charVarcharAsString` already exists. I think the two options serve different purposes. `spark.sql.legacy.charVarcharAsString` disables Spark's CHAR/VARCHAR type semantics broadly by treating CHAR/VARCHAR as STRING. This restores older Spark behavior, but it also disables length checks and CHAR padding semantics, so it is a coarse-grained legacy compatibility switch. The proposed option is narrower. It only changes the read-side representation of CHAR values by trimming trailing spaces when explicitly enabled. It does not affect VARCHAR or STRING, and it does not disable write-side CHAR/VARCHAR length checks. This allows users to keep Spark's stricter CHAR/VARCHAR type handling while opting into MySQL-compatible CHAR retrieval behavior. This is useful for migration/upgrade scenarios where users want to preserve standard CHAR/VARCHAR validation in Spark, but need the returned CHAR values to match MySQL's default behavior or previous string-like query results more closely. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
