yaooqinn commented on a change in pull request #31253:
URL: https://github.com/apache/spark/pull/31253#discussion_r560975231
##########
File path:
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/util/CharVarcharCodegenUtils.java
##########
@@ -47,19 +47,23 @@ public static UTF8String charTypeReadSideCheck(UTF8String
inputStr, int limit) {
}
public static UTF8String varcharTypeWriteSideCheck(UTF8String inputStr, int
limit) {
- if (inputStr != null && inputStr.numChars() <= limit) {
- return inputStr;
- } else if (inputStr != null) {
- // Trailing spaces do not count in the length check. We need to retain
the trailing spaces
- // (truncate to length N), as there is no read-time padding for varchar
type.
- // TODO: create a special TrimRight function that can trim to a certain
length.
- UTF8String trimmed = inputStr.trimRight();
- if (trimmed.numChars() > limit) {
- throw new RuntimeException("Exceeds varchar type length limitation: "
+ limit);
- }
- return inputStr.substring(0, limit);
- } else {
+ if (inputStr == null) {
return null;
+ } else {
+ int numChars = inputStr.numChars();
+ if (numChars <= limit) {
+ return inputStr;
+ } else {
+ // Trailing spaces do not count in the length check. We need to retain
the trailing spaces
+ // (truncate to length N), as there is no read-time padding for
varchar type.
+ int tailSpacesMayNeedTrim = numChars - limit;
+ int lengthAfterTrim = inputStr.effectiveLength(tailSpacesMayNeedTrim);
+ if (lengthAfterTrim == limit) {
Review comment:
oh...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
