rednaxelafx commented on code in PR #37252:
URL: https://github.com/apache/spark/pull/37252#discussion_r927846790
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/UnsafeRowUtils.scala:
##########
@@ -74,13 +79,36 @@ object UnsafeRowUtils {
if ((row.getLong(index) >> 32) != 0L) return false
case _ =>
}
- case (_, index) if row.isNullAt(index) =>
- if (row.getLong(index) != 0L) return false
+ case (field, index) if row.isNullAt(index) =>
+ field.dataType match {
+ case dt: DecimalType if !UnsafeRow.isFixedLength(dt) =>
+ // See special case in UnsafeRowWriter.write(int, Decimal, int,
int) and
+ // UnsafeRow.setDecimal(int, Decimal, int).
+ // A variable-length Decimal may be marked as null while having
non-zero offset and
+ // zero length. This allows the field to be updated (i.e. mutable
variable-length data)
+ val (offset, size) = getOffsetAndSize(row, index)
+ if (size != 0 || // size must be zero
+ offset != 0 && // offsetAndSize may be entirely zero if
row.setNullAt(index),
+ // otherwise range check offset
+ (offset < bitSetWidthInBytes + 8 * row.numFields ||
+ offset + size > rowSizeInBytes)) {
Review Comment:
Thank you very much for the suggestion! The suggested version of Line 94 is
going to exceed 100 character line width limit, so please allow me to still
break the predicate into multiple lines. But I'll extract the inlilne comments
to above the `if` as you suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
