SEPURI-SAI-KRISHNA commented on code in PR #19646:
URL: https://github.com/apache/hudi/pull/19646#discussion_r3818414252
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/DataSkippingUtils.scala:
##########
@@ -160,11 +160,12 @@ object DataSkippingUtils extends Logging {
Option.empty
})
- // Filter "colA = null"
- // Translates to "colA_nullCount = null" for index lookup
- case EqualNullSafe(attrRef: AttributeReference, litNull @ Literal(null,
_)) =>
+ // Filter "colA <=> null"
+ // Translates to "colA_nullCount = null or colA_nullCount > 0" for index
lookup
Review Comment:
Good point — `= NULL` is exactly the thing this patch is fixing, so spelling
it that way in the comment is confusing. Updated all five comments this PR
touched to `colA_nullCount IS NULL OR colA_nullCount > 0`.
I also went one line further and gave the neighbouring `is not null`
comments the same treatment (`colA_nullCount IS NULL OR colA_valueCount IS NULL
OR ...`), since they describe the `Or(Or(IsNull(numNullExpr),
IsNull(valueCountExpr)), ...)` that arm builds and had the same `= null`
wording. Happy to drop that hunk if you'd rather keep the diff to the lines
this PR already changed.
Comment-only change, no behavior difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
