codope commented on code in PR #12575:
URL: https://github.com/apache/hudi/pull/12575#discussion_r1904088832
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/DataSkippingUtils.scala:
##########
@@ -109,137 +127,187 @@ object DataSkippingUtils extends Logging {
// corresponding column in the Column Stats Index
val targetExprBuilder: Expression => Expression =
swapAttributeRefInExpr(sourceExpr, attrRef, _)
genColumnValuesEqualToExpression(colName, valueExpr,
targetExprBuilder)
- }
+ }.orElse({
+ hasNonIndexedCols.set(true)
Review Comment:
On second thought, it’s safe to continue using `set(true)` because the goal
is simply to track whether **any** non-indexed column exists. This is being set
only when non-indexed column is detected. It does not need conditional updates,
as setting it to true multiple times is harmless.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
