Copilot commented on code in PR #12626:
URL: https://github.com/apache/gluten/pull/12626#discussion_r3868695221
##########
gluten-delta/src/main/scala/org/apache/gluten/execution/DeltaScanTransformer.scala:
##########
@@ -166,6 +167,20 @@ case class DeltaScanTransformer(
override def withNewPushdownFilters(filters: Seq[Expression]):
BasicScanExecTransformer =
copy(pushDownFilters = Some(filters))
+
+ // Vanilla Delta CDF translates V1 filters with nested-predicate pushdown
disabled.
+ override protected def supportNestedPredicatePushdownForDisplay: Boolean = {
+ if (isCdfScan) {
+ false
+ } else {
+ super.supportNestedPredicatePushdownForDisplay
+ }
+ }
+
+ private def isCdfScan: Boolean = relation.location match {
+ case _: CdcAddFileIndex | _: TahoeRemoveFileIndex | _:
TahoeChangeFileIndex => true
+ case _ => false
+ }
Review Comment:
`isCdfScan` is expanded to treat `TahoeChangeFileIndex` as a CDF scan for
display translation, but the DV safety guard (`cdfFilesHaveDeletionVectors`)
still only handles `TahoeRemoveFileIndex` and `CdcAddFileIndex`. If
`TahoeChangeFileIndex` is used for CDF scans on DV-enabled tables, this change
could bypass the native fallback and expose incorrect CDF results. Please align
the CDF file-index classification used for display with the classification used
for DV validation (either include `TahoeChangeFileIndex` in the DV guard or
centralize the predicate and reuse it in both places).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
