Copilot commented on code in PR #13042:
URL: https://github.com/apache/gluten/pull/13042#discussion_r4032233589
##########
gluten-delta/src/main/scala/org/apache/gluten/extension/OffloadDeltaScan.scala:
##########
@@ -55,9 +57,8 @@ case class OffloadDeltaScan(enableNativeDmlRowIndexScan:
Boolean) extends Offloa
case scan: FileSourceScanExec if
shouldFallbackSpark34DeletionVectorScan(scan) =>
FallbackTags.add(scan, "fallback Spark 3.4 Delta DV scan")
scan
- case scan: FileSourceScanExec
- if shouldFallbackDeletionVectorScanWithoutMetadataRowIndex(scan) =>
- FallbackTags.add(scan, "fallback Delta DV scan without metadata row
index")
+ case scan: FileSourceScanExec if
shouldFallbackGeneratedDeletionVectorMetadataScan(scan) =>
Review Comment:
This case runs before the existing `DeltaScanUtils.isDeltaScan` case and
tags any `FileSourceScanExec` whose schema happens to contain one of these
names. An ordinary Parquet scan can have a user column named
`__delta_internal_row_index` or `__delta_internal_is_row_deleted`;
`OffloadOthers` honors the tag and skips native conversion, so setting this
Delta config to false can unnecessarily force unrelated scans onto Spark.
Restrict this fallback to Delta scans.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
