HwangDongJun opened a new issue, #12712:
URL: https://github.com/apache/gluten/issues/12712
### Backend
VL (Velox)
### Bug description
[Expected behavior]: When `spark.gluten.sql.columnar.batchscan=true` and
reading from an Iceberg table, Gluten's native scan validation
(`VeloxBackend.validateScanExec` → `validateScheme`) should check whether the
table's storage filesystem scheme is supported by the registered native (Velox)
filesystems, and fall back to vanilla Spark if not — the same way it does for
DSv1/DSv2 file-based scans via `FileIndex.rootPaths`.
[Actual behavior]: `IcebergScanTransformer.getRootPathsInternal`
(`gluten-iceberg/src/main/scala/org/apache/gluten/execution/IcebergScanTransformer.scala:179`)
is hardcoded to return `Seq.empty`, behind a `// TODO: get root paths from
table.` comment:
```scala
// TODO: get root paths from table.
override def getRootPathsInternal: Seq[String] = Seq.empty
```
`validateScheme()` in `VeloxBackend.scala` treats an empty `rootPaths` as
"nothing to check" (only validates `if (filteredRootPaths.nonEmpty && ...)`),
so scheme validation is silently skipped for every Iceberg scan regardless of
the table's actual filesystem. Gluten will not correctly fall back to vanilla
Spark for Iceberg tables on a scheme unsupported by the native build; any
resulting failure instead surfaces later inside native code, in a much less
clear form.
Suggested fix — return the Iceberg table's base location, mirroring how
DSv1/DSv2 scans expose their root paths:
```scala
override def getRootPathsInternal: Seq[String] = {
table match {
case t: SparkTable => Seq(t.table().location())
case _ => Seq.empty
}
}
```
We found this while investigating an unrelated S3A native-scan issue. We
have a small patch ready (verified via local compile) and are happy to open a
PR referencing this issue, along with a regression test.
Possibly related (unconfirmed): #7528, a native SIGSEGV on GCS + Iceberg
with no established root cause.
This issue was written with the assistance of AI (used to help organize and
phrase the investigation notes).
### Gluten version
main branch
### Spark version
Spark-4.0.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
