LuciferYang opened a new issue, #12998:
URL: https://github.com/apache/gluten/issues/12998
### Description
`backends-bolt` is not built by any CI job. The profile is
`activeByDefault=false` (`pom.xml:2085-2088`), `.github/workflows/` has no
reference to it, and the one script that does (`dev/gen-all-config-docs.sh:31`)
is not wired into a workflow. As a result the module drifted away from the
shared APIs without anyone noticing, and by the time #12454 merged it no longer
compiled at all: it still called seven `SparkShims` methods removed by #12954
and two symbols removed by #12981, and overrode two `BackendSettingsApi`
settings that no longer exist. The compile fix (PR link to follow) repairs
those twelve compile errors, but the drift it exposed is wider than the build
break.
This issue collects what is left. Everything below is pre-existing and none
of it comes from that fix.
### 1. `-Pbackends-bolt -Pdelta` still does not compile
`BoltDeltaComponent.scala:40` calls `OffloadDeltaScan()` with no arguments,
but the case class has taken a parameter since it grew deletion-vector support:
```scala
case class OffloadDeltaScan(enableNativeDmlRowIndexScan: Boolean) extends
OffloadSingleNode
```
There is no default value, so `mvn test-compile -Pspark-3.4 -Pscala-2.13
-Pbackends-bolt -Piceberg -Piceberg-test -Pdelta` fails with `not enough
arguments for method apply`. Velox passes `new
VeloxDeltaConfig(c.sqlConf).enableNativeDmlRowIndexScan`
(`VeloxDeltaComponent.scala:56-58`); bolt has no `BoltDeltaConfig`, so this
needs a decision rather than a mechanical copy. Passing `false` is the
conservative option: `shouldFallbackDeletionVectorDmlScan` then falls the scan
back rather than offloading it, and since bolt has not compiled with `-Pdelta`
since the parameter was added, there is no prior behaviour to preserve.
### 2. Native write drops the session Hadoop conf
`BoltTransformerApi.scala:113` builds the native write parameters from
`write.caseInsensitiveOptions` only. Velox first merges the session Hadoop conf
(`VeloxTransformerApi.scala:159-165`):
```scala
val writeOptions = Option(write.session).map { session =>
val hadoopConf =
session.sessionState.newHadoopConfWithOptions(write.options)
CaseInsensitiveMap(hadoopConf.iterator().asScala.map(e => e.getKey ->
e.getValue).toMap)
}.getOrElse(write.caseInsensitiveOptions)
```
`nativeConf` looks up the Parquet writer settings by key, so on bolt
anything set through `spark.hadoop.*` never reaches the native writer.
Reproduce with `set spark.hadoop.parquet.enable.dictionary=false` followed by
an `INSERT`: the file is written with Gluten's defaults instead, and differs
from both vanilla Spark and Velox. No error and no fallback, so it is silent.
`parquet.block.size`, `parquet.page.size`, `parquet.writer.version` and the
codec levels behave the same way.
### 3. Bloom filter rewrite is injected at the wrong point
`BoltRuleApi.scala:77` registers `BloomFilterMightContainJointRewriteRule`
at `injectPreTransform`. Velox moved it to `injectPre` for GLUTEN-12013
(`VeloxRuleApi.scala:83`), with a comment explaining why: when
`ExpandFallbackPolicy` promotes a stage fallback to a whole-stage fallback, the
reversion strips rewrites applied at `injectPreTransform`. That can leave one
stage producing Spark-format bloom filter bytes while another still reads them
as native-format, which fails with "Unsupported BloomFilter version".
`BoltRuleApi.scala:125` injects the same `ExpandFallbackPolicy`, so the path is
reachable on bolt too. The fix is the one-word move to `injectPre`.
### 4. Smaller divergences from Velox
- `validateDataTypes` in `BoltBackend.scala:331` inspects only top-level
`StructField`s, where Velox uses a recursive `hasUnsupportedType`
(`VeloxBackend.scala:362`). Writing `array<interval year to month>` passes
bolt's check and reaches the native writer; Velox falls back.
- `GenerateTransformStageId()` is never injected. Velox
(`VeloxRuleApi.scala:139`) and CH (`CHRuleApi.scala:149`) both inject it, so on
bolt `WholeStageTransformer.transformStageId` keeps its default and stage ids
in explain output are not unique.
- `BoltIteratorApi.scala:118` has no equivalent of Velox's
`needMetadataColumns` short-circuit (`VeloxIteratorApi.scala:124-136`), so a
scan with no metadata columns still allocates one empty map per file. Cost only.
### 5. Spark 3.3 leftovers
#12902 removed Spark 3.3 support, but #12454 brought some of it back:
`gluten-ut/spark33/src/test/scala/org/apache/gluten/utils/bolt/{BoltSQLQueryTestSettings,BoltTestSettings}.scala`,
plus four `spark33/` directories under
`backends-bolt/src/test/resources/tpch-approved-plan/` (`v1`, `v1-ras`,
`v1-bhj`, `v1-bhj-ras`). Nothing builds `gluten-ut/spark33` any more, so these
are dead files.
### 6. gluten-ut only wires bolt for 3.5
`gluten-ut/spark35/src/test/backends-bolt/` exists; `spark40` and `spark41`
have only `backends-clickhouse` and `backends-velox`. So `-Pbackends-bolt
-Pspark-ut` compiles on 3.4 and 3.5 but not on 4.0 or 4.1, where the shared
suites cannot resolve `GlutenColumnarWriteTestSupport` and friends.
### What would stop this recurring
A CI job that compiles `-Pbackends-bolt` on the supported Spark versions
would have caught the twelve errors the fix addresses, and would catch item 1
today. Whether that job also runs bolt's suites is a separate question, since
the native library needs Linux plus GCC 10-12 or Clang 16
(`docs/bolt-quick-start.md`).
Happy to split this into separate issues if that suits the bolt owners
better.
cc @taiyang-li @FelixYBW @zhztheplayer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
