malinjawi commented on code in PR #13338:
URL: https://github.com/apache/gluten/pull/13338#discussion_r4312983554
##########
docs/get-started/VeloxDelta.md:
##########
@@ -1,62 +1,632 @@
-# Delta Lake Feature Support Status in Apache Gluten (Velox Backend)
+---
+layout: page
+title: Delta Lake Support in Velox Backend
+nav_order: 13
+parent: Getting-Started
+---
-This document summarizes the support status of **Delta Lake table features**
when used with **Apache Gluten (Velox backend)**.
+# Delta Lake Support in Velox Backend
-## Supported Spark / Delta combinations
+Gluten's Velox backend can accelerate eligible scans and query operators over
Delta Lake tables. Read
+acceleration does not require native writes. Native Delta writes are
experimental and disabled by
+default. This guide covers the supported Spark and Delta Lake versions, setup,
feature support and
+known limitations.
-| Spark profile | Spark version | Scala version | Delta Lake version | Status |
+## How a Delta query runs
+
+Take a Delta table partitioned by `sale_date`:
+
+```sql
+SELECT SUM(amount)
+FROM sales
+WHERE sale_date = DATE '2026-10-01'
+ AND amount > 100;
+```
+
+```mermaid
+flowchart LR
+ subgraph Planning["Spark driver"]
+ Snapshot["1. Delta snapshot<br/>Schema + active files"]
+ Select["2. Delta file pruning<br/>Partitions + file statistics"]
+ Plan["3. Spark plan<br/>Gluten offload checks"]
+ Snapshot --> Select --> Plan
+ end
+
+ subgraph Execution["Spark executors"]
+ Scan["4. Velox scan<br/>Reads the selected files"]
+ Compute["5. Velox operators<br/>Filter + SUM"]
+ Spark["Spark operators<br/>Operators Gluten does not offload"]
+ Scan --> Compute
+ Compute <-->|"Row and columnar conversions"| Spark
+ end
+
+ subgraph Storage["Delta table"]
+ Log[("_delta_log<br/>Metadata + file statistics")]
+ Data[("Parquet files")]
+ end
+
+ Result["6. Query result"]
+
+ Log --> Snapshot
+ Plan -->|"Offloaded operators"| Scan
+ Plan -->|"Other operators"| Spark
+ Data --> Scan
+ Data --> Spark
+ Compute --> Result
+ Spark --> Result
+```
+
+Delta Lake loads the snapshot and prunes partitions and files while Spark
plans the query. Gluten then
+offloads the operators Velox supports, and the rest run on Spark, so one query
can mix Velox and Spark
+operators. Delta Lake keeps the table metadata, the transaction log and
commits.
+
+## Supported Spark and Delta Lake versions
+
+| Spark | Delta Lake | Delta Lake jar | Read | Native write | Tested by |
+|---|---|---|---|---|---|
+| 3.4.4 | 2.4.0 | `io.delta:delta-core_2.12:2.4.0` | Offload. Deletion vector
scans fall back. | None | Gluten suites |
+| 3.5.5 | 3.3.2 | `io.delta:delta-spark_2.12:3.3.2` (`_2.13` for Scala 2.13) |
Offload | Experimental, off by default | Gluten suites, copied Delta Lake
suites (Scala 2.12 only) |
+| 4.0.2 | 4.0.1 | `io.delta:delta-spark_2.13:4.0.1` | Offload | Experimental,
off by default | Gluten suites, copied Delta Lake suites tagged
`ExtendedSQLTest` |
+| 4.1.1 | 4.1.0 | `io.delta:delta-spark_4.1_2.13:4.1.0` | Offload |
Experimental, off by default | Gluten suites, nightly Delta Lake 4.2.0 job |
+| 4.2 | None on main | | None | None | None |
+
+Each Spark profile from 3.4 to 4.1 builds against the one Delta Lake release
in the table. On Spark 4.0 and 4.1, the Read and Native write columns assume
ANSI mode off (see [ANSI mode on Spark 4.x](#ansi-mode-on-spark-4x)). The
copied Delta Lake suites run with native write on and
`spark.gluten.sql.fallbackUnexpectedMetadataParquet=true`, which is off by
default.
+
+Spark 4.1 has the most tests, because the nightly job runs on it. That job
builds Delta Lake 4.2.0 from source against Spark 4.1.0, which is newer than
the Delta Lake 4.1.0 in the table, and runs its Spark test suites. It puts a
Gluten bundle compiled against Spark 4.1.1 and Delta Lake 4.1.0 on the test
classpath. It replaces Delta Lake's `DeltaSQLCommandTest` with Gluten's copy,
which loads Gluten with native write on and the non-default
`spark.gluten.sql.fallbackUnexpectedMetadataParquet=true`. It then runs
`.github/workflows/util/delta-spark-ut/apply-delta-test-patches.sh`. That
script cherry-picks two Delta Lake test changes that are not in 4.2.0,
delta-io/delta#7104 and #7105, so Delta Lake's scan checks accept Gluten's scan
node. It also sets a row-group row limit in two test fixtures. The same job
also runs on pull requests that change Gluten's Delta code. "Seen on 4.1" on
this page refers to that job.
+
+The job's known failures are listed in
`.github/workflows/util/delta-spark-ut/known-failures.txt`. Four entries are
two 2-billion-row deletion vector tests, run in two suites, that the job
force-fails. Some entries check a query plan that changes once Gluten offloads
it. Read the README in that directory before treating an entry as a Gluten bug.
+
+On Spark 4.1, Gluten's own suites check deletion vector, change data feed,
column mapping, time travel and partition filter reads, and a partitioned
native `save(path)`. Other native writes on Spark 4.1 are checked only by the
nightly job.
+
+Delta Lake support for Spark 4.2 is in progress in
[#13271](https://github.com/apache/gluten/pull/13271).
+
+### Build and run
+
+Delta Lake support is compiled in only with the `-Pdelta` Maven profile. Build
the native library first, as described in [Gluten with Velox Backend](Velox.md).
+
+```bash
+./build/mvn clean install -Pbackends-velox -Pspark-3.5 -Pdelta -DskipTests
+./build/mvn clean install -Pbackends-velox -Pspark-4.1 -Pscala-2.13 -Pjava-17
-Pdelta \
+ -DskipTests -Dmaven.compiler.release=17
+```
+
+`dev/buildbundle-veloxbe.sh` does not pass `-Pdelta`, so its bundle has no
Delta Lake support. `dev/release/build-release.sh` passes `-Pdelta` only for
Spark 3.4 and 3.5, so its Spark 4.0 and 4.1 bundles have no Delta Lake support
either. No Gluten bundle contains the Delta Lake jar, including one built with
`-Pdelta`. Add Delta Lake with `--packages` and the coordinate from the table
above, which also adds its `delta-storage` dependency. When
`io.delta.sql.DeltaSparkSessionExtension` is not on the classpath, Gluten logs
`Excluding runtime-incompatible component` and leaves its Delta support off.
From code.
+
+Native writes to HDFS, S3, GCS or ABFS paths need the native library built
with `--enable_hdfs=ON`, `--enable_s3=ON`, `--enable_gcs=ON` or
`--enable_abfs=ON`. Without that connector, scans of those paths fall back and
native writes fail (see [Write problems](#write-problems)). From code.
+
+For example, on Spark 3.5:
+
+```bash
+spark-shell \
+ --packages io.delta:delta-spark_2.12:3.3.2 \
+ --jars gluten-velox-bundle-spark3.5_2.12-<os>_<arch>-<version>.jar \
+ --conf spark.plugins=org.apache.gluten.GlutenPlugin \
+ --conf spark.memory.offHeap.enabled=true \
+ --conf spark.memory.offHeap.size=4g \
+ --conf
spark.shuffle.manager=org.apache.spark.shuffle.sort.ColumnarShuffleManager \
+ --conf spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension \
+ --conf
spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog
+```
+
+On Spark 4.0 or 4.1, use the Gluten bundle and the Delta Lake jar for that
version, and add `--conf spark.sql.ansi.enabled=false`.
+
+### ANSI mode on Spark 4.x
+
+Spark 4.0 and 4.1 turn ANSI mode on by default. With Gluten's default
`spark.gluten.sql.ansiFallback.enabled=true`, every query then falls back,
Delta scans and native writes included. From code. Turning ANSI mode off also
changes Spark's own behavior for invalid input. For example, `CAST('a' AS INT)`
returns NULL instead of failing.
+
+```properties
+spark.sql.ansi.enabled=false
+```
+
+Gluten's Delta tests run with ANSI mode off in CI. The nightly Delta Lake job
turns ANSI mode off in the test setup that loads Gluten. That setup and some
Gluten Delta suites also set `spark.gluten.sql.ansiFallback.enabled=false`. On
Spark 3.x, turning ANSI mode on makes every query fall back in the same way.
From code.
+
+Keeping ANSI on and setting `spark.gluten.sql.ansiFallback.enabled=false`
removes that fallback. Gluten passes ANSI mode to Velox. Velox does not
implement ANSI behavior for every expression, so some invalid input returns a
result where Spark fails
([#10134](https://github.com/apache/gluten/issues/10134)). For Delta Lake, only
the nightly Delta Lake job tests that combination, in the Delta Lake 4.2.0
suites that turn ANSI on. It has a known MERGE problem (see [MERGE, UPDATE and
DELETE problems](#merge-update-and-delete-problems)). Cast overflow errors
raised in Velox have a different exception type (see [Error
types](#error-types)).
+
+## Support Status
+The following values show the Delta Lake support status:
+
+| Value | Description
|
+|----------------|----------------------------------------------------------------------------|
+| Offload | Runs in the Velox backend. For writes, Velox writes the
data files. |
+| PartialOffload | Some operators run in Velox and the rest fall back.
|
+| Fallback | Runs on Spark or Delta Lake.
|
+| Exception | Fails instead of falling back.
|
+| ResultMismatch | Can return wrong results or write wrong files.
|
+
+Not tested: no test covers the case.<br>
+From code: the status comes from reading the source. No test that runs in CI
checks it.<br>
+Seen on 4.1: the nightly Delta Lake job showed the result, with Delta Lake
4.2.0 on Spark 4.1.0.<br>
+Reported on 3.4, Reported on 4.1: the linked issue reports the result on that
Spark version. No CI test checks it.<br>
+No label: a test in the Gluten repository that runs in CI checks the status on
the stated versions.<br>
+
+## Adding catalogs
+Fallback
+
+Catalog operations run on the driver. From code.
+
+## Creating a table
+Fallback
+
+```sql
+CREATE TABLE t (id INT, name STRING) USING delta;
+```
+
+A table created without `AS SELECT` is a metadata-only commit. There is
nothing to offload. From code.
+
+Offload
+
+```sql
+SET spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite=true;
+CREATE TABLE t2 USING delta AS SELECT * FROM t;
+```
+
+With `spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite=true` on
Spark 3.5 to 4.1, Velox writes the data files of `CREATE TABLE ... AS SELECT`
and `REPLACE TABLE ... AS SELECT` when the target catalog is a DeltaCatalog.
Seen on 4.1. On 3.5 and 4.0 this is from code. On Spark 4.x, also apply [ANSI
mode on Spark 4.x](#ansi-mode-on-spark-4x).
+
+## Writing
+PartialOffload
+
+```sql
+INSERT INTO t SELECT id, name FROM source WHERE length(name) = 1;
+```
+
+Delta's writer writes the files of `INSERT`, with or without native write. The
`SELECT` can run in Velox. From code.
+
+Fallback
+
+```sql
+INSERT INTO t VALUES (1, 'a'), (2, 'b'), (3, 'c');
+```
+
+With the default `spark.gluten.sql.columnar.localTableScan=false`, the
`VALUES` scan runs on Spark. Delta's writer writes the files. From code.
+
+Offload
+
+```sql
+SET spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite=true;
+UPDATE t SET name = 'b' WHERE id = 2;
+```
+
+On Spark 3.5 to 4.1, set
`spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite=true` to have
Velox write the data files of the commands marked Offload in [Write
commands](#write-commands). Velox writes the files of this UPDATE. Seen on 4.1.
On 3.5 and 4.0 this is from code. On Spark 4.x, also apply [ANSI mode on Spark
4.x](#ansi-mode-on-spark-4x).
+
+Native write does not check the codec, the path scheme or the partition column
types before it runs. Unsupported values make the task fail or write wrong
files. Read [Write problems](#write-problems) before turning it on.
+
+### Checking the plan
+
+A command that Gluten wraps for native write shows a `GlutenDelta` node, for
example `Execute GlutenDelta UpdateCommand` or `GlutenDelta
AtomicCreateTableAsSelect`. Without the prefix, Delta's writer wrote the files.
+
+```scala
+val df = spark.sql("UPDATE delta.`/data/t` SET name = 'bb' WHERE id = 2")
+df.queryExecution.executedPlan.treeString.contains("GlutenDelta ")
+```
+
+The node also shows when Velox writes no data files. This happens for a DELETE
that writes only deletion vectors or only metadata, and for a Spark 4.x write
whose schema Velox cannot write, such as one with a VARIANT column. From code.
For `DataFrameWriter` calls, read the command's plan from the Spark UI SQL tab
or a `QueryExecutionListener`. Each write also reports a nested
`deltaTransactionalWrite` execution for the query that feeds it. That plan
never has the prefix, whichever writer ran. From code.
+
+### Write commands
+
+The Status column assumes Spark 3.5 to 4.1 with native write on. Otherwise
every row is at best PartialOffload. In PartialOffload rows, Delta writes the
files and the query that feeds the write can run in Velox.
+
+Gluten's CI checks that a partitioned `save(path)` on 3.5 to 4.1 and OPTIMIZE
on 3.5 run as Gluten native write commands. It also checks on 3.5 and 4.0 that
a DELETE that rewrites a file runs in Gluten's transaction and that its scan
runs in Velox. On 3.5 and 4.0, the copied Delta Lake suites also run UPDATE,
DELETE, `save(path)` and `CREATE TABLE ... AS SELECT` with native write on.
They check results only. On 4.1, the nightly job shows every Offload row except
DELETE going through Velox's writer. The other Offload cases are from code.
+
+| SQL | DataFrame API | Status | Notes |
+|---|---|---|---|
+| | `df.write.format("delta").save(path)` | Offload | Every save mode, with or
without `partitionBy`. |
+| `CREATE TABLE ... AS SELECT` | `writeTo(t).create()`, `saveAsTable(t)` when
`t` does not exist and the mode is not overwrite | Offload | Needs a
DeltaCatalog target. |
+| `REPLACE TABLE ... AS SELECT`, `CREATE OR REPLACE TABLE ... AS SELECT` |
`writeTo(t).replace()`, `writeTo(t).createOrReplace()`, `saveAsTable(t)` in
overwrite mode | Offload | Needs a DeltaCatalog target. |
+| `UPDATE` | `DeltaTable.update()` | Offload | On a table with deletion
vectors, Velox writes only the updated rows. |
+| `DELETE` | `DeltaTable.delete()` | Offload, PartialOffload when it writes
deletion vectors (from code on 4.1), Fallback when it is metadata-only (from
code) | Velox writes the rewritten files. A deletion-vector DELETE and a
metadata-only DELETE write no data files. |
+| `OPTIMIZE`, `OPTIMIZE ... WHERE` |
`DeltaTable.optimize().executeCompaction()`, with or without `where(...)` |
Offload | Bin-packing, by path or by name. |
+| `OPTIMIZE ... ZORDER BY`, `OPTIMIZE ... FULL`, `OPTIMIZE` on a
liquid-clustered table | | PartialOffload | From code. |
+| `REORG TABLE ... APPLY (PURGE)`, `REORG TABLE ... APPLY (UPGRADE UNIFORM
...)` | | PartialOffload | From code. |
+| `INSERT INTO ... SELECT`, `INSERT OVERWRITE`, `INSERT INTO ... REPLACE
WHERE` | | PartialOffload | From code. To append natively, use `save(path)` in
append mode. |
+| | `insertInto(t)`, `saveAsTable(t)` in append mode on an existing table,
`writeTo(t).append()`, `overwrite(condition)`, `overwritePartitions()` |
PartialOffload | From code. |
+| `MERGE INTO` | | PartialOffload | Scans, the DML target scan and joins can
run in Velox. On Spark 3.5 and 4.0, Gluten's tests check that a deletion-vector
MERGE runs a scan of the target table in Velox. The other cases are from code.
Native MERGE write is in progress in
[#13324](https://github.com/apache/gluten/pull/13324). |
+| | `writeStream.format("delta")` | PartialOffload | Inside `foreachBatch`,
batch writes follow the rows above. From code. |
+
+With native write on, `DESCRIBE HISTORY` operation metrics of `save(path)`,
UPDATE, DELETE and `CREATE TABLE ... AS SELECT` match Delta Lake in
`DescribeDeltaHistorySuite`. Seen on 4.1. Three unpartitioned tests in
`DeleteMetricsSuite` and `UpdateMetricsSuite` expect two added files and get
one. The job sets `spark.default.parallelism=1`, which plans the five small
input files as one scan partition (from code).
+
+### Write features
+
+| Feature | Status | Notes |
+|---|---|---|
+| Optimized write | Offload | In the commands marked Offload, the rebalancing
shuffle runs in Velox. If it fails validation, Delta's shuffle is used. Both
are from code. `OPTIMIZE` never uses it. |
+| Auto compaction | Offload after native writes, PartialOffload after other
writes | Compacts natively written tables (seen on 4.1). The auto compaction
tests that fail in the nightly job fail because its
`spark.default.parallelism=1` makes their first write produce one file (from
code). After a native write, the compaction runs in a Gluten transaction (from
code). After INSERT, MERGE or streaming writes, Delta's writer compacts (from
code). |
+
+## Reading
+### Read data
+Offload/Fallback
+
+| Table type | Spark 3.4 / Delta Lake 2.4 | Spark 3.5 to 4.1 |
+|---|---|---|
+| Unpartitioned | Offload | Offload |
+| Partitioned | Offload | Offload |
+| Column mapping, name mode | Offload | Offload |
+| Column mapping, id mode | Offload, matched by physical name. Known
ResultMismatch cases: see [Read problems](#read-problems). | Offload, matched
by physical name. Known ResultMismatch cases: see [Read
problems](#read-problems). |
+| Files with deletion vectors | Fallback | Offload |
+| Change data feed, no deletion vectors in range | Offload. PartialOffload
when the table allows deletion vectors and the range has `_change_data` files,
because those scans then fall back (from code). | Offload |
+| Change data feed, deletion vectors in range | Fallback, PartialOffload when
a file scan inside Delta's change data feed scan reads no files with deletion
vectors and no `_change_data` files (from code) | Fallback, PartialOffload when
a file scan inside Delta's change data feed scan reads no files with deletion
vectors (from code) |
+
+```sql
+SELECT count(1) AS count, name
+FROM t
+GROUP BY name;
+```
+
+Reads need no Gluten option. With `spark.sql.adaptive.enabled=false`,
`EXPLAIN` shows the native scan as `FileDeltaScanTransformer` and `EXPLAIN
FORMATTED` shows `DeltaScanTransformer`. With AQE on, the default, `EXPLAIN`
and `EXPLAIN FORMATTED` of a query with a shuffle, such as this one, print the
plan from before Gluten offloads it. The Spark UI plan graph of a finished
query shows `DeltaScanTransformer`. Delta prunes partitions and skips files by
statistics while it plans the query, before the scan runs.
+
+| Read | Status | Notes |
+|---|---|---|
+| By table name, ``delta.`/path` `` or `spark.read.format("delta").load(path)`
| Offload | Same native scan. |
+| `VERSION AS OF` | Offload | Tested on Spark 3.4 to 4.1. |
+| `TIMESTAMP AS OF`, `versionAsOf`, `timestampAsOf` | Offload | From code.
Delta Lake's `DeltaTimeTravelSuite` passes (seen on 4.1). |
+| Streaming source | Offload | Each micro-batch is a Delta file scan. From
code. Delta Lake's `DeltaSourceSuite` passes (seen on 4.1). |
+| Dynamic partition pruning | Not tested | |
+
+### Deletion vectors
+Offload
+
+```sql
+OPTIMIZE t;
+ALTER TABLE t SET TBLPROPERTIES ('delta.enableDeletionVectors' = true);
+DELETE FROM t WHERE id IN (3, 4);
+SELECT * FROM t;
+```
+
+On Spark 3.5 to 4.1, Velox drops the deleted rows in the scan. This needs
Delta's `spark.databricks.delta.deletionVectors.useMetadataRowIndex=true`, the
default. The DML target scan, which finds the rows for new deletion vectors,
also runs in Velox. Gluten tests check the scan on Spark 3.5 to 4.1 and the DML
target scan on Spark 3.5 and 4.0. Delta writes the deletion-vector files.
+
+Fallback
+
+```sql
+SET spark.databricks.delta.deletionVectors.useMetadataRowIndex=false;
+SELECT * FROM t;
+RESET spark.databricks.delta.deletionVectors.useMetadataRowIndex;
+```
+
+| Case | Behavior |
+|---|---|
+| Spark 3.5 to 4.1 with `useMetadataRowIndex=false` | A scan that reads
Delta's generated `__delta_internal_is_row_deleted` or
`__delta_internal_row_index` column falls back with `fallback Delta scan
requiring generated DV metadata`. Delta adds the first column when any file in
the scanned table version, micro-batch, DML file set or `OPTIMIZE` bin has a
deletion vector, so `OPTIMIZE` and `REORG TABLE ... APPLY (PURGE)` scans of
those files also fall back (from code). It adds the second to DML target scans
that write deletion vectors. Other scans offload. Tested on Spark 3.5 and 4.0. |
+| Spark 3.4, normal reads | A read falls back with `fallback Spark 3.4 Delta
DV scan` when a file it selected has a deletion vector. Tested on Spark 3.4. A
read whose selected files have none offloads (from code). |
+| Spark 3.4, other scans, such as DML target scans, `OPTIMIZE` scans,
streaming micro-batches and change data feed scans of `_change_data` files |
Fall back whenever the table protocol allows deletion vectors. Change data feed
scans of added or removed files fall back only when a file they read has a
deletion vector. From code. |
+| `spark.databricks.delta.stats.skipping=false` | Delta Lake itself rejects
reads of a table with deletion vectors with `Cannot work with a non-pinned
table snapshot of the TahoeFileIndex` (seen on 4.1). Before Delta Lake 4.1.0,
the table feature alone triggers it (from code). |
+
+The fallback reasons show in [Gluten UI](GlutenUI.md).
+
+### Change data feed
+Offload
+
+```sql
+CREATE TABLE c (id INT, name STRING) USING delta
+ TBLPROPERTIES ('delta.enableChangeDataFeed' = true);
+INSERT INTO c VALUES (1, 'a'), (2, 'b');
+UPDATE c SET name = 'bb' WHERE id = 2;
+SELECT id, name, _change_type, _commit_version
+FROM table_changes('c', 1)
+WHERE id = 2;
+```
+
+Gluten expands a batch change data feed read into Delta file scans that run in
Velox with the filters pushed down. Delta also adds an empty placeholder scan
to the plan. That scan falls back when the table schema has a MAP at any
nesting level (from code). If the expansion fails, Gluten logs a warning and
the read uses Delta's change data feed scan. From code.
+
+PartialOffload
+
+```sql
+SET
spark.gluten.sql.columnar.backend.velox.delta.enableChangeDataFeedScan=false;
+SELECT * FROM table_changes('c', 1);
+```
+
+A read whose version range adds or removes a file with a deletion vector uses
Delta's change data feed scan. Insert-only ranges of a table with deletion
vectors enabled still run in Velox. With
`spark.gluten.sql.columnar.backend.velox.delta.enableChangeDataFeedScan=false`,
every batch change data feed read uses Delta's scan. Delta's scan reads the
data files in a query of its own that Gluten also plans, and `EXPLAIN` does not
show it (from code). That query's file scans can run in Velox, except scans of
added or removed files that have deletion vectors and, on Spark 3.4, scans of
`_change_data` files when the table allows deletion vectors (from code).
Streaming change data feed reads ignore this option. They run as Delta file
scans. A scan whose files have deletion vectors falls back, and on Spark 3.4 so
does a scan of `_change_data` files when the table allows deletion vectors
(from code).
+
+### Read metadata
+Offload
+
+```sql
+SELECT id, _metadata.file_path, _metadata.row_index
+FROM t
+WHERE _metadata.row_index = 2;
+```
+
+The native scan fills the `_metadata` file fields and `input_file_name()`
(from code), and `_metadata.row_index`. On Spark 3.5 to 4.1, Delta Lake exposes
`_metadata.row_index` only on tables that allow deletion vectors or have row
tracking, so the example needs `t` from [Deletion vectors](#deletion-vectors)
(from code). A query that selects only `_metadata.file_name` can fail when the
table has a BIGINT data column named `file_name` (seen on 4.1, see [Read
problems](#read-problems)). Gluten tests check `_metadata.row_index` in the
native scan on Spark 3.5 and 4.0, and Delta Lake tests that read it pass (seen
on 4.1). Spark 3.4 is not tested. On row-tracking tables, `_metadata.row_id`
and `_metadata.row_commit_version` are read natively (from code). In the
nightly job, they return correct values except in the cases in [Read
problems](#read-problems) (seen on 4.1).
+
+Fallback
+
+```sql
+DESCRIBE HISTORY t;
+```
+
+`DESCRIBE HISTORY` reads each commit file on Spark, through Delta's LogStore
in a `mapPartitions` job. When Delta replays the log for a snapshot, every scan
under `_delta_log`, checkpoints included, stays on Spark with the reason
`fallback Delta _delta_log scan`. From code.
+
+## Table features
+Offload/Fallback
+
+The Read column covers Spark 3.5 to 4.1. It covers Spark 3.4 only where the
row names it. The Native write column applies to Spark 3.5 to 4.1 with native
write on.
+
+| Feature | Read | Native write | Notes |
+|---|---|---|---|
+| Deletion vectors | Offload, Fallback on Spark 3.4 | PartialOffload |
Gluten's tests check deletion vector DELETE and MERGE results with native write
on, on Spark 3.5 and 4.0. Which writer writes each file is from code. See
[Deletion vectors](#deletion-vectors). |
+| Change data feed | Offload, also on Spark 3.4. Ranges with deletion vectors
fall back. On Spark 3.4, scans of `_change_data` files also fall back when the
table allows deletion vectors (from code). | Offload | UPDATE and DELETE write
the `_change_data` files with Velox (from code). Change data feed reads of
their changes are correct (seen on 4.1). See [Change data
feed](#change-data-feed). |
+| Column mapping, name mode | Offload, also on Spark 3.4 | ResultMismatch |
Files have no Parquet field IDs. Seen on 4.1. See [Write
problems](#write-problems). |
+| Column mapping, id mode | Offload, also on Spark 3.4. ResultMismatch when
field IDs differ from physical names (seen on 4.1). | ResultMismatch, Exception
when Spark's reader reads the files (seen on 4.1) | Gluten matches columns by
physical name. See [Known problems](#known-problems). |
+| Identity columns | Offload | Offload | Seen on 4.1. High-water marks are
correct. The projection that generates identity values runs on Spark (from
code). |
+| Generated columns | Offload | Offload | Seen on 4.1. No Gluten test covers
generated columns. Velox computes the generated values when it supports the
expression (from code). Partition filters derived from them reach the native
scan. `DATE_FORMAT` results for years above 9999 differ (see [Write
problems](#write-problems)). |
+| CHECK constraints, NOT NULL | Offload | Offload | Delta checks the rows on
the JVM before Velox writes them. Gluten's tests cover NOT NULL on Spark 3.5
and 4.0. Seen on 4.1. |
+| Row tracking | Offload, Exception for `_metadata.base_row_id` filters,
ResultMismatch for `_metadata.row_id` when older files lack base row IDs |
Offload | Seen on 4.1. See [Read problems](#read-problems). |
+| Liquid clustering | Offload | PartialOffload | Reads use the same native
scan as other tables. Native commands write to clustered tables. `OPTIMIZE`,
which clusters the data, uses Delta's writer. All three are from code. |
+| UniForm (IcebergCompatV1, V2) | | ResultMismatch, Exception when partitioned
| The write status is from code. No test reads a UniForm table. |
+| Type widening | Offload, Exception for DATE to TIMESTAMP_NTZ | Offload,
Exception for FLOAT or DOUBLE partition columns, for TIMESTAMP_NTZ partition
values with sub-millisecond digits and for some writes of TIMESTAMP_NTZ data
columns | Seen on 4.1. Spark 3.5 and 4.0 are not covered. |
+| TIMESTAMP_NTZ | PartialOffload, also on Spark 3.4 | Offload, Exception for
some writes (seen on 4.1) | See [DataType](#datatype). |
+| Variant, including shredding | Fallback | Fallback | Spark 4.x. From code.
See [DataType](#datatype). |
+| Collations | Fallback | | Spark 4.x. From code. See [DataType](#datatype). |
+| materializePartitionColumns | | ResultMismatch | Spark 4.1. From code. |
+
+## DataType
+
+The Read column applies to Spark 3.4 to 4.1. On all of them, Gluten's tests
read INT, BIGINT, DECIMAL(12,2), STRING, DATE, TIMESTAMP, TIMESTAMP_NTZ,
STRUCT, ARRAY and MAP columns natively. The other read statuses are from code,
unless a row says otherwise. The native write columns apply to Spark 3.5 to 4.1
with native write on. On Spark 3.5 and 4.0, Gluten's tests write BOOLEAN, INT,
BIGINT, STRING, STRUCT and ARRAY columns natively. The other native write
statuses are seen on 4.1, unless a row says they are from code.
+
+| Type | Read | Native write | Native write as partition column | Notes |
|---|---|---|---|---|
-| `spark-3.5` | Spark 3.5.x | 2.12 | 3.3.x | Supported |
-| `spark-4.0` | Spark 4.0.x | 2.13 | 4.0.x | Supported |
-
-Native Delta write is supported in both Spark 3.5 and Spark 4.0 profiles. The
difference between
-the two rows above is the Spark/Delta compatibility target (Spark 3.5 + Delta
3.3 vs Spark 4.0 +
-Delta 4.0), not a native-write capability gap.
-
-## Build and runtime notes
-
-Build Gluten with Delta support by enabling `-Pdelta` together with the Velox
backend profile and a Spark profile.
-
-- Spark 3.5 build example:
- - `mvn clean package -Pbackends-velox -Pdelta -Pspark-3.5 -DskipTests`
-- Spark 4.0 build example:
- - `mvn clean package -Pbackends-velox -Pdelta -Pspark-4.0 -Pscala-2.13
-Pjava-17 -DskipTests`
-
-Native Delta write is controlled by:
-
-- `spark.gluten.sql.columnar.backend.velox.delta.enableNativeWrite`
- - Default: `false`
- - Type: experimental
-
-Native change data feed scan offload is controlled by:
-
-- `spark.gluten.sql.columnar.backend.velox.delta.enableChangeDataFeedScan`
- - Default: `true`
- - Type: experimental
-
-The native DELETE/UPDATE/MERGE deletion-vector target scan is controlled by:
-
-- `spark.gluten.sql.columnar.backend.velox.delta.enableNativeDmlRowIndexScan`
- - Default: `true`
- - Type: experimental
- - When disabled, the DML target scan that produces file paths and row
indexes for
- deletion-vector writes stays on Spark; other scans are unaffected.
-
-| Feature | Delta minWriterVersion | Delta minReaderVersion | Iceberg
format-version | Feature type | Supported by Gluten (Velox) |
-|---|---:|---:|---:|---|---|
-| Basic functionality | 2 | 1 | 1 | Writer | Yes |
-| CHECK constraints | 3 | 1 | N/A | Writer | No |
-| Change data feed | 4 | 1 | N/A | Writer | Yes |
-| Generated columns | 4 | 1 | N/A | Writer | Partial |
-| Column mapping | 5 | 2 | N/A | Reader and writer | Yes |
-| Identity columns | 6 | 1 | N/A | Writer | Yes |
-| Row tracking | 7 | 1 | 3 | Writer | Partial |
-| Deletion vectors | 7 | 3 | 3 | Reader and writer | Partial |
-| TimestampNTZ | 7 | 3 | 1 | Reader and writer | No |
-| Liquid clustering | 7 | 3 | 1 | Reader and writer | Yes |
-| Iceberg readers (UniForm) | 7 | 2 | N/A | Writer | Not tested |
-| Type widening | 7 | 3 | N/A | Reader and writer | Partial |
-| Variant | 7 | 3 | 3 | Reader and writer | Not tested |
-| Variant shredding | 7 | 3 | 3 | Reader and writer | Not tested |
-| Collations | 7 | 3 | N/A | Reader and writer | Not tested |
-| Protected checkpoints | 7 | 1 | N/A | Writer | Not tested |
+| BOOLEAN | Offload | Offload | Offload | |
+| TINYINT, SMALLINT, INT, BIGINT | Offload | Offload | Offload | |
+| FLOAT, DOUBLE | Offload | Offload | Exception | See [Write
problems](#write-problems). |
+| DECIMAL | Offload | Offload | Offload | Precision above 18 included. |
+| STRING | Offload | Offload | Offload | |
+| CHAR, VARCHAR | Offload | Offload | | Stored as STRING. Delta checks the
length on the JVM before the native write. Both statuses are from code. Reads
return correct results (seen on 4.1). |
+| BINARY | Offload | Offload | Not tested | The read status is from code. One
Delta Lake test writes a table with only a BINARY column, checks its file
statistics and runs a DELETE that filters on that column (seen on 4.1). |
+| DATE | Offload | Offload | Offload | |
+| TIMESTAMP | Offload, ResultMismatch for sub-millisecond partition values
(seen on 4.1) | Offload | Offload, Exception for sub-millisecond values | The
partition column status is from code. See [Known problems](#known-problems). |
+| TIMESTAMP_NTZ | PartialOffload | Offload, Exception for some writes (seen on
4.1) | Offload, Exception for sub-millisecond values (seen on 4.1) | Filters,
sorts, joins and other operators whose input or output includes a TIMESTAMP_NTZ
column fall back, even when they do not use that column. Scans, aggregates,
shuffles and some projections can run in Velox. A scan whose pushed filter has
a TIMESTAMP_NTZ literal falls back (from code). Gluten's tests check on Spark
3.4 to 4.1 that a scan of the column runs in Velox, and that a filter on the
column and a sort on another column cause a fallback. The other operator rules
are from code. |
+| STRUCT, ARRAY, MAP | Offload | Offload | | |
+| VARIANT | Fallback | Fallback | | Spark 4.x. Shredded variant reads fall
back too. Both statuses are from code. |
+| Collated STRING | Fallback | | | Spark 4.x. Scans that read the column fall
back. Delta Lake 4.0.1 and 4.1.0 reject a collated column in a table schema
with `DELTA_UNSUPPORTED_DATA_TYPES`. They cannot write to tables that have the
`collations` feature. From code. |
+
+A plan node falls back when its output contains a type Velox does not support,
such as VARIANT. With the default
`spark.gluten.sql.columnar.backend.velox.parquetUseColumnNames=true`, a scan
that does not select such a column can still offload. From code. Delta Lake
does not allow interval columns.
+
+## Format
+PartialOffload
+
+Delta data files are Parquet, and Gluten reads them with Velox's Parquet
reader. Velox writes them only for native write commands on Spark 3.5 to 4.1.
Otherwise Delta's Parquet writer writes them.
+
+### Natively written files
+
+| Property | Behavior |
+|---|---|
+| Compression | snappy (default), gzip, zstd (always level 3), lz4 (Hadoop
framing), none. From code. Other codecs: see [Write problems](#write-problems).
|
+| Row groups | Sized by `spark.gluten.sql.columnar.parquet.write.blockSize`
and `spark.gluten.sql.native.parquet.write.blockRows`. `parquet.*` settings are
ignored. From code. |
+| TIMESTAMP | INT64 microseconds. `spark.sql.parquet.outputTimestampType` is
ignored. Delta Lake's writer uses INT96 by default. Spark reads both. From
code. |
+| Parquet field IDs | Not written, for column-mapped and UniForm tables too.
Seen on 4.1 for column mapping, from code for UniForm. See [Write
problems](#write-problems). |
+| Spark schema footer entry | `org.apache.spark.sql.parquet.row.metadata` is
not written ([#13288](https://github.com/apache/gluten/issues/13288)). Seen on
4.1. |
+| Nested field names | Lowercased below the top level when
`spark.sql.caseSensitive=false`. Spark and Gluten match them
case-insensitively. A case-sensitive reader can miss them. From code. |
+| Page index | Always written. From code. |
+| `spark.sql.parquet.writeLegacyFormat=true` | Applies to DECIMAL only. Arrays
and maps keep the standard layout. From code. |
+| File statistics | numRecords, min, max and nullCount are computed in Velox
(checked on Spark 3.5). Gluten's tests check the values against the rows in
each file on Spark 3.5 to 4.1. Most tests in Delta's data-skipping suites pass
(seen on 4.1). Of the 34 that fail, 22 check statistics and expect specific
rows in each file after `repartition(2)`. The other 12 hit the
`_metadata.file_name` failure in [Read problems](#read-problems). |
+| Data file paths | Come from Delta's committer, so names and random prefixes
follow Delta. File names seen on 4.1, random prefixes from code. |
+
+## SQL
+
+| Statement | Status | Notes |
+|---|---|---|
+| `SELECT` | Offload | See [Reading](#reading). |
+| `table_changes()` | Offload | Ranges with deletion vectors fall back. See
[Change data feed](#change-data-feed). |
+| `CREATE TABLE ... AS SELECT`, `REPLACE TABLE ... AS SELECT` | Offload | With
native write on, Spark 3.5 to 4.1, and a DeltaCatalog target. Seen on 4.1, from
code on 3.5 and 4.0. See [Creating a table](#creating-a-table). |
+| `INSERT`, `UPDATE`, `DELETE`, `MERGE`, `OPTIMIZE`, `REORG` | | See [Write
commands](#write-commands). |
+| `CREATE TABLE` without `AS SELECT`, `DROP TABLE`, `DESCRIBE` | Fallback |
They read and write no data files. From code. |
+| `ALTER TABLE` | Fallback, PartialOffload when it reads data | Most forms
read and write no data files. `ADD CONSTRAINT` scans the table, and that scan
can run in Velox. On Spark 3.5 to 4.1, `ALTER COLUMN ... SYNC IDENTITY` also
scans it, and `DROP FEATURE` and `SET TBLPROPERTIES ('delta.columnMapping.mode'
= 'none')` can rewrite data files with Delta's writer. From code. |
+| `VACUUM`, `RESTORE`, `SHALLOW CLONE` | | They write no Parquet data files.
From code. |
+| `CONVERT TO DELTA` | PartialOffload | Writes no Parquet data files. Unless
the statement has `NO STATISTICS` or
`spark.databricks.delta.stats.collect=false` is set, it scans the Parquet files
for statistics. That scan can run in Velox. From code. |
+
+## Schema evolution
+PartialOffload
+
+The `mergeSchema` and `overwriteSchema` rows need native write on Spark 3.5 to
4.1. Otherwise Delta writes the files.
+
+| Change | Status | Notes |
+|---|---|---|
+| `mergeSchema` with `save(path)` | Offload | Velox writes the evolved schema
(from code). On Spark 3.5 and 4.0, the copied `DeltaSuite` checks the results
of a `mergeSchema` overwrite. Delta Lake's `SchemaEnforcementWithPathSuite`
passes (seen on 4.1). |
+| `overwriteSchema` with `save(path)` | Offload | Delta applies the change
before the write. From code. Delta Lake's `SchemaEnforcementWithPathSuite`
passes (seen on 4.1). |
+| Schema evolution in SQL `INSERT` | PartialOffload | From code. |
+| Schema evolution in `MERGE` | PartialOffload, Exception for some STRUCT map
key changes | The merged table contents match Delta Lake. Some STRUCT map key
changes fail in Velox (see [MERGE, UPDATE and DELETE
problems](#merge-update-and-delete-problems)). The two TIMESTAMP_NTZ widening
tests fail before the MERGE runs, in a setup write of TIMESTAMP_NTZ values.
Seen on 4.1. |
+| `RENAME COLUMN` with column mapping | Offload | Reads after top-level and
nested renames return correct results. After a top-level rename, partition
pruning and file skipping still work. |
+| `DROP COLUMN` with column mapping | Offload | From code. Delta Lake's
`DeltaDropColumnSuite` passes (seen on 4.1). |
+| Type widening | Offload, Exception for DATE to TIMESTAMP_NTZ and for native
writes to FLOAT or DOUBLE partition columns | Seen on 4.1. See [Table
features](#table-features). |
+| `spark.sql.parquet.mergeSchema=true` | Fallback | Every Parquet scan falls
back, Delta scans included. From code. |
+
+## Known problems
+
+### Read problems
+
+| Problem | Status | Versions | Issue |
+|---|---|---|---|
+| A column default stored as `EXISTS_DEFAULT` metadata, for example after
`CONVERT TO DELTA`, reads as NULL for rows in files without that column. |
ResultMismatch | Seen on 4.1. #11494 reports the same bug for plain Parquet on
3.4 to 4.1. | [#11494](https://github.com/apache/gluten/issues/11494) |
+| In column mapping id mode, when field IDs no longer match the Parquet column
names, Gluten returns the column with the matching physical name. Delta Lake
returns NULL when no file column has that ID (seen on 4.1). It returns the
column the ID points to when another column has it (from code). |
ResultMismatch | 3.4 to 4.1. Seen on 4.1 for the NULL case, from code
otherwise. | [#12241](https://github.com/apache/gluten/issues/12241) |
+| On a TIMESTAMP-partitioned table with six partition values from .1 to
.123456 seconds, `count(DISTINCT ts)` returns 3, as if the values were
truncated to milliseconds. Equality filters on the column still return the
right rows. | ResultMismatch | Seen on 4.1 | |
+| Reading a data column widened from DATE to TIMESTAMP_NTZ fails on files
written before the change with `Converted type DATE is not allowed for
requested type TIMESTAMP UTC`. A partition column widened the same way reads
correctly. | Exception | Seen on 4.1 | |
+| A query that selects only `_metadata.file_name` from a table with a BIGINT
data column named `file_name` reads the data column and fails with `Converted
type BIGINT is not allowed for requested type VARCHAR`. | Exception | Seen on
4.1 | |
+| An equality filter on `_metadata.base_row_id` fails with `Synthesized column
'base_row_id' failed filter validation` when a scanned file has a different
base row ID. | Exception | Seen on 4.1 | |
+| On a row-tracking table whose older files lack base row IDs, selecting
`_metadata.row_id` returns rows. Delta raises an error. Delta Lake's test
creates this state with a manual metadata commit. `ALTER TABLE ... SET
TBLPROPERTIES` backfills base row IDs when it enables row tracking. |
ResultMismatch | Seen on 4.1 | |
+| Reading or deleting from a 2-billion-row table with deletion vectors can
grow native memory until the JVM is killed. Delta Lake's two 2-billion-row
tests reproduce it. The nightly job force-fails both tests. | Exception |
Reported on 4.1 | [#12387](https://github.com/apache/gluten/issues/12387) |
+| With `spark.databricks.delta.deletionVectors.useMetadataRowIndex=true`, the
default, selecting `__delta_internal_is_row_deleted` from a relation built by
hand over `DeltaParquetFileFormat` drops the deleted rows and returns 1 for the
rows it keeps. The column is not in the table schema, so SQL and DataFrame
queries over the table cannot select it. | ResultMismatch | Seen on 4.1 | |
+| Scans of a table with many small files can be slower than vanilla Spark on
EBS storage. In the report, two of five TPC-DS SF1000 queries take 2.2 and 3.4
times as long as on vanilla Spark, with 182,300 files in one table. A
maintainer did not see the slowdown on a host with local NVMe storage. | |
Reported on 3.4 | [#10778](https://github.com/apache/gluten/issues/10778) |
+
+### Write problems
+
+These need native write on, unless a row says otherwise.
+
+| Problem | Status | Versions | Issue |
+|---|---|---|---|
+| A write partitioned by a FLOAT or DOUBLE column fails with `Unsupported
partition type`. | Exception | 3.5 to 4.1, seen on 4.1 |
[#13138](https://github.com/apache/gluten/issues/13138) |
+| A write to a path scheme the native library cannot write fails at task time.
This covers custom Hadoop file systems, and S3, GCS, HDFS or ABFS in a build
without that connector. | Exception | 3.5 to 4.1, seen on 4.1 (custom file
system), from code (missing connector) |
[#4963](https://github.com/apache/gluten/issues/4963) |
+| Files written for a column mapping table, in name or id mode, have no
Parquet field IDs, which the Delta Lake protocol requires. Delta Lake and
Gluten read name-mode files by physical name. Readers that match by field ID
cannot. | ResultMismatch | 3.5 to 4.1, seen on 4.1 | |
+| Spark's own reader fails on natively written id-mode files with `Spark read
schema expects field Ids, but Parquet file schema doesn't contain any field
Ids`. Spark uses this reader when a scan falls back or Gluten is off. With
`spark.sql.parquet.fieldId.read.ignoreMissing=true`, Spark skips this check and
returns NULL for the data columns of these files (from code). That setting is
not tested with these files. | Exception | 3.5 to 4.1, seen on 4.1 | |
+| Files written for a UniForm table have no Parquet field IDs, so the Iceberg
metadata points at files without IDs. No error is raised. | ResultMismatch |
3.5 to 4.1, from code | [#12957](https://github.com/apache/gluten/pull/12957) |
+| A write to a partitioned UniForm table fails with `The file schema type
should be equal with the input rowvector type`. | Exception | 3.5 to 4.1, from
code | |
+| Writes to a table with the `materializePartitionColumns` feature leave the
partition columns out of the data files. | ResultMismatch | 4.1, from code | |
+| Some writes of a TIMESTAMP_NTZ data column fail with `The file schema type
should be equal with the input rowvector type`. Delta Lake's type widening
tests fail this way when they write `CAST(value AS TIMESTAMP_NTZ)`. Delta
Lake's data skipping tests also write that cast and pass. On Spark 4.x,
`spark.gluten.sql.columnar.backend.velox.enableTimestampNtzValidation=true`
sends TIMESTAMP_NTZ writes to Delta's writer. It is not tested as a workaround.
| Exception | Seen on 4.1 | |
+| A partitioned write fails in Velox's `PartitionIdGenerator` when a TIMESTAMP
or TIMESTAMP_NTZ partition value has sub-millisecond digits. | Exception | 3.5
to 4.1, seen on 4.1 (TIMESTAMP_NTZ), from code (TIMESTAMP) | |
+| A generated partition column such as `DATE_FORMAT(ts, 'yyyy-MM')` stores
`12345-07` for the year 12345. Spark stores `+12345-07`. Not checked with
native write off. | ResultMismatch | Seen on 4.1 | |
+| The `compression` option set to brotli or lzo fails the write. So does
`spark.sql.parquet.compression.codec` when the option is unset. The write fails
even with the Hadoop codec class on the classpath. Without that class, Delta's
writer fails too, and Spark does not ship it. | Exception | 3.5 to 4.1, from
code | |
+| The `compression` option or `spark.sql.parquet.compression.codec` set to
lz4_raw writes Snappy data into a `.lz4raw.parquet` file. | ResultMismatch |
3.5 to 4.1, from code | |
+| `maxRecordsPerFile` on an unpartitioned write counts rows but writes whole
batches. When the limit is smaller than a batch, the whole batch goes into one
file. The other files opened for that batch are 0-byte files that are not valid
Parquet. | ResultMismatch | 3.5 to 4.1, from code | |
+| `spark.sql.maxConcurrentOutputFileWriters` above 0 makes a partitioned write
with unsorted input fail with `Underlying columnar data is inaccessible from
BatchCarrierRow`. | Exception | 3.5 to 4.1, from code | |
+| An unpartitioned native write whose first task gets no rows, for example a
write of an empty DataFrame, leaves a 0-byte file that is not valid Parquet.
Reading the table directory as plain Parquet can then fail with `is not a
Parquet file (length is too low: 0)`. With statistics on, the file stays on
disk until `VACUUM`. With `spark.databricks.delta.stats.collect=false`, Delta
commits it. The last two sentences are from code. | ResultMismatch | 3.5 to
4.1, seen on 4.1 | |
+| A statistics aggregation that Velox cannot offload fails the write with
`ClassCastException`. With the default
`spark.gluten.sql.columnar.backend.velox.enableTimestampNtzValidation=false`,
the TIMESTAMP_NTZ case from the issue no longer fails on main. Tested on Spark
3.5, seen on 4.1, from code on 4.0. On Spark 3.5 with that option set to true,
the TIMESTAMP_NTZ case still fails (from code). | Exception | 3.5 to 4.1, from
code | [#12538](https://github.com/apache/gluten/issues/12538) |
+
+### MERGE, UPDATE and DELETE problems
+
+MERGE does not use native write, so the MERGE rows also apply with native
write off (from code).
Review Comment:
I double-checked this one and the statement holds as MERGE isn't wrapped for
native write, so Delta's writer still writes its files (none of the MERGE
traces on main go through Gluten's writer; #13324 adds the native path which is
still WIP). These problems come from the parts of MERGE that run in Velox, so
they show up either way.
I reworded it to tie it to the Write commands table, and fixed the deletion
vector row that made it look like MERGE writes natively.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
