malinjawi opened a new pull request, #12131:
URL: https://github.com/apache/gluten/pull/12131
## What changes were proposed in this pull request?
This PR adds the JVM-side Delta deletion-vector scan handoff on top of the
native Delta DV reader support from #12040.
Main pieces:
- Materialize Delta deletion-vector payloads on the JVM side and pass them
to native scan execution.
- Preserve Delta scan behavior while stripping Spark's DV predicate only
after the native payload handoff is available.
- Add Delta 3.3 and Delta 4.0 scan handoff shims and tests.
- Add fallback/reporting visibility for unsupported DV scan paths.
Delta 2.4 support is intentionally not included in this PR.
## How was this patch tested?
- `./dev/run-scala-test.sh --force
-Pjava-17,spark-3.5,backends-velox,hadoop-3.3,spark-ut,delta -pl backends-velox
-s org.apache.gluten.execution.VeloxDeltaSuite`
- 19/19 passed
- `./dev/run-scala-test.sh --force
-Pjava-17,spark-4.0,scala-2.13,backends-velox,hadoop-3.3,spark-ut,delta -pl
backends-velox -s org.apache.gluten.execution.VeloxDeltaSuite`
- 19/19 passed
- `git diff --check upstream/main..HEAD`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
