felipepessoto opened a new pull request, #13561:
URL: https://github.com/apache/gluten/pull/13561
<!--
Thank you for submitting a pull request! Here are some tips:
1. For first-time contributors, please read our contributing guide:
https://github.com/apache/gluten/blob/main/CONTRIBUTING.md
2. If necessary, create a GitHub issue for discussion beforehand to avoid
duplicate work.
3. If the PR is specific to a single backend, include [VL] or [CH] in the PR
title to indicate the
Velox or ClickHouse backend, respectively.
4. If the PR is not ready for review, please mark it as a draft.
-->
## What changes are proposed in this pull request?
<!--
Provide a clear and concise description of the changes introduced in this PR.
Ensure the PR description aligns with the code changes, especially after
updates.
If applicable, include "Fixes #<GitHub_Issue_ID>" to automatically close the
corresponding issue
when the PR is merged.
-->
**Layer 2 of 3**, extracted from #13133 at
`b25dc154cab6e5df33ad8f65465cc15e3b89dbd8`. Depends on #13560 at
`e7bf0b610839570dcc2e9302bea30853eea3a679`; neither PR is superseded or closed
by this draft.
**Review the [layer-only
comparison](https://github.com/felipepessoto/gluten/compare/e7bf0b610839570dcc2e9302bea30853eea3a679...d2d3b3756d8a3db7ed076721f75db40c36e36e89)**:
11 files, 714 insertions, 41 deletions. GitHub native stacks do not support
these fork heads, so this upstream PR targets `apache/gluten:main`. Its normal
PR diff includes layer 1 until that prerequisite lands; this is not a native
stack or a layer-only normal PR diff. The branch retains layer 1 directly above
underlying main `e63d43ef4f677c860b2a3c27f768572d46725a8c`.
- Wrap Java-backed row/GPU input iterator callbacks before Velox Driver's
lossy generic catch. Preserve the existing core carrier through both Velox
`exception_ptr` and standard nested exceptions, using runtime/system
classification so TRY cannot suppress a failed callback.
- Recover the original throwable across cursor advancement and lazy output
loading. Keep five native regressions, six real-JVM
iterator/metadata/identity/describer/end-of-input cases, and six strict Delta
CHECK/NOT NULL/replaceWhere cases. Delta tests compare class, structured
metadata, SQLSTATE, message and cause, verify unchanged committed
version/files/rows, and check native Parquet footer identity.
- Keep the JVM suite and Scalastyle source wiring Velox-only. Remove the 20
remaining `followAnsiEnabled=false` batch-cast baseline entries after targeted
execution; the selected main already removed the other 14 nested-map entries
from the original source change. All other baseline entries are preserved.
No core carrier API/implementation changes in this layer.
Allocation-listener wrapping and its three native/two Scala reservation tests
remain for layer 3. Native cast translation (#13135), partition materialization
(#13502), and hybrid Java/C++ stack traces (#9642) remain out of scope. The
Delta suite keeps `writePartitionColumnsToParquet=false`.
## How was this patch tested?
<!--
Describe how the changes were tested, if applicable.
Include new tests to validate the functionality, if necessary.
For UI-related changes, attach screenshots to demonstrate the updates.
-->
All results below were executed against this layer's code, not reused from
the complete original implementation. An isolated copy of the native cache was
updated to the selected base's exact Velox pin, `dft-2026_11_05` /
`eddeec2f17fcb4bf750f2c24bb82232e690e2c5d`. Later-layer/materialization code
was removed and the affected native objects and both JNI libraries rebuilt
before Maven. Both Spark bundles were checked against the rebuilt library
hashes, using normal two-library JNI resource loading.
| Local check | Result |
|---|---|
| `VeloxJavaExceptionTest` via CTest | 5/5 passed |
| Spark 3.5 / Scala 2.12 reactor and package | Passed |
| Spark 3.5 `GlutenJniExceptionSuite` / `GlutenDeltaNativeConstraintSuite` |
6/6 + 6/6 passed |
| Spark 4.1 / Scala 2.13 reactor and package | Passed |
| Spark 4.1 `GlutenJniExceptionSuite` / `GlutenDeltaNativeConstraintSuite` |
6/6 + 6/6 passed |
| Exact upstream Delta v4.4.1 batch-cast/nested-map cases | 34/34 passed,
original assertions unchanged |
| Repository baseline gate on those 34 reports | 0 regressions, 0 stale
now-passing entries |
| clang-format 15, cmake-format 0.6.13, Spotless apply/check, both
Scalastyle profiles, license headers, diff checks | Passed |
All 24 JVM cases ran with JDK 17 and `-Xcheck:jni`, without canceled or
skipped cases. Existing JNI class-descriptor/shuffle-metric warnings and Spark
shutdown-thread warnings were observed; these runs are not claimed to be
warning-free. Native build parallelism was capped at four and all commands had
hard timeouts. GPU execution and the full 8-shard x 4-fork Delta matrix were
not run locally; the 34 upstream cases ran with one shard/fork.
## Was this patch authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has been used in the process of authoring this
patch, please include the
phrase: 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
Generated-by: GitHub Copilot 1.0.102-0.unstable.r40949665530.gab1eef5 (GPT-6
Astra, gpt-6-astra)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
