felipepessoto opened a new pull request, #13133:
URL: https://github.com/apache/gluten/pull/13133
<!--
Thank you for submitting a pull request! Here are some tips:
1. For first-time contributors, please read our contributing guide:
https://github.com/apache/gluten/blob/main/CONTRIBUTING.md
2. If necessary, create a GitHub issue for discussion beforehand to avoid
duplicate work.
3. If the PR is specific to a single backend, include [VL] or [CH] in the PR
title to indicate the
Velox or ClickHouse backend, respectively.
4. If the PR is not ready for review, please mark it as a draft.
-->
## What changes are proposed in this pull request?
<!--
Provide a clear and concise description of the changes introduced in this PR.
Ensure the PR description aligns with the code changes, especially after
updates.
If applicable, include "Fixes #<GitHub_Issue_ID>" to automatically close the
corresponding issue
when the PR is merged.
-->
Preserve the original Java throwable when a Java-backed native input
iterator fails. Today `checkException` retains only its printed stack trace,
Velox's generic native-exception handler can flatten it again, and
`JNI_METHOD_END` constructs a message-only `GlutenException`. This loses the
actual Java exception class, cause chain, and structured Spark error metadata.
- Add the backend-neutral `gluten::JavaException(JNIEnv*, jthrowable)`
carrier. A shared RAII global reference keeps the throwable alive across native
exception copies and threads. Local references and diagnostic strings are
released; cleanup attaches a thread only when necessary and does not detach a
caller-owned JNI thread.
- Capture callback exceptions through the existing `checkException` API and
restore the same object with JNI `Throw`, before the existing native-exception
fallback. An already-pending JNI allocation failure is not overwritten. Failure
to describe the original exception does not replace it.
- Wrap CPU and cuDF Java-backed input callbacks using Velox's existing
`exception_ptr` constructor. Use a runtime wrapper so a failed Java callback is
not treated as a suppressible `TRY` expression error. Recover Java carriers
through Velox and standard nested wrappers at the output boundary, including
lazy-vector loading. Other native errors are rethrown unchanged.
- Add focused core reference-lifetime tests, Velox wrapper/lazy-output
tests, and `GlutenJniExceptionSuite` under `gluten-ut` for real iterator/JNI
identity, Spark metadata, nested callbacks, Errors, and diagnostic failures.
Document the JNI exception contract.
This is the generic transport fix only: no native cast-error classification,
Delta assertion changes, or missing-error ARRAY/MAP changes. The original
throwable is retained, not reconstructed from its message.
## How was this patch tested?
<!--
Describe how the changes were tested, if applicable.
Include new tests to validate the functionality, if necessary.
For UI-related changes, attach screenshots to demonstrate the updates.
-->
**Completed on Windows/WSL with isolated portable tooling:**
- Compiled the actual `JavaException.cc` and `JavaExceptionTest.cc` with GCC
13, C++20, `-Wall -Wextra -Werror`: **7/7 tests passed**. Coverage includes
shared lifetime, local/global reference accounting, detached-thread cleanup,
already-attached thread ownership, describer/frame failures, and pending
allocation errors.
- Built an additional standalone JNI harness against the actual carrier and
`JniExceptionDescriber`, then ran JDK 17 with `-Xcheck:jni`: **504 real JVM
round trips passed**, checking original object/class/cause/suppressed
exceptions/stack trace/custom structured metadata. Includes Java-caller and
detached native-worker callbacks, concurrent calls, checked exceptions, Errors,
a failing stack-trace describer, and normal return. This is generic JNI
validation, not Spark/Velox integration.
- `dev/format-cpp-code.sh` using clang-format **15.0.7**, followed by a
dry-run check of the changed C++ files: passed.
- Maven-wrapper Spotless apply/check scoped to
`GlutenJniExceptionSuite.scala`, with Spark 3.5/Scala 2.12/backend profiles:
passed.
- Actual CI `.github/workflows/util/license-header.py` fix/check on all
changed files: passed. `git diff --check`: passed.
**Remaining verification / why this is a draft:**
- Full native CMake configuration successfully detects the portable
compiler, then stops because the pinned Velox checkout is absent
(`ep/build-velox/build/velox_ep/scripts/setup-helper-functions.sh`). No full
native dependency provisioning was started on the shared host.
- Consequently, `velox_java_exception_test`, the full `gluten-ut`/Spark JNI
suite, and the 40 affected Delta CI cases have **not** been executed. No claim
is made that those 40 cases now pass.
- The full `dev/format-scala-code.sh` was attempted but encounters a
pre-existing Windows symlink-as-text file,
`src-delta40/.../GlutenParquetFileFormat.scala`. The changed Scala suite passes
scoped Spotless instead.
- `dev/check.py header main --fix` was attempted but references the absent
`dev/license-header.py`; the repository's actual CI license-header helper was
used instead. Formatter-only changes outside this patch were removed; only the
scoped implementation/tests/documentation are committed.
## Was this patch authored or co-authored using generative AI tooling?
<!--
If generative AI tooling has been used in the process of authoring this
patch, please include the
phrase: 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
Generated-by: GitHub Copilot 1.0.87-0 (GPT-6 Astra, model ID: gpt-6-astra)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
