viirya opened a new issue, #6434:
URL: https://github.com/apache/datafusion-comet/issues/6434
### What is the problem the feature request solves?
The native JNI entry points in `native/core/src/execution/jni_api.rs` and
`native/core/src/lib.rs`
mix three concerns in one function body: converting JNI arguments
(`JLongArray`, `JString`,
`JByteBuffer`, ...), the actual native logic, and raising the JVM exception
on failure. As a result:
- The native logic of most entry points cannot be unit-tested or benchmarked
without a JVM, even
when it never calls back into the JVM (for example shuffle block decoding
or the feature and
object store checks).
- The mapping from `CometError` / `SparkError` to the JVM exception (class,
message, Spark error
JSON, rethrown Java throwables, panic backtraces) is interleaved with the
JNI calls that throw it
in `throw_exception`, so the classification itself can only be tested
through a JVM.
### Describe the potential solution
- Split the exception raising into a JNI-free classification step, which
describes the exception
a failed native call surfaces as (with a status code and a serializable
form), and a JNI step
that throws it. Every entry point goes through the classification via
`try_unwrap_or_throw`,
with exception classes and messages unchanged.
- Turn each `Java_org_apache_comet_*` export into a thin wrapper that
converts JNI arguments into
plain Rust values and calls a core function without JNI types in its
signature.
- Do this incrementally: the error classification plus a few entry points
first, then the
remaining entry points that do not call back into the JVM, then the
argument handling of the
plan entry points (`createPlan`, `executePlan`, `releasePlan`,
`setShufflePartitionPusher`),
which keep depending on JVM callbacks.
### Additional context
No change to the JVM side or to the exceptions users see.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
