andygrove opened a new pull request, #77:
URL: https://github.com/apache/datafusion-java/pull/77
## Which issue does this PR close?
- Part of #33 (work items 1 and 3). Cross-build CI matrix, Sonatype/Maven
Central wiring, and release docs are deferred to follow-up PRs.
## Rationale for this change
`NativeLibraryLoader` currently calls
`System.loadLibrary("datafusion_jni")`, so consumers need to set
`java.library.path` or install the `.so`/`.dylib` system-wide before importing
the library. That's a barrier to publishing a Maven-Central artifact that just
works.
This PR adds the runtime piece of the fat-JAR design from #33: the loader
can pull a platform-specific native library out of the JAR's resource tree, and
the Maven build copies the locally-built `datafusion_jni` into that tree before
packaging.
## What changes are included in this PR?
`NativeLibraryLoader`:
- Tries `System.loadLibrary` first so users can still override the bundled
library via `java.library.path` / `LD_LIBRARY_PATH`.
- On `UnsatisfiedLinkError`, detects the host OS/arch, looks up
`org/apache/datafusion/<os>/<arch>/lib<name>.<ext>` on the classpath, extracts
it to `$TMPDIR/datafusion-java/<sha256>/`, and loads via `System.load`.
Hash-based directory naming lets concurrent JVMs sharing a temp directory
converge on the same file; extraction uses `ATOMIC_MOVE` to avoid racing
writers clobbering each other.
- OS/arch detection lives in a new package-private `Platform` class that
maps the OS/arch combinations from the design in #33: `linux/amd64`,
`linux/aarch64`, `darwin/x86_64`, `darwin/aarch64` (with a stub for
`windows`/`.dll` to make the future Windows port a build-matrix change).
Maven (`core/pom.xml`):
- Per-platform profiles activate by host OS/arch and set `datafusion.lib.os`
/ `datafusion.lib.arch` / `datafusion.lib.filename`.
- A new `maven-antrun-plugin` execution at `process-classes` copies
`native/target/<profile>/lib<name>.<ext>` into
`target/classes/org/apache/datafusion/<os>/<arch>/lib<name>.<ext>`, failing
fast with a clear message if the native crate hasn't been built.
`-Ddatafusion.native.profile=release` switches the source from `debug` to
`release`.
- Drops `-Djava.library.path=...` from the surefire argline (and the
equivalent argument from the examples module's `exec-maven-plugin`) so the test
path now exercises the same resource-extraction code path users will hit.
`--add-opens=java.base/java.nio=ALL-UNNAMED` stays.
## Are these changes tested?
- New `PlatformTest` covers OS detection (linux/darwin/windows), arch
detection across `amd64`/`x86_64`/`x64`/`aarch64`/`arm64` aliases,
unsupported-platform error handling, library filename mapping per OS, and
resource paths matching the spec in #33 exactly.
- Existing JNI-using integration tests now load the library via the
resource-extraction path (since `-Djava.library.path` is gone). Full local `mvn
test` is green on macOS aarch64 (229 tests, 13 skipped, 0 failures). Verified
that the JAR contains
`org/apache/datafusion/darwin/aarch64/libdatafusion_jni.dylib` and that the
extracted copy lands under `$TMPDIR/datafusion-java/<sha256>/`.
Other host platforms (linux/amd64, linux/aarch64, darwin/x86_64) will be
exercised once the cross-build CI matrix lands.
## Are there any user-facing changes?
The `NativeLibraryLoader.loadLibrary()` public API is unchanged. Behavior is
strictly broader — code that previously needed `java.library.path` set still
works, and code that depended only on the bundled lib now works too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
