robertbuessow opened a new pull request, #594: URL: https://github.com/apache/arrow-julia/pull/594
## Summary This PR adds support for the [Arrow C Data Interface](https://arrow.apache.org/docs/format/CDataInterface.html), enabling zero-copy data exchange between Julia and C/C++/Python runtimes in the same process. Key additions and fixes: - **C Data Interface implementation** (`src/cdatainterface.jl`): `Arrow.from_c_data` / `Arrow.to_c_data` for importing and exporting Arrow arrays and tables across the C ABI boundary. Handles all Arrow types: primitive, boolean, list, fixed-size list, map, struct, union, dict-encoded, and nested types. - **Defensive resource management**: `CDataHandle` tracks C-side memory lifetime. The GC finalizer uses `jl_safe_printf` instead of `@error` (task switches are forbidden in finalizers) and an atomic counter instead of a non-thread-safe `Ref{Int}`. The finalizer also calls the C release callback as a safety net, so C resources are freed even when `release_c_data` is not called explicitly. - **ABI compatibility test**: compiles a C probe at test time using `offsetof()` and compares every field offset of `ArrowSchema` and `ArrowArray` against Julia's `fieldoffset()`, catching any struct layout divergence between Julia and the C compiler. - **Explicit release in tests**: all tests that import C data now call `release_c_data` explicitly, and a final `@testset "no unexpected resource leaks"` asserts `UNRELEASED_HANDLE_COUNT` only increases by the one intentional leak test. - **Bug fixes**: double-free on import, type narrowing errors, `BoundsError` when dict-encoding `CategoricalArray`s with missing values. ## Test plan - [ ] `julia --project -e 'using Pkg; Pkg.test()'` passes with no leak warnings - [ ] `@testset "struct field offsets match C ABI"` — 19 assertions all green - [ ] `@testset "no unexpected resource leaks"` — counter == initial + 1 - [ ] No `Arrow.CDataHandle GC'd without explicit release_c_data` output during the test run (except inside the `redirect_stderr` block of the intentional-leak test) 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
