fresh-borzoni opened a new pull request, #136:
URL: https://github.com/apache/fluss-rust/pull/136
### Purpose
**Linked issue:** close #133
Fix memory leak in Python bindings caused by `Box::leak` usage when
writing Arrow batches. The old implementation leaked memory for every string
value written, and had poor performance due to row-by-row conversion with
O(rows × cols) Python calls.
### Brief change log
**Problem:**
- `write_arrow_batch()` used `Box::leak` to convert Python strings to
`&'static str` for `Datum::String`, causing permanent memory leaks
- Row-by-row extraction required O(rows × cols) Python→Rust conversions
- Held GIL during blocking I/O operations
**Solution:**
- Replaced row-by-row conversion with **Arrow C Data Interface (FFI)**
zero-copy approach
- Use `FromPyArrow::from_pyarrow_bound()` to share Arrow buffers between
Python and Rust without copying
- Call `append_arrow_batch()` API directly (columnar batch operation)
- Use `py.detach()` to release GIL during async I/O
**Impact:**
- ✅ Zero memory leaks
- ✅ Better performance
**Changes:**
- `bindings/python/src/table.rs`: Rewrote `write_arrow_batch()` method
- Added Arrow FFI imports at module level
- Removed all `Box::leak` usage and manual row conversion
- Changed from row-by-row `append()` to columnar `append_arrow_batch()`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
