fresh-borzoni opened a new pull request, #127:
URL: https://github.com/apache/fluss-rust/pull/127
### Purpose
New feature for analytics use cases. Returns Arrow batches directly
instead of wrapped records.
Linked issue: close #27
Add `poll_batches()` API to LogScanner for direct Arrow RecordBatch access
without row-by-row iteration overhead.
### Brief change log
- Add `poll_batches()` method to `LogScanner` that returns
`Vec<RecordBatch>` directly
- Add poll mode tracking with `AtomicU8` to prevent mixing `poll()` and
`poll_batches()` (causes data loss)
- Extend `CompletedFetch` trait with `fetch_batches()` method
- Add `LogRecordBatch::record_batch()` for direct batch extraction
- Add `Error::IllegalState` variant for poll mode violations
### Tests
**Integration Tests:**
- `test_poll_batches_basic` - Basic functionality and data correctness
- `test_poll_batches_empty` - Timeout on empty table
- `test_poll_batches_with_projection` - Field projection support
- `test_poll_and_poll_batches_mixed_returns_error` - Both directions of
poll mode mixing
### API and Format
**New API:**
- `LogScanner::poll_batches(timeout: Duration) -> Result<Vec<RecordBatch>>`
**Breaking:** None (additive only)
**Storage:** No changes
### Documentation
**Important:** Cannot mix `poll()` and `poll_batches()` on same scanner -
create new scanner to switch methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
