timsaucer opened a new issue, #22337: URL: https://github.com/apache/datafusion/issues/22337
## Question `FFI_RecordBatchStream` in `datafusion/ffi/src/record_batch_stream.rs` exposes `poll_next` and `schema` and a clone/release pair, but does not appear to carry a `library_marker_id` field — unlike most other `FFI_X` structs in the crate. Confirm whether this is intentional. ## Why it matters Without `library_marker_id`, the local-bypass optimization is unavailable: when a `RecordBatchStream` originates from the same library that consumes it, conversion back to `Box<dyn RecordBatchStream>` cannot short-circuit to the original concrete type and instead always wraps it in a `Foreign` adapter that pays the FFI hop on every `poll_next`. If by design (e.g., streams are always consumed by a different library than the one that produced them, or the bypass is unsafe for `Stream` types specifically), this is a no-op — please document the rationale here and we will update the `datafusion-ffi` skill accordingly so the audit checklist no longer flags it. ## Possible outcomes 1. **Intentional omission.** Document why in the wrapper's module-level doc comment, and update the skill so the marker-id check is listed as \"not required for `FFI_RecordBatchStream` because <reason>\". 2. **Genuine gap.** Add `library_marker_id: extern \"C\" fn() -> usize` and the corresponding marker-id check in `From<&FFI_RecordBatchStream> for Box<dyn RecordBatchStream>`. ABI change → `api change` label, target `main` only, no back-port to `branch-<major>`, integration test required. --- Generated from an audit performed for PR #22327 (datafusion-ffi agent skill). If the intentional-omission outcome applies, please propose the skill update along with the close-as-not-planned. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
