kumarUjjawal commented on issue #22337:
URL: https://github.com/apache/datafusion/issues/22337#issuecomment-4507381143
Claude:
**`FFI_RecordBatchStream` does not follow the Foreign-adapter pattern used
elsewhere.**
| | Other `FFI_*` (table_provider, udaf/groups_accumulator, udtf,
catalog_provider_list, proto/physical_extension_codec) |
`FFI_RecordBatchStream` |
|---|---|---|
| Adapter struct | `ForeignX` wraps `FFI_X` | None — `grep -r
ForeignRecordBatchStream datafusion/` returns zero hits |
| Trait impl site | On the `ForeignX` adapter | Directly on
`FFI_RecordBatchStream` itself: `impl RecordBatchStream`
(`record_batch_stream.rs:149`), `impl Stream` (`record_batch_stream.rs:189`) |
| Return-path conversion | `From<FFI_X> for Box<dyn Trait>`, consults
`marker_id` inside | Only the forward `From<SendableRecordBatchStream> for
FFI_RecordBatchStream` (`record_batch_stream.rs:63`) exists |
| Consumer code | Calls `.into()` and gets the right concrete type |
`Pin::new(Box::new(stream)) as SendableRecordBatchStream` at
`execution_plan.rs:438` (and test at `record_batch_stream.rs:243`) |
So `library_marker_id` would have no place to be read in the current API —
there is no `From<FFI_RecordBatchStream> for SendableRecordBatchStream` and no
`ForeignRecordBatchStream` adapter. The field would be ABI surface with no
consultation site.
**A local bypass is feasible but is a separate ABI change.**
The natural site is `FFI_ExecutionPlan::execute` at
`execution_plan.rs:430-439`: a marker check could swap
`Pin::new(Box::new(stream))` for extracting `private_data.rbs` directly, saving
the per-poll fn-pointer indirection in same-library use.
Worth tracking, but as an enhancement — not under the same "missing
marker_id" framing as the other types, because here the marker is one piece of
a larger bypass that doesn't exist yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
