wiedld opened a new pull request, #7800: URL: https://github.com/apache/arrow-datafusion/pull/7800
## Which issue does this PR close? Prerequisite for the cascading merge. Part of #7181 . ## Rationale for this change Before this change, the buffered stream abstractions only handled record batch streams (`SendableRecordBatchStream`). Instead, would like to be able to buffer streams of other outputs too. Ideally using the same receiver abstraction. Later on, this will be used to buffer the stream of each merge node in the cascaded tree. Specifically, [this spawn_buffered_merge()](https://github.com/apache/arrow-datafusion/blob/9b1019853fb3d71f6355ab1d850089332592b3d8/datafusion/physical-plan/src/sorts/cascade.rs#L297) will be wrapped around each merge nodes ([here](https://github.com/apache/arrow-datafusion/blob/9b1019853fb3d71f6355ab1d850089332592b3d8/datafusion/physical-plan/src/sorts/cascade.rs#L144) for the leaves and [here](https://github.com/apache/arrow-datafusion/blob/9b1019853fb3d71f6355ab1d850089332592b3d8/datafusion/physical-plan/src/sorts/cascade.rs#L164) for the non-leaves). ## What changes are included in this PR? * change ReceiverStreamBuilder and ReceiverStream to handle a generic stream of data. * define an adaptor interface to be used for any input-specific `call()` on the data. * implement this adaptor for record batches (as needed). ## Are these changes tested? Is a refactor. Not new functionality. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
