Github user JoshRosen commented on the issue:
https://github.com/apache/spark/pull/21346
Summary of key changes (WIP; notes to self):
> Summary of changes:
>
> - Introduce a new `UploadStream` RPC which is sent to push a large
payload as a stream (in contrast, the pre-existing `StreamRequest` and
`StreamResponse` RPCs are used for pull-based streaming).
> - Generalize `RpcHandler.receive()` to support requests which contain
streams.
> - Generalize `StreamInterceptor` to handle both request and response
messages (previously it only handled responses).
> - Introduce `StdChannelListener` to abstract away common logging logic in
`ChannelFuture` listeners.
Question: is this effectively dead code at this point? In other words, this
PR just adds the lower-level pieces but there's nothing currently using the new
API? So this patch as of now has no behavior change and actual functional
changes impacting queries / actual usage will come later when we wire this up
to the block replicator?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
