robertwb commented on a change in pull request #11235: [WIP] Pull bundle
descriptors.
URL: https://github.com/apache/beam/pull/11235#discussion_r398805859
##########
File path: model/fn-execution/src/main/proto/beam_fn_api.proto
##########
@@ -81,6 +81,13 @@ service BeamFnControl {
returns (
// A stream of instructions requested of the SDK to be performed.
stream InstructionRequest) {}
+
+ rpc GetProcessBundleDescriptor(GetProcessBundleDescriptorRequest) returns (
Review comment:
The current Control must be a stream because (1) it is server-initiated
requests on a client-initiated connection and (2) in streaming pipelines, it we
want it to be high-volume, low-latency, possibly pipelined. However, using
streams in this way vs. "standard" RPC methods comes at a cost (in both code
and conceptual complexity).
As far as ordering, it is preferable to have a protocol that is not
sensitive to this kind of thing. (In fact, this eliminates the one need for
ordering in our Control stream, where bundles needed to be registered before
they were used (and this didn't fit well with pattern of dispatching
instructions to threadpool-backed handlers in the SDKs which required extra
locking/waiting anyway). Progress and splitting requests were already
asynchronous.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
