robertwb commented on a change in pull request #11270: [BEAM-9639][BEAM-9608]
Improvements for FnApiRunner
URL: https://github.com/apache/beam/pull/11270#discussion_r407663602
##########
File path:
sdks/python/apache_beam/runners/portability/fn_api_runner/execution.py
##########
@@ -367,6 +413,73 @@ def _build_process_bundle_descriptor(self):
state_api_service_descriptor=self.state_api_service_descriptor(),
timer_api_service_descriptor=self.data_api_service_descriptor())
+ def commit_output_views_to_state(self):
+ """Commit bundle outputs to state to be consumed as side inputs later.
+
+ Only the outputs that should be side inputs are committed to state.
+ """
+ data_side_input = {} # type: DataSideInput
+ for pcoll, si_ids in self.stage.downstream_side_inputs.items():
+ for (consumer_transform_name, tag), access_pattern in si_ids.items():
+ data_side_input[consumer_transform_name, tag] = (
+ translations.create_buffer_id(pcoll), access_pattern)
+ self.execution_context.commit_side_inputs_to_state(data_side_input)
+
+ def extract_bundle_inputs(self):
Review comment:
extract_bundle_inputs_and_outputs?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
