ASF GitHub Bot logged work on BEAM-3741:

                Author: ASF GitHub Bot
            Created on: 13/Mar/18 18:30
            Start Date: 13/Mar/18 18:30
    Worklog Time Spent: 10m 
      Work Description: lukecwik commented on a change in pull request #4743: 
[BEAM-3741] Proto changes for splitting over Fn API
URL: https://github.com/apache/beam/pull/4743#discussion_r174239903

 File path: model/fn-execution/src/main/proto/beam_fn_api.proto
 @@ -182,6 +182,48 @@ message ProcessBundleDescriptor {
state_api_service_descriptor = 7;
+// Represents a partition of the bundle into two bundles: a "primary" and
+// a "residual", with the following properties:
+// - The work in primary and residual doesn't overlap, and combined, adds up
+//   to the work in the current bundle if the split hadn't happened.
+// - The current bundle, if it keeps executing, will have done none of the
+//   work under residual roots.
+// - The current bundle, if no further splits happen, will have done exactly
+//   the work under primary_roots.
+// For more rigorous definitions see https://s.apache.org/beam-breaking-fusion
+message BundleSplit {
+  // One of the root applications specifying the scope of work for a bundle.
+  message Application {
+    // (Required) The primitive transform to which to pass the element
+    string ptransform_id = 1;
+    // (Required) Name of the transform's input to which to pass the element.
+    string input_id = 2;
+    // (Required) The encoded element to pass to the transform.
+    bytes element = 3;
+    // Approximate lower bounds on timestamps of elements that this PTransform
+    // will produce into each of its output PCollections, when invoked on this
+    // element. Keyed by the transform's local output name.
+    map<string, int64> output_watermarks = 4;
 Review comment:
   The issue I see is that if you perform a split, how do you hold the input 
watermark of these applications so that state/side inputs aren't garbage 
collected, timers don't fire too early (watermark based timers).
   It may be that we need both or we could say that the input watermark is 
always a lower bound for the output watermarks.
   The change looks fine for now but I think this is an important point to 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

Issue Time Tracking

    Worklog Id:     (was: 80002)
    Time Spent: 4h  (was: 3h 50m)

> Proto changes for splitting over Fn API
> ---------------------------------------
>                 Key: BEAM-3741
>                 URL: https://issues.apache.org/jira/browse/BEAM-3741
>             Project: Beam
>          Issue Type: Sub-task
>          Components: beam-model
>            Reporter: Eugene Kirpichov
>            Assignee: Eugene Kirpichov
>            Priority: Major
>          Time Spent: 4h
>  Remaining Estimate: 0h

This message was sent by Atlassian JIRA

Reply via email to