[
https://issues.apache.org/jira/browse/BEAM-7951?focusedWorklogId=349368&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-349368
]
ASF GitHub Bot logged work on BEAM-7951:
----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Nov/19 21:25
Start Date: 25/Nov/19 21:25
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #9979: [BEAM-7951]
Allow runner to configure customization WindowedValue coder.
URL: https://github.com/apache/beam/pull/9979#discussion_r350426810
##########
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##########
@@ -1338,3 +1359,44 @@ message ExecutableStagePayload {
string local_name = 2;
}
}
+
+// Window information in WindowedValue
+message WindowInfo {
+ // timestamp in milliseconds
+ int64 timestamp = 1;
+ repeated BoundedWindow bounded_windows = 2;
+ PaneInfo pane_info = 3;
+}
+
+// Represents window information assigned to data elements
+message BoundedWindow {
Review comment:
This limits the set of window types and the number of windows that can be
specified. I think we should go with a model where we encode a "dummy element"
which is replaced by the actual element when decoding. For example, the payload
would be:
```
message ParameterizedWindowedValuePayload {
// A specification of a windowed value coder where the element coder is
always "beam:coder:bytes:v1".
string coder_id;
// Contains an encoded windowed value with an empty byte[] element. This
elements windowing
// information should be copied to all elements that are decoded with this
coder.
bytes value;
}
```
This allows us to use the full range of window encodings and the already
defined pane/timestamp encodings.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 349368)
Time Spent: 2h 50m (was: 2h 40m)
> Allow runner to configure customization WindowedValue coder such as
> ValueOnlyWindowedValueCoder
> -----------------------------------------------------------------------------------------------
>
> Key: BEAM-7951
> URL: https://issues.apache.org/jira/browse/BEAM-7951
> Project: Beam
> Issue Type: Sub-task
> Components: java-fn-execution
> Reporter: sunjincheng
> Assignee: sunjincheng
> Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> The coder of WindowedValue cannot be configured and it’s always
> FullWindowedValueCoder. We don't need to serialize the timestamp, window and
> pane properties in Flink and so it will be better to make the coder
> configurable (i.e. allowing to use ValueOnlyWindowedValueCoder)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
