[
https://issues.apache.org/jira/browse/BEAM-7951?focusedWorklogId=364868&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-364868
]
ASF GitHub Bot logged work on BEAM-7951:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Dec/19 05:40
Start Date: 31/Dec/19 05:40
Worklog Time Spent: 10m
Work Description: sunjincheng121 commented on pull request #10464:
[BEAM-7951] Supports multiple inputs/outputs for wire coder settings.
URL: https://github.com/apache/beam/pull/10464#discussion_r362151892
##########
File path: model/pipeline/src/main/proto/beam_runner_api.proto
##########
@@ -1362,4 +1347,27 @@ message ExecutableStagePayload {
// (Required) The local name of this timer for the PTransform that
references it.
string local_name = 2;
}
+
+ // Settings that decide the coder type of wire coder.
+ message WireCoderSetting {
+ // (Required) The URN of the wire coder.
+ // Note that only windowed value coder or parameterized windowed value
coder are supported.
+ string urn = 1;
+
+ // (Optional) The data specifying any parameters to the URN. If
+ // the URN is beam:coder:windowed_value:v1, this may be omitted. If the
URN is
+ // beam:coder:param_windowed_value:v1, the payload is an encoded windowed
+ // value using the beam:coder:windowed_value:v1 coder parameterized by
+ // a beam:coder:bytes:v1 element coder and the window coder that this
+ // param_windowed_value coder uses.
+ bytes payload = 2;
+
+ // (Required) The target(PCollection or Timer) this setting applies to.
+ oneof target {
Review comment:
Do you think we should add the WireCoderSetting property for each input and
output edge explicitly in the proto?
I was thinking the setting is not a necessary properties for each edge so
added the wire_coder_settings as an ExecutableStage level property. The
wire_coder_settings can be an empty list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 364868)
Time Spent: 13h (was: 12h 50m)
> Allow runner to configure customization WindowedValue coder such as
> ValueOnlyWindowedValueCoder
> -----------------------------------------------------------------------------------------------
>
> Key: BEAM-7951
> URL: https://issues.apache.org/jira/browse/BEAM-7951
> Project: Beam
> Issue Type: Sub-task
> Components: java-fn-execution
> Reporter: sunjincheng
> Assignee: sunjincheng
> Priority: Major
> Time Spent: 13h
> Remaining Estimate: 0h
>
> The coder of WindowedValue cannot be configured and it’s always
> FullWindowedValueCoder. We don't need to serialize the timestamp, window and
> pane properties in Flink and so it will be better to make the coder
> configurable (i.e. allowing to use ValueOnlyWindowedValueCoder)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
