[
https://issues.apache.org/jira/browse/BEAM-7951?focusedWorklogId=354602&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-354602
]
ASF GitHub Bot logged work on BEAM-7951:
----------------------------------------
Author: ASF GitHub Bot
Created on: 05/Dec/19 19:05
Start Date: 05/Dec/19 19:05
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #9979: [BEAM-7951]
Allow runner to configure customization WindowedValue coder.
URL: https://github.com/apache/beam/pull/9979#discussion_r354492422
##########
File path: sdks/python/apache_beam/coders/coders.py
##########
@@ -1136,6 +1136,62 @@ def __hash__(self):
common_urns.coders.WINDOWED_VALUE.urn, WindowedValueCoder)
+class ParamWindowedValueCoder(WindowedValueCoder):
+ """A coder used for parameterized Windowed values."""
+
+ def __init__(self, payload, components):
+ super(ParamWindowedValueCoder, self).__init__(components[0], components[1])
+ self._wrapped_value_coder = components[0]
+ self.window_coder = components[1]
+ self.payload = payload
+ self.components = components
+
+ def _create_impl(self):
+ return coder_impl.ParamWindowedValueCoderImpl(
+ self._wrapped_value_coder.get_impl(),
+ self.window_coder.get_impl(),
+ self.payload)
+
+ def is_deterministic(self):
+ return all(c.is_deterministic() for c in [self._wrapped_value_coder,
+ self.window_coder])
+
+ def as_cloud_object(self, coders_context=None):
Review comment:
This is meant to support Dataflow explicitly and is similar to the proto
conversion.
You'll need to add one to java in
[CloudObjectTranslators](https://github.com/apache/beam/blob/a06b7a6cbad2386b7bee37d3be0d7f3f5b8bfaff/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/CloudObjectTranslators.java#L244)
and wire it together.
You'll need to add the encoded payload as part of the cloud object as well
and I suggest you add it as a 'payload' field. Please use the same byte string
-> json compatible string conversion found elsewhere in these as_cloud_object
conversions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 354602)
> Allow runner to configure customization WindowedValue coder such as
> ValueOnlyWindowedValueCoder
> -----------------------------------------------------------------------------------------------
>
> Key: BEAM-7951
> URL: https://issues.apache.org/jira/browse/BEAM-7951
> Project: Beam
> Issue Type: Sub-task
> Components: java-fn-execution
> Reporter: sunjincheng
> Assignee: sunjincheng
> Priority: Major
> Time Spent: 5h
> Remaining Estimate: 0h
>
> The coder of WindowedValue cannot be configured and it’s always
> FullWindowedValueCoder. We don't need to serialize the timestamp, window and
> pane properties in Flink and so it will be better to make the coder
> configurable (i.e. allowing to use ValueOnlyWindowedValueCoder)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
