[
https://issues.apache.org/jira/browse/BEAM-3759?focusedWorklogId=81744&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81744
]
ASF GitHub Bot logged work on BEAM-3759:
----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Mar/18 06:43
Start Date: 19/Mar/18 06:43
Worklog Time Spent: 10m
Work Description: charlesccychen commented on a change in pull request
#4763: [BEAM-3759] Add support for PaneInfo in WindowedValues
URL: https://github.com/apache/beam/pull/4763#discussion_r175341743
##########
File path: sdks/python/apache_beam/coders/coders.py
##########
@@ -878,6 +877,27 @@ def __hash__(self):
common_urns.INTERVAL_WINDOW_CODER, IntervalWindowCoder)
+class PaneInfoCoder(FastCoder):
+ """Coder for an PaneInfo descriptor."""
+
+ def _create_impl(self):
+ return coder_impl.PaneInfoCoderImpl()
+
+ def is_deterministic(self):
+ return True
+
+ def as_cloud_object(self):
+ raise NotImplementedError(
+ 'The PaneInfoCoder should not be explicitly constructed in a pipeline '
+ 'graph.')
+
+ def __eq__(self, other):
+ return type(self) == type(other)
+
+ def __hash__(self):
+ return hash(type(self))
+
+
Review comment:
<!--thread_id:cc_173305213_t;
commit:aa1b931750aa409e1d26e603fccd0fd82b9484da; resolved:1-->
<!--section:context-quote-->
> **robertwb** wrote:
> Rather than introducing a PaneInfoCoder, just instantiate the
PaneInfoCoderImpl in WindowedValueCoderImpl's constructor. (There's a TODO to
remove the parameterizablility of the TimestampCoder as well.) Actually, we
could consider just making {en,de}code_pane_info methods on WindowedValueCoder
itself.
<!--section:body-->
I am going to get rid of PaneInfoCoder but keep the PaneInfoCoderImpl,
because it's more natural to keep it separate, as it would clutter the
WindowedValueCoderImpl with encode/decode_paneinfo_to_stream and
estimate_paneinfo_size.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 81744)
Time Spent: 3h (was: 2h 50m)
> Add support for PaneInfo descriptor in Python SDK
> -------------------------------------------------
>
> Key: BEAM-3759
> URL: https://issues.apache.org/jira/browse/BEAM-3759
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Affects Versions: 2.3.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> The PaneInfo descriptor allows a user to determine which particular
> triggering emitted a value. This allows the user to differentiate between
> speculative (early), on-time (at end of window) and late value emissions
> coming out of a GroupByKey. We should add support for this feature in the
> Python SDK.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
