[
https://issues.apache.org/jira/browse/BEAM-3759?focusedWorklogId=81742&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81742
]
ASF GitHub Bot logged work on BEAM-3759:
----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Mar/18 06:43
Start Date: 19/Mar/18 06:43
Worklog Time Spent: 10m
Work Description: charlesccychen commented on a change in pull request
#4763: [BEAM-3759] Add support for PaneInfo in WindowedValues
URL: https://github.com/apache/beam/pull/4763#discussion_r175341749
##########
File path: sdks/python/apache_beam/utils/windowed_value.py
##########
@@ -32,6 +32,107 @@
from apache_beam.utils.timestamp import Timestamp
+class PaneInfoTiming(object):
+ """The timing of a PaneInfo."""
+
+ EARLY = 0
+ ON_TIME = 1
+ LATE = 2
+ UNKNOWN = 3
+
+
+class PaneInfo(object):
+ """Describes the trigger firing information for a given WindowedValue."""
+
+ def __init__(self, is_first, is_last, timing, index, nonspeculative_index):
+ self._is_first = is_first
+ self._is_last = is_last
+ self._timing = timing
+ self._index = index
+ self._nonspeculative_index = nonspeculative_index
+ self._encoded_byte = self._get_encoded_byte()
+
+ def _get_encoded_byte(self):
+ byte = 0
+ if self.is_first:
+ byte |= 1
+ if self.is_last:
+ byte |= 2
+ byte |= self.timing << 2
+ return byte
+
+ @staticmethod
+ def from_encoded_byte(encoded_byte):
+ assert encoded_byte in _BYTE_TO_PANE_INFO
+ return _BYTE_TO_PANE_INFO[encoded_byte]
+
+ # Because common PaneInfo objects are cached, it is important that the value
+ # is immutable. We therefore explicitly enforce this here with read-only
+ # properties.
+
+ @property
+ def is_first(self):
+ return self._is_first
+
+ @property
+ def is_last(self):
+ return self._is_last
+
+ @property
+ def timing(self):
+ return self._timing
+
+ @property
+ def index(self):
+ return self._index
+
+ @property
+ def nonspeculative_index(self):
+ return self._nonspeculative_index
+
+ @property
+ def encoded_byte(self):
+ return self._encoded_byte
+
+ def __repr__(self):
+ return ('PaneInfo(first: %r, last: %r, timing: %s, index: %d, '
+ 'nonspeculative_index: %d)') % (self.is_first, self.is_last,
+ self.timing, self.index,
+ self.nonspeculative_index)
+
+ def __eq__(self, other):
+ if self is other:
+ return True
+ return (self.is_first == other.is_first and
+ self.is_last == other.is_last and
+ self.timing == other.timing and
+ self.index == other.index and
+ self.nonspeculative_index == other.nonspeculative_index)
+
+
+def _construct_pane_info_map():
+ result = {}
Review comment:
<!--thread_id:cc_173303970_t;
commit:aa1b931750aa409e1d26e603fccd0fd82b9484da; resolved:1-->
<!--section:context-quote-->
> **robertwb** wrote:
> Make this a list, not a map.
<!--section:body-->
Done.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 81742)
Time Spent: 2h 40m (was: 2.5h)
> Add support for PaneInfo descriptor in Python SDK
> -------------------------------------------------
>
> Key: BEAM-3759
> URL: https://issues.apache.org/jira/browse/BEAM-3759
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
> Affects Versions: 2.3.0
> Reporter: Charles Chen
> Assignee: Charles Chen
> Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> The PaneInfo descriptor allows a user to determine which particular
> triggering emitted a value. This allows the user to differentiate between
> speculative (early), on-time (at end of window) and late value emissions
> coming out of a GroupByKey. We should add support for this feature in the
> Python SDK.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
