[
https://issues.apache.org/jira/browse/BEAM-7473?focusedWorklogId=344732&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-344732
]
ASF GitHub Bot logged work on BEAM-7473:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Nov/19 00:53
Start Date: 16/Nov/19 00:53
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10118: [BEAM-7473]
Pack RangeTracker into restriction
URL: https://github.com/apache/beam/pull/10118#discussion_r347064287
##########
File path: sdks/python/apache_beam/io/iobase.py
##########
@@ -1487,10 +1497,10 @@ def split(self, element, restriction):
# Invoke source.split to get initial splitting results.
source_bundles = self._source.split(self._desired_chunk_size)
for source_bundle in source_bundles:
- yield source_bundle
+ yield (source_bundle, None)
def restriction_size(self, element, restriction):
- return restriction.weight
+ return restriction[0].weight
def restriction_coder(self):
return coders.DillCoder()
Review comment:
Thinking about this some more, I think it'd be safer and cleaner to create
an actual _SDFBoundedSourceRestriction class that holds the SourceBundle and
the (lazily-initialized) underlying restriction rather than using a tuple. Its
`__reduce__` method could return only the former, and its constructor only take
the former. Many of the methods here would simply delegate to it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 344732)
Time Spent: 20m (was: 10m)
> Update RestrictionTracker within Python to not be required to be thread safe
> ----------------------------------------------------------------------------
>
> Key: BEAM-7473
> URL: https://issues.apache.org/jira/browse/BEAM-7473
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core, sdk-py-harness
> Reporter: Luke Cwik
> Assignee: Boyuan Zhang
> Priority: Major
> Fix For: 2.17.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The commit
> [https://github.com/apache/beam/commit/8faffb2bcf28ccab5e9a95322743cc60df65077c#diff-ed95abb6bc30a9ed07faef5c3fea93f0]
> modified the Java SDK removed the need for users to be thread safe and
> instead made the framework provide the necessary locking around a restriction
> tracker.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
