[
https://issues.apache.org/jira/browse/BEAM-9577?focusedWorklogId=412784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412784
]
ASF GitHub Bot logged work on BEAM-9577:
----------------------------------------
Author: ASF GitHub Bot
Created on: 31/Mar/20 02:35
Start Date: 31/Mar/20 02:35
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #11203: [BEAM-9577]
Define and implement dependency-aware artifact staging service.
URL: https://github.com/apache/beam/pull/11203#discussion_r400604546
##########
File path: sdks/python/apache_beam/runners/portability/artifact_service.py
##########
@@ -284,12 +284,38 @@ def _open(self, path, mode='r'):
# The dependency-aware artifact staging and retrieval services.
-def _queue_iter(queue, end_token):
- while True:
- item = queue.get()
- if item is end_token:
- break
- yield item
+class _QueueIter(object):
+
+ _END = object()
+
+ def __init__(self):
+ self._queue = queue.Queue()
+
+ def put(self, item):
+ self._queue.put(item)
+
+ def done(self):
+ self._queue.put(self._END)
+ self._queue.put(StopIteration)
+
+ def abort(self, exn=None):
+ if exn is None:
+ exn = sys.exc_info()[1]
+ self._queue.put(self._END)
+ self._queue.put(exn)
+
+ def __iter__(self):
+ return self
+
+ def __next__(self):
+ item = self._queue.get()
Review comment:
so clever
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 412784)
Time Spent: 6h (was: 5h 50m)
> Update artifact staging and retrieval protocols to be dependency aware.
> -----------------------------------------------------------------------
>
> Key: BEAM-9577
> URL: https://issues.apache.org/jira/browse/BEAM-9577
> Project: Beam
> Issue Type: Improvement
> Components: beam-model
> Reporter: Robert Bradshaw
> Assignee: Robert Bradshaw
> Priority: Major
> Time Spent: 6h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
