tvalentyn commented on a change in pull request #14137:
URL: https://github.com/apache/beam/pull/14137#discussion_r589853882
##########
File path:
sdks/python/apache_beam/runners/portability/fn_api_runner/worker_handlers.py
##########
@@ -750,48 +745,47 @@ def host_from_worker(self):
def start_worker(self):
# type: () -> None
- with SUBPROCESS_LOCK:
- try:
- _LOGGER.info('Attempting to pull image %s', self._container_image)
- subprocess.check_call(['docker', 'pull', self._container_image])
- except Exception:
- _LOGGER.info(
- 'Unable to pull image %s, defaulting to local image if it exists' %
- self._container_image)
- self._container_id = subprocess.check_output([
- 'docker',
- 'run',
- '-d',
- # TODO: credentials
- '--network=host',
- self._container_image,
- '--id=%s' % self.worker_id,
- '--logging_endpoint=%s' % self.logging_api_service_descriptor().url,
- '--control_endpoint=%s' % self.control_address,
- '--artifact_endpoint=%s' % self.control_address,
- '--provision_endpoint=%s' % self.control_address,
+ try:
+ _LOGGER.info('Attempting to pull image %s', self._container_image)
+ subprocess.check_call(['docker', 'pull', self._container_image])
Review comment:
I think it makes sense to keep the lock to avoid concurrent pulls of the
same image.
In general, it's better to separate such potentially non-trivial changes in
separate PRs, but thanks for putting it in it's own commit.
##########
File path:
sdks/python/apache_beam/runners/interactive/interactive_environment.py
##########
@@ -172,12 +171,7 @@ def __init__(self):
self._computed_pcolls = set()
# Always watch __main__ module.
self.watch('__main__')
- # Do a warning level logging if current python version is below 3.6.
- if sys.version_info < (3, 6):
- self._is_py_version_ready = False
- _LOGGER.warning('Interactive Beam requires Python 3.5.3+.')
- else:
- self._is_py_version_ready = True
+ self._is_py_version_ready = True
Review comment:
We can probably let go of this variable in another change.
##########
File path:
sdks/python/apache_beam/runners/interactive/display/pcoll_visualization_test.py
##########
@@ -20,7 +20,6 @@
from __future__ import absolute_import
Review comment:
Just a note - we can let go of these imports from __future__ in another
change.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
