Philip Zeyliger has posted comments on this change. ( http://gerrit.cloudera.org:8080/10234 )
Change subject: Speed up Python dependencies. ...................................................................... Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/10234/1/infra/python/deps/pip_download.py File infra/python/deps/pip_download.py: http://gerrit.cloudera.org:8080/#/c/10234/1/infra/python/deps/pip_download.py@165 PS1, Line 165: results.append(pool.apply_async(download_package, args=[pkg_name.strip(), pkg_version.strip()])) > You can use "git diff HEAD^ | impala-flake8 --diff" and similar patterns to Done http://gerrit.cloudera.org:8080/#/c/10234/1/infra/python/deps/pip_download.py@173 PS1, Line 173: x.get() > What happens if the download fails for some reason? Will this be detected? Yes, it works correctly, I believe. >>> from multiprocessing.pool import ThreadPool >>> p = ThreadPool() >>> def f(a): ... raise Exception("hey") ... >>> x = p.apply_async(f, [1]) >>> x.get() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/philip/anaconda/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value Exception: hey -- To view, visit http://gerrit.cloudera.org:8080/10234 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7cbf622adb7d037f1a53c519402dcd8ae3c0fe30 Gerrit-Change-Number: 10234 Gerrit-PatchSet: 1 Gerrit-Owner: Philip Zeyliger <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Brown <[email protected]> Gerrit-Reviewer: Philip Zeyliger <[email protected]> Gerrit-Reviewer: Taras Bobrovytsky <[email protected]> Gerrit-Comment-Date: Fri, 27 Apr 2018 20:17:03 +0000 Gerrit-HasComments: Yes
