Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/22128 )
Change subject: IMPALA-13585: Make pip_download.py interruptible ...................................................................... IMPALA-13585: Make pip_download.py interruptible infra/python/deps/pip_download.py use multiprocessing.pool.ThreadPool where each thread calls wget. It is also wrap the download_package function with retry wrapper. When there is a network issue happen, pressing Ctrl+C does not immediately terminate pip_download.py and all its children. Thus, the script appears to hang. This patch make pip_download.py to capture SIGINT and pass it as cancellation event to all threads. It is changed to run with python3. All flake8 issues are also fixed. Testing: - Manually run `buildall.sh -cmake_only` and interrupt it in the middle of pip_download.py execution. Verify that script terminate immediately. Change-Id: I6f293dd8f3fcf3cffa17a4a44627a41d67b7dc91 Reviewed-on: http://gerrit.cloudera.org:8080/22128 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M infra/python/deps/pip_download.py 1 file changed, 35 insertions(+), 14 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/22128 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I6f293dd8f3fcf3cffa17a4a44627a41d67b7dc91 Gerrit-Change-Number: 22128 Gerrit-PatchSet: 3 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Jason Fehr <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
