Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-5181: Extract PYPI metadata from a webpage ......................................................................
IMPALA-5181: Extract PYPI metadata from a webpage There were some build failures due to a failure to download a JSON file containing package metadata from PYPI. We need to switch to downloading this from a PYPI mirror. In order to be able to download the metadata from a PYPI mirror, we need be able to extract the data from a web page, because PYPI mirrors do not always have a JSON interface. We implement a regex based html parser in this patch. Also, we increase the number of download attempts and randomly vary the amount of time between each attempt. Testing: - Tested locally against PYPI and a PYPI mirror. - Ran a private build that passed (which used a PYPI mirror). Change-Id: If3845a0d5f568d4352e3cc4883596736974fd7de Reviewed-on: http://gerrit.cloudera.org:8080/6579 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Impala Public Jenkins --- M infra/python/deps/pip_download.py 1 file changed, 57 insertions(+), 33 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/6579 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: If3845a0d5f568d4352e3cc4883596736974fd7de Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Taras Bobrovytsky <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: David Knupp <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Taras Bobrovytsky <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
