I hit this issue. Your diff seems to work.

*Fredy Wijaya* | Software Engineer
e. fwij...@cloudera.com
cloudera.com <https://www.cloudera.com>

[image: Cloudera] <https://www.cloudera.com/>

[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
------------------------------

On Mon, Apr 16, 2018 at 12:54 PM, Philip Zeyliger <phi...@cloudera.com>
wrote:

> I've only seen this once so far, but I ran into the following. The way our
> pip_download.py script works, it doesn't follow redirects.
>
> I'm testing the two-line diff below.
>
>
>
> 2018-04-16 10:27:32.113099 Getting package info from
> https://pypi.python.org/simple/allpairs/
> 2018-04-16 10:27:32.113258 Downloading AllPairs-2.0.1.tar.gz from
> https://pypi.python.org/packages/04/44/cb85d029b33ddfa7b5a27ae523434c
> 2230f11a89d1f0c73203e9007a6eb0/AllPairs-2.0.1.tar.gz
> 2018-04-16 10:27:32.113320 ('http error', 302, 'Found',
> <httplib.HTTPMessage instance at 0x7f180a42a128>)
> 2018-04-16 10:27:32.113359 Sleeping for 15 seconds before retrying
>
>
>
> diff --git infra/python/deps/pip_download.py
> infra/python/deps/pip_download.py
> index 6fbb683..0cce9e9 100755
> --- infra/python/deps/pip_download.py
> +++ infra/python/deps/pip_download.py
> @@ -28,7 +28,7 @@ import re
>  import sys
>  from random import randint
>  from time import sleep
> -from urllib import urlopen, URLopener
> +from urllib import urlopen, FancyURLopener
>
>  NUM_DOWNLOAD_ATTEMPTS = 8
>
> @@ -114,7 +114,7 @@ def download_package(pkg_name, pkg_version):
>        expected_digest):
>      print 'File with matching digest already exists, skipping
> {0}'.format(file_name)
>      return True
> -  downloader = URLopener()
> +  downloader = FancyURLopener()
>    pkg_url = '{0}/packages/{1}'.format(PYPI_MIRROR, path)
>    print 'Downloading {0} from {1}'.format(file_name, pkg_url)
>    downloader.retrieve(pkg_url, file_name)
>

Reply via email to