** Also affects: python-pip (Ubuntu Bionic) Importance: Undecided Status: New
** Description changed: + [Impact] + + * The --extra-index-url feature is not working when an index doesn't + contain all the packages in the dependency set. + + [Test Plan] + # apt install python3-venv + # python3 -m venv /tmp/test3env + # /tmp/test3env/bin/pip install wheel + # /tmp/test3env/bin/pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple pytopo==1.6.1 + + A 404 error is the failure. + A successful install is success. + + [Where problems could occur] + + * Changes were cherry-picked from upstream trunk, where they are still present. + * It's entirely possible that other de-bundling bugs will be fixed by the same changes. + + [Other Info] + + * https://github.com/pypa/pip/pull/6113 has a good explanation of the bug. + + [Original Bug Report] + I originally wrote this as a comment on https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1822842 , but I guess it makes sense to open a new separate bug since the other report was already closed. Pasting my comment contents below. - ~$ lsb_release -rd - Description: Ubuntu 18.04.2 LTS - Release: 18.04 - - ~$ apt-cache policy python3-pip - python3-pip: - Installed: 9.0.1-2.3~ubuntu1.18.04.1 - Candidate: 9.0.1-2.3~ubuntu1.18.04.1 - Version table: - *** 9.0.1-2.3~ubuntu1.19.04.1 500 - 500 http://se.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 packages - 500 http://se.archive.ubuntu.com/ubuntu bionic-updates/universe i386 packages - 100 /var/lib/dpkg/status - 9.0.1-2 500 - 500 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 packages - 500 http://se.archive.ubuntu.com/ubuntu bionic/universe i386 packages + ~$ lsb_release -rd + Description: Ubuntu 18.04.2 LTS + Release: 18.04 + + ~$ apt-cache policy python3-pip + python3-pip: + Installed: 9.0.1-2.3~ubuntu1.18.04.1 + Candidate: 9.0.1-2.3~ubuntu1.18.04.1 + Version table: + *** 9.0.1-2.3~ubuntu1.19.04.1 500 + 500 http://se.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 packages + 500 http://se.archive.ubuntu.com/ubuntu bionic-updates/universe i386 packages + 100 /var/lib/dpkg/status + 9.0.1-2 500 + 500 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 packages + 500 http://se.archive.ubuntu.com/ubuntu bionic/universe i386 packages --- It seems like `9.0.1-2.3~ubuntu1.18.04.1` breaks the `--extra-index-url` functionality of PIP. In my understanding, the idea is for `--extra-index-url` to provide a PyPI-compliant repository that offers a few additional packages, allowing PIP to fall back onto the instance configured as `--index-url` (defaults to upstream PyPI). This has been the case with earlier versions, allowing us to host an internal PyPI repository containing a subset of packages, as well as a generic local caching PyPI mirror. In `9.0.1-2.3~ubuntu1.18.04.1`, it seems like the fallback mechanisms do not work properly, instead failing if _either_ the `--index-url` _or_ `--extra-index-url` instances lack the package. --- With `~/.config/pip/pip.conf` as: - [global] - index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple - extra-index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_extra/simple + [global] + index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple + extra-index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_extra/simple With `python3-pip` version `9.0.1-2.3~ubuntu1.18.04.1`, I get a session like: - ~$ python3 -m venv env/pip-latest - ~$ . env/pip-latest/bin/activate - (pip-latest) ~$ pip --version - pip 9.0.1 from /home/dandersson/env/pip-latest/lib/python3.6/site-packages (python 3.6) - (pip-latest) ~$ pip install hpt - Collecting hpt - Exception: - Traceback (most recent call last): - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main - status = self.run(options, args) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run - wb.build(autobuilding=True) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build - self.requirement_set.prepare_files(self.finder) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files - ignore_dependencies=self.ignore_dependencies)) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file - require_hashes - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link - self.link = finder.find_requirement(self, upgrade) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement - all_candidates = self.find_all_candidates(req.name) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates - for page in self._get_pages(url_locations, project_name): - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages - page = self._get_page(location) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page - return HTMLPage.get_page(link, session=self.session) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page - resp.raise_for_status() - File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status - raise HTTPError(http_error_msg, response=self) - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple/hpt/ + ~$ python3 -m venv env/pip-latest + ~$ . env/pip-latest/bin/activate + (pip-latest) ~$ pip --version + pip 9.0.1 from /home/dandersson/env/pip-latest/lib/python3.6/site-packages (python 3.6) + (pip-latest) ~$ pip install hpt + Collecting hpt + Exception: + Traceback (most recent call last): + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main + status = self.run(options, args) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run + wb.build(autobuilding=True) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build + self.requirement_set.prepare_files(self.finder) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files + ignore_dependencies=self.ignore_dependencies)) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file + require_hashes + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link + self.link = finder.find_requirement(self, upgrade) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement + all_candidates = self.find_all_candidates(req.name) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates + for page in self._get_pages(url_locations, project_name): + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages + page = self._get_page(location) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page + return HTMLPage.get_page(link, session=self.session) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page + resp.raise_for_status() + File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status + raise HTTPError(http_error_msg, response=self) + requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple/hpt/ PIP fails with a traceback due to not finding `hpt` on the `--index-url` instance -- but the point of giving `--extra-index-url` is that `hpt` resides on that instance. Trying to install a package that should be present on the `--index-url` instance: - (pip-latest) ~$ pip install requests - Collecting requests - Exception: - Traceback (most recent call last): - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main - status = self.run(options, args) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run - wb.build(autobuilding=True) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build - self.requirement_set.prepare_files(self.finder) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files - ignore_dependencies=self.ignore_dependencies)) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file - require_hashes - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link - self.link = finder.find_requirement(self, upgrade) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement - all_candidates = self.find_all_candidates(req.name) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates - for page in self._get_pages(url_locations, project_name): - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages - page = self._get_page(location) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page - return HTMLPage.get_page(link, session=self.session) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page - resp.raise_for_status() - File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status - raise HTTPError(http_error_msg, response=self) - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_extra/simple/requests/ + (pip-latest) ~$ pip install requests + Collecting requests + Exception: + Traceback (most recent call last): + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main + status = self.run(options, args) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run + wb.build(autobuilding=True) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build + self.requirement_set.prepare_files(self.finder) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files + ignore_dependencies=self.ignore_dependencies)) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file + require_hashes + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link + self.link = finder.find_requirement(self, upgrade) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement + all_candidates = self.find_all_candidates(req.name) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates + for page in self._get_pages(url_locations, project_name): + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages + page = self._get_page(location) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page + return HTMLPage.get_page(link, session=self.session) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page + resp.raise_for_status() + File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status + raise HTTPError(http_error_msg, response=self) + requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_extra/simple/requests/ Now it fails with a traceback since `requests` does not exist on the `--extra-index-url` instance, but the purpose of that instance is to provide a small subset of extra packages, not to be a full mirror. Changing `~/.config/pip/pip.conf` to: - [global] - index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple + [global] + index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple removing the `--extra-index-url` instance altogether, it behaves as expected, not finding the `hpt` package, but being able to install `requests`: - (pip-latest) ~$ pip install hpt - Collecting hpt - Exception: - Traceback (most recent call last): - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main - status = self.run(options, args) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run - wb.build(autobuilding=True) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build - self.requirement_set.prepare_files(self.finder) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files - ignore_dependencies=self.ignore_dependencies)) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file - require_hashes - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link - self.link = finder.find_requirement(self, upgrade) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement - all_candidates = self.find_all_candidates(req.name) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates - for page in self._get_pages(url_locations, project_name): - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages - page = self._get_page(location) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page - return HTMLPage.get_page(link, session=self.session) - File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page - resp.raise_for_status() - File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status - raise HTTPError(http_error_msg, response=self) - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple/hpt/ + (pip-latest) ~$ pip install hpt + Collecting hpt + Exception: + Traceback (most recent call last): + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main + status = self.run(options, args) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/commands/install.py", line 353, in run + wb.build(autobuilding=True) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/wheel.py", line 749, in build + self.requirement_set.prepare_files(self.finder) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 380, in prepare_files + ignore_dependencies=self.ignore_dependencies)) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_set.py", line 554, in _prepare_file + require_hashes + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/req/req_install.py", line 278, in populate_link + self.link = finder.find_requirement(self, upgrade) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 465, in find_requirement + all_candidates = self.find_all_candidates(req.name) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 423, in find_all_candidates + for page in self._get_pages(url_locations, project_name): + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 568, in _get_pages + page = self._get_page(location) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 683, in _get_page + return HTMLPage.get_page(link, session=self.session) + File "/home/dandersson/env/pip-latest/lib/python3.6/site-packages/pip/index.py", line 795, in get_page + resp.raise_for_status() + File "/home/dandersson/env/pip-latest/share/python-wheels/requests-2.18.4-py2.py3-none-any.whl/requests/models.py", line 935, in raise_for_status + raise HTTPError(http_error_msg, response=self) + requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple/hpt/ It is expected to not be able to install `hpt`, but PIP should not fail with a traceback, but just with a diagnostic message like `No matching distribution found for hpt`. Installing a common package will correctly use the `--index-url` instance: - (pip-latest) ~$ pip install requests - Collecting requests - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl - Collecting idna<2.9,>=2.5 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl - Collecting certifi>=2017.4.17 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl - Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl - Collecting chardet<3.1.0,>=3.0.2 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl - Installing collected packages: idna, certifi, urllib3, chardet, requests - Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3 + (pip-latest) ~$ pip install requests + Collecting requests + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl + Collecting idna<2.9,>=2.5 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl + Collecting certifi>=2017.4.17 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl + Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl + Collecting chardet<3.1.0,>=3.0.2 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl + Installing collected packages: idna, certifi, urllib3, chardet, requests + Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3 --- Downgrading `python3-pip` to `9.0.1-2`: - ~$ sudo apt install python3-pip=9.0.1-2 python-pip-whl=9.0.1-2 - Reading package lists... Done - Building dependency tree - Reading state information... Done - The following packages will be DOWNGRADED: - python-pip-whl python3-pip - 0 upgraded, 0 newly installed, 2 downgraded, 0 to remove and 5 not upgraded. - Need to get 1 493 kB of archives. - After this operation, 226 kB disk space will be freed. - Do you want to continue? [Y/n] - Get:1 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pip all 9.0.1-2 [114 kB] - Get:2 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 python-pip-whl all 9.0.1-2 [1 379 kB] - Fetched 1 493 kB in 0s (4 177 kB/s) - dpkg: warning: downgrading python3-pip from 9.0.1-2.3~ubuntu1.18.04.1 to 9.0.1-2 - (Reading database ... 196581 files and directories currently installed.) - Preparing to unpack .../python3-pip_9.0.1-2_all.deb ... - Unpacking python3-pip (9.0.1-2) over (9.0.1-2.3~ubuntu1.18.04.1) ... - dpkg: warning: downgrading python-pip-whl from 9.0.1-2.3~ubuntu1.18.04.1 to 9.0.1-2 - Preparing to unpack .../python-pip-whl_9.0.1-2_all.deb ... - Unpacking python-pip-whl (9.0.1-2) over (9.0.1-2.3~ubuntu1.18.04.1) ... - Setting up python-pip-whl (9.0.1-2) ... - Setting up python3-pip (9.0.1-2) ... - Processing triggers for man-db (2.8.3-2ubuntu0.1) ... + ~$ sudo apt install python3-pip=9.0.1-2 python-pip-whl=9.0.1-2 + Reading package lists... Done + Building dependency tree + Reading state information... Done + The following packages will be DOWNGRADED: + python-pip-whl python3-pip + 0 upgraded, 0 newly installed, 2 downgraded, 0 to remove and 5 not upgraded. + Need to get 1 493 kB of archives. + After this operation, 226 kB disk space will be freed. + Do you want to continue? [Y/n] + Get:1 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pip all 9.0.1-2 [114 kB] + Get:2 http://se.archive.ubuntu.com/ubuntu bionic/universe amd64 python-pip-whl all 9.0.1-2 [1 379 kB] + Fetched 1 493 kB in 0s (4 177 kB/s) + dpkg: warning: downgrading python3-pip from 9.0.1-2.3~ubuntu1.18.04.1 to 9.0.1-2 + (Reading database ... 196581 files and directories currently installed.) + Preparing to unpack .../python3-pip_9.0.1-2_all.deb ... + Unpacking python3-pip (9.0.1-2) over (9.0.1-2.3~ubuntu1.18.04.1) ... + dpkg: warning: downgrading python-pip-whl from 9.0.1-2.3~ubuntu1.18.04.1 to 9.0.1-2 + Preparing to unpack .../python-pip-whl_9.0.1-2_all.deb ... + Unpacking python-pip-whl (9.0.1-2) over (9.0.1-2.3~ubuntu1.18.04.1) ... + Setting up python-pip-whl (9.0.1-2) ... + Setting up python3-pip (9.0.1-2) ... + Processing triggers for man-db (2.8.3-2ubuntu0.1) ... and retrying the first attempt with `~/.config/pip/pip.conf` as: - [global] - index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple - extra-index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_extra/simple + [global] + index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple + extra-index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_extra/simple yields a session like: - ~$ python3 -m venv env/pip-previous - ~$ . env/pip-previous/bin/activate - (pip-previous) ~$ pip --version - pip 9.0.1 from /home/dandersson/env/pip-previous/lib/python3.6/site-packages (python 3.6) - (pip-previous) ~$ pip install hpt - Collecting hpt - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_extra/packages/hpt/0.1.397/hpt-0.1.397.tar.gz - Collecting argh (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/06/1c/e667a7126f0b84aaa1c56844337bf0ac12445d1beb9c8a6199a7314944bf/argh-0.26.2-py2.py3-none-any.whl - Collecting pyyaml==5.1 (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/9f/2c/9417b5c774792634834e730932745bc09a7d36754ca00acf1ccd1ac2594d/PyYAML-5.1.tar.gz - Collecting argcomplete (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/4d/82/f44c9661e479207348a979b1f6f063625d11dc4ca6256af053719bbb0124/argcomplete-1.10.0-py2.py3-none-any.whl - Collecting distro (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/ea/35/82f79b92fa4d937146c660a6482cee4f3dfa1f97ff3d2a6f3ecba33e712e/distro-1.4.0-py2.py3-none-any.whl - Collecting requests (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl - Collecting termcolor (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz - Collecting cilib==v0.1.397 (from hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_extra/packages/cilib/0.1.397/cilib-0.1.397.tar.gz - Collecting idna<2.9,>=2.5 (from requests->hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl - Collecting chardet<3.1.0,>=3.0.2 (from requests->hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl - Collecting certifi>=2017.4.17 (from requests->hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl - Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl - Collecting hvac (from cilib==v0.1.397->hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/7b/2c/b95b551769459406ca440788c32d699820f4c875020ff95c6e3844eee6d3/hvac-0.9.2-py2.py3-none-any.whl - Collecting psycopg2-binary (from cilib==v0.1.397->hpt) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/b1/35/75c9c2d9cfc073ab6c42b2d8e91ff58c9b99f4ed7ed56b36647642e6080e/psycopg2_binary-2.8.3-cp36-cp36m-manylinux1_x86_64.whl - [...SNIP...] - Successfully installed argcomplete-1.10.0 argh-0.26.2 certifi-2019.6.16 chardet-3.0.4 cilib-0.1.397 distro-1.4.0 hpt-0.1.397 hvac-0.9.2 idna-2.8 psycopg2-binary-2.8.3 pyyaml-5.1 requests-2.22.0 termcolor-1.1.0 urllib3-1.25.3 + ~$ python3 -m venv env/pip-previous + ~$ . env/pip-previous/bin/activate + (pip-previous) ~$ pip --version + pip 9.0.1 from /home/dandersson/env/pip-previous/lib/python3.6/site-packages (python 3.6) + (pip-previous) ~$ pip install hpt + Collecting hpt + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_extra/packages/hpt/0.1.397/hpt-0.1.397.tar.gz + Collecting argh (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/06/1c/e667a7126f0b84aaa1c56844337bf0ac12445d1beb9c8a6199a7314944bf/argh-0.26.2-py2.py3-none-any.whl + Collecting pyyaml==5.1 (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/9f/2c/9417b5c774792634834e730932745bc09a7d36754ca00acf1ccd1ac2594d/PyYAML-5.1.tar.gz + Collecting argcomplete (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/4d/82/f44c9661e479207348a979b1f6f063625d11dc4ca6256af053719bbb0124/argcomplete-1.10.0-py2.py3-none-any.whl + Collecting distro (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/ea/35/82f79b92fa4d937146c660a6482cee4f3dfa1f97ff3d2a6f3ecba33e712e/distro-1.4.0-py2.py3-none-any.whl + Collecting requests (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl + Collecting termcolor (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/8a/48/a76be51647d0eb9f10e2a4511bf3ffb8cc1e6b14e9e4fab46173aa79f981/termcolor-1.1.0.tar.gz + Collecting cilib==v0.1.397 (from hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_extra/packages/cilib/0.1.397/cilib-0.1.397.tar.gz + Collecting idna<2.9,>=2.5 (from requests->hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl + Collecting chardet<3.1.0,>=3.0.2 (from requests->hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl + Collecting certifi>=2017.4.17 (from requests->hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl + Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl + Collecting hvac (from cilib==v0.1.397->hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/7b/2c/b95b551769459406ca440788c32d699820f4c875020ff95c6e3844eee6d3/hvac-0.9.2-py2.py3-none-any.whl + Collecting psycopg2-binary (from cilib==v0.1.397->hpt) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/b1/35/75c9c2d9cfc073ab6c42b2d8e91ff58c9b99f4ed7ed56b36647642e6080e/psycopg2_binary-2.8.3-cp36-cp36m-manylinux1_x86_64.whl + [...SNIP...] + Successfully installed argcomplete-1.10.0 argh-0.26.2 certifi-2019.6.16 chardet-3.0.4 cilib-0.1.397 distro-1.4.0 hpt-0.1.397 hvac-0.9.2 idna-2.8 psycopg2-binary-2.8.3 pyyaml-5.1 requests-2.22.0 termcolor-1.1.0 urllib3-1.25.3 PIP now correctly resolves the respective packages to their respective instances, using `--extra-index-url` for the packages that are _only_ present there, and falling back to `--index-url` for everything else. Changing `~/.config/pip/pip.conf` to: - [global] - index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple + [global] + index-url = https://<user>:<api-token>@artifactory.internal/artifactory/api/pypi/PyPI_mirror/simple cleaning out the virtual environment and trying again: - (pip-previous) ~$ deactivate - ~$ rm -rf env/pip-previous/ - ~$ python3 -m venv env/pip-previous - ~$ . env/pip-previous/bin/activate - (pip-previous) ~$ pip --version - pip 9.0.1 from /home/dandersson/env/pip-previous/lib/python3.6/site-packages (python 3.6) - (pip-previous) ~$ pip install hpt - Collecting hpt - Could not find a version that satisfies the requirement hpt (from versions: ) - No matching distribution found for hpt - (pip-previous) ~$ pip install requests - Collecting requests - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl - Collecting chardet<3.1.0,>=3.0.2 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl - Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl - Collecting certifi>=2017.4.17 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl - Collecting idna<2.9,>=2.5 (from requests) - Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl - Installing collected packages: chardet, urllib3, certifi, idna, requests - Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3 + (pip-previous) ~$ deactivate + ~$ rm -rf env/pip-previous/ + ~$ python3 -m venv env/pip-previous + ~$ . env/pip-previous/bin/activate + (pip-previous) ~$ pip --version + pip 9.0.1 from /home/dandersson/env/pip-previous/lib/python3.6/site-packages (python 3.6) + (pip-previous) ~$ pip install hpt + Collecting hpt + Could not find a version that satisfies the requirement hpt (from versions: ) + No matching distribution found for hpt + (pip-previous) ~$ pip install requests + Collecting requests + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl + Collecting chardet<3.1.0,>=3.0.2 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl + Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/e6/60/247f23a7121ae632d62811ba7f273d0e58972d75e58a94d329d51550a47d/urllib3-1.25.3-py2.py3-none-any.whl + Collecting certifi>=2017.4.17 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/69/1b/b853c7a9d4f6a6d00749e94eb6f3a041e342a885b87340b79c1ef73e3a78/certifi-2019.6.16-py2.py3-none-any.whl + Collecting idna<2.9,>=2.5 (from requests) + Using cached https://artifactory.internal/artifactory/api/pypi/PyPI_mirror/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl + Installing collected packages: chardet, urllib3, certifi, idna, requests + Successfully installed certifi-2019.6.16 chardet-3.0.4 idna-2.8 requests-2.22.0 urllib3-1.25.3 --- In summary, from my point-of-view, the `9.0.1-2.3~ubuntu1.18.04.1` version breaks all functionality related to `--extra-index-url`, as well as regresses the behaviour when a package is not found when using `--index-url`. I will be happy to provide any additional information that can help. ** Patch added: "python-pip_9.0.1-2.3~ubuntu1.18.04.5.debdiff" https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1833229/+attachment/5493956/+files/python-pip_9.0.1-2.3~ubuntu1.18.04.5.debdiff ** Changed in: python-pip (Ubuntu Bionic) Status: New => In Progress ** Changed in: python-pip (Ubuntu Bionic) Assignee: (unassigned) => Stefano Rivera (stefanor) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1833229 Title: `--extra-index-url` not working for PIP To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/python-pip/+bug/1833229/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs