On Tue, Feb 5, 2013 at 9:06 AM, Holger Krekel <holger.kre...@gmail.com>wrote:
> > On Tue, Feb 5, 2013 at 2:13 PM, Lennart Regebro <rege...@gmail.com> wrote: > >> On Tue, Feb 5, 2013 at 2:02 PM, Holger Krekel <holger.kre...@gmail.com> >> wrote: >> > Dropping the crawling over external pages needs _much_ more than just a >> few >> > months deprecation warnings, rather years. There are many packages out >> > there, and it would break people's installations. >> >> No it won't. Nothing gets uninstalled. What stops working is >> installing those packages with pip/easy_install. And that will start >> again as soon as the maintainer uploads the last version to PyPI, >> which she/he is likely to do quite quickly after people start >> complaining. >> >> > I wouldn't assume that maintainers are easily reachable. I've contacted > at least three people of different (>1K downloads) packages which never > responded. > > And of course, i didn't mean to imply that already installed packages > would suddenly break. Rather that installation instructions like "use pip > install X" will just fail with some dependency "Y" not getting installed. > Or getting installed in some random lower version which might contain evil > bugs (including security bugs). For exmaple, the referenced "lockfile" > project has a "0.2" release on pypi, but is currently at 0.9. > > >> > I certainly agree, though, that the current client-side crawling is a >> > nuisance and makes for unreliability of installation procedures. I >> think we >> > should move the crawling to the server side and cache packages. >> >> That will mean that a man in the middle-attack might poison PyPI's >> cache. I don't think that's a feasible path forward. >> >> > Like i said (you snipped that part of the mail), it's a matter of policy. > Externally available packages could be downloaded at once, and not on > demand. Such a download and checksumming could be repeated over a period > of time and from different machines. Of course a remotely stored package > could already be compromised - but such a possibility always exists (even > if an author signs a package with PGP - his machine might be infiltrated, > or the Jenkins build systems performing automated releases etc.). > > Packages does not need to be "cached", as they are not supposed to >> change. If you change the package you should really release a new >> version. (Unless you made a mistake and discovered it before anyone >> actually downloaded it). So what you are proposing is really that PyPI >> downloads the package from an untrusted source, if the maintainer >> doesn't upload it. I prefer that we demand that the maintainer upload >> it. >> >> > I actually think it might make sense to forbid referencing external files > for _future_ pypi uploads (except "#egg=" references probably). The > maintainer trying to do that, then gets a clear error and instructions how > to proceed. She is just trying to get something out, so we have her > attention. > > Changing pip/distribute-easy_install defaults to require an option for > installing packages coming from link rel-types of "download" or "homepage" > might make sense as well. > > In the end, however, none of this prevents MITM attacks between a > downloader and pypi.python.org. Or between the uploader and > pypi.python.org (using basic auth over http often). Signing methods like > https://wiki.archlinux.org/index.php/Pacman-key are key. If a signature > is available (also at a download_url site), then we can exclude undetected > tampering. And there might not be a need to break currently working > package releases. > > It certainly makes sense to fortify python packaging and installation > procedures, but i'd like a bit more of a systematic view on it, including > reviews from security-focused people and a somewhat incremental verified > approach to turn it real and used. > > best, > holger > As long as you are trusting PyPI itself, a PyPI-hosted/signed/timestamped SHA2 hash of the file to be downloaded from an external host would be enough to detect tampering over time. pip could come with a copy of PyPI's ssl certificate, verifying that it was identical to the expected cert rather than signed by one of 100s of trusted CAs.
_______________________________________________ Catalog-SIG mailing list Catalog-SIG@python.org http://mail.python.org/mailman/listinfo/catalog-sig
