Even ignoring the malicious possibility there is a probably greater chance
of accidental mistakes:
- company sets up internal index using pip's multi-index support and hosts
various modules
- someone quite innocently uploads something with the same name, never
version, to pypi
- company installs now use that unknown code
devpi avoids this (I would recommend it over multi-index for companies
anyway) by having a white list system for packages that might be pulled
from upstream that would clash with internal packages.
As Nick's mentioned, a signing infrastructure - tied to the index
registration of a name - could solve this problem.
There still remains the usability issue of unsophisticated users running
into external indexes and needing to cope with that in one of a myriad of
ways as evidenced by the PEP. One solution proposed and refined at the
EuroPython gathering today has PyPI caching packages from external indexes
*for packages registered with PyPI*. That is: a requirement of registering
your package (and external index URL) with PyPI is that you grant PyPI
permission to cache packages from your index in the central index - a
scenario that is ideal for users. Organisations not wishing to do that
understand that they're the ones causing the pain for users.
An extension of this proposal is quite elegant; to reduce the pain of
migration from the current approach to the new, we implement that caching
right now, using the current simple index scraping. This ensures the
packages are available to all clients throughout the transition period.
The transition issue was enough for those at the meeting today to urge me
to reject the PEP.
Richard
On 24 July 2014 12:40, Vladimir Diaz <vladimir.v.d...@gmail.com> wrote:
> In metadata 2.0 even with package signing you end up where I can have you
> install “django-foobar” which depends on “FakeDjango”, which provides
> “Django”, and then for all intents and purposes you have a “Django” package
> installed.
>
> Can you go into more detail? Particularly, the part where "FakeDjango"
> provides Django.
>
> Richard Jones mentions the case where an external index provides an
> "updated release" and tricks the updater into installing a compromised
> "Django." Is this the same thing?
>
>
> On Thu, Jul 24, 2014 at 4:55 AM, Richard Jones <r1chardj0...@gmail.com>
> wrote:
>
>> Thanks for responding, even from your sick bed.
>>
>> This message about users having to view and understand /simple/ indexes
>> is repeated many times. I didn't have to do that in the case of PIL. The
>> tool told me "use --allow-external PIL to allow" and then when that failed
>> it told me "use --allow-unverified PIL to allow". There was no needing to
>> understand why, nor any reading of /simple/ indexes.
>> Currently most users (I'm thinking of people who install PIL once or
>> twice) don't need to edit configuration files, and with a modification we
>> could make the above process interactive. Those ~3000 packages that have
>> internal and external packages would be slow, yes.
>>
>> This PEP proposes a potentially confusing break for both users and
>> packagers. In particular, during the transition there will be packages
>> which just disappear as far as users are concerned. In those cases users
>> will indeed need to learn that there is a /simple/ page and they will need
>> to view it in order to find the URL to add to their installation invocation
>> in some manner. Even once install tools start supporting the new mechanism,
>> users who lag (which as we all know are the vast majority) will run into
>> this.
>>
>> On the devpi front: indeed it doesn't use the mirroring protocol because
>> it is not a mirror. It is a caching proxy that uses the same protocols as
>> the install tools to obtain, and then cache the files for install. Those
>> files are then presented in a single index for the user to use. There is no
>> need for multi-index support, even in the case of having multiple staging
>> indexes. There is a need for devpi to be able to behave just like an
>> installer without needing intervention, which I believe will be possible in
>> this proposal as it can automatically add external indexes as it needs to.
>>
>> I talked to a number of people last night and I believe the package
>> spoofing concept is also a vulnerability in the Linux multi-index model
>> (where an external index provides an "updated release" of some core package
>> like libssl on Linux, or perhaps requests in Python land). As I understand
>> it, there is no protection against this. Happy to be told why I'm wrong, of
>> course :)
>>
>>
>> Richard
>>
>> _______________________________________________
>> Distutils-SIG maillist - Distutils-SIG@python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
>>
>>
>
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
