Thanks everyone who helped make this happen.
>From my perspective* I believe the ssh upload mechanism was added to
address security issues around the basic-auth-over-http method used
historically. Now uploads *may* be done over https, and those using the ssh
method can move over to using twine or pip upload, I think that it's
reasonable to discontinue support for ssh uploads.
Richard
* as the guy who will be hassled if its loss is noticed ;)
On 26 January 2014 10:38, Donald Stufft <don...@stufft.io> wrote:
> Today (Sat Jan 25, 2014) the Infrastructure team has migrated PyPI to new
> infrastructure.
>
> The old infrastructure was:
>
> - a single database server managed by OSUOSL
> - a pair of load balancers shared by all of the python.org services
> hosted on
> OSUOSL
> - a single backend VM that served as everything else for PyPI
>
> The VM that was acting as the backend server from PyPI was partially hand
> configured and partially setup to be managed by chef. Additionally it had
> an
> issue that caused it to kernel panic every so often which had been the
> cause of
> a number of downtimes in the last few months. Because it was primarily
> configured and administered by hand and because the way it was set up it
> was
> not feasible to have any sort of failover or spare.
>
> The new infrastructure is:
>
> - 2 Web VMs
> - 2 Database servers in Master/Slave Configuration
> - 2 PgPool Servers pooling connections to the database servers and load
> balancing reads across them.
> - 2 GlusterFS servers backed by Cloud Block Storage acting as the file
> storage
> for package and package docs
> - 1 metrics server to handle updating the download counts as they come in
> from
> Fastly
>
> All of the VMs are hosted on Rackspace’s Public Cloud and have their
> configuration completely controlled and managed using Salt. Going forward
> this
> will allow us to easily scale out as required or kill malfunctioning
> servers
> and spin up new ones easily. Additionally the setup has been setup so that
> where possible there is two servers performing the same role, ideally in an
> Active/Active configuration but at least in a Master/Slave configuration.
> This
> should allow PyPI to be far more stable moving forward and make downtimes
> much
> easier to recover from.
>
> The services are still fronted by Fasty’s CDN and in the new infrastructure
> we’ve removed our load balancer and have replaced it by having Fastly
> handle
> the load balancing for us. Additionally we’ve recently setup a static
> mirror of
> PyPI that is updated once every minute. This is hosted on Rackspace cloud
> as
> well but in a separate data center from the rest of PyPI. Fastly is
> configured
> to fall back to this static mirror in the case that neither of the two web
> heads are functioning. This should ensure that even in the event of a
> catastrophic failure of the PyPI service that the bulk of package
> installations
> should hopefully remain working.
>
> The bad news, (and the “Breakage” from the subject) is that while the new
> infrastructure was being planned out, built, and migrated to the “pypissh”
> package was forgotten. The pypissh package is an alternative way to upload
> packages to PyPI however it is very difficult, because of the way it
> works, to
> provide HA support for it as we’ve set up for everything else. We don’t
> have
> any numbers for how many people are actively using this package but looking
> at a roughly 2 week chunk of time in PyPI’s download history, the pypissh
> package was downloaded 7 times by a browser, and 7 times by pip. All other
> downloads were caused by the mirroring system.
>
> As of right now pypissh is non functional and due to the difficulty in
> HAing
> and monitoring the current setup and because it is apparently has a very
> small set of users we would like to effectively kill off this particular
> service. Additionally the benefits of pypissh have been reduced now that
> PyPI
> is available over a TLS connection with a well trusted certificate. My
> question
> to you is, is this something that distutils-sig is willing to have happen?
> If
> we are to re-enable pypissh we’ll need to write a new solution to doing it
> that
> can be properly HA’d and we’d prefer to put our efforts into improving
> things
> for a much larger set of people.
>
> So yea, PyPI should be loads more stable and more reliable now.
>
> -----------------
> Donald Stufft
> PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372
> DCFA
>
>
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG@python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
>
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
https://mail.python.org/mailman/listinfo/distutils-sig
