"Martin v. Löwis" wrote: >> I think it overlaps a bit the PEP goal, which is to set up a network >> of mirrors, >> and have them listed in the PyPI DNS so clients can switch from one >> mirror >> to another.(and even do geoloc!) > > JFTR, this already exists. a.mirrors.pypi.python.org and > b.mirrors.pypi.python.org are already there and could be used by clients. > >> Well maybe this is the best path to follow right now, as it will be >> done faster, >> without having to interact with much people to set it up, so it's a >> quick win. > > My main worry (besides the client integration) is statistics: I do want > to get download statistics. So anybody implementing it would have to > find a way of fetching the download numbers from Amazon.
Download statistics are readily available from Amazon Cloudfront, so no worries: you'll get statistics for all edge server downloads. http://docs.amazonwebservices.com/AmazonCloudFront/latest/DeveloperGuide/index.html?AccessLogs.html >> But it will probably kill the mirroring protocol idea from the PEP in >> the process, >> which I think is superior in the long term since it provides a >> standardized ground >> for the community to set up mirrors independently from pypi.python.org. > > I also remain skeptical that this cloud idea is useful at all. Amazon > Cloudfront is a *beta* service. So they aren't sure themselves whether > it works correctly - and there have been reports about two-day outages > of EC2, for bitbucket.org. There also have been complaints about the > available bandwidth. So I'm not sure whether replacing a single point of > failure with a different one is actually improving anything. Amazon Cloudfront uses S3 as basis for the service, S3 has been around for years and has a very stable uptime: http://www.readwriteweb.com/archives/amazon_s3_exceeds_9999_percent_uptime.php Cloudfront itself has been around since Nov 2008. You can check their current online status using this panel: http://status.aws.amazon.com/ Apart from the gained availability and outsourced management, we'd also get faster downloads in most parts of the world, due to the local caching Cloudfront is applying (and this can be used to further increase the availability, since we can control the expiry time of those local copies). So in summary we are replacing a single point of failure with N points of failure (with N being the number of edge caching servers they use). Regaring the bitbucket problem you mentioned: EC2 is their virtual server service, which we don't use. The bitbucket problems originated from a) a DDoS attack on their virtual servers running on EC2 and b) a problem with the Amazon EBS, which is their virtualized SAN, and was related to the way the DDoS was done (EBS and the DDoS attack both used UDP): http://blog.bitbucket.org/2009/10/04/on-our-extended-downtime-amazon-and-whats-coming/ """ And to re-iterate, the problem wasn’t really Amazon EC2 or EBS, it was isolated to our case, due to the nature of the attack. """ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jun 14 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ 2010-07-19: EuroPython 2010, Birmingham, UK 34 days to go ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ _______________________________________________ Catalog-SIG mailing list Catalog-SIG@python.org http://mail.python.org/mailman/listinfo/catalog-sig