Some interesting discussion going on this week at Eclipse.org (none of this is specifically related to SVN):
http://eclipsewebmaster.blogspot.ca/2012/06/problems-of-scale.html Which spawned: https://bugs.eclipse.org/bugs/show_bug.cgi?id=381598 https://bugs.eclipse.org/bugs/show_bug.cgi?id=381620 Among others. There could be some Serf relevance here, although to be honest I think the way Neon works with HTTPv1 in many ways may be even more relevant. The net effect is just the impact that a seemingly small HTTP request can have on a highly trafficked site where those requests start to add up. The main problem here is that the Eclipse update process looks for a small index file that is currently always a HTTP 404. This results in 1.5GB of network packets per day which basically amounts to 152 minutes per day of their available bandwidth. The 404 itself is not the problem, the bandwidth usage would be worse if the file existed. The issue is more the design of the protocol and the discovery it does each time it runs. This is probably similar to all of the HEAD and PROPFIND requests we do in our protocol. They are seeing similar bandwidth issues resulting from the usage of HEAD in their protocol. It looks like the webmaster would like them to explore doing more with a single request. Someone that understands how our REPORT requests work might want to throw some commentary into: https://bugs.eclipse.org/bugs/show_bug.cgi?id=381620 To explain how a single HTTP request could stream back multiple files to the client. It looks like this would mean adopting WebDAV on their servers and clients though. I am not sure if there is anything for us to learn, but I suspect there is. If nothing else, it is interesting to see the impact that a seemingly small and simple request can have when it is multiplied by millions of users. If there are ways we can continue the work we started with HTTPv2 and eliminate more HTTP requests there would be benefits. For example, we still issue a lot of HEAD and PROPFIND requests during update and checkout. Maybe we could include more information in the REPORT response to eliminate the need for those additional requests. -- Thanks Mark Phippard http://markphip.blogspot.com/
