On 2010-06-15, James A. T. Rice wrote: > Perhaps if the user stats page is causing people to reduce the quality of > the ti...@home effort because they're more interested in their own stats > than they are in ti...@home quality, then one solution is to remove the > user stats page entirely? It looks like from the amount of time my clients > have spent in 'waiting 30s server load too high', that there's no shortage > of clients who would do the work anyway.
Currently clients upload .zip files of individuals pngs which the server puts together to .tileset files. This used to not be a bottleneck at all when the amount of changes was lower, but it proves to be a bottleneck now. The server has support for uploading .tileset files directly doing away with the bottleneck. And the clients have experimental support for creating those .tileset files. All we need to do is to test the .tileset file upload and switch clients to produce them directly. This would do away with the waiting queue completly as the upload would not require anything but a few basic sanity checks. The changes needed are entirely on the client side, and I would welcome any effort to start uploading .tileset files directly. > Having had a look isn't this - karosm - the bottleneck in the whole > system, and the reason that clients are sitting idle most of the time > already? > > It seems to be doing: > a) distributing work to the clients > b) receiving completed work from the client > c) sole webserver for the osmarender layer > > Anything else I've missed? Perhaps these three things could be split out > some? Upload processing has become a bottleneck and that could be pushed to the clients as outlined above. Beyond that splitting out things is a bit tough: 1) Distributing work to the clients is not that much effort and uploading a tileset file needs to update the current request database anyway. 2) Receiving the completed work is not that bad either, the upload processing is a bottleneck by now. 3) Given that we had about 5TB of uploads/months last I checked, splitting the upload and the tileserver would be quite difficult as the network bandwidth required would be quite tough. A couple years ago we tried to use squid caches to cache tiles by mirrors but that was not overly successful. A couple mirrors were always offline, and as edits are supposed to show up soon, a long expiration duration would be showing stale maps. > Can this be fixed? It's probably fairly important in improving the > throughput of the ti...@home clients. Let me look at that again... Sebastian
pgpZ8KdnfRVEx.pgp
Description: PGP signature
_______________________________________________ Tilesathome mailing list [email protected] http://lists.openstreetmap.org/listinfo/tilesathome
