Jiri Klement wrote: >> Instead of proclaiming a bottleneck, why not show us? > > You can test on [EMAIL PROTECTED] server how much faster is copying one big > file > than hounders of small files.
No doubt that this file operation will be quicker, but is that all we'd need to test for? We need to 1)Upload (no change) 2)Check if the tileset is complete (ie, peek at every tile in the uploaded blob) 3a) move the tile to its end destination (if it's a complete tileset) 3b) or extract the tiles and insert them into the destination blob if only a single tile, or a few tiles, or an incomplete tileset has been uploaded (means copying over the whole binary blob, and inserting the new tiles into it) I'd expect that especially 3b is much worse in the tileset-per-file system. And then you have to consider tile serving. Right now, it's one file per tile, no PHP etc involved. Apache and the OS can do all their file system tricks, e.g. caching often viewed z12 tiles in memory etc. The one-tileset-per-file approach would require you to open the tilesetfile, extract the data blob, wrap it into a PNG structure, and serve it. If I browse the z12 layer, I don't see any caching advantages, as rather than opening one file per tile, it has to open one tileset file per tile and deliver parts of it. I'd expect poorer caching results as the bigger tileset files fall out of memory quicker, not better ones. Admittedly at z13, you'd only have to open one file for every 2 z13 tiles and it get's better at higher zoom levels. Still you would have to cache whole tilesets rather than just the often-used tiles in memory, so there are arguments for both sides. > My tests shows that copying 100 tilesets (700kB, 138 actuall tiles) > over nfs on LAN needs 8 seconds for 1 file per tileset and 65 seconds > for one file per tile. As we are getting away from NFS connected disks in favor of local disks pretty soon now (hopefully), I wouldn't recommend optimizing the whole thing for NFS access. I agree that unzipping a tileset with 1600 disk seeks is more expensive than just pushing one big file. I still haven't been convinced though that *serving* tiles, would be quicker this way though. Actually, the whole tile serving seems to perform pretty well, I haven't heard of any bottlenecks. And it's pretty simple, which is a plus. I don't think it's very obvious or easy to claim the superiority of one system over the other. The existing system has the advantage that, errr, it exists. :-) So, I back up deelkar here, "show us" and we'll be happy to use the better performing system. spaetz _______________________________________________ Tilesathome mailing list [email protected] http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome
