Florian Lohoff wrote: > On Sat, Jun 14, 2008 at 03:14:34PM +0200, Frederik Ramm wrote: >> * Will tile download be done with PHP (has this been covered in >> the benchmarks - compare executing PHP code to extract data from >> a large tile with having Apache load the tile directly), or >> will we have an Apache module for that? > > I thought about this - Basically the apache module could be storage > agnostic. This means we get a tile name/offset/coordinates and ask a in > memory index database which returns a filename, offset and length to serve. > > So the only software which needs to be storage aware would be the > indexer which takes the uploaded tileset and recreates/updates parts of > the index.
I am not sure I fully understand your proposal, so maybe it is very similar or better, but the structure proposed by Jiri nd described at http://wiki.openstreetmap.org/index.php/Tiles%40home/Tileset_as_one_file seems pretty reasonable. For this there is no need for an index database and as such should keep the module very simple. All the module has to do is check that the zoom level is between 12 and 17. If so, calculate the z12 tileset coordinates. Check if a full tileset file exists. If any of these fail, fall-back to what the server does now. If all the checks succeed, then the offset within this file can be calculated with reading about 8bytes from the beginning of the file (hopefully cached due to common access). So overall I would guess the overhead should be minimal and be outwayed by the benefits shown in the previous artificial benchmarks. I guess how well it really performs one can only tell once one has implemented it, but given the current status of the tah server which seems to not even keep up with the changed tiles let alone the huge priority 3 queue, I think it would be worth trying it out. I cannot guarantee anything yet, but I hope to at least attempt to implement this in case no one else gives it a try. It doesn't seem to hard. But given I have never written an Apache module, maybe I am simply to naive ;-) > > I thought about it and my idea was to let [EMAIL PROTECTED] clients upload > tilesets as > a tar.gz. The tileserver then would simply gunzip the tar and let the > tar exist as the storage. So the indexer would read tars and the apache > module would serve from the tars. Is it necessary to zip any of this? The PNGs should be close to optimally compressed already anyway and the extra header at the beginning is only 5.3kb for a z12-z17 tileset, so even if that could be compressed, overall the gain would probably be less than a percent. In this case, the upload code for a complete tileset would simply be putting the complete file at the correct position. No extra processing is needed at all. (apart from updating the request queue and user stats). Dealing with incomplete tilesets shouldn't be too difficult either. If there is no tileset file for the incomplete tileset, then can simply use the current upload mechanism and the fall-back code in the tile serving will automatically take care of it. In this case you don't get the benefits of the single tileset file, but also no additional overhead is occurred and hardly any extra code is needed. If there is a tileset file for the incomplete tileset, then one can still use the same old mechanism, but would have to mark all tiles in the new tileset invalid in the index of the old tileset file. The changes to the client would still be very easy. I have written a quick perl script to take the temp directory created by the current tah client and create a tileset file from it. So this could be hooked into the client before uploading. So I think all of the problems Frederik mentioned should be solvable, but who knows what shit hits the fan once one tries to code things up ;-) Kai > > If it makes sense to even concatenate tars to even large files needs to > be tried but the frontend code in the apache module does not need to > know. > > The module could even fall through to alternative storage in case the > index does not know about the tile (transition to storage-ng) > > As a starting base one could have a look at the apache mod_ziplook > which enables apache to serve from zip files directly. > > http://pihl.kumpu.org/ziplook/ > > Flo > > > ------------------------------------------------------------------------ > > _______________________________________________ > Tilesathome mailing list > [email protected] > http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome _______________________________________________ Tilesathome mailing list [email protected] http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome
