spaetz schrieb:
> On Sun, Jun 01, 2008 at 10:00:22AM +0200, Gerhard Schmidt wrote:
>> i have done quite some thinking lately how to speed up the [EMAIL PROTECTED]
>> upload. I've come up with an complete new structure of the server System.
>
> Great to hear that you want to improve the [EMAIL PROTECTED] server. I'll add
> comments below, to sum it up, I think your redesign is a little
> overengineered.
>
>> 1. Database (MySQL)
>> 2. Reflector (Apache or squid)
>> 3. TileServer (plain apache)
>
>> One of the Reflector server (choose per round robin or by some
>> intelligent code in the OSM Javascript code) gets the request looks the
>> tile up in the Database and returns the location of the tile via
>> redirect the the client who connects to the indicated TileServer and
>> gets the Tile there.
>
> Why would you want to have several Refelctor servers that would need to look
> up a central mySQL database (and if you have distributed mySQLs, you would
> need to synchronize all of them on each upload)? And why issue a redirect
> that the browser needs to follow for each single tile? That sounds horribly
> complex and slow.
Read Only Replication can be done by MySQL it self. With some tweaking
you can do rw replication as well but thats not need for this use case.
> If you want to make it distributed in some way, the first suggestion I would
> make is to simply have a separate tile server for each layer. Currently we
> have:
writing a complete new system that hit the wall again sometime in the
future isn't my understanding on developing scalable systems.
> "tile" (regular), "maplint", "lowzoom", and "caption" (currently only -en,
> potentially many more). If you simply had one server responsible for one (or
> a few) layers, you could easily distribute the upload-load between 2-4
> servers.
The load will be extremely uneven. 90% of the request will got to the
regular server and the rest 10% are distributed about the 3 other servers.
> One of the current beauties of the [EMAIL PROTECTED] server is that usually a
> tile is being served as one single file system look up, no database or php
> involved. I just checked the apache log, today from 07:41 - 08:19 we served
> 125366 tiles (just viewing), while we had 3198 "/Upload/" requests. So
> keeping viewing as cheap as possible should be a priority.
It is right now because empty tiles are searched recursively upwards.
> I don't want to discourage you from writing a completely decentralized tile
> server, but splitting layers among servers sounds much easier to me.
The Database Redirecting isn't that expensive. I have done an
redirecting server for an ad banner system which handles 27k request per
minute on 4
i386 machines (7 Years ago i don't recall what they where exactly). An
the Redirect logic was far more expensive than what we will have to do.
We will have to do one or max two queries per tile an these queries can
be cached easily as in most cases more than one tile for the same
block are requested. So there is much room for cache gain (in the
redirector).
The Tileserver is far less complex than today.
> Plus, OSM doesn't have a few dozen servers in the attic that could be used
> for all this, so besides coding you would also find people who run
> "deflectors", "mysqls", and "tah servers". IMHO too much complexity for what
> is required.
Maybe i have :-) No there will be no need for news servers at start. I
think the existing server will perform with the new system as fast as it
does today while running all three parts of the system. The Upload will
be faster because the need to replace the tiles in existing and well
read directories will be gone (thats the problem as i See it. Replacing
the tiles in the directories does take to long).
The gain is that we have ways to scale if demand rises (and rising
demand is what we need for the success of the project) If the system
grows to slow we just add another Server an the system scales top both
of them.
When the system is grown to 3 or 4 Servers we dedicate one as database
and redirector and the others do TileServer.
And there will be the possibility to distribute the system to other RZ
if someone donates Server ans Bandwidth. But its not need do do that at
start.
Regard
Estartu
--
-------------------------------------------------
Gerhard Schmidt | E-Mail: [EMAIL PROTECTED]
TU-München |
WWW & Online Services |
Tel: 089/289-25270 |
Fax: 089/289-25257 | PGP-Publickey auf Anfrage
_______________________________________________
Tilesathome mailing list
[email protected]
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/tilesathome
