On 02/11/2010 12:52 PM, Peter Körner wrote: > River Tarnell schrieb: >> Peter Körner: >>> It's more important to think about where the diff-import should be >>> running from. I'd suggest trying it from willow but keep thinking about >>> maybe shifting it to ptolemy.. >> >> Does importing one minute's diffs really take so much memory than 1GB isn't >> enough? From what I remember of osm2pgsql, it took at least several minutes >> to use that much memory.
My guess would be that for diffs, it won't need that much memory, as the cache won't be effective during imports anyway. The cache stores the lat/lon values of nodes in a blocked-sparse array at the id-th entry in the table. When building ways and relations, Osm2pgsql will either query the each node in O(1) from the cache or require a SQL query over the network. It is heavily optimized for initial import and on diffs, the cache will mostly not be populated anyway. Therefore I guess the memory doesn't matter in this case, but the extra latency of the network vs local process might. On the other hand on a local lan, that will presumably be less than a single disk seek, so not really an issue either. > > No i don't think it will. As I said I'd try it from willow but I can > imagine, that eg. expiring tiles won't work this easy when osm2pgsql > does not run on the same machine as mod_tile. Expiring tiles is a different matter, but osm2pgsql doesn't expire the tiles directly so shouldn't effect where it has to run. Instead it spits out a textual list of tiles that should be considered expired, that then needs to be handled separately. The actual expiry script will most likely need to be run on the same machine as the (meta)tiles stored though, as it does require to touch and stat quite a lot of files. Especially with the 200 different language styles. But if the meta tile directory is writable from the login servers, we can experiment with what setup works best first before moving it over to ptolemy. Kai > > Peter > > _______________________________________________ > Maps-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/maps-l _______________________________________________ Maps-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/maps-l
