2009/11/25 Peter Körner <[email protected]>: > Colin Marquardt schrieb: >> 2009/11/25 Peter Körner <[email protected]>: >> >>> Atm. the load on cassini is around 20 and the diff imports are fighting >>> to stay up. I think it's cmarq's hillshade-generator takes the most >>> resources on cassini. top reports ~57.1% IO waiting so I think there's a >>> lot of disk going on. >> >> This must have gotten worse as generate_tiles goes to lower zoom >> levels. I can kill it and maybe run with a single thread or even some >> artificial slowdown tonight. Right now it's three threads, which >> wasn't a problem so far. > I don't think that this is necessary atm. I just wanted to talk about > this upcoming issue. Can you estimate how long it will keep running with > the current parameters? Getting some days out of sync is not that big > problem (we're not expiring tiles currently anyway).
Maybe two more days, but it's really no problem to slow it down a bit. The maximum run time of the current job is bounded anyway with cassini being repurposed. > >>> I tried to do a "du -hs /mnt/user-store/osm_hillshading" but ir ran >>> loooong until I canceled it. >>> >> >> Well, I generate hillshading for the land mass of Europe (plus the UK) >> right now, so there isn't such a lot of empty sea tiles. > Okay, cool. > > > It would >> indeed be nice to generate metatiles instead of the single PNGs >> though. > Yes, it would. Is there a tool to pack the existing tiles into metatiles? I just found that there is a tool called convert_meta: http://svn.openstreetmap.org/applications/utils/mod_tile/readme.txt I hope it will work with the paletted PNGs I'm using. >> I'm also not rendering down to zoom 18. Pre-rendering those >> hillshading tiles (up to a certain zoom level at least) in general is >> not bad IMO since they are very much static (they don't contain any >> changing information), so will be useful for many maps for the next >> several years. > Yes I see that, too, but rendering on the fly and then caching the > results is a compromise between disk space and cpu power. ACK. I'll try to get the necessary tool support. > It's not the absolute "max number" I'm worried about but the decreasing > performance when working with a lot of files in a directory (and maybe > the inode usage). This could both be solved by using metatiles. I see. Let's try convert_meta on a subset of the data tonight and see how it goes. Cheers Colin _______________________________________________ Maps-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/maps-l
