Colin Marquardt wrote: > 2009/11/25 Peter Körner <[email protected]>: > >> Atm. the load on cassini is around 20 and the diff imports are fighting >> to stay up. I think it's cmarq's hillshade-generator takes the most >> resources on cassini. top reports ~57.1% IO waiting so I think there's a >> lot of disk going on. > > Load is now back to ~4 without me changing anything. Maybe it was some > interaction with other processes?
Just looking at the munin graphs, (i.e. not looking at more detailed stats), my guess would be that it is probably not the hillshading , but rather the standard mod_tile /renderd setup that is causing the load spikes. Those load spikes correlate quite strongly with renderd queue lengths and rendering times. Currently I think renderd is set to use 8 threads. So if there is a render queue, all 8 threads will try to run, each connecting to a separate postgres process. Therefore, there are about 16 processes/threads that try to run and thus depending on how much of the wait time gets accounted to load, a load of 20 doesn't sound all that unreasonable. I think the problem is that the current setup is too slow to render low zooms, with single metatiles taking partly over a minute to render. This is therefore clearly not feasible to render on the fly. However, with the 200+ styles of cassini, the fraction of low zoom tiles to high zoom tiles is skewed in the wrong direction. Once you get down to zoom level 12 or so, it starts just about being fast enough to render on the fly. I think it is therefore necessary to pre-render all tiles to zoom level of probably at least 9 - 10. One can use render_list to pre-render in the background through renderd, thus playing nicely with the mod_tiles on-the-fly rendering. It will probably also be necessary to not expire the low zoom tiles as frequently to not need to rerender those as much. The OSM mapnik server basically doesn't expire lowzoom tiles at all, other than after a full DB reimport. Given that probably most of the name localisation still happens on large features like country / major city names, the low zoom tiles probably change much more though. So perhaps the the lowzoom tiles can be bunched up and only expired once a day and then immediately pre-rendered again. render_old can probably handle this. It goes through all of the previously rendered metatiles and checks if they are marked dirty. If yes, it submits them for background rendering. It might still make sense though to convert the hillshading tiles to metatiles though. It might be that the program convert_meta allows to convert individual pngs into metatiles. But I haven't used convert_meta so I don't know if it actually does that. Kai > > Cheers > Colin > > _______________________________________________ > Maps-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/maps-l _______________________________________________ Maps-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/maps-l
