On Sat, Apr 17, 2010 at 12:09:52PM +0100, Kai Krueger wrote: > On 04/17/2010 11:29 AM, Frederik Ramm wrote: > > Kai, > > > > Kai Krueger wrote: > >> This is less of a question though regarding using renderd vs tirex but > >> more of a question of where the future lies and thus for which to > >> develop new features. It would be a shame and rather inefficient to > >> put in effort to develop features for something that turns out to be a > >> dead end > > > > What particular feature did you want to work on? > > It was somewhat intended as a more general question than a specific > feature, but what sparked the question was the potential need to extend > the rendering to support a lot of styles (200+) as is currently being > discussed on the wikimedia maps list. At > http://cassini.toolserver.org/tile-browse/ there are renderings for all > the 278 languages wikipedia supports. With renderd each of the rendering > threads has a copy of all the mapnik styles. As each style takes up > about 5 - 6 Mb, this results in 278*8*5.5 = 12Gb of ram. Not directly > desirable. So thoughts exist to implement some form of on demand loading > and unloading of styles, as it is unlikely that all those styles will be > needed all the time and/or share memory accross rendering > threads/processes. So the question somewhat was where to do this > potential development. But as it is only thoughts at the moment it will > probably depend on who ever gets to actually implementing this. Hence my > more general question of which of the two seems more "recomended/future > proof" for potential developments like that.
Having 278 styles that only differ in a few places where the different languages are involved certainly seems wasteful. We have already thought about the on-demand-loading of styles and it is probably something we want to implement for Tirex. There is a different use case that also needs that: When you are working on and testing a new style, you don't want to restart your renderer all the time to reload styles. Thats a different problem, but it would need a similiar solution, ie load styles on use. But in your case it might not solve the problem. Depending on how often tiles are asked for for the different languages, you might still end up with either most of the styles in memory or otherwise need to constantly reload the styles which takes time. There are many ways we could approach this: One would be to implement something in Mapnik so that it can allow variables in style files. Then you would only have one style file for all languages and somehow feed the language in separately. Another approach would be to render the map in one step and the labels in a second step and merge them. You still had one style file for each language, but they would be much smaller, because they contain only the labels. This merging could be done in the renderer. Bascially it would call Mapnik twice and merge the images. Or maybe we can modify Mapnik so that it can not only render into a new empty bitmap, but also into an existing one. Something like this. Most of these approaches are probably easier with Tirex, because we separated the queue management from the rendering. And, as I said in my other mail, I am currently working on making the rendering backend more configurable and flexible. The idea is that you can configure for any map, which rendering backend to call with which parameters. So it should be easy to experiment with a new backend. Jochen -- Jochen Topf [email protected] http://www.remote.org/jochen/ +49-721-388298 _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
