On Apr 15, 2010, at 2:02 PM, Kai Krueger wrote: > On 04/15/2010 02:38 PM, River Tarnell wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Ævar Arnfjörð Bjarmason: >>> Those servers need a lot of love. People other than me have been >>> doing >>> some work on it recently. >> >> I've been distracted by other things the last couple of weeks, but I >> plan to do some working on getting renderd running properly next >> week. >> We may need some additional programming to make it feasible to run it >> with a lot of styles (e.g. loading styles on demand instead of at >> startup). > > It appears as if renderd/Mapnik uses up about 5 - 6 Mb of Ram per > instance.
Note, that a minor memory leak in XML loading was fixed in the Mapnik 0.7.0 release: http://trac.mapnik.org/ticket/473 http://trac.mapnik.org/changeset/1506 > Part of the problem may be that renderd uses a new instance > per style per rendering thread. Yes, this certainly uses more memory, but it is an intended feature AFAIK. The mapnik.Map object is not (by design) intended to shared between threads. > So if you have 250 styles and 8 > rendering threads you end up needing about 250*8*5.5Mb ~ 11 Gb of Ram, > which sounds roughly like the numbers given for cassini. > Yes, I can see this being a problem with that many styles! :) > If Mapniks Map Object could be reused for all of the rendering > threads, > that would reduce memory by a factor of 8. It would still be a lot, > but > at least get back into the feasible range. But someone more familiar > with Mapniks internals would have to comment on that. In my experience with python-based servers (deployed with mod_wsgi), it works fine sharing a global map object if deployed in multiprocess mode (rather than multithreaded). I'd be interested to hear from Jon Burgess or Artem about alternative approaches for threaded apps in C. Kai, could you post this question to mapnik-devel? Or, if you are not subscribed, I can - just let me know. > > Otherwise implementing a load on demand with a time based expiry for > map > styles might not be a bad idea, given that most of the 250 odd styles > will likely very seldom be used and sounds like it wouldn't be too > hard > to implement. I think that's great idea. Dane > > Kai > >> >> - river. >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.10 (HP-UX) >> >> iEYEARECAAYFAkvHFuIACgkQIXd7fCuc5vJWyQCfcoW6A81hZYUAFmnNKTLXOWoH >> ccUAnA59GN6GxIfUHNdlGnY9LHGykK/j >> =YzWW >> -----END PGP SIGNATURE----- >> >> _______________________________________________ >> Maps-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/maps-l > > > _______________________________________________ > Maps-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/maps-l _______________________________________________ Maps-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/maps-l
