Hi Dan, Buildbot eight manage a cache of unpickled builds, which is per builder http://docs.buildbot.net/0.8.12/manual/cfg-global.html#caches The doc here suggest the default cache is 500 builds, which looks very big to me you can try to reduce that value On my eight prod, I had to tweak that value a lot in order to compromise speed VS memory. The 3GB usage is indeed a common number (and often much more)
The pickle problem and its scalability issues is the main reason why we started the nine project. For the comparaison, I started a simple master.cfg with 2000 builders: https://paste.fedoraproject.org/460567/89908147/ It runs happily on my mac with 150MB of memory. Pierre Le lun. 24 oct. 2016 à 16:53, Dan Kegel <[email protected]> a écrit : > 15 days of uptime, about 1450 builders, and twisted is using 1.7GB of > *physical* ram. > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 1196 buildbot 20 0 3124m 1.7g 4456 S 10 14.8 1515:22 twistd > > Aw, what the heck, RAM is cheap these days... still, it's a bit surprising. > > > On Mon, May 16, 2016 at 2:35 PM, Dan Kegel <[email protected]> wrote: > > What's normal RAM consumption for twisted on buildbot master for 0.8.12? > > With 1359 builders, I'm seeing 1.4GB. If memory use is proportional > > to builders, that's 100MB/builder. > > > > Here's the output of top after two days of uptime. This is steady > > state; if someone clicks on the waterfall, CPU spikes to 277%, but ram > > doesn't climb. > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 1223 buildbot 20 0 2681m 1.4g 4340 D 0 17.7 130:18.76 twistd > _______________________________________________ > users mailing list > [email protected] > https://lists.buildbot.net/mailman/listinfo/users >
_______________________________________________ users mailing list [email protected] https://lists.buildbot.net/mailman/listinfo/users
