On Wed, Sep 16, 2009 at 10:59 PM, Yun Huang Yong <[email protected]> wrote: > > Martin Sarsale wrote: >> On Wed, Sep 16, 2009 at 1:13 AM, Yun Huang Yong <[email protected]> wrote: >>> martin sarsale wrote: >>>> We have it running since a few days (uptime: 1114539) and I just found >>>> out that it's using a lot of memory even if the queue is almost empty: >>>> >>>> 8376 beanstalkd 20 0 1813m 1.7g 388 S 0 44.1 18:19.30 >>>> beanstalkd >>>> >>>> this is the output of stats: >>>> >>>> current-jobs-ready: 2917 >>>> current-jobs-reserved: 143 >>>> current-jobs-buried: 8803 >>>> max-job-size: 10000000 >>> Hmmm these stats seem to indicate that your queues are not really >>> "empty"? In total that's almost 12,000 jobs. How large are the jobs? >> >> jobs are tiny, definitely much smaller than 141666b/each (1.7g/12000). >> We had to raise max-job-size because we were using in a different way before. > > What OS are you on? > > Do your queue lengths spike much? > > If its Linux I wonder if you are bumping into memory fragmentation > issues. In brief testing under Linux many months ago we found that in > certain conditions beanstalkd's in-memory size would grow steadily over > time if the queue sizes are allowed to spike. We think this is because > of the way the pq/ms structures double in size when growth is needed. > As they're allocated on the heap and semi-permanent you end up with an > ever increasing high water mark over time. We're not sure if this > problem would really manifest in real world usage since I haven't seen > anyone mention it before but you can link against jemalloc to make it go > away. > > http://www.canonware.com/~ttt/2008/08/stand-alone-jemalloc-for-linux.html > > Of course an even simpler explanation might be a memory leak somewhere... :) >
lol! well, if it happens again, I'll write and see if I can help debugging this. tnx -- Martin Sarsale msn: [email protected] jabber: [email protected] twitter: http://twitter.com/runixo linkedin: http://www.linkedin.com/in/msarsale sumavisos: http://www.sumavisos.com blog: http://runa.tumblr.com --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en -~----------~----~----~----~------~----~------~--~---
