I should note that I am using the beanstalk-client ruby gem. On Wed, Nov 11, 2009 at 1:18 PM, Al-Faisal El-Dajani < [email protected]> wrote:
> What Keith said was correct, the clients were leaking connections which was > leading to my problem. What follows is an explanation of how the leak > occurred, and how I fixed it. > > The whole problem was in the way I reserved messages from the queue. I used > to get messages like so: > message = tube.reserve(2) rescue nil > > The entire problem was the that I specified a timeout for the reserve call. > Tracing the code, you'll find that method reserve in class Pool calls method > send_to_rand_conn which in turn calls method call_wrap. > Within method call_wrap, if an exception was encountered, It removes the > address of the connection from the list of addresses WITHOUT closing the > connection, and raises an exception. And that was the entire problem. > > Whenever reserve(2) was called and did not find a message in the tube, it > would raise a TimedOut exception, which leads to removing the connection > address. Any future call to put or reserve would establish a new connection, > because the address no longer exists. Since the old connection was not > closed, and a new one was being created, this led to a leakage in > connections which would soon prohibit new connections from being made. > > There are 3 solutions to this problem: > 1) Before removing the address, make sure that the connection is closed. > 2) Create a different rescue stub to handle TimedOut exceptions (I went > with this option as it was far simpler) > 3) Do not use reserve with timeout. > > I hope I made my problem clear now, and hope that a fix for this becomes > part of the official gem soon. > > Cheers, > > > On Wed, Nov 11, 2009 at 12:09 PM, Jaume Sabater <[email protected]>wrote: > >> On Tue, Nov 10, 2009 at 3:53 PM, Al-Faisal El-Dajani >> <[email protected]> wrote: >> >> > I can not find any logs for beanstalk that might contain information >> > about why this is happening. If somebody can offer some help or tell >> > me where to look to figure this out it would be deeply appreciated, >> > thanx. >> >> Sorry for hijacking this thread. About a couple of weeks ago I sent a >> mail to this list asking for the way beanstalkd logs information. I >> think it was Keith that replied that he had not been able to find any >> logger fast enough to support the way beanstalkd works. Just wanted to >> note that this is the perfect example of logging put into good use. >> Perhaps when beanstalkd is in live, heavily loaded environments what >> you said to me is true, but if not under very heavy load, the use of >> normal logging procedures would help "debugging". >> >> Just my two cents anyway. >> >> -- >> Jaume Sabater >> http://linuxsilo.net/ >> >> "Ubi sapientas ibi libertas" >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "beanstalk-talk" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<beanstalk-talk%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/beanstalk-talk?hl=. >> >> >> > > > -- > Al-Faisal El-Dajani > 10/6 > -- Al-Faisal El-Dajani 10/6 -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=.
