I think we've got this figured out. It looks like what probably happened was
the job timed out so it went back on the queue.
On Apr 2, 2012, at 9:38 AM, Roy Smith wrote:
> We're using pybeanstalk 0.11.1, python 2.6.5, Ubuntu 10.04.3 LTS. The main
> loop of our multi-threaded beanstalk client looks like:
>
> while True:
> m = mq.reserve()
> jid = m['jid']
> try:
> # process job
>
>
> except Exception:
> logger.error("skipped job %d" % jid)
> finally:
> logger.info("deleted job %d" % jid)
> mq.delete(jid)
>
> Every once in a while (yesterday it happened once in 60k jobs processed), the
> delete() call in the finally block raises a NotFound exception. I've
> included a stack dump below.
>
> 2012-04-02T04:56:57+00:00 cluster1.songza.com 2012-04-02 04:56:57,497 -
> scrobble_mill - Caught NotFound('Server returned: NOT_FOUND',) in
> <Thread(Thread-9, started 139969929115392)>
> Traceback (most recent call last):
> File "/home/songza/deploy/current/scrobble/scrobble_mill.py", line 98, in
> listen_wrapper
> listen(host, port)
> File "/home/songza/deploy/current/scrobble/scrobble_mill.py", line 162, in
> listen
> mq.delete(jid)
> File "/usr/lib/pymodules/python2.6/beanstalk/serverconn.py", line 94, in
> caller
> *getattr(protohandler, 'process_%s' % attr)(*args, **kw))
> File "/usr/lib/pymodules/python2.6/beanstalk/serverconn.py", line 68, in
> _do_interaction
> return self._get_response(handler)
> File "/usr/lib/pymodules/python2.6/beanstalk/serverconn.py", line 58, in
> _get_response
> res = handler(recv)
> File "/usr/lib/pymodules/python2.6/beanstalk/protohandler.py", line 56, in
> __call__
> return self.__h(val)
> File "/usr/lib/pymodules/python2.6/beanstalk/protohandler.py", line 68, in
> handler
> checkError(response)
> File "/usr/lib/pymodules/python2.6/beanstalk/errors.py", line 35, in
> checkError
> raise err
> NotFound: Server returned: NOT_FOUND
>
> Looking over the logs, it appears that two different threads got the same job
> in their reserve() calls. Is this a bug? I was assuming that beanstalk was
> thread safe, but maybe not?
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "beanstalk-talk" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/beanstalk-talk/-/GMkJoHdsoOgJ.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/beanstalk-talk?hl=en.
---
Roy Smith
[email protected]
--
You received this message because you are subscribed to the Google Groups
"beanstalk-talk" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/beanstalk-talk?hl=en.
