I forgot to mention - in my second message, the consumer IS running (which is why it picks up the first job at all). If it is, in fact, expected behavior for beanstalkd to discard jobs when the consumer is occupied, is there at least a way for me to figure out if a job I am about to put on the queue will get discarded?
On Dec 4, 12:56 pm, Saikat Chakrabarti <[email protected]> wrote: > Hm, this seems stranger than I thought. I just tried doing a very > simple stress test by submitting 6 jobs in quick succession, and my > first job got picked up and processed, but the other 5 just got > dropped. Am I setting things up incorrectly? My producer does the > following: > > connection = serverconn.ServerConn(BEANSTALKD_QUEUE_ADDRESS, > BEANSTALKD_QUEUE_PORT) > connection.job = job.Job > new_job = job.Job(data=str(j.id), conn=connection, > ttr=BEANSTALKD_QUEUE_TTR, tube=BEANSTALKD_QUEUE_TUBE) > new_job.Queue() > > where j.id is a number i set before that code > > my consumer does: > def main(): > connection = serverconn.ServerConn > (settings.BEANSTALKD_QUEUE_ADDRESS, settings.BEANSTALKD_QUEUE_PORT) > connection.job = job.Job > connection.watchlist = [settings.BEANSTALKD_QUEUE_TUBE] > while True: > print "Polling" > j = connection.reserve() > print "Got job" > export_id = int(j.data) > time.sleep(15) > j.Finish() > > if __name__ == '__main__': > main() > > Using print statements in my producer, I see that the jobs are > actually being received, but my consumer just processes the first > one. Do I need to somehow tell beanstalkd not to discard jobs? > > On Dec 4, 2:57 am, Saikat Chakrabarti <[email protected]> wrote: > > > > > Using pybeanstalk, I just tried submitting some jobs to beanstalkd > > while my consumer wasn't running. When I checked the queue by > > telnetting to localhost:11300 (where beanstalkd is running) and typed > > stats, I get: > > > current-jobs-urgent: 0 > > current-jobs-ready: 0 > > current-jobs-reserved: 0 > > current-jobs-delayed: 0 > > current-jobs-buried: 0 > > > but my total jobs is increasing every time I submit a job. Does > > beanstalkd automatically discard jobs if no one is listening to the > > tube the job comes in on? > > > Thanks, > > Saikat -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
