this has been discussed elsewhere i think. what i remember is that rather than storing the actual data, store the reference to the data. eg, store the data in the hard disk and store the full path to the data with maybe the host name ie.URI.
deepak On Nov 30 2009, 1:36 pm, Pierre-Yves Kerembellec <[email protected]> wrote: > > I have a webapp that makes a backend request which takes a while to > > process, and I was thinking of using beanstalkd for this. Ideally, I > > wanted to do this: > > > 1) When the request comes in, put a job on the beanstalkd queue, > > returning the job id to the client. > > 2) A job consumer on the web server picks up this job and starts > > processing it, and while processing it, changes this job's data with > > the percent complete. I'm not sure if this is possible though - I > > couldn't find anything in the docs that suggests that I can edit the > > data of a job already added on the queue. > > 3) The web client keeps querying every 2 seconds or so with the job id > > to check the completion percentage (and update the view on the client > > based on this percentage). I was hoping to do this with peek. > > 4) Once the job is done, the consumer puts a URL in the job's data > > which the client, upon querying, gets - this points to a resource on > > the web server that the client can then download. > > > My question is - can this actually be done? Specifically, I couldn't > > find if it's possible to edit the data of a job already on the queue. > > If I can't do this this way, does anyone have any suggestions for how > > they have solved similar problems? > > We choosed to store everything related to a job in a Redis server, sitting > next to the beanstalkd server. > This way, you may associate arbitrary data to any job, because beanstalkd is > not equipped (in purpose) > to do so. > > Regards, > Pierre-Yves -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
