Update. Tried PULL queues. Some entries arrive > 10 seconds later into the queue. This completely won't work for my application. This was occuring when the total adds to the queue was around 180 over a 1-5 second
On Monday, July 30, 2012 2:54:37 PM UTC-4, Richard wrote: > > Well, I need performance of at least 1k /second throughput including > non-batch add from many F1's and batch dequeue/reaping. > > On Monday, July 30, 2012 10:40:23 AM UTC-4, David Hardwick wrote: >> >> We do pull queues on backends, and use the pipeline library to insure >> reliability. >> >> >> >> On Jul 29, 7:42 pm, Jeff Schnitzer <[email protected]> wrote: >> > On Sun, Jul 29, 2012 at 7:19 AM, Richard <[email protected]> wrote: >> > >> > > How can I do this reliably in a 10 second window ? I have had using >> a PULL >> > > queue suggested, but I don't want to do all the work of converting >> the app >> > > over if it will be just as unreliable (I remember posts about >> tasksqueue's >> > > getting "stuck").... >> > >> > The task queues that have gotten "stuck" in the past were push queues; >> > basically, the machines that do the pushing fell behind (or were >> > temporarily suspended due to problems). I haven't heard of any >> > equivalent problem with pull queues. The queue would have to start >> > throwing errors on service calls, or "losing" tasks... probably not >> > impossible, but on par with "datastore requests failing". It would be >> > a major failure. >> > >> > Does anyone have any comments about the reliability of pull queues? >> > >> > I'm only just now starting to work with pull queues. While I do allow >> > that my push queues may not fire in a timely manner, I'm engineering >> > to expect pull queues to always be available. >> > >> > Jeff >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/0Fi8cr5ZK6UJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
