Hi HalcyonDays, In this case the system will add another token every 1/50th of a second.
Note that the taskqueue system may batch up requests so that even though you will receive a new token every 1/50th of a second it may burst 2-3 (well, actually up to 5) tasks at a time. On 2 February 2011 07:35, HalcyonDays <[email protected]> wrote: > Thanks for replying... > > So in my example of rate=50/s and bucket-size=5: > When no tasks are in the queue, clearly the bucket fills up and has 5 > tokens. > If I add 100 tasks, the first 5 will execute immediately and take all > 5 tokens... but now the bucket is empty. > > I have specified a rate of 50 tokens to be added to the bucket every > second... but there are two ways that could happen: > 1) 50 tokens are deposited every 1 second. > 2) 1 token is deposited every 1/50th of a second. > > If the first is true, then by using a bucket of size 5, no more than 5 > tasks will ever be able to execute in a second because tokens are only > being added 50 at a time to a bucket that can only hold 5 tokens. > If the second is true, 5 will burst initially, but then there will be > a steady stream of 1 task every 50th of a second. > > From what you said in your post: "the token-adding resolution is the > unit of the rate you set..." it sounds like the first is true... which > would mean that the task queue is essentially ALWAYS bursting. I > guess that's not a huge deal, but means that to truly execute 50 tasks > a second, I have to bump my bucket-size to 50 as well, in which case > the task queue essentially starts 50 simultaneous tasks instead of > staging them across the entire second... the behavior I would prefer > is the second... > > On Feb 1, 1:44 pm, Robert Kluin <[email protected]> wrote: >> Hi, >> The rate is like an average upper bound; in other words, the >> token-adding resolution is the unit of the rate you set. And, it is >> 'bursty,' so if you set a rate of 50/M you might very well have 50 >> tasks execute in the first second then the queue will not execute any >> tasks for about 59 seconds. Setting a higher bucket_size will let the >> queue 'burst' at a higher rate, I usually see that after a queue has >> emptied. >> >> At least that's how I've seen the task-queue behave. >> >> Robert >> >> >> >> On Mon, Jan 31, 2011 at 14:57, HalcyonDays <[email protected]> wrote: >> > So I've read through a couple of things about the Task Queue's Token- >> > Bucket system, and I've taken a look at the wikipedia article to no >> > avail, so forgive me if this question has been answered elsewhere: >> >> > What is the resolution of the clock that deposits tokens into the >> > bucket for each queue? Is its minimum resolution "per second?" >> > i.e. If I say, "I want my rate to be 10/s," are 10 tokens deposited >> > into the bucket every second, or is one token added every 10th of a >> > second? >> >> > It seems like if the resolution is only per second, the rate is >> > limited not only by the user's specification, but also by the bucket >> > size... >> >> > For example: If I have a specified rate of 50/s but a bucket size of 5 >> > and the system deposits 50 tokens into the bucket every second, are >> > the additional 45 just thrown away? Does my rate actually become 5/s? >> >> > I'm specifically using the Java app engine, if there's a difference >> > between the Task Queues in Python/Java. >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Greg Darke -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
