Thanks for replying... So in my example of rate=50/s and bucket-size=5: When no tasks are in the queue, clearly the bucket fills up and has 5 tokens. If I add 100 tasks, the first 5 will execute immediately and take all 5 tokens... but now the bucket is empty.
I have specified a rate of 50 tokens to be added to the bucket every second... but there are two ways that could happen: 1) 50 tokens are deposited every 1 second. 2) 1 token is deposited every 1/50th of a second. If the first is true, then by using a bucket of size 5, no more than 5 tasks will ever be able to execute in a second because tokens are only being added 50 at a time to a bucket that can only hold 5 tokens. If the second is true, 5 will burst initially, but then there will be a steady stream of 1 task every 50th of a second. >From what you said in your post: "the token-adding resolution is the unit of the rate you set..." it sounds like the first is true... which would mean that the task queue is essentially ALWAYS bursting. I guess that's not a huge deal, but means that to truly execute 50 tasks a second, I have to bump my bucket-size to 50 as well, in which case the task queue essentially starts 50 simultaneous tasks instead of staging them across the entire second... the behavior I would prefer is the second... On Feb 1, 1:44 pm, Robert Kluin <[email protected]> wrote: > Hi, > The rate is like an average upper bound; in other words, the > token-adding resolution is the unit of the rate you set. And, it is > 'bursty,' so if you set a rate of 50/M you might very well have 50 > tasks execute in the first second then the queue will not execute any > tasks for about 59 seconds. Setting a higher bucket_size will let the > queue 'burst' at a higher rate, I usually see that after a queue has > emptied. > > At least that's how I've seen the task-queue behave. > > Robert > > > > On Mon, Jan 31, 2011 at 14:57, HalcyonDays <[email protected]> wrote: > > So I've read through a couple of things about the Task Queue's Token- > > Bucket system, and I've taken a look at the wikipedia article to no > > avail, so forgive me if this question has been answered elsewhere: > > > What is the resolution of the clock that deposits tokens into the > > bucket for each queue? Is its minimum resolution "per second?" > > i.e. If I say, "I want my rate to be 10/s," are 10 tokens deposited > > into the bucket every second, or is one token added every 10th of a > > second? > > > It seems like if the resolution is only per second, the rate is > > limited not only by the user's specification, but also by the bucket > > size... > > > For example: If I have a specified rate of 50/s but a bucket size of 5 > > and the system deposits 50 tokens into the bucket every second, are > > the additional 45 just thrown away? Does my rate actually become 5/s? > > > I'm specifically using the Java app engine, if there's a difference > > between the Task Queues in Python/Java. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
