It looks like you recently got your wish for this excellent feature. Am I
right in thinking that this can (and should) be used to address the
challenges with high throughput updates that still exist? I know the 2010
Google I/O presentation by Brett Slatkin has been referenced by many folks,
but there seem to be some issues with that, specifically the Eventual
Consistency of the HRD. A push/pull queue based implementation might look
like this:
For each unit of work:
-Write work to Pull queue with tag=BatchID
-Write named 'fan-in' task to Push Queue for execution <batch period ms> in
the future, containing the BatchID
When the named task executes:
-Lease all available tasks from the Pull queue using the tag=BatchID
-Aggregate the work
-Apply the update to the Datastore
Is a feasible/appropriate/recommended use of the Pull Queue tag mechanism?
On Monday, May 16, 2011 9:53:03 PM UTC-4, Jason Collins wrote:
>
> As pointed out by all-around-smart-dude Robert Kluin, it would be
> great if you could place arbitrary tags on a task when submitting to a
> pull queue, and then lease the tasks back out with that tag. This
> would provide a great, built-in mechanism to do groupings, etc.
>
> This would yield:
>
> q = taskqueue.Queue('pull0')
> q.add(taskqueue.Task(payload=payload_str, method='PULL',
> tag='mytag'))
>
> and
>
> q = taskqueue.Queue('pull0')
> sometasks = q.lease_tasks(3600, 100, tag='mytag')
>
> where "tag" is an optional kwarg in both cases.
>
> Star http://code.google.com/p/googleappengine/issues/detail?id=5061,
> or discuss there.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/WEfUTqDVHcAJ.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
