A question re. 'stable storage'... ...is it possible that might be opened up in a more general way in the future? Even with limitations on how much an app can store at a time or whatnot?
Memcache is nice, but the unpredictability of it makes it a no-no for me in some cases. But if there was some kind of temporary stable storage that was as fast to write to as it is to add a task to the queue, that could be a nice layer in between datastore and memcache. Sorry to go off topic... On Apr 9, 10:26 am, "Nick Johnson (Google)" <[email protected]> wrote: > Hi, > > On Fri, Apr 9, 2010 at 5:16 AM, dhruvbird <[email protected]> wrote: > > Hello folks at Google, > > I was wondering if you would be open to describing certain internals > > of the Task Queue so that we can get a better understanding of it, and > > potentially use the learnings to replicate it for use as a Task/ > > Message Queue. We have a requirement for a task queue, so I was > > wondering if there is a chance to understand what appengine is using. > > You might want to look into using a ready-made solution such > ashttp://www.rabbitmq.com/ > > > I have a few specific questions too. I'll just go ahead and write > > them out, so if it is okay, then you could reply to those. > > > 1. Are the tasks persisted on stable storage when they are added to > > the queue? > > Yes. > > > 2. Is there special handling for named tasks and tasks with an ETA? > > All tasks have an ETA - the default is just 'now'. Named tasks have special > handling in that they result in tombstone records to prevent duplicate names > being enqueued. > > > 3. Can you disclose numbers as to how well a task queue instance on a > > single machine performs (with reliable message delivery and no ETA/ > > named tasks)? > > Sorry, no. > > > 4. What is the typical load on a single task queue server instance? > > We can't disclose that, either. > > > 5. What is used to communicate between different task queue instances > > and between the appengine API and the task queue (protobuf, etc..??) > > Google uses Protocol Buffers extensively. > > > 6. Are you planning to open source the task queue server any time soon/ > > No. Our task queue depends on a large amount of internal infrastructure, and > open sourcing it would require open sourcing all of that, too. > > -Nick Johnson > > > > > > > Do you use a message queue that is available off the shelf and build > > stuff around it? > > > Regards, > > -Dhruv. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%2Bunsubscrib > > [email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: > Registered in Dublin, Ireland, Registration Number: 368047 > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
