Vince, this is great! I hadn't been watching my own issue, so didn't see this until now. Thanks so much!
Only enhancement I would suggest is to enable multiple deferred queues. For example. I would like to defer tasks in an email throttle queue separately from a general background queue. Perhaps the Deferrable interface could have a getQueueName() method, or Deferred.defer could have an additional signature defer(Deferrable task, String queueName). Thanks again, /dmc On Nov 10, 4:37 pm, Vince Bonfanti <[email protected]> wrote: > I've faxed over the CLA and created issue #2381 for this: > > http://code.google.com/p/googleappengine/issues/detail?id=2381 > > Here are the changes I've made since the original implementation: > > - The Deferrable.doTask method is now declared to throw Exception > (instead of PermanentTaskFailure). This allows implementations to > throw any subclass of Exception in order to have the task retried. > (PermanentTaskFailure remains a subclass of Exception). > > - The Deferrable.doTask method no longer takes any arguments. > > - Removed the TaskHolder nested class since it's no longer needed. > > - Allow an extra 240 bytes for task overhead; in testing the actual > overhead was 101 bytes, so this will leave some head-room. In > addition, the IllegalArgumentException that's thrown if the task size > is exceeded is caught (and logged), and the task it then written to > the datastore. > > - An exception is logged if payload is not an instance of Deferrable. > > - If the task is written to the datastore and there's a subsequent > failure to queue the task, the datastore entity is deleted. > > - Try twice for datatstore puts and deletes in case of > DatastoreTimeoutException. > > - Try twice when queuing a task in case of TransientFailureException. > > - Added comments. > > Let me know if you have additional feedback. > > Vince > > On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google) > > <[email protected]> wrote: > > Hi Vince, > > > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti <[email protected]> wrote: > > >> Hi Nick, > > >> Thanks for your feedback. Yes, I'm very interested in seeing this > >> included in the official SDK. What are the steps from here? > > > See this doc: > >http://groups.google.com/group/google-appengine/web/how-to-submit-a-p... > > >> See responses to specific questions inserted below. > > >> Vince > > >> > - Perhaps make PermanentTaskFailure a runtime exception, which would > >> > eliminate the need to declare it on methods that throw it. > > >> That's fine. I generally prefer checked exceptions, but if the > >> AppEngine team prefers unchecked exceptions, I'm happy to make this > >> change. > > > I wouldn't say we prefer them in general - but personally, I think common > > uses of PermanentTaskFailure fit into the same sort of category as out of > > memory errors, divisions by zero, and so forth - they're not predictable at > > design-time. > > >> > - If you do the above, you could make the Deferrable interface extend > >> > Runnable, as suggested in the meeting. I'm not sure that's appropriate, > >> > since the implications of Runnable are different. > > >> I think Runnable is not appropriate. I also looked at Callable, but in > >> the end thought a new interface--Deferrable--was the best option. > > > Fair enough. > > >> > - I'm not sure there's any need to be able to pass arguments to the > >> > Deferrable, since users will already have to declare a deferrable class > >> > and > >> > instantiate it. I'm open to convincing, though. > > >> I put the arguments there to mimic the Python API and then realized > >> later they're not really needed. Removing the arguments also > >> simplifies the code by eliminating the need for the TaskHolder > >> class--the Deferrable class could be serialized directly. I'm happy to > >> remove the arguments if that's what the AppEngine team decides is > >> best. > > > Since the Java version is inevitably different due to the need for a custom > > class instead of a function object, I'd vote to drop the arguments. > > >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in > >> > Python, the actual limit seems to include some overhead that varies from > >> > task to task. > > >> No, I haven't tested this yet. > > >> > - If payload isn't an instance of TaskHolder, the task will silently > >> > return > >> > - it should log an error. > > >> I think the only way payload won't be an instance of TaskHolder is if > >> the deserialize() method returns null, in which case an error has > >> already been logged. But I'll double-check this. > > > The other possibility is that something else serialized an object and sent > > it to your task handler - or possibly, that a future version of your own > > code did so. I think logging an error (but returning a 200 status code) > > would be the best policy, so these sorts of situations are debuggable. > > > -Nick Johnson > > >> > Are you interested in having this included in the official SDK? > >> > -Nick > >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <[email protected]> > >> > wrote: > > >> >> This looked like an interesting problem, and I already had most of the > >> >> pieces in place, so here's my first attempt, which is implemented in a > >> >> single class (also attached, along with some test files): > > >> >> http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... > > >> >> I'm more than happy to contribute this for inclusion in the SDK. > > >> >> Some caveats: > > >> >> 1) Because of issue #2097 > >> >> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this > >> >> doesn't work on the development server (it does work in production). > >> >> So go star that issue! > > >> >> 2) I've only done minimal testing on the production server (see the > >> >> attached test files). > > >> >> 3) This post is the only documentation that's currently available. > > >> >> First, add the following your your web.xml: > > >> >> <servlet> > >> >> <servlet-name>Deferred</servlet-name> > > >> >> <servlet-class>com.newatlanta.appengine.taskqueue.Deferred</servlet-class> > >> >> </servlet> > >> >> <servlet-mapping> > >> >> <servlet-name>Deferred</servlet-name> > >> >> <url-pattern>/_ah/queue/deferred</url-pattern> > >> >> </servlet-mapping> > > >> >> Second, define the "deferred" queue within queue.xml (use whatever > >> >> rate you want): > > >> >> <queue> > >> >> <name>deferred</name> > >> >> <rate>10/s</rate> > >> >> </queue> > > >> >> Next, create a class that implements the > >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the > >> >> "doTask()" method of this class is where you implement your task > >> >> logic. > > >> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer() > >> >> method to queue up your task: > > >> >> TestDeferred testDeferred = new TestDeferred(); // implements > >> >> Deferrable > >> >> Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 ); > > >> >> (Note that it would be possible to pass arguments as attributes to > >> >> your Deferrable instance, rather than using varargs; I'm not sure it > >> >> makes much difference which you choose). > > >> >> Just as for the Python implementation, if the arguments size exceeds > >> >> 10KB, the arguments are stored in a datastore entity, which is then > >> >> deleted when the task is executed. Also, your doTask() method can > >> >> throw a PermanentTaskFailure exception to halt retries; any other > >> >> exceptions cause the task to be retried. > > >> >> Let me know if you find this useful, have any questions or encounter > >> >> any problems. > > >> >> Vince > > >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <[email protected]> > >> >> wrote: > >> >> > Hi David. This may be coming to Java eventually, but it hasn't been > >> >> > started > >> >> > yet. If you or anyone else is interested in contributing, let me > >> >> > know. > > >> >> > - Jason > > >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler > >> >> > <[email protected]> > >> >> > wrote: > > >> >> >> Re:http://code.google.com/appengine/articles/deferred.html > > >> >> >> Will this be coming to AppEngine for Java? > > >> >> >> David Chandler > >> >> >>http://turbomanage.wordpress.com > > >> > -- > >> > Nick Johnson, Developer Programs Engineer, App Engine > >> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > >> > Number: > >> > 368047 > > > -- > > Nick Johnson, Developer Programs Engineer, App Engine > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > > 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=.
