I've faxed over the CLA and created issue #2381 for this: http://code.google.com/p/googleappengine/issues/detail?id=2381
Here are the changes I've made since the original implementation: - The Deferrable.doTask method is now declared to throw Exception (instead of PermanentTaskFailure). This allows implementations to throw any subclass of Exception in order to have the task retried. (PermanentTaskFailure remains a subclass of Exception). - The Deferrable.doTask method no longer takes any arguments. - Removed the TaskHolder nested class since it's no longer needed. - Allow an extra 240 bytes for task overhead; in testing the actual overhead was 101 bytes, so this will leave some head-room. In addition, the IllegalArgumentException that's thrown if the task size is exceeded is caught (and logged), and the task it then written to the datastore. - An exception is logged if payload is not an instance of Deferrable. - If the task is written to the datastore and there's a subsequent failure to queue the task, the datastore entity is deleted. - Try twice for datatstore puts and deletes in case of DatastoreTimeoutException. - Try twice when queuing a task in case of TransientFailureException. - Added comments. Let me know if you have additional feedback. Vince On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google) <[email protected]> wrote: > Hi Vince, > > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti <[email protected]> wrote: >> >> Hi Nick, >> >> Thanks for your feedback. Yes, I'm very interested in seeing this >> included in the official SDK. What are the steps from here? > > See this doc: > http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk > >> >> See responses to specific questions inserted below. >> >> Vince >> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would >> > eliminate the need to declare it on methods that throw it. >> >> That's fine. I generally prefer checked exceptions, but if the >> AppEngine team prefers unchecked exceptions, I'm happy to make this >> change. > > I wouldn't say we prefer them in general - but personally, I think common > uses of PermanentTaskFailure fit into the same sort of category as out of > memory errors, divisions by zero, and so forth - they're not predictable at > design-time. >> >> > - If you do the above, you could make the Deferrable interface extend >> > Runnable, as suggested in the meeting. I'm not sure that's appropriate, >> > since the implications of Runnable are different. >> >> I think Runnable is not appropriate. I also looked at Callable, but in >> the end thought a new interface--Deferrable--was the best option. > > Fair enough. > >> >> > - I'm not sure there's any need to be able to pass arguments to the >> > Deferrable, since users will already have to declare a deferrable class >> > and >> > instantiate it. I'm open to convincing, though. >> >> I put the arguments there to mimic the Python API and then realized >> later they're not really needed. Removing the arguments also >> simplifies the code by eliminating the need for the TaskHolder >> class--the Deferrable class could be serialized directly. I'm happy to >> remove the arguments if that's what the AppEngine team decides is >> best. > > Since the Java version is inevitably different due to the need for a custom > class instead of a function object, I'd vote to drop the arguments. > >> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in >> > Python, the actual limit seems to include some overhead that varies from >> > task to task. >> >> No, I haven't tested this yet. >> >> > - If payload isn't an instance of TaskHolder, the task will silently >> > return >> > - it should log an error. >> >> I think the only way payload won't be an instance of TaskHolder is if >> the deserialize() method returns null, in which case an error has >> already been logged. But I'll double-check this. > > The other possibility is that something else serialized an object and sent > it to your task handler - or possibly, that a future version of your own > code did so. I think logging an error (but returning a 200 status code) > would be the best policy, so these sorts of situations are debuggable. > > -Nick Johnson >> >> > Are you interested in having this included in the official SDK? >> > -Nick >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <[email protected]> >> > wrote: >> >> >> >> This looked like an interesting problem, and I already had most of the >> >> pieces in place, so here's my first attempt, which is implemented in a >> >> single class (also attached, along with some test files): >> >> >> >> >> >> >> >> http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java >> >> >> >> I'm more than happy to contribute this for inclusion in the SDK. >> >> >> >> Some caveats: >> >> >> >> 1) Because of issue #2097 >> >> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this >> >> doesn't work on the development server (it does work in production). >> >> So go star that issue! >> >> >> >> 2) I've only done minimal testing on the production server (see the >> >> attached test files). >> >> >> >> 3) This post is the only documentation that's currently available. >> >> >> >> First, add the following your your web.xml: >> >> >> >> <servlet> >> >> <servlet-name>Deferred</servlet-name> >> >> >> >> >> >> <servlet-class>com.newatlanta.appengine.taskqueue.Deferred</servlet-class> >> >> </servlet> >> >> <servlet-mapping> >> >> <servlet-name>Deferred</servlet-name> >> >> <url-pattern>/_ah/queue/deferred</url-pattern> >> >> </servlet-mapping> >> >> >> >> Second, define the "deferred" queue within queue.xml (use whatever >> >> rate you want): >> >> >> >> <queue> >> >> <name>deferred</name> >> >> <rate>10/s</rate> >> >> </queue> >> >> >> >> Next, create a class that implements the >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the >> >> "doTask()" method of this class is where you implement your task >> >> logic. >> >> >> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer() >> >> method to queue up your task: >> >> >> >> TestDeferred testDeferred = new TestDeferred(); // implements >> >> Deferrable >> >> Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 ); >> >> >> >> (Note that it would be possible to pass arguments as attributes to >> >> your Deferrable instance, rather than using varargs; I'm not sure it >> >> makes much difference which you choose). >> >> >> >> Just as for the Python implementation, if the arguments size exceeds >> >> 10KB, the arguments are stored in a datastore entity, which is then >> >> deleted when the task is executed. Also, your doTask() method can >> >> throw a PermanentTaskFailure exception to halt retries; any other >> >> exceptions cause the task to be retried. >> >> >> >> Let me know if you find this useful, have any questions or encounter >> >> any problems. >> >> >> >> Vince >> >> >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) <[email protected]> >> >> wrote: >> >> > Hi David. This may be coming to Java eventually, but it hasn't been >> >> > started >> >> > yet. If you or anyone else is interested in contributing, let me >> >> > know. >> >> > >> >> > - Jason >> >> > >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler >> >> > <[email protected]> >> >> > wrote: >> >> >> >> >> >> Re: http://code.google.com/appengine/articles/deferred.html >> >> >> >> >> >> Will this be coming to AppEngine for Java? >> >> >> >> >> >> David Chandler >> >> >> http://turbomanage.wordpress.com >> >> >> >> >> >> >> >> > >> >> > >> >> > > >> >> > >> >> >> >> >> > >> > >> > >> > -- >> > Nick Johnson, Developer Programs Engineer, App Engine >> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration >> > Number: >> > 368047 >> > >> > > >> > >> >> > > > > -- > Nick Johnson, Developer Programs Engineer, App Engine > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~----------~----~----~----~------~----~------~--~---
