Hi Nick, I'm taking this off-list for now.
I've visited codereview.appspot.com, but am confused. I tried creating a new issue, but: - what do I specify for "SVN base"? - it will only let me upload a patch file (not a java source file), but I'm not sure how to create a patch for a brand-new file? Can you offer some help? thanks. Vince On Tue, Nov 10, 2009 at 5:12 PM, Nick Johnson (Google) <[email protected]> wrote: > Hi Vince, > Thanks for doing this! Could you upload your code to codereview.appspot.com > and link to it from the issue? We can continue the discussion there. > -Nick > > On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti <[email protected]> wrote: >> >> I've faxed over the CLA and created issue #2381 for this: >> >> http://code.google.com/p/googleappengine/issues/detail?id=2381 >> >> Here are the changes I've made since the original implementation: >> >> - The Deferrable.doTask method is now declared to throw Exception >> (instead of PermanentTaskFailure). This allows implementations to >> throw any subclass of Exception in order to have the task retried. >> (PermanentTaskFailure remains a subclass of Exception). >> >> - The Deferrable.doTask method no longer takes any arguments. >> >> - Removed the TaskHolder nested class since it's no longer needed. >> >> - Allow an extra 240 bytes for task overhead; in testing the actual >> overhead was 101 bytes, so this will leave some head-room. In >> addition, the IllegalArgumentException that's thrown if the task size >> is exceeded is caught (and logged), and the task it then written to >> the datastore. >> >> - An exception is logged if payload is not an instance of Deferrable. >> >> - If the task is written to the datastore and there's a subsequent >> failure to queue the task, the datastore entity is deleted. >> >> - Try twice for datatstore puts and deletes in case of >> DatastoreTimeoutException. >> >> - Try twice when queuing a task in case of TransientFailureException. >> >> - Added comments. >> >> Let me know if you have additional feedback. >> >> Vince >> >> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google) >> <[email protected]> wrote: >> > Hi Vince, >> > >> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti <[email protected]> >> > wrote: >> >> >> >> Hi Nick, >> >> >> >> Thanks for your feedback. Yes, I'm very interested in seeing this >> >> included in the official SDK. What are the steps from here? >> > >> > See this doc: >> > >> > http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk >> > >> >> >> >> See responses to specific questions inserted below. >> >> >> >> Vince >> >> >> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would >> >> > eliminate the need to declare it on methods that throw it. >> >> >> >> That's fine. I generally prefer checked exceptions, but if the >> >> AppEngine team prefers unchecked exceptions, I'm happy to make this >> >> change. >> > >> > I wouldn't say we prefer them in general - but personally, I think >> > common >> > uses of PermanentTaskFailure fit into the same sort of category as out >> > of >> > memory errors, divisions by zero, and so forth - they're not predictable >> > at >> > design-time. >> >> >> >> > - If you do the above, you could make the Deferrable interface extend >> >> > Runnable, as suggested in the meeting. I'm not sure that's >> >> > appropriate, >> >> > since the implications of Runnable are different. >> >> >> >> I think Runnable is not appropriate. I also looked at Callable, but in >> >> the end thought a new interface--Deferrable--was the best option. >> > >> > Fair enough. >> > >> >> >> >> > - I'm not sure there's any need to be able to pass arguments to the >> >> > Deferrable, since users will already have to declare a deferrable >> >> > class >> >> > and >> >> > instantiate it. I'm open to convincing, though. >> >> >> >> I put the arguments there to mimic the Python API and then realized >> >> later they're not really needed. Removing the arguments also >> >> simplifies the code by eliminating the need for the TaskHolder >> >> class--the Deferrable class could be serialized directly. I'm happy to >> >> remove the arguments if that's what the AppEngine team decides is >> >> best. >> > >> > Since the Java version is inevitably different due to the need for a >> > custom >> > class instead of a function object, I'd vote to drop the arguments. >> > >> >> >> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in >> >> > Python, the actual limit seems to include some overhead that varies >> >> > from >> >> > task to task. >> >> >> >> No, I haven't tested this yet. >> >> >> >> > - If payload isn't an instance of TaskHolder, the task will silently >> >> > return >> >> > - it should log an error. >> >> >> >> I think the only way payload won't be an instance of TaskHolder is if >> >> the deserialize() method returns null, in which case an error has >> >> already been logged. But I'll double-check this. >> > >> > The other possibility is that something else serialized an object and >> > sent >> > it to your task handler - or possibly, that a future version of your own >> > code did so. I think logging an error (but returning a 200 status code) >> > would be the best policy, so these sorts of situations are debuggable. >> > >> > -Nick Johnson >> >> >> >> > Are you interested in having this included in the official SDK? >> >> > -Nick >> >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti <[email protected]> >> >> > wrote: >> >> >> >> >> >> This looked like an interesting problem, and I already had most of >> >> >> the >> >> >> pieces in place, so here's my first attempt, which is implemented in >> >> >> a >> >> >> single class (also attached, along with some test files): >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java >> >> >> >> >> >> I'm more than happy to contribute this for inclusion in the SDK. >> >> >> >> >> >> Some caveats: >> >> >> >> >> >> 1) Because of issue #2097 >> >> >> (http://code.google.com/p/googleappengine/issues/detail?id=2097), >> >> >> this >> >> >> doesn't work on the development server (it does work in production). >> >> >> So go star that issue! >> >> >> >> >> >> 2) I've only done minimal testing on the production server (see >> >> >> the >> >> >> attached test files). >> >> >> >> >> >> 3) This post is the only documentation that's currently available. >> >> >> >> >> >> First, add the following your your web.xml: >> >> >> >> >> >> <servlet> >> >> >> <servlet-name>Deferred</servlet-name> >> >> >> >> >> >> >> >> >> >> >> >> <servlet-class>com.newatlanta.appengine.taskqueue.Deferred</servlet-class> >> >> >> </servlet> >> >> >> <servlet-mapping> >> >> >> <servlet-name>Deferred</servlet-name> >> >> >> <url-pattern>/_ah/queue/deferred</url-pattern> >> >> >> </servlet-mapping> >> >> >> >> >> >> Second, define the "deferred" queue within queue.xml (use whatever >> >> >> rate you want): >> >> >> >> >> >> <queue> >> >> >> <name>deferred</name> >> >> >> <rate>10/s</rate> >> >> >> </queue> >> >> >> >> >> >> Next, create a class that implements the >> >> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; >> >> >> the >> >> >> "doTask()" method of this class is where you implement your task >> >> >> logic. >> >> >> >> >> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer() >> >> >> method to queue up your task: >> >> >> >> >> >> TestDeferred testDeferred = new TestDeferred(); // implements >> >> >> Deferrable >> >> >> Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 ); >> >> >> >> >> >> (Note that it would be possible to pass arguments as attributes to >> >> >> your Deferrable instance, rather than using varargs; I'm not sure it >> >> >> makes much difference which you choose). >> >> >> >> >> >> Just as for the Python implementation, if the arguments size exceeds >> >> >> 10KB, the arguments are stored in a datastore entity, which is then >> >> >> deleted when the task is executed. Also, your doTask() method can >> >> >> throw a PermanentTaskFailure exception to halt retries; any other >> >> >> exceptions cause the task to be retried. >> >> >> >> >> >> Let me know if you find this useful, have any questions or encounter >> >> >> any problems. >> >> >> >> >> >> Vince >> >> >> >> >> >> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) >> >> >> <[email protected]> >> >> >> wrote: >> >> >> > Hi David. This may be coming to Java eventually, but it hasn't >> >> >> > been >> >> >> > started >> >> >> > yet. If you or anyone else is interested in contributing, let me >> >> >> > know. >> >> >> > >> >> >> > - Jason >> >> >> > >> >> >> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler >> >> >> > <[email protected]> >> >> >> > wrote: >> >> >> >> >> >> >> >> Re: http://code.google.com/appengine/articles/deferred.html >> >> >> >> >> >> >> >> Will this be coming to AppEngine for Java? >> >> >> >> >> >> >> >> David Chandler >> >> >> >> http://turbomanage.wordpress.com >> >> >> >> >> >> >> >> >> >> >> > >> >> >> > >> >> >> > > >> >> >> > >> >> >> >> >> >> >> >> > >> >> > >> >> > >> >> > -- >> >> > Nick Johnson, Developer Programs Engineer, App Engine >> >> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration >> >> > Number: >> >> > 368047 >> >> > >> >> > > >> >> > >> >> >> >> >> > >> > >> > >> > -- >> > Nick Johnson, Developer Programs Engineer, App Engine >> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration >> > Number: >> > 368047 >> > >> > > >> > >> >> > > > > -- > Nick Johnson, Developer Programs Engineer, App Engine > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~----------~----~----~----~------~----~------~--~---
