We had a chat about what would be desirable in a Twisted-based async web client api.
Background: * https://dev.launchpad.net/LEP/WebservicePerformance We can get inspiration from existing Python remote-object-like libraries: * http://www.lothar.com/tech/papers/PyCon-2003/pb-pycon/pb.html * http://twistedmatrix.com/documents/10.0.0/api/twisted.protocols.amp.html Some things: * Application startup should not require an http roundtrip (assuming we have a reasonably current wadl and auth token - true at the moment in launchpadlib). * If the client can reasonably be expected to know the URL of an object ('/bzr', or /bugs/1) then it should just use this, without doing a redundant call merely to be able to traverse through that object. * Anything that does a remote call should be explicit and return a Deferred. * Therefore, traversing object links so they need to be fetched separately ought to be something the client explictly asks for * Nothing should implicitly do more than one remote call: for instance, after putting an object to the server, we should not automatically read it back in a separate http call. (It's fine of course to build higher-level operations that do multiple calls, but there should be a clear one-to-one layer.) * Explicit is better than implicit: for net operations, and for modifications of remote operations. * Objects that do remote calls should be very obvious. (Perhaps put them all onto a LaunchpadServer object, passing objects as parameters.) * Proxy objects are just value objects, with little magic. (For instance, we don't unwrap _link fields into magic pointers.) * Rather than mutating object fields then saving them, we might have an explicit update operation, which can return a Deferred for its completion. * For collections, we will probably deliver them into a consumer object. This can probably opt-in to see batch boundaries but otherwise it will get all pages. It will get gotObjects() that gets the whole batch, and the default can then split them and deliver them one by one. * Getting a coherent list across multiple calls is hard; the best solution is probably to make the server fast enough that it can return large lists, and secondarily to have higher-level client code that will heal tearing between batch pages by matching up unique ids. * If Launchpad adds the ws.expand parameter, we can have a way to pass it. Rather than this pre-populating attributes that would otherwise be lazy, it will instead add attributes to the passive objects. For instance, if you don't ask for expansion you will get task.bug_link; if you do expand that you will also have task.bug which is a local reference to another real passive object. * We could possibly provide a synchronous layer on top of this. * This seems to be able to accommodate the <https://dev.launchpad.net/LEP/WebservicePerformance> approach of first giving you a huge set of URLs, then later doing separate batched calls to get the details. * Although the client code clearly shows when it's doing network io, it doesn't need to expose exactly what HTTP method is being used, which is an irrelevant detail. (Case in point, some conceptually-read methods might use POST to send large parameters.) So, sketchy, using inlineCallbacks: me = yield lp.get_object(lp.me) # gives you back a fully-populated passive object print me.full_name class ConsumeAndPrintBugs(object): def objectReceived(self, bug): print bug.title done = yield lp.get_collection(me.assigned_bugs_link, ConsumeAndPrintBugs()) me = yield me.update(full_name='Martin "poolie" Pool') urls = yield me.callRemote('getArchiveSubscriptionURLs') # make a url without fetching anything bzr_url = lp.get_object_link(lp.Products, 'bzr') # now get it class PrintTaskTitles(object): def objectReceived(self, task): # need another roundtrip to get the bug d = lp.get_object(task.bug) d.addCallback(self.print_bug) def print_bug(self, bug): print bug.title bzr = lp.get_object(bzr) lp.call_collection_method(bzr, 'searchTasks', PrintTaskTitles()) # we can have a convenience method that folds up all the collection objects into one list, if you don't care to get them individually; this gives a Deferred producing a list all_bugs = yield lp.call_collection_method_to_list(bzr, 'searchTasks') # time passes... print len(all_bugs) # should be correct; and you now have all the data -- Martin _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
