Thanks for this mail. I loved it. And for a novice like me at the network level, it sounds exciting. Stef
On Sep 4, 2011, at 8:22 PM, Sven Van Caekenberghe wrote: > Towards better HTTP client usage > > At the last Pharo Sprint in Lille (July 8th), Stéphane and I were at one > point trying to remove some old HTTPSocket usage (which indirectly uses Zn) > and replace it with direct and clean Zn usage. We hadn't much time left and > didn't get very far. But I realized afterwards that although technically > everything was there to write good HTTP client code, it was way too difficult > (it required to much code on the user's behalf). > > Hence I decided to try write a new client that would support this usage much > better and encourage better HTTP client usage in general. Another goal of > this new client is to ultimately replace all other clients currently in Zn > (these different client are creating some confusion among users). > > ZnNeoClient (a temporary name, to be renamed to ZnClient in the future) is a > single object with a lot of (convenience) API to build, execute and process > HTTP client requests. It is somewhat similar to Gofer like classes. It is > also somewhat comparable to ZnHttpClient but it contains even more > functionality. > > The new tests in ZnNeoClientTests show some of the ways the client can be > used. In the current Zn version all ZnClient class functionality as well as > all ZnHTTPSocketFacade (and thus HTTPSocket and thus the rest of the image) > has already been reimplemented using ZnNeoClient. These are good examples to > look at. > > At one point I plan to write some proper documentation, when the design > settles down a bit, but here are some examples: > > The simplest possible usage: > > ZnNeoClient new get: 'http://zn.stxfx.eu/zn/small.html'. > > This is shorthand for: > > ZnNeoClient new > url: 'http://zn.stxfx.eu/zn/small.html'; > get. > > Which is actually shorthand for: > > ZnNeoClient new > url: 'http://zn.stxfx.eu/zn/small.html'; > get; > contents. > > If you know upfront that you will do only one request, you can help conserve > resources by doing: > > ZnNeoClient new > beOneShot; > get: 'http://zn.stxfx.eu/zn/small.html'. > > Connections are kept open whenever possible unless #beOneShot is chosen. > Multiple requests (possibly to the same host) can be issued using the same > client instance. At the end, #close should be send to the client, but garbage > collection cleans up as well. > > Specifying URLs as strings can sometimes be tricky when special characters > such as spaces are involved, for this ZnNeoClient has a whole URL and request > construction API. Here is an example: > > ZnNeoClient new > http; > host: 'www.google.com'; > addPath: 'search'; > queryAt: 'q' put: 'Pharo Smalltalk'; > get. > > There is also API for specifying headers, forms, multipart uploads and so on. > See the unit tests, the ZnClient class side as well as the ZnHTTPSocketFacade > for examples. > > The result of an HTTP request is an HTTP response, modelled by ZnResponse. > Hence, you can ask a client for the last #response, #entity, #contents or > test for #isSuccess. > > Which brings us to the problem that it is not easy to properly set up and > handle the various things that can go wrong. It is here that ZnNeoClient aims > to make a big contribution. > > Assume the following example: at some URL there is a list of numbers as lines > in a text file that we want to download and use. Let's start simple: > > ^ ZnNeoClient new > get: 'http://www.example.com/numbers.txt'. > > When all is well, this will return the text file as string. Let's add parsing: > > ^ ZnNeoClient new > contentReader: [ :entity | > (entity contents lines do: [ :each | > Integer readFrom: each ifFail: [ nil ] ]) > select: [ :each | each notNil ] ]; > get: 'http://www.example.com/numbers.txt'. > > This will make sure we get a (possibly empty) list of numbers. The problem > is, when number.txt is not found by the server, we don't deal properly with > that situation. We can fix that: > > ^ ZnNeoClient new > enforceHttpSuccess: true; > contentReader: [ :entity | > (entity contents lines do: [ :each | > Integer readFrom: each ifFail: [ nil ] ]) > select: [ :each | each notNil ] ]; > get: 'http://www.example.com/numbers.txt'. > > Now, a non-success code will throw an error. We should also make sure that we > do get a text/plain document back, and we want a uniform error handler > reaction: > > ^ ZnNeoClient new > enforceHttpSuccess: true; > enforceAcceptContentType: true; > accept: ZnMimeType textPlain; > contentReader: [ :entity | > (entity contents lines do: [ :each | > Integer readFrom: each ifFail: [ nil ] ]) > select: [ :each | each notNil ] ]; > ifFail: [ :exception | > self log: exception printString, ' while fetching numbers'. > ^ #() ]; > get: 'http://www.example.com/numbers.txt'. > > The failBlock will be execute on any exception, including ZnHttpUnsuccessful > and ZnUnexpectedContentType. > > What about unreliable networking in general: we need a timeout, but better > still, we could retry the request once or twice to cover for (possibly > transient) networking/server problems. > > ^ ZnNeoClient new > timeout: 15; > numberOfRetries: 1; > retryDelay: 2; > enforceHttpSuccess: true; > enforceAcceptContentType: true; > accept: ZnMimeType textPlain; > contentReader: [ :entity | > (entity contents lines do: [ :each | > Integer readFrom: each ifFail: [ nil ] ]) > select: [ :each | each notNil ] ]; > ifFail: [ :exception | > self log: exception printString, ' while fetching numbers'. > ^ #() ]; > get: 'http://www.example.com/numbers.txt'. > > If anything goes wrong, there will be one retry after a delay of 2 seconds. > > Although there are sensible defaults for all options, it will probably make > sense to group the options, like this: > > ^ ZnNeoClient new > systemPolicy; > accept: ZnMimeType textPlain; > contentReader: [ :entity | > (entity contents lines do: [ :each | > Integer readFrom: each ifFail: [ nil ] ]) > select: [ :each | each notNil ] ]; > ifFail: [ :exception | > self log: exception printString, ' while fetching numbers'. > ^ #() ]; > get: 'http://www.example.com/numbers.txt'. > > Finding the proper defaults and policies will take some trial and error. > Another strategy is factory method like ZnClient class>>#client or > ZnHTTPSocketFacade class>>#client. > > Here is another real-world example, invoking a REST service that returns JSON > (you need the JSJsonParser class that comes with Seaside): > > ^ ZnNeoClient new > systemPolicy; > beOneShot; > url: 'http://easy.t3-platform.net/rest/geo-ip'; > queryAt: 'address' put: '81.83.7.35'; > accept: ZnMimeType applicationJson; > contentReader: [ :entity | > (JSJsonParser parse: entity contents) at: #country ]; > ifFail: [ nil ]; > get. > > Or as a utility method: > > XYZUtils class>>#countryForIpAddress: ipAddressString ifFail: failBlock > ^ ZnNeoClient new > systemPolicy; > beOneShot; > url: 'http://easy.t3-platform.net/rest/geo-ip'; > queryAt: 'address' put: ipAddressString; > accept: ZnMimeType applicationJson; > contentReader: [ :entity | > (JSJsonParser parse: entity contents) at: #country ]; > ifFail: failBlock; > get > > That's it for now. There is still some implementation and testing work to be > done. As always, all feedback is welcome. > > Sven > >
