On Mon, 2006-10-09 at 13:26 -0400, Sam Berlin wrote: > I agree with Robert that it is much easier to go from an event driven > model to a blocking model. If the first layer that HttpNIO exposes is > blocking, there'd need to be additional hacking below that in order to > remove the blocking / thread-based layer. On the other hand, if the > first layer it exposes is non-blocking, it's relatively trivial to add > a thread ontop of that and expose an additional blocking layer. > > It is difficult to think of many scenarios that require (or a better > with) non-blocking I/O, but I would caution against excluding them > from HttpClient's scope. If HttpClient 4.0 had been ready a year or > so ago (with an exposed non-blocking layer), we would definitely have > used it in LimeWire as the basis of file-transfers. As-is, we > invented our own minimalistic non-blocking state-based http transport > for downloads. > > If the non-blocking layer is there, I guarantee that folks will be > able to find a use for it. Whereas if only a blocking layer is there, > those developers looking for the high-performance asyncronous model > will have to go elsewhere. >
Sam, Robert, et al Simply for practical reasons while there are only two guys hacking (me and Roland) we ought not spread out efforts too thin. A full-blown even-driven API will take time to get right. I think it is more important to get HttpCore ALPHA3 release that covers 95% of use cases out the door rather sooner than later. Beyond that it is just a matter of priorities and available time. Oleg > Sam > > On 10/9/06, Robert Olofsson <[EMAIL PROTECTED]> wrote: > > > > Since my proxy is almost fully nio/event based I would like to share > > a few comments. > > > > Oleg Kalnichevski wrote: > > > I am still quite skeptical about usefulness of a fully event-driven HTTP > > > transport for one simple reason: asynchronous (non-blocking) I/O > > > transport makes no sense of what so ever if the process of content > > > generation or content consumption is asynchronous (blocking). > > > > There are many things that may block here are a few examples: > > *) DNS-lookup > > *) File reading writing > > *) Database access > > *) All higher level api:s that only give you a stream. > > *) Calls to Runtime.exec > > > > That DNS lookups are also totally single threaded in native code in > > some systems does not make things better. > > > > > If one > > > needs a worker thread to generate / process content anyways, what is the > > > point of having an even driven transport? > > > > Agreed. > > One objection here may be that you do not need one worker thread for all > > of the content generation, but that usually does not make things better. > > You will still need the worker thread for the _slow_and_blocking_ > > operation. > > So if content generation/modification uses any of the above then using > > workers simplify things a lot. > > > > > I see only a few scenarios > > > where the third choice (event callbacks) may prove advantageous, > > > primarily in HTTP proxies and gateways. > > > > Except that http proxies does lots of dns lookups so they will block > > a lot. My proxy spawns worker thread only when they need to, but it > > complicates some part of the code. > > > > That my proxy also modifies the content and caches the data will mean > > lots of other blocking calls in some of the code paths. > > > > > I think ultimately we need both options. I suggest we start with the > > > second option, release ALPHA3 and then consider implementing the third > > > option before ALPHA4 / BETA1. > > > > One thing to keep in mind: > > It is easier to go from event driven to a blocking model than to do the > > reverse. This may be an argument to go for number 3 (full nio). > > If you go for 3 then make it easy to use a few selector-threads > > otherwise the system will use only 1 cpu (or 1 core). > > > > /robo > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
