On Tue, Oct 02, 2001 at 07:02:04AM -0700, Aaron Bannert wrote: > On Tue, Oct 02, 2001 at 02:30:26AM -0700, Greg Stein wrote: > > On Mon, Oct 01, 2001 at 11:54:44PM -0700, Justin Erenkrantz wrote: > > > Sure. I'd like to make it able to speak any protocol that > > > has similar functionality to HTTP but isn't quite HTTP/1.1 - i.e. > > > try to be as protocol-agnostic as we can make it. > > > > Euh... let's do HTTP/1.1 *today*. At some point in the future, we can look > > back and see what kinds of tweaks/changes are necessary to meet other needs. > > Just remember that our charter is based solely on an HTTP client library. > > *Not* a general purpose "http-like" protocol library. > > > > Say "oh, but protocol X uses request/response, too... let's add that!" is > > out of bounds for 1.0 :-) > > I'm going to have to sort-of agree with Justin on this one. Although I don't > think we should go all out for a generic request/response meta-protocol API, > it would be in our best interest to be as protocol-agnostic as possible, at > least under the umbrella of HTTP.
We may all be agreeing here. All that we can do at this point is just move forwards. If some particular change or line of development seems to be moving the library "too far" from its original charter, then we can discuss it then. *if* it even comes up :-) >... > > See my previous response to Ian, and my notes about the initial proposal. I > > really think we can create an API that easily moves between the push/pull > > and sync/async models. >... > I'll agree that we can mix sync/async here (will the distinction be fragile?), I'm not sure what you mean by "fragile" here. Do you mean in the sense that we can't mix the modes easily? Or that you must set it up "just so" do do one mode, yet "like this" for the other mode? And that it won't be obvious which, so you'd usually end up with a broken setup? Personally, I think we can have an API that easily flows and can handle the different modes. > but I'm not quire sure I understand how this will work internally to the > filters. I think one of the bigger hurdles in handling push vs. pull on > input filters is where the buffers come from. I'll have to think about this > some more though... See one of my other posts, where I talk about turning a push filter into a pull filter. Essentially, a person should always write their filter as a "push" filter (like the output filters in httpd). If we want to run the filter in a "pull" mode (something wants to pull data through a filter chain), then we have a way to "flip" the chain and push data through in a way that ends up as a "pull". Let's say that you want to operate the response stack in a pull mode (the app is saying "give me more of the response"). The app calls serf_get_response() to get a brigade with more of the response. Inside that function, the network code creates a brigade with some socket data. That data is then pushed through the response filters. Whatever pops out the other side is then returned from serf_get_response(). We may have some issues with limiting the amount of data which is processed. For example, if the app requests 200 bytes, but after pushing data through the response stack, we end up with 300 bytes... we can easily save aside the extra 100 bytes. But what happens if that happens *internally* to the stack? A bit of thought needed there. [ in particular, if F1 generates more data than F2 needs/wants, then should F2 set it aside? what if F2 is tossed and F3 is brought in, and should have received that data? ] >... > I'm still trying to get a grip on this filters stuff, so correct me if I'm > wrong here. To me it seems that filters blend the lines between the push > vs. pull models. Euh... no, I don't think they blend it. They are used in both models, but they are pretty much used differently. > A filter doesn't really care if the library or the app > triggered the pass on the filter. Correct. It should live in its own little world. It shouldn't know what is before/after itself, it shouldn't know why it was invoked, it should have all of the context it needs, etc. > Also, since some filters may be provided > by our library (chunk, ssl, basic auth, various header manipulations, etc) > and some will be provided by the application (flood will have a content-length > verification filter, a webservices thingy could render some xml, etc), and Yes, although auth isn't really a filter. There is a lot of high-level processing related to auth (think about failing auth, asking the app for auth data, then re-submitting the request; a filter can't do that). > since it seems that most of the power of our library will be in the filters, > I don't see why the application will care if it's push or pull. The app will have data pushed at it, or the app will pull data from the library. Transparent to the app, some filters will execute in there. >... > That brings up a good point. We have an opportunity to come up with a > scheme for the ordering of filters that could potentially be more flexible > than the one for HTTP. (I think they are under more constraints). Suggestions? Filters should operate independent of what comes before/after themselves. That means that a higher-level entity is responsible for ordering them properly since a filter can't examine its neighbors and place itself accordingly. That said: even though a filter can operate "anywhere", it can also describe itself to some extent to the library/app about where it "should" go. This is the model used by httpd (filter type). However, httpd has a lot of extra goo for adding/removing those things. I don't know how much of that is needed. >... > > The serf will have both chunking and dechunking code. Since the request > > could be "pushed" at the library, chunking is required. It is also possible > > to give a brigade to the serf and it could compute the length and send a > > request with a Content-Length header. > > > > On the response side, we'll definitely have dechunking. > > Stuff to add to the STATUS :) Hmm? We have a lot of this in the feature/design roadmap. What exactly needs to be added, where? >... > > I generally agree that it is an advanced concept and possibly able to > > relegate to the app, but I also think that many apps will want multiple > > connections and multiple outstanding requests. Thus, a pool of connections. > > Something to be built upon our base library. We could have /core be the very base library. Then we could add the session groups and things in /session or something. Not really sure. It would kind of suck to have a single .c file per directory, so this might simply go into /core. (i.e. it really depeneds on how much code is necessary for some of the various feature groups) Cheers, -g -- Greg Stein, http://www.lyra.org/
