Hi - To put spiders in a separate thread pool, mapping requests to
thread pools by url doesn't work. You have to map by regular
expression on the User-Agent header or by IP address, assuming you can
figure out where Google's robots live. That's why I was suggesting
that the dispatching algorithm needs to be a registered proc.
Jim
>
> > The reason thread pools would be interesting to me personally is to be
> > able to control quality of service better. For example, I may want a
> > pool of 5 threads to handle search engine spidering requests, and a
> > pool of 5 threads to handle PHP requests, 5 threads to handle a
> > particular application that uses a different database, 5 threads to
> > handle CGI requests, 10 threads to TCL requests, 5 to handle image
> > serving (fastpath), 5 to handle all other requests. I'd want to
> > dispatch the requests to the various thread pools myself, using TCL
> > code. The protocol and communication method (TCP vs UDP) part are
> > all the same here - HTTP.
>
>
> > Perhaps this can all be done today with AS 4 using multiple IP
> > addresses - I dunno.
>
> >From sample-config.tcl ... you can map by URL, which means that by proper
> partition of your URL-space you can do all you discuss above but shove
> requests from spiders into their own thread pool.
>
> #
> # Example: Multiple connection thread pools.
> #
> # To enable:
> #
> # 1. Define one or more thread pools.
> # 2. Configure pools as with the default server pool.
> # 3. Map method/URL combinations to the pools
> #
> # All unmapped method/URL's will go to the default server pool.
> #
> #ns_section ns/server/server1/pools
> #ns_section slow "Slow requests here."
> #ns_section fast "Fast requests here."
> #
> #ns_section ns/server/server1/pool/slow
> #ns_param map {POST /slowupload.adp}
> #ns_param maxconnections 100 ;# Max connections to put on queue
> #ns_param maxdropped 0 ;# Shut down if dropping too many conns
> #ns_param maxthreads 20 ;# Tune this to scale your server
> #ns_param minthreads 0 ;# Tune this to scale your server
> #ns_param threadtimeout 120 ;# Idle threads die at this rate
> #
> #ns_section ns/server/server1/pool/fast
> #ns_param map {GET /faststuff.adp}
> #ns_param maxthreads 10
> #
>
>
> --
> AOLserver - http://www.aolserver.com/
>
> To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with
> the
> body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field
> of your email blank.
>
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the
body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of
your email blank.
