Re: [racket-users] web-server dispatch and places

[George Neuner]

On 12/2/2015 7:47 AM, Jay McCarthy wrote:

Hi George,

The output port (and input port) that a response gets is the real 
port, meaning that it has a file-descriptor and can be shared across 
places. I suggest having one serve/servlet place that does the 
dispatching, forwards the port (perhaps using response/output if you 
know the headers and if not, using a new function that I can help you 
write) over to another place depending on what the API call is, that 
other place does the actual writing, the original place waits for the 
completion and then returns (thus closing/recycling the connection). 
I'd be happy to look at your code and point you to the internal 
functions you might need to write out the response properly.


Jay



Hi Jay,

Thanks for confirming about the port, and also for the offer of help.

In almost all cases the return headers are static: almost all 
server->client communication is JSON formatted - regardless of the 
disposition of the function call , its wrapping HTTP request succeeds.  
Errors (if any) are returned in the JSON data.


But there is one function which returns binary image files.  It's 
designed to mimic file download so it can be used directly in an HTML 
image tag.  Might need a spot of help with that one.  It currently looks 
like:


:
; DBMS fetch image file into 'result'
:
(send/back
 (if success
 (response/full 200 #"OK"
(current-seconds)
#"application/octet-stream"
'()
`(,result))

 (response/full 404 #"REQUESTED OBJECT NOT FOUND"
(current-seconds)
#"application/text"
'()
'())
 ))



The only other potential difficulty I see would be if a place(d) 
function needed to set a cookie on the client.  At least for now, none 
of the functions I would want to separate need to do that.



I'm going to have to investigate places more and develop a framework on 
which to build my (core) distributed version.  I have to address some 
things like sharing log files [relatively easy I think] and maybe 
sharing of a DBMS connection pool.  It's an open question whether it's 
better to share a connection pool (if possible) among a small number of 
multi-threaded places, or to create a large (pool equivalent) number of 
single-threaded places each keeping one connection open.


George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] web-server dispatch and places

[Jay McCarthy]

Hi George,

The output port (and input port) that a response gets is the real port,
meaning that it has a file-descriptor and can be shared across places. I
suggest having one serve/servlet place that does the dispatching, forwards
the port (perhaps using response/output if you know the headers and if not,
using a new function that I can help you write) over to another place
depending on what the API call is, that other place does the actual
writing, the original place waits for the completion and then returns (thus
closing/recycling the connection). I'd be happy to look at your code and
point you to the internal functions you might need to write out the
response properly.

Jay


On Wed, Dec 2, 2015 at 1:08 AM, George Neuner  wrote:

> On 12/1/2015 9:51 PM, Robby Findler wrote:
>
>> You probably thought of this, but could you hand the ports themselves
>> over to the worker places after the dispatch but before any other
>> processing happened? Or maybe just make each place run the entire
>> server and then hand the ports over before dispatching and dispatch
>> inside each worker place?
>>
>
> Hi Robby,
>
> I didn't think of that exactly.  Using serve/servlet, I don't know how to
> get the port before I wind up in a handler function.  Once inside a
> handler, AFAIK, the port is only accessible from response/output (or a
> custom response function which I haven't attempted).
>
> My initial idea was to have each place run serve/servlet using its own set
> of dispatch rules.  I was thinking about having one place handle all the
> low volume requests plus some background tasks, and having a number of
> identical places handle the high volume requests.
>
> I did think of having just one place perform dispatch, and using handler
> stubs such as:
>
> (define (high-volume-function request)
> (response/output
> (lambda (port)
> (let [(ch (get-a-worker))
>   (msg (hash 'req request 'port port))
>  ]
> (place-channel-put/get ch msg)
> ))
> :
> ; other response/output stuff
> :
> )
> )
>
> and while doing most processing in the worker place.  Perhaps sanity
> checking in the dispatch place before delegating.
>
> But I'm not positive this approach would work.  Is that the actual TCP
> port in response/output, or a local bytestring that will buffer the data in
> the dispatch place before sending it on to the client? [Have to model that
> and find out].  And dealing with a whole new set of potential errors due to
> using places could get interesting.
>
> I can't afford to have data being copied/buffered unnecessarily. RAM isn't
> an issue, but processing performance is.  No point to relaying something
> from place to place only to have both collectors have to recycle it later.
>
> The biggest issue I'm facing is that there's no practical way to limit the
> data.  The primary function of the application is to be a specialized DBMS
> search engine.  Most realistic queries are expected to produce < 250KB ...
> but poorly targeted queries combined with [not my choice] an option to
> "show all results" (as opposed to in "pages") have potential to produce
> many Mbytes.  Meanwhile the database - and with it the largest possible
> result - keeps growing. The user base also is expected to grow
> significantly in the next year.
>
>
> Most of the time, the single core application handles its load just fine.
> I have it limited to 16 concurrent requests and most searches return in a
> few seconds with the odd large one absorbed nicely even when traffic is
> saturating.  But 99.9994% of all requests are some kind of search - every
> other function combined lurks in the rounding error.   Just a handful of
> concurrent large queries can bog down the server for minutes while the
> application struggles with post-processing and packaging results for the
> clients.   Apache and the DBMS are apportioned carefully over the available
> cores - the limitation to more users and/or bigger searches on the current
> hardware now is the application.
>
> So my application will be going "places" :-)   ... it's only a matter of
> exactly how.
>
>
> George
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Jay McCarthy
Associate Professor
PLT @ CS @ UMass Lowell
http://jeapostrophe.github.io

   "Wherefore, be not weary in well-doing,
  for ye are laying the foundation of a great work.
And out of small things proceedeth that which is great."
  - D 64:33

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an 

[racket-users] web-server dispatch and places

[George Neuner]

Hi all,

I'm not sure exactly how to phrase my questions, so let me give a bit of 
background.


I have a fairly large (and growing) web middleware application that uses 
serve/servlet and currently exposes about 90 function URLs, all of the 
form "/api//".   The application grew precipitously 
and right now there is a lot of duplication in the code - there are many 
instances of groups of functions within a "category" that can be 
implemented via a common parameter driven function.  I was hoping to 
refactor to use symbol-arg dispatch here practical.


I know the code can be refactored that way.  The problem is the 
"practical" part.



There are a small number of heavily used URL functions which I would 
like to separate out into a "place" which can be replicated to take 
advantage of multiple cores.  I have some good reasons [not relevant 
here] for not wanting to use separate processes.


Within a single servlet instance I know that dispatch rules are 
evaluated in order: e.g., in


:
[ ("api" "foo" "static")#:method "get" do-static  ]
[ ("api" "foo" (symbol-arg) #:method "get" do-dynamic ]
:

there is no problem handling "/api/foo/static" separately from other foo 
related URLs



However, continuing with this example, if  do-static  were moved into a 
separate place _listening on the same port_, then the symbol-arg version 
might erroneously capture and fail a request for the "static" version.



Making the places listen on separate ports is logistically complicated.  
My application hides behind Apache, which would need reconfiguration for 
every "special" ProxyPass URL whenever the application changed how it 
dispatched.  Although possible, mucking with Apache is inconvenient and 
yet-another-thing to remember to do.  It would be better if everything 
could be handled entirely within the Racket application.


I thought of keeping all the dispatch in one place, handing off specific 
functions to "worker" places.  But that too is problematic: the 
application is, in part, a specialized search engine ... the functions I 
want to separate can, and do, produce large amounts of data that would 
be impractical to relay through a common dispatcher place.


I thought of passing the TCP port to the worker place so it could 
respond directly.  I know the port is accessible inside response/output, 
but in that context it seems like it might be difficult to use a 
place(d) function to generate the response data. [Maybe not, I haven't 
tried it, but I'm concerned about trapping / handling / logging errors.]



It seems to make the most sense to have places dispatch their own set of 
requests.  However, if I want them to use the same listen port, it 
appears that I have to fully enumerate dispatch rule URLs - negating the 
possibility of using symbol-arg dispatch to handle commonalities.  
Obviously I can dispatch to stubs that redirect to a common function 
where possible, but it seems like there should be a less convoluted 
solution.


Am I missing something (obvious or not) ?Other, better ideas?


In any event, I thank everyone for "listening".  Just venting sometimes 
is helpful. :-)

George

--
You received this message because you are subscribed to the Google Groups "Racket 
Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [racket-users] web-server dispatch and places

[Robby Findler]

You probably thought of this, but could you hand the ports themselves
over to the worker places after the dispatch but before any other
processing happened? Or maybe just make each place run the entire
server and then hand the ports over before dispatching and dispatch
inside each worker place?

Robby

On Tue, Dec 1, 2015 at 5:18 PM, George Neuner  wrote:
> Hi all,
>
> I'm not sure exactly how to phrase my questions, so let me give a bit of
> background.
>
> I have a fairly large (and growing) web middleware application that uses
> serve/servlet and currently exposes about 90 function URLs, all of the form
> "/api//".   The application grew precipitously and right
> now there is a lot of duplication in the code - there are many instances of
> groups of functions within a "category" that can be implemented via a common
> parameter driven function.  I was hoping to refactor to use symbol-arg
> dispatch here practical.
>
> I know the code can be refactored that way.  The problem is the "practical"
> part.
>
>
> There are a small number of heavily used URL functions which I would like to
> separate out into a "place" which can be replicated to take advantage of
> multiple cores.  I have some good reasons [not relevant here] for not
> wanting to use separate processes.
>
> Within a single servlet instance I know that dispatch rules are evaluated in
> order: e.g., in
>
> :
> [ ("api" "foo" "static")#:method "get" do-static  ]
> [ ("api" "foo" (symbol-arg) #:method "get" do-dynamic ]
> :
>
> there is no problem handling "/api/foo/static" separately from other foo
> related URLs
>
>
> However, continuing with this example, if  do-static  were moved into a
> separate place listening on the same port, then the symbol-arg version might
> erroneously capture and fail a request for the "static" version.
>
>
> Making the places listen on separate ports is logistically complicated.  My
> application hides behind Apache, which would need reconfiguration for every
> "special" ProxyPass URL whenever the application changed how it dispatched.
> Although possible, mucking with Apache is inconvenient and yet-another-thing
> to remember to do.  It would be better if everything could be handled
> entirely within the Racket application.
>
> I thought of keeping all the dispatch in one place, handing off specific
> functions to "worker" places.  But that too is problematic: the application
> is, in part, a specialized search engine ... the functions I want to
> separate can, and do, produce large amounts of data that would be
> impractical to relay through a common dispatcher place.
>
> I thought of passing the TCP port to the worker place so it could respond
> directly.  I know the port is accessible inside response/output, but in that
> context it seems like it might be difficult to use a place(d) function to
> generate the response data.  [Maybe not, I haven't tried it, but I'm
> concerned about trapping / handling / logging errors.]
>
>
> It seems to make the most sense to have places dispatch their own set of
> requests.  However, if I want them to use the same listen port, it appears
> that I have to fully enumerate dispatch rule URLs - negating the possibility
> of using symbol-arg dispatch to handle commonalities.  Obviously I can
> dispatch to stubs that redirect to a common function where possible, but it
> seems like there should be a less convoluted solution.
>
> Am I missing something (obvious or not) ?Other, better ideas?
>
>
> In any event, I thank everyone for "listening".  Just venting sometimes is
> helpful.  :-)
> George
>
> --
> You received this message because you are subscribed to the Google Groups
> "Racket Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to racket-users+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.