Hi Brian,
There are a couple of ways of managing this. One way is to do something
like the load-balancing pattern in the ZMQ guide. You will have a
single push/pull socket to get requests from m2 and then will need to do
req/rep from the workers. The workers can reply directly to mongrel2,
so you'll want to do a dummy REP once you get a request:
+--REQ/REP-->Worker--PUB/SUB--> Mongrel2
|
Mongrel2 --PUSH/PULL--> Broker +--REQ/REP-->Worker--PUB/SUB--> Mongrel2
|
+--REQ/REP-->Worker--PUB/SUB--> Mongrel2
As a disclaimer, I haven't actually tried the above, but it should work
fine, and is what the ZMQ people suggest for load balancing.
What I *have* done is this:
A single process with one thread that reads on the push/pull; then I use
language native queues to distribute to worker threads. You can then
either have a the replies go back directly from the worker threads, or
have another thread that sends all the responses back. Just remmeber
that zmq sockets can't be shared between threads safely, so each worker
will need it's own PUB socket if you want them to send directly to
Mongrel2 -- I didn't know that the first time I treid this, and it didn't
end well :).
-Jason
On 16:11 Fri 13 Sep , Brian McQueen wrote:
> I've got handlers running slow long running backend jobs. Â They start
> up and return a response and close the connection, and then they waits
> for their slow jobs to finish before they goes back to conn.recv.Â
> While a slow job is running other handlers are connected to the queues
> and receiving. Â I'm seeing that the distribution of the requests still
> seems to round robin into the busy handler which is not even at
> conn.recv. Â The other two handlers are sitting there waiting. Â They
> are fast and not busy and waiting, and they will each take a request,
> and then mongrel2 stops like its trying to direct the next request in a
> round robin fashion to the busy handler, which is not even at
> conn.recv, but instead is busy doing a big backend job.
> What's going on here? Â How should I debug this?
> I need the requests to go the the ready handlers and not return to the
> slow handler until its back at conn.recv.
