Why not just use Ruby's Queue class, and do io in your worker threads,
sizing your pool appropriately to fill cpu while waiting on io?

On Jul 21, 2016 11:10 PM, "Arindam Mukherjee" <arindam.muker...@gmail.com>
wrote:

> I am trying to build a task broker using EventMachine, thread pools and a
> fibers. I wanted to validate the approach and ask questions about some
> challenges I am facing.
>
> The task broker runs a thread-pool (perhaps not EM.defer) to which
> individual tasks are assigned.
> The tasks are I/O bound, but occasionally may do CPU bound activity.
>
> Proposed design:
> 1. EventMachine event loop runs in its own thread.
> 2. Each task is represented by a Ruby Proc / lambda. Worker threads from a
> thread pool service each task.
> 3. A worker thread runs the task in a fiber. There is a global EM::Queue
> for shipping I/O work to the EM loop. Each task creates a channel that the
> EM loop uses to ship data back to the task. 4. The task subscribes to the
> channel it creates. It does a Fiber.yield after shipping I/O to the EM
> loop. And in the subscription callback, it resumes itself.
>
> This way, using Fibers to represent tasks, I hoped to have single threads
> multiplexing multiple tasks. But I am somehow not able to pull this off. Is
> this a viable approach?
>
> Arindam
>
> --
> You received this message because you are subscribed to the Google Groups
> "EventMachine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to eventmachine+unsubscr...@googlegroups.com.
> To post to this group, send email to eventmachine@googlegroups.com.
> Visit this group at https://groups.google.com/group/eventmachine.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"EventMachine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to eventmachine+unsubscr...@googlegroups.com.
To post to this group, send email to eventmachine@googlegroups.com.
Visit this group at https://groups.google.com/group/eventmachine.
For more options, visit https://groups.google.com/d/optout.

Reply via email to