Hi -
First, I'm excited to see your initial prototype.
This is an area I've been interested in, but haven't
been able to work much on myself. The rest of this
email goes into some design questions for integrating
libevent2 with Chapel tasks.
I'm just wondering - in your HTTP server, how
does libevent2 interact with the Chapel tasking layers?
Is your example functionally single-threaded? (i.e. just
using 1 Chapel task)?
In particular, I'm hoping that we can start to build
an HTTP and sockets network programming standard module.
Since Chapel has user-level tasks, I think it makes sense
to consider how to integrate something like libevent2 with
them. Since Chapel is a parallel language, I would expect
a networking standard module to interact in useful
ways with Chapel tasks.
The behavior that is appealing to me would be for blocking
network I/O to suspend the current task until some data is
ready. The idea being that then event-based programs can be
written as a combination of serial, blocking tasks. This
idea is used in other languages (although cases I know
of do this only within 1 thread).
Thinking about libevent2, one would presumably want
a limited number (maybe just 1) of event queues. Then,
the server starts accepting requests using the event
queue. We'd want to write the handling for each network
connection in its own Chapel task. (Note that I expect
in some cases it is better to run all network operations
as "events" - but this kind of programming is harder
to manage than sequential, blocking tasks).
Now, the problem that libevent2 solves, the "C10K problem",
is that it's probably not efficient for each connection
to be handled in a blocking way, as far as the OS is
concerned. So, if we want to offer a simple/serial
taks-per-connection model, each task handling a connection
needs to somehow translate a blocking connection-handler routine
into a series of non-blocking operations that work with
libevent2.
This brings up several design questions:
* Should there be 1 libevent2 event queue (event_base)
or many? Should there be 1 per system, one per socket,
one per core?
* Should the libevent2 system use its own "worker pthread"
for guaranteeing progress? Or should Chapel tasks
periodically call a non-blocking event_base_dispatch?
Not all combinations of answers to these questions necessarily
make sense. I can imagine two specific designs:
1) there is 1 libevent2 event queue per system, and
a pthread runs this event queue by calling event_base_dispatch().
Any Chapel-based event in the event queue would be implemented by
signalling a blocked task that it could continue.
I'd expect that would be built with Chapel sync variables.
(Ie. a task that is blocked on 'read' will actually
register an event on that socket and then block
waiting for a sync variable. The registered event
simply sets the sync variable, so the task will
be scheduled again when a core is not busy).
2) there is 1 libevent2 event queue per core/pthread.
In this design, each network-blocked task periodically calls
event_base_loop() in a non-blocking way while trying
to make progress. As with (1), it would interact with
Chapel's tasking layer to run tasks that have the
network data ready (possibly by using sync variables
as I described above).
I'm sure other designs are possible. Anyway, I think it would
be really interesting to investigate this design space a little
bit. Perhaps there is some reason why it does not make sense
to present a task-per-connection model to Chapel users. Or,
perhaps we can find a way to enable parallel Chapel programs to
work with the network in a way that is both easy-to-use and also
efficient. I think we need some experimentation to find out.
Cheers,
-michael
On 12/5/16, 4:14 PM, "Marcos Cleison Silva Santana"
<[email protected]> wrote:
>Dear Chapel Users,
>
>
>I created a simple HTTP server for Chapel using Libevent2 embedded HTTP
>server. The module is in the beginning and I am myself a newbie in Chapel
>(please, forgive me if you find something non pro in the code). But I
>think that would be good to share what
> I am coding.
>Please, see the repo:
>
>
>https://github.com/marcoscleison/chapel-http
>
>
>
>Now, we can serve chapel string to the browser from anonymous functions
>bound by an HTTP request.
>I will add session management and file serving to the Http module.
>
>
>I am open to suggestions. Please, feel free to share your ideas that will
>help me.
>
>
>Thank you!
>
>
>Dr. Marcos Cleison
>
------------------------------------------------------------------------------
Developer Access Program for Intel Xeon Phi Processors
Access to Intel Xeon Phi processor-based developer platforms.
With one year of Intel Parallel Studio XE.
Training and support from Colfax.
Order your platform today.http://sdm.link/xeonphi
_______________________________________________
Chapel-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-users
