Hi, Michael, Greg
I totally agree that we should minimize the runtime changes, especially the
4 different task implementations. Many things rely on the runtime task
interface. Modify the task interface will be quite risky. We should find a
way to walk around the problem unless we have to. There are tradeoffs about
how net library talks to the task implementations and how they schedule the
IO tasks. I'll try to minimize the related code changes. Maybe sync
variables would just fit in a basic version. I think we should take my
proposal as an experiment or a draft net library. If it works fine, then
move on this way, make the decision, stabilize the API.
--zhao
2016-03-22 4:13 GMT+08:00 Michael Ferguson <[email protected]>:
> Hi Greg -
>
> On 3/21/16, 4:08 PM, "Greg Titus" <[email protected]> wrote:
>
> >Hi Michael, Zhao --
> >
> >Are there good reasons for solving this in C code in the runtime rather
> >than in Chapel code in, say, the modules? Put another way, is there a
> >lack of Chapel language features that, if they were present, would let
> >us implement this more naturally in module code?
>
> I don't think we have enough of this task figured out to know
> what would go in the runtime vs. the modules. I generally
> agree it's better to put code in the modules. In this case,
> part of what we hope to do is to interact intelligently with
> the tasking system, and that might mean changing it (but
> then again it might not - perhaps sync variables and/or
> chpl_task_yield are sufficient). I don't think Zhao or I
> were trying to say that we knew we needed to change the
> tasking layer.
>
> -michael
>
>
> >
> >(I always worry about the maintenance overhead of adding to the runtime
> >interfaces, especially for things like the tasking layer, which has many
> >implementations. Any feature we add to the runtime tasking interface
> >has to be implemented 4 times, right now. We may get rid of 1 of those
> >implementations soon, but we don't have plans to go below that.)
> >
> >greg
> >
> >
> >On Mon, 21 Mar 2016, Michael Ferguson wrote:
> >
> >> Hi Zhao -
> >>
> >>> Would you take this as a possible approach? Am I proposing something
> >>>too
> >>> big for a three-
> >>> month job? What would you say? Any suggestions?
> >>
> >> What you are describing below matches well with my expectations and
> >> desires.
> >> I really think that the task-blocking but "evented" under the hood
> >> implementation
> >> is what we will need for good performance and productivity.
> >>
> >> I think it would be reasonable to focus on qthreads and possibly
> >> massivethreads.
> >> I'm not particularly familiar with massivethreads, but I believe it
> >>already
> >> has some support for this type of network I/O polling.
> >>
> >> Anyway, I have tried to guide others thinking about the web-server task
> >> to focus most on making it easy to write a web server that has good
> >> performance.
> >> I'm not particularly concerned about the features of the web server
> >>that is
> >> produced at the end.
> >>
> >> Cheers,
> >>
> >> -michael
> >>
> >> On 3/21/16, 12:01 PM, "赵亮" <[email protected]> wrote:
> >>
> >>> Hi, everyone.
> >>>
> >>> I'm a postgraduate student in Xidian University, Xi'an, China. I would
> >>> like to propose the web-server task for Google Summer of Code. I'll
> >>>share
> >>> some basic ideas about the task-aware socket/net library. Need your
> >>> feedbacks.
> >>>
> >>> I used to program a lot in C/C++. I would like to program in C/C++,
> >>> especially when it comes to performance issues. I had an internship as
> >>> backend developer at Alibaba Group, working on our distributed file
> >>> system. We have a huge amount of data. C/C++
> >>> are good at this, high performance. I learned a lot though the way. But
> >>> the traditional network programming model is not quite human-oriented.
> >>> It's full of callbacks, and would be a hell if you try to do some
> >>> debugging. I'll definitely not vote for exposing
> >>> the raw select/poll-like API to our end users as chapel's net library.
> >>> "We deserve something better". Chapel is a productive programming
> >>> language. I don't think I would call it productive if I still have to
> >>> deal with all these kind of callbacks. I'm thinking we should provide
> >>> something like this as a modern programming
> >>> language:
> >>>
> >>> ln = net.Listern("tcp", ":80")
> >>> for conn in ln.Accept() do
> >>> begin handleConnectin(conn)
> >>>
> >>> We only expose blocking API to chapel user while handling non-blocking
> >>> async networking underneath(in the task layer). This's a much more
> >>> productive approach. The coroutine way is basically something modern
> >>> language like go-lang provided. I have checked
> >>> out the implementation in golang, gevent, tornado during the past days.
> >>> They basically did the something: task-switching when the blocking API
> >>> get called, using the select/poll API on a separate task/thread to
> >>>notify
> >>> the upcoming IO event, though they differ
> >>> from different schedule strategies.
> >>> Chapel has several task
> >>> implementation
> >>>
> >>><http://chapel.cray.com/docs/latest/usingchapel/tasks.html
> >s(light-weigh
> >>>t
> >>> qthreads, fifo pthread, massivethreads). I have glam through the
> >>>qthreads
> >>> source code. It has built-in async system call scheduling support. But
> >>> currently, chapel's qthreads task bindings and the standard library
> >>> hasn't leveraged
> >>> qthreads' built-in async system call feature. There would be a lot of
> >>> decision-making, API design, and coding if I try to d o the
> >>> implementation. But also, the outcome will be remarkable. The network
> >>> programming in chapel will be a pleasure with a huge performance
> >>> improvement. And writing a web server would be quite easy. The
> >>> fifo/pthread implementation could just use the blocking system call.
> >>> Leave the schedule job to the OS. Haven't do much investigation with
> >>>the
> >>> massivethreads library yet.
> >>>
> >>>
> >>> Would you take this as a possible approach? Am I proposing something
> >>>too
> >>> big for a three-month job? What would you say? Any suggestions?
> >>>
> >>>
> >>>
> >>> The mailing list archive has some related discussion. [tasking layer
> >>>and
> >>> asynchronous gets
> >>> <https://sourceforge.net/p/chapel/mailman/message/27398790/>] [request
> >>> for review:
> >>> net moudule
> >>> <https://sourceforge.net/p/chapel/mailman/message/31744487/>].
> >>>
> >>>
> >>> --zhao
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>-------------------------------------------------------------------------
> >>-----
> >> Transform Data into Opportunity.
> >> Accelerate data analysis in your applications with
> >> Intel Data Analytics Acceleration Library.
> >> Click to learn more.
> >> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
> >> _______________________________________________
> >> Chapel-developers mailing list
> >> [email protected]
> >> https://lists.sourceforge.net/lists/listinfo/chapel-developers
>
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
_______________________________________________
Chapel-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-developers
