Request feedback on my draft proposal based on this idea
https://docs.google.com/document/d/1Mpp5ocvRmermVvVOEJtEcAioIrMiy_aAzkih3V_eLyU/edit?usp=sharing
--zhao
2016-03-22 11:27 GMT+08:00 赵亮 <[email protected]>:
> Hi, Michael, Greg
>
> I totally agree that we should minimize the runtime changes, especially
> the 4 different task implementations. Many things rely on the runtime task
> interface. Modify the task interface will be quite risky. We should find a
> way to walk around the problem unless we have to. There are tradeoffs about
> how net library talks to the task implementations and how they schedule the
> IO tasks. I'll try to minimize the related code changes. Maybe sync
> variables would just fit in a basic version. I think we should take my
> proposal as an experiment or a draft net library. If it works fine, then
> move on this way, make the decision, stabilize the API.
>
> --zhao
>
>
>
>
>
> 2016-03-22 4:13 GMT+08:00 Michael Ferguson <[email protected]>:
>
>> Hi Greg -
>>
>> On 3/21/16, 4:08 PM, "Greg Titus" <[email protected]> wrote:
>>
>> >Hi Michael, Zhao --
>> >
>> >Are there good reasons for solving this in C code in the runtime rather
>> >than in Chapel code in, say, the modules? Put another way, is there a
>> >lack of Chapel language features that, if they were present, would let
>> >us implement this more naturally in module code?
>>
>> I don't think we have enough of this task figured out to know
>> what would go in the runtime vs. the modules. I generally
>> agree it's better to put code in the modules. In this case,
>> part of what we hope to do is to interact intelligently with
>> the tasking system, and that might mean changing it (but
>> then again it might not - perhaps sync variables and/or
>> chpl_task_yield are sufficient). I don't think Zhao or I
>> were trying to say that we knew we needed to change the
>> tasking layer.
>>
>> -michael
>>
>>
>> >
>> >(I always worry about the maintenance overhead of adding to the runtime
>> >interfaces, especially for things like the tasking layer, which has many
>> >implementations. Any feature we add to the runtime tasking interface
>> >has to be implemented 4 times, right now. We may get rid of 1 of those
>> >implementations soon, but we don't have plans to go below that.)
>> >
>> >greg
>> >
>> >
>> >On Mon, 21 Mar 2016, Michael Ferguson wrote:
>> >
>> >> Hi Zhao -
>> >>
>> >>> Would you take this as a possible approach? Am I proposing something
>> >>>too
>> >>> big for a three-
>> >>> month job? What would you say? Any suggestions?
>> >>
>> >> What you are describing below matches well with my expectations and
>> >> desires.
>> >> I really think that the task-blocking but "evented" under the hood
>> >> implementation
>> >> is what we will need for good performance and productivity.
>> >>
>> >> I think it would be reasonable to focus on qthreads and possibly
>> >> massivethreads.
>> >> I'm not particularly familiar with massivethreads, but I believe it
>> >>already
>> >> has some support for this type of network I/O polling.
>> >>
>> >> Anyway, I have tried to guide others thinking about the web-server task
>> >> to focus most on making it easy to write a web server that has good
>> >> performance.
>> >> I'm not particularly concerned about the features of the web server
>> >>that is
>> >> produced at the end.
>> >>
>> >> Cheers,
>> >>
>> >> -michael
>> >>
>> >> On 3/21/16, 12:01 PM, "赵亮" <[email protected]> wrote:
>> >>
>> >>> Hi, everyone.
>> >>>
>> >>> I'm a postgraduate student in Xidian University, Xi'an, China. I
>> would
>> >>> like to propose the web-server task for Google Summer of Code. I'll
>> >>>share
>> >>> some basic ideas about the task-aware socket/net library. Need your
>> >>> feedbacks.
>> >>>
>> >>> I used to program a lot in C/C++. I would like to program in C/C++,
>> >>> especially when it comes to performance issues. I had an internship as
>> >>> backend developer at Alibaba Group, working on our distributed file
>> >>> system. We have a huge amount of data. C/C++
>> >>> are good at this, high performance. I learned a lot though the way.
>> But
>> >>> the traditional network programming model is not quite human-oriented.
>> >>> It's full of callbacks, and would be a hell if you try to do some
>> >>> debugging. I'll definitely not vote for exposing
>> >>> the raw select/poll-like API to our end users as chapel's net library.
>> >>> "We deserve something better". Chapel is a productive programming
>> >>> language. I don't think I would call it productive if I still have to
>> >>> deal with all these kind of callbacks. I'm thinking we should provide
>> >>> something like this as a modern programming
>> >>> language:
>> >>>
>> >>> ln = net.Listern("tcp", ":80")
>> >>> for conn in ln.Accept() do
>> >>> begin handleConnectin(conn)
>> >>>
>> >>> We only expose blocking API to chapel user while handling non-blocking
>> >>> async networking underneath(in the task layer). This's a much more
>> >>> productive approach. The coroutine way is basically something modern
>> >>> language like go-lang provided. I have checked
>> >>> out the implementation in golang, gevent, tornado during the past
>> days.
>> >>> They basically did the something: task-switching when the blocking
>> API
>> >>> get called, using the select/poll API on a separate task/thread to
>> >>>notify
>> >>> the upcoming IO event, though they differ
>> >>> from different schedule strategies.
>> >>> Chapel has several task
>> >>> implementation
>> >>>
>> >>><http://chapel.cray.com/docs/latest/usingchapel/tasks.html
>> >s(light-weigh
>> >>>t
>> >>> qthreads, fifo pthread, massivethreads). I have glam through the
>> >>>qthreads
>> >>> source code. It has built-in async system call scheduling support. But
>> >>> currently, chapel's qthreads task bindings and the standard library
>> >>> hasn't leveraged
>> >>> qthreads' built-in async system call feature. There would be a lot of
>> >>> decision-making, API design, and coding if I try to d o the
>> >>> implementation. But also, the outcome will be remarkable. The network
>> >>> programming in chapel will be a pleasure with a huge performance
>> >>> improvement. And writing a web server would be quite easy. The
>> >>> fifo/pthread implementation could just use the blocking system call.
>> >>> Leave the schedule job to the OS. Haven't do much investigation with
>> >>>the
>> >>> massivethreads library yet.
>> >>>
>> >>>
>> >>> Would you take this as a possible approach? Am I proposing something
>> >>>too
>> >>> big for a three-month job? What would you say? Any suggestions?
>> >>>
>> >>>
>> >>>
>> >>> The mailing list archive has some related discussion. [tasking layer
>> >>>and
>> >>> asynchronous gets
>> >>> <https://sourceforge.net/p/chapel/mailman/message/27398790/>]
>> [request
>> >>> for review:
>> >>> net moudule
>> >>> <https://sourceforge.net/p/chapel/mailman/message/31744487/>].
>> >>>
>> >>>
>> >>> --zhao
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>> >>
>>
>> >>-------------------------------------------------------------------------
>> >>-----
>> >> Transform Data into Opportunity.
>> >> Accelerate data analysis in your applications with
>> >> Intel Data Analytics Acceleration Library.
>> >> Click to learn more.
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
>> >> _______________________________________________
>> >> Chapel-developers mailing list
>> >> [email protected]
>> >> https://lists.sourceforge.net/lists/listinfo/chapel-developers
>>
>>
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
_______________________________________________
Chapel-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/chapel-developers
