Hey Evgenii, One month ago I gave a talk in Minsk regarding HPX in general (available in Russian here: https://corehard.by/2016/02/15/conf2016-hpx/). You also may be interested in some other ideas described in stellar gsoc page. There are lots of them. I have some preliminary work done for one — https://github.com/STEllAR-GROUP/hpx/wiki/GSoC-2016-Project-Ideas#a-c-runtime-replacement <https://github.com/STEllAR-GROUP/hpx/wiki/GSoC-2016-Project-Ideas#a-c-runtime-replacement> There are some other projects which I would really like to look at/mentor but don’t have enough time, e.g. implementing parcelport over websockets and etc. So if you are interested, feel free to get in contact!
Sincerely, Anton. > On Mar 16, 2016, at 3:27 PM, Hartmut Kaiser <[email protected]> wrote: > > Hey Evgenii, > > Cc'ing my answer to hpx-users > >> My name's Evgenii. I'm a second year student of BSUIR. >> I want to take part in GSOC 2016, I didn't participated before. >> >> I want to implement a Map/Reduce Framework. >> I have a good knowledge of C, C++, Github, Gerrit. I have no experience in >> Open Source, >> >> I think that this task may be divided into 3 parts >> 1. RPC (Remote Procedure Call) >> 2. Map >> 3. Reduce >> >> I think that a good start will be to develop application, that will apply >> a filter to the image. >> For example, i will divide my image for 4 part, then sent to nodes, apply >> filter, get image, and finally, fold image. As a result this application >> will be a good base to start work on the framework. > > Welcome to GSoC 2016! > > We already responded to another student who is looking into this project. > Here is the essence of this conversation: > >> 1) Instead of Map/Reduce it would be much more rewarding to implement >> Google Dataflow Model as it would provide efficient handling of both batch >> processing and real time stream processing. > > Yes! Good thinking. > >> 2) Along with Dataflow model, I would also borrow some of the features >> from MillWheel [1] and FlumeJava [2] (features such as Fault-tolerance, >> running efficient data parallel pipelines, etc). > > Perfect. Do you have something more concrete in mind? Any use cases? Design > ideas? > >> 3) Construct an execution model as directed graph which would make better >> optimisation than Map/Reduce, this approach would be useful as complex >> optimisation would require multiple map/reduce steps. > > Nod, that's what dataflow/async/futures etc. can give you. The only thing to > keep in mind is that the execution tree generated by those is implicit, i.e. > not directly accessible. In our experience this is not a problem, however. > >> Finally, I would really appreciate if you could please look into above >> steps and further help me with reviews and other possible >> idea's/approach's for the project :) > > We definitely will be here to discuss things as you start putting out ideas, > questions, suggestions, etc. I think you have already started looking at HPX > itself, if not - it might be a good time to start doing so. > > > The above is still true. The idea we had was to find a way to substitute > Map/Reduce with a new, dataflow-based programming model which overcomes the > disadvantages of Map/Reduce (it imposes at least 2 global barriers onto the > computation, thus reducing the possible parallelism, etc.). In general, I'd > ask you to keep poking at the problem and to keep developing your design. It > might be a good idea if you tried to outline things in a bit more detail for > us to really grasp what you're up to. > > HTH > Regards Hartmut > --------------- > http://boost-spirit.com > http://stellar.cct.lsu.edu > > > > _______________________________________________ > hpx-users mailing list > [email protected] > https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
_______________________________________________ hpx-users mailing list [email protected] https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
