Hey Aalekh, > I am an final year undergraduate student at Jaypee Institute Of > Information Technology-Noida, India and through this mail I would like to > propose my interest in "Implementing a Map/Reduce Framework" as a GSOC- > 2016 student. > > In my past experience I have successfully completed GSOC-2015 with MariaDB > Foundation, have also contributed/worked with many different open source > organisations like Mozilla (as Mozilla Winter Of Security student), OWASP > (as Winter Code Sprint Student) , ArangoDB, Snowplow Analytics (both as > software engineering intern), MediaWiki, etc and worked(as software > engineering intern) with many other organisations. > > For the purpose of the project following are my initial approach: > > 1) Instead of Map/Reduce it would be much more rewarding to implement > Google Dataflow Model as it would provide efficient handling of both batch > processing and real time stream processing.
Yes! Good thinking. > 2) Along with Dataflow model, I would also borrow some of the features > from MillWheel [1] and FlumeJava [2] (features such as Fault-tolerance, > running efficient data parallel pipelines, etc). Perfect. Do you have something more concrete in mind? Any use cases? Design ideas? > 3) Construct an execution model as directed graph which would make better > optimisation than Map/Reduce, this approach would be useful as complex > optimisation would require multiple map/reduce steps. Nod, that's what dataflow/async/futures etc. can give you. The only thing to keep in mind is that the execution tree generated by those is implicit, i.e. not directly accessible. In our experience this is not a problem, however. > Finally, I would really appreciate if you could please look into above > steps and further help me with reviews and other possible > idea's/approach's for the project :) We definitely will be here to discuss things as you start putting out ideas, questions, suggestions, etc. I think you have already started looking at HPX itself, if not - it might be a good time to start doing so. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu > > Thank you, > > Aalekh Nigam > [email protected] > https://in.linkedin.com/in/aalekh-nigam-a7962064 > > [1] http://research.google.com/pubs/pub41378.html > [2] http://research.google.com/pubs/pub35650.html _______________________________________________ hpx-users mailing list [email protected] https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
