I prototyped a simple MapReduce runner that can executes Read.Bounded+ParDo+Combine: https://github.com/peihe/incubator-beam/tree/mr-runner
I am still working on View support, and will give another update once I can get WordCount runs. On Sat, Jul 15, 2017 at 4:45 AM, Vikas RK <[email protected]> wrote: > Thanks Pei, left a few comments, but this looks exciting! > > -Vikas > > On 12 July 2017 at 21:52, Jean-Baptiste Onofré <[email protected]> wrote: > > > Hi, > > > > I will push my branch with the current state of the mapreduce runner. > > > > Regards > > JB > > > > > > On 07/13/2017 04:47 AM, Pei HE wrote: > > > >> Thanks guys! > >> > >> I replied Kenn's comments, and looking forward to more feedbacks and > >> suggestions. > >> > >> Also, could we add a mapreduce-runner branch? > >> > >> Thanks > >> -- > >> Pei > >> > >> > >> On Sat, Jul 8, 2017 at 12:42 AM, Kenneth Knowles <[email protected] > > > >> wrote: > >> > >> Very cool to see this. Commenting a little on the doc. > >>> > >>> On Fri, Jul 7, 2017 at 8:41 AM, Jean-Baptiste Onofré <[email protected]> > >>> wrote: > >>> > >>> Hi Pei, > >>>> > >>>> I also pumped some ideas and part of code from Crunch for the > MapReduce > >>>> runner. > >>>> > >>>> I will push my changes on my github branch and share with you. > >>>> > >>>> Let me take a look on your doc. > >>>> > >>>> Regards > >>>> JB > >>>> > >>>> > >>>> On 07/07/2017 03:11 PM, Pei HE wrote: > >>>> > >>>> Hi all, > >>>>> While JB is working on MapReduce Runner BEAM-165 > >>>>> <https://issues.apache.org/jira/browse/BEAM-165>, I have spent time > >>>>> reading > >>>>> Apache Crunch code and drafted Beam MapReduce Runner One-Pager > >>>>> <https://docs.google.com/document/d/10jJ8pBTZ10rNr_IO5YnggmZ > >>>>> ZG1MU-F47sWg8N6xkBM0/edit#heading=h.bewnehqnt4zd> > >>>>> (mostly > >>>>> around ParDo/Flatten fusion support, and with many missing details). > >>>>> > >>>>> I would like to start the discussion, and get people's attention of > >>>>> supporting MapReduce in Beam. > >>>>> > >>>>> Feel free to make comments and suggestions on that doc. > >>>>> > >>>>> Thanks > >>>>> -- > >>>>> Pei > >>>>> > >>>>> > >>>>> -- > >>>> Jean-Baptiste Onofré > >>>> [email protected] > >>>> http://blog.nanthrax.net > >>>> Talend - http://www.talend.com > >>>> > >>>> > >>> > >> > > -- > > Jean-Baptiste Onofré > > [email protected] > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > >
