I added you to the Jira "Contributors" role, so you should be able to self-assign the ticket now.
*From: *Pulasthi Supun Wickramasinghe <[email protected]> *Date: *Tue, May 14, 2019 at 10:55 AM *To: *Maximilian Michels, <[email protected]> *Cc: *dev Hi, > > Thanks Kenn and Max for the information. Will read up a little more and > discuss with the Twister2 team before deciding on which route to take. I > also created an issue in BEAM JIRA[1], but I cannot assign this to my self > would someone be able to assign the issue to me. Thanks in advance. > > [1] https://issues.apache.org/jira/browse/BEAM-7304 > > Best Regards > Pulasthi > > On Tue, May 14, 2019 at 6:19 AM Maximilian Michels <[email protected]> wrote: > >> Hi Pulasthi, >> >> Great to hear you're planning to implement a Twister2 Runner. >> >> If you have limited time, you probably want to decide whether to build a >> "legacy" Java Runner or a portable one. They are not fundamentally >> different but there are some tricky implementation details for the >> portable Runner related to the asynchronous communication with the SDK >> Harness. >> >> If you have enough time, first implementing a "legacy" Runner might be a >> good way to learn the Beam model and subsequently creating a portable >> Runner should not be hard then. >> >> To get an idea of the differences, check out the Flink source code: >> - FlinkStreamingTransformTranslators (Java "legacy") >> - FlinkStreamingPortablePipelineTranslator (portable) >> >> Feel free to ask questions here or on Slack. >> >> Cheers, >> Max >> >> On 14.05.19 05:11, Kenneth Knowles wrote: >> > Welcome! This is very cool to hear about. >> > >> > A major caveat about https://beam.apache.org/contribute/runner-guide/ is >> >> > that it was written when Beam's portability framework was more of a >> > sketch. The conceptual descriptions are mostly fine, but the pointers >> to >> > Java helper code will lead you to build a "legacy" runner when it is >> > better to build a portable runner from the start*. >> > >> > We now have four portable runners in various levels of completeness: >> > Spark, Flink, Samza, and Dataflow. I have added some relevant people to >> > the CC for emphasis. You might also join >> > https://the-asf.slack.com/#beam-portability though I prefer the dev >> list >> > since it gives visibility to a much greater portion of the community. >> > >> > Kenn >> > >> > *volunteers welcome to update the guide to emphasize portability first >> > >> > *From: *Pulasthi Supun Wickramasinghe <[email protected] >> > <mailto:[email protected]>> >> > *Date: *Mon, May 13, 2019 at 11:03 AM >> > *To: * <[email protected] <mailto:[email protected]>> >> > >> > Hi All, >> > >> > I am Pulasthi a Ph.D. student at Indiana University. We are planning >> > to develop a beam runner for our project Twister2 [1] [2]. Twister2 >> > is a big data framework which supports both batch and stream >> > processing. If you are interested you can find more information on >> > [2] or read some of our publications [3] >> > >> > I wanted to share our intent and get some guidance from the beam >> > developer community before starting on the project. I was planning >> > on going through the code for Apache Spark and Apache Flink runners >> > to get a better understanding of what I need to do. It would be >> > great if I can get any pointers on how I should approach this >> > project. I am currently reading through the runner-guide >> > <https://beam.apache.org/contribute/runner-guide/>. >> > >> > Finally, I assume that I need to create a JIRA issue to track the >> > progress of this project, right?. I can create the issue but from >> > what I read from the contribute section I would need some permission >> > to assign it to my self, I hope someone would be able to help me >> > with that. Looking forward to working with the Beam community. >> > >> > [1] https://github.com/DSC-SPIDAL/twister2 >> > [2] https://twister2.gitbook.io/twister2/ >> > [3] https://twister2.gitbook.io/twister2/publications >> > >> > Best Regards, >> > Pulasthi >> > -- >> > Pulasthi S. Wickramasinghe >> > PhD Candidate | Research Assistant >> > School of Informatics and Computing | Digital Science Center >> > Indiana University, Bloomington >> > cell: 224-386-9035 >> > >> > > > -- > Pulasthi S. Wickramasinghe > PhD Candidate | Research Assistant > School of Informatics and Computing | Digital Science Center > Indiana University, Bloomington > cell: 224-386-9035 >
