Once I am back home from Berlin next week, I was looking at working on Responder-Flink, Flink's awesome Streaming capabilities (its not mini-batch like the other hyped ones) makes Flink the perfect distributed engine to implement the Wideskies algorithm.
It would help for Pirk-noobs like me and others if there was a Pirk 101 (via Google hangout maybe). On Wed, Sep 14, 2016 at 2:55 PM, Ellison Anne Williams < eawilli...@apache.org> wrote: > Darin - That would be great! > > Sounds like it will take a bit of time to get it together. > > In the meantime/very near term, we could provide a step-by-step > AWS/GCP/Azure instructions for bringing up a small cluster, running the > distributed tests, and debugging. Admittedly, most of this is handled in > the AWS/GCP/Azure documentation, but, in my experience, the documentation > is confusing and very time consuming to get through the first time. > > The goal is for smart people who want to contribute to Pirk (who don't work > on these systems often) to be able to get up and running quickly to test > their improvements and additions to the codebase. > > On Wed, Sep 14, 2016 at 8:46 AM, Darin Johnson <dbjohnson1...@gmail.com> > wrote: > > > Regarding (ii) I think docker-compose might help with distributed testing > > for development and Terraform scripts for benchmark work in AWS/GCE. I > > might be doing this soon - if interested I can share both. > > > > On Sep 14, 2016 8:06 AM, "Tim Ellison" <t.p.elli...@gmail.com> wrote: > > > > > I'm reluctant to dive into more Pirk dev at the moment because > > > > > > (i) there was plenty of discussion around the release about > > > restructuring the code base into a number of modules. I figure that > > > will be a disruptive change best undertaken with most folks "out of the > > > pool". > > > > > > (ii) I have failed to set up a reliable distributed testing system for > > > myself, and wholeheartedly agree that distrib testing is important for > > > all devs. Hopefully I can fix that as improved instructions come > along, > > > though perhaps we should be approaching infra to get a shared Pirk > > > cluster available for testing? > > > > > > (iii) other stuff gets in the way :-) > > > > > > I realise that these are within my powers to address, and I'm still > keen > > > to participate in moving Pirk forward; so what is the next hill to > > > conquer for the project? > > > > > > Regards, > > > Tim > > > > > >
