Hi Chris, If you guys think it would be helpful/interesting to the meetup audience, my colleague Tim and I would be interested in talking about how we use Airflow to coordinate our Spark jobs on EMR.
I wrote the EMR tooling <https://github.com/apache/incubator-airflow/pull/1630> (hook, operators, sensors) for Airflow so hopefully this could be a good video documentation of how to use it. Tim wrote many of our Spark jobs and has more context on how/why we use Airflow in this way for it. Let me know if you think this would be good. Happy to put together a short abstract. Thanks, Rob On Thu, Oct 20, 2016 at 8:43 AM, Chris Riccomini <[email protected]> wrote: > Hey all, > > We have one more slot to fill for the next meetup at WePay on November 16: > > http://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/ > 234778571/ > > The slot is for 15m + 5m Q&A, so it's not too much of a commitment. > Would you like to speak? If so, let me know! > > Cheers, > Chris >
