RE: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Ross Gardler
companies. There are no statistics that say a podling can or can’t graduate, only behaviours. Ross Sent from Windows Mail *From:* Ted Dunning *Sent:* 19 February 2013 16:13 *To:* Ross Gardler *CC:* general@incubator.apache.org *Subject:* Re: [PROPOSAL] Tez to join Apache Incubator Sorry... I was

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Ted Dunning
Sorry... I was clear on the need for real diversity on graduation, but was unclear on the need for diversity at the start. And, of course, the project would be unable to graduate with just this list of committers. But that is really just what you said in the first sentence. Incubation is the tim

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Arun C Murthy
Thanks Sebastian. The scope includes allowing for a complex DAG within the same 'job' and, as such, it generalizes MapReduce to look more like Stratosphere/Hyracks. The goal is to help better Hive/Pig/Cascading/Crunch etc. Hope that helps. thanks, Arun On Feb 19, 2013, at 1:23 AM, Sebastian S

RE: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Ross Gardler
? Ross Sent from Windows Mail *From:* Ted Dunning *Sent:* 19 February 2013 05:41 *To:* general@incubator.apache.org *Subject:* Re: [PROPOSAL] Tez to join Apache Incubator This seems like a reasonable project (basically it is the long fabled map-reduce-reduce or MCR* in google terminology). But

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Sebastian Schelter
Hi, This proposal looks very interesting to me. What exactly is the scope of Tez? Does it aim to be a general data flow system such as Stratosphere[1] or Hyracks[2]? Or will it still be executing Map and Reduce tasks, that are composable in a more flexible manner? Best, Sebastian [1] http://dl.a

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Avik Dey
The Tez incubator proposal seems to have a lot in common with the work on https://issues.apache.org/jira/browse/OOZIE-1178 > It is useful to have a workflow application master, which will be capable > of running a DAG of jobs. The workflow client submits a DAG request to the > AM and then the AM w

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-18 Thread Ted Dunning
This seems like a reasonable project (basically it is the long fabled map-reduce-reduce or MCR* in google terminology). But it is *very* heavy with Hortonworks developers. By my count, the proportion is over half from HW with only token representation from other companies: 13 Hortonworks 4

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-18 Thread Josh Wills
So happy to see this project coming about! I think you have plenty of committers already, but I am looking forward to creating a Crunch backend that executes against Tez-- let me know if there's anything I can to do help. J On Mon, Feb 18, 2013 at 8:29 PM, Arun C Murthy wrote: > Folks, > > I'