Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Avik Dey
The Tez incubator proposal seems to have a lot in common with the work on https://issues.apache.org/jira/browse/OOZIE-1178 It is useful to have a workflow application master, which will be capable of running a DAG of jobs. The workflow client submits a DAG request to the AM and then the AM

RE: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Ross Gardler
? Ross Sent from Windows Mail *From:* Ted Dunning ted.dunn...@gmail.com *Sent:* 19 February 2013 05:41 *To:* general@incubator.apache.org *Subject:* Re: [PROPOSAL] Tez to join Apache Incubator This seems like a reasonable project (basically it is the long fabled map-reduce-reduce or MCR

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Arun C Murthy
Thanks Sebastian. The scope includes allowing for a complex DAG within the same 'job' and, as such, it generalizes MapReduce to look more like Stratosphere/Hyracks. The goal is to help better Hive/Pig/Cascading/Crunch etc. Hope that helps. thanks, Arun On Feb 19, 2013, at 1:23 AM, Sebastian

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Ted Dunning
Sorry... I was clear on the need for real diversity on graduation, but was unclear on the need for diversity at the start. And, of course, the project would be unable to graduate with just this list of committers. But that is really just what you said in the first sentence. Incubation is the

RE: [PROPOSAL] Tez to join Apache Incubator

2013-02-19 Thread Ross Gardler
] Tez to join Apache Incubator Sorry... I was clear on the need for real diversity on graduation, but was unclear on the need for diversity at the start. And, of course, the project would be unable to graduate with just this list of committers. But that is really just what you said in the first

[PROPOSAL] Tez to join Apache Incubator

2013-02-18 Thread Arun C Murthy
Folks, I'd like to propose adding Tez to the Apache Incubator: http://wiki.apache.org/incubator/TezProposal Essentially, it's the next step to improve projects in the Apache Hadoop ecosystem such as Apache Hive, Apache Pig, Cascading (ASL2, but not ASF project) by providing a more complex

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-18 Thread Josh Wills
So happy to see this project coming about! I think you have plenty of committers already, but I am looking forward to creating a Crunch backend that executes against Tez-- let me know if there's anything I can to do help. J On Mon, Feb 18, 2013 at 8:29 PM, Arun C Murthy a...@hortonworks.com

Re: [PROPOSAL] Tez to join Apache Incubator

2013-02-18 Thread Ted Dunning
This seems like a reasonable project (basically it is the long fabled map-reduce-reduce or MCR* in google terminology). But it is *very* heavy with Hortonworks developers. By my count, the proportion is over half from HW with only token representation from other companies: 13 Hortonworks 4