Hi folks, Siddharth, Bikas and I have been working on looking at how to re-factor/clean up the tez-engine layer ( i.e the Input/Output/Processor and Task constructs ) to allow non-MR processors to be built and allow easy introduction of new inputs and outputs.
I posted an initial design/overview draft on TEZ-398 ( https://issues.apache.org/jira/secure/attachment/12600691/TEZ-398-Engine-Design.pdf ). Please feel free to add your comments to the jira. If you are interested in working on any parts of the changes, comment on either the jira and/or mailing lists. If there are folks interested, we can set up a meeting where we can discuss the proposed design. Given that these changes are going to make the current implementation unstable for folks that are starting to use to tez ( in hive/pig ), we plan to create a new branch ( likely to be named TEZ-398 ) to work on these engine changes. thanks -- Hitesh
