[ https://issues.apache.org/jira/browse/HIVE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933170#action_12933170 ]
Arun C Murthy commented on HIVE-1107: ------------------------------------- +1 on the direction to get Pig and Hive to use common infrastructure for DAG execution. {quote} 1) A way to serialize and exchange this DAG (e.g. Avro, JSON, XML) 2) A service to execute the DAG and ensure it runs to completion {quote} +1 Some more: # Ability to modify the DAG on the fly, potentially in reaction to execution of parents of the nodes. # Maybe shared infrastructure for ability to restart the necessary components of the DAG etc. ---- I agree with Russell that Oozie seems too complicated for this task. Potentially, as Zheng suggested, a generalized form of JobControl from Map-Reduce could be the answer, it could be something that Pig, Hive and potentially even Oozie can co-opt. > Generic parallel execution framework for Hive (and Pig, and ...) > ---------------------------------------------------------------- > > Key: HIVE-1107 > URL: https://issues.apache.org/jira/browse/HIVE-1107 > Project: Hive > Issue Type: New Feature > Components: Query Processor > Reporter: Carl Steinbach > > Pig and Hive each have their own libraries for handling plan execution. As we > prepare to invest more time improving Hive's plan execution mechanism we > should also start to consider ways of building a generic plan execution > mechanism that is capable of supporting the needs of Hive and Pig, as well as > other Hadoop data flow programming environments. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.