[
https://issues.apache.org/jira/browse/HIVE-1107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12933177#action_12933177
]
Arun C Murthy commented on HIVE-1107:
-------------------------------------
I think Russel did a good job explaining it.
I'll add some more:
I believe, mid-to-long term, that Pig/Hive will get significantly smarter about
the way they construct MR jobs - they will want to run some of the nodes in the
DAG, wait for their output (e.g. a sampler) and then make ever more complicated
decisions to modify the DAG. I believe Oozie isn't the right tool to be using
for this purpose. Also, there is no need to force Oozie either, people can use
Azkaban etc. for workflow.
> Generic parallel execution framework for Hive (and Pig, and ...)
> ----------------------------------------------------------------
>
> Key: HIVE-1107
> URL: https://issues.apache.org/jira/browse/HIVE-1107
> Project: Hive
> Issue Type: New Feature
> Components: Query Processor
> Reporter: Carl Steinbach
>
> Pig and Hive each have their own libraries for handling plan execution. As we
> prepare to invest more time improving Hive's plan execution mechanism we
> should also start to consider ways of building a generic plan execution
> mechanism that is capable of supporting the needs of Hive and Pig, as well as
> other Hadoop data flow programming environments.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
