[
https://issues.apache.org/jira/browse/PIG-120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12573394#action_12573394
]
Stefan Groschupf commented on PIG-120:
--------------------------------------
Alan, sorry I'm not sure if I can follow you. In general I see 3 kind of how we
can run pig, LocalExecutionEngine, HadoopExecutionEngine - map reduce using a
hadoop cluster and HadoopExecutionEngine - map reduce using hadoops
localJobRunner.
This is very very helpful for debugging and profiling since the
HadoopExecutionEngine is used but all runs in the same jvm.
This patch makes it possible by not using a port in case the name node and job
tracker are "local" and also not opening a remote proxy to the jobtracker in
that case.
Makes that sense?
> support hadoop map reduce in loal mode
> --------------------------------------
>
> Key: PIG-120
> URL: https://issues.apache.org/jira/browse/PIG-120
> Project: Pig
> Issue Type: Bug
> Reporter: Stefan Groschupf
> Attachments: PIG-120_v_1.patch
>
>
> Currently pig support mapreduce and local as execution modes.
> LocalExecutionEngine is used for local and HExecutionEngine for map reduce.
> HExecutionEngine always expect that hadoop runs as cluster with a name node
> and jobtracker listing on a port.
> Though, hadoop can also run in a local mode (LocalJobRunner) this would give
> several advantages.
> First it would speed up the test suite significant. Second it would be
> possible to debug map reduce plans easily.
> For example we was able to debug and reproduce PIG-110 with this method.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
