Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Pig Wiki" for change 

The following page has been changed by Shravan Narayanamurthy:

New page:
= Local Execution Mode using LocalJobRunner from Hadoop =
Currently we have an additional Local execution engine which handles the local 
execution mode. The physical plan for this has to be built separately and its 
execution should be done differently. This also introduces different operators 
for local and map reduce.

Instead of this, we can reuse Hadoop's LocalJobRunner to execute the same Map 
Reduce Physical plans locally. So we compile the logical plan into a map reduce 
physical plan and create the jobcontrol object corresponding to the mapred 
plan. We just need to write a separate launcher which will submit the job to 
the LocalJobRunner instead of submitting to an external Job Tracker.

== Pros ==
* Code Reuse
* No need to write and maintain
   * Different operators
   * Different logical to physical tranlators
   * Different launchers
* The current framework does not have any progress reporting. With this 
approach we will have it at no extra cost.

== Cons ==
* Not sure how stable LocalJobRunner is.
* Found some bugs in hadoop-15 on it which makes it practically useless for us 
right now.
* These have been fixed however in hadoop-16
* Not sure how this will affect Example generator

Reply via email to