Hey,

> 2. Local mode and other parallel frameworks
>
> <snip>
> Pigs Live Anywhere
>
> Pig is intended to be a language for parallel data processing. It is not
> tied to one particular parallel framework. It has been implemented first
> on hadoop, but we do not intend that to be only on hadoop.
> </snip>
>
> Are we still holding onto this? What about local mode? Local mode is not
> being treated on equal footing with that of Hadoop for practical
> reasons. However, users expect things that work on local mode to work
> without any hitches on Hadoop.
>
> Are we still designing the system assuming that Pig will be stacked on
> top of other parallel frameworks?
>

FWIW, I appreciate this philosophical stance from Pig. Allowing locally
tested scripts to be migrated to the cluster without breakage is a noble
goal, and keeping the option of (one day) developing an alternative
execution environment for Pig that runs over HDFS but uses a richer physical
set of operators than MapReduce would be great.

Of course, those of you who are running Pig in production will have a much
better sense of the feasibility, rather than desirability, of this
philosophical stance.

Later,
Jeff

Reply via email to