Re: Is there any document about the JobControlCompiler
Jeff, Chris Olston answered this a while back: http://markmail.org/thread/xnwutstlftnyycxs (by the way, MarkMail is awesome for searching mailing list archives. Highly recommended.) There are some changes that have to do with sampling and multi-store, but that email will give you the general idea. Also, remember you can always get the MR plan by running describe on a relation. Hope this helps -Dmitriy On Wed, Jul 8, 2009 at 6:24 PM, zhang jianfeng zjf...@gmail.com wrote: Hi all, I found that the following script will be converted into 3 mapreduce jobs: A = *LOAD* '/user/zjffdu/input.txt' *USING* PigStorage(); B = *GROUP* A *BY* $0; B = *FOREACH* B *GENERATE* *group*,COUNT($1); B = *ORDER* B *BY* $1; *DUMP* B; I am very interested to know How Pig compile the script to jobs, reading the source code is a way, but If there’s any document, that would be better. Does anyone know where can I find the related documents ? Or is there any JIRA item related to this ? Thank you in advance. Jeff Zhang.
Re: Is there any document about the JobControlCompiler
Dmitriy , Thank you for your help. On Thu, Jul 9, 2009 at 9:34 AM, Dmitriy Ryaboy dvrya...@cloudera.comwrote: Jeff, Chris Olston answered this a while back: http://markmail.org/thread/xnwutstlftnyycxs (by the way, MarkMail is awesome for searching mailing list archives. Highly recommended.) There are some changes that have to do with sampling and multi-store, but that email will give you the general idea. Also, remember you can always get the MR plan by running describe on a relation. Hope this helps -Dmitriy On Wed, Jul 8, 2009 at 6:24 PM, zhang jianfeng zjf...@gmail.com wrote: Hi all, I found that the following script will be converted into 3 mapreduce jobs: A = *LOAD* '/user/zjffdu/input.txt' *USING* PigStorage(); B = *GROUP* A *BY* $0; B = *FOREACH* B *GENERATE* *group*,COUNT($1); B = *ORDER* B *BY* $1; *DUMP* B; I am very interested to know How Pig compile the script to jobs, reading the source code is a way, but If there’s any document, that would be better. Does anyone know where can I find the related documents ? Or is there any JIRA item related to this ? Thank you in advance. Jeff Zhang.