I'm not aware of anyway to do that. I think you're also missing the spirit of Pig. Pig is meant to be a data workflow language. Describe a workflow for your data using PigLatin and Pig will then compile your script to MapReduce jobs. The number of MapReduce jobs that it generates is the smallest number of jobs (based on the optimizers) that Pig thinks it needs to complete the workflow.
Why do you want to control the number of MR jobs? On Tue, Oct 15, 2013 at 10:07 AM, ey-chih chow <eyc...@gmail.com> wrote: > Thanks everybody. Is there anyway we can programmatically control the > number of M-R jobs that a Pig script will generate, similar to write M-R > jobs in Java? > > Best regards, > > Ey-Chih Chow > > > On Tue, Oct 15, 2013 at 6:14 AM, Shahab Yunus <shahab.yu...@gmail.com > >wrote: > > > And Geert's comment about using external-to-Pig approach reminds me that, > > then you have Netflix's PigLipstick too. Nice visual tool for actual > > execution and stores job history as well. > > > > Regards, > > Shahab > > > > > > On Tue, Oct 15, 2013 at 8:51 AM, Geert Van Landeghem <g...@foundation.be > > >wrote: > > > > > You can also use ambrose to monitor execution of your pig script at > > > runtime. Remark: from pig-0.11 on. > > > > > > It show you the DAG of MR jobs and which are currently being executed. > As > > > long as pig-ambrose is connected to the execution of your script > > (workflow) > > > you can replay the workflow. > > > > > > -- > > > kind regards, > > > Geert > > > > > > > > > > > > > > > On 15-okt.-2013, at 14:43, Shahab Yunus <shahab.yu...@gmail.com> > wrote: > > > > > > > Have you tried using ILLUSTRATE and EXPLAIN command? As far as I > know, > > I > > > > don't think they give you the exact number as it depends on the > actual > > > data > > > > but I believe you can interpret it/extrapolate it from the > information > > > > provided by these commands. > > > > > > > > Regards, > > > > Shahab > > > > > > > > > > > > On Tue, Oct 15, 2013 at 3:57 AM, ey-chih chow <eyc...@gmail.com> > > wrote: > > > > > > > >> Hi, > > > >> > > > >> I have a Pig script that has two group-by statements on the the > input > > > data > > > >> set. Is there anybody knows how many M-R jobs the script will > > generate? > > > >> Thanks. > > > >> > > > >> Best regards, > > > >> > > > >> Ey-Chih Chow > > > >> > > > > > > > > >
