I thought that pig built a logical plan for every bag that the user defines, and that for next commands it would only combine the previous ones with the new ones.
2010/10/24 Daniel Dai <[email protected]> > Not sure if I understand your question. Logical plan will be created when > you "store", "dump" or "run/exec" a script. > Don't those commands make the compilation of a logical plan into a physical one? I thought it went something like this . . . Logical plan is merely a internal structure for your Pig script. If you have > script: > > A = load 'xxxx'; > ---> A logical plan is made. > B = load 'yyyy'; > ---> B logical plan is made. > C = join A by $0, B by $0; > ---> C logical plan is made by joining plans A and B which would produce the logical plan you pointed out > > Logical plan will be: > load load > \ / > join > > I thought this was the way it worked because of the paper, have you guys change it? if so, is there any new documentation on it please? So my question would be if there are any other commands like the LOAD command that originate a new logical plan because all the rest will be just combinations of the previous logical plans (sorry I mistyped in my first email, I was referening to the LOAD command and not to the STORE one). Thanks Daniel! > We then further transform logical plan to map-reduce plan, which will > decide which map reduce plan each operator goes. > > Daniel > > -----Original Message----- From: Renato Marroquín Mogrovejo > Sent: Sunday, October 24, 2010 12:59 PM > To: [email protected] > Subject: Understanding Logical Plan > > > Hi everyone, > > I know Pig builds a logical plan for every bag that the user defines, and > that every new bag will have as its logical plan the combination of the > input bags' logical plans and the comands it uses. Is there any other > command that creates a logical plan? I guess the one that creates the first > logical plan is the STORE comand. And how are the other commands' logical > plans combined? For example if I have a join between two relations, it will > create a single logical plan that contains the loading logical plans, but > how does Pig decide which relation goes where? Or is that a part that > happens while compiling the logical plan into MapReduce and not in the > logical plan? > Thanks in advanced. > > Renato M. >
