OK. That sounds good. By the way, the documentation doesn't say whether the
following is impossible: Suppose I have a long script and I have several of
such stages where I want the prior execution to be completed before moving on,
could I have more "execs"
For example:
<Pig Stuff that needs to be run prior to exec 1 >
exec;
< Pig Stuff that needs to be run prior to exec 2 >
exec:
So on..
Would the presence of the first exec in the portion of the pig script prior to
the second exec affect the second exec when it is executed?
Thanks
Esh
On Aug 10, 2011, at 11:28 AM, Thejas Nair wrote:
> You can use the exec command to force pig run the portion of script before
> the exec, before it runs rest of it.
>
> it will look like -
> STORE doc_ids INTO '$TEMP';
> exec;
> modifieddocs = FOREACH docs GENERATE myUDF('$TEMP', doc_id);
>
> This is not ideal, as there might be other parts of the script that could
> have been run in parallel. It would be useful to enhance the syntax to
> support this kind of implicit dependency.
> I have created a jira to track this -
> https://issues.apache.org/jira/browse/PIG-2212 . (Feel free to contribute
> patches ! :) )
>
>
> The documentation of exec (http://pig.apache.org/docs/r0.9.0/cmds.html#exec)
> needs to be fixed, i have created a jira for that -
> https://issues.apache.org/jira/browse/PIG-2211
>
>
>
> Thanks,
> Thejas
>
>
> On 8/10/11 9:10 AM, Eshwaran Vijaya Kumar wrote:
>> All,
>> I have some data that I would like to store into a file and then load it
>> in a UDF to do some operations in the next pig statement.
>> For example,
>> doc_ids = FOREACH docs GENERATE doc_id;
>> STORE doc_ids INTO '$TEMP';
>> modifieddocs = FOREACH docs GENERATE myUDF('$TEMP', doc_id);
>>
>> where myUDF loads doc_ids stored in '$TEMP' and does some operation using
>> $TEMP and doc_id. Now I need to make sure that the "STORE doc_ids INTO
>> '$TEMP';" occurs before the FOREACH statement, so that loading the index
>> occurs smoothly. Is there anyway to guarantee that that can happen?
>>
>> Thanks
>> Esh
>
