[
https://issues.apache.org/jira/browse/PIG-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Santhosh Srinivasan updated PIG-335:
------------------------------------
Attachment: PIG_335.patch
The attached patch, PIG_335.patch adds the lineage feature to PIG. The patch
adds more than 50 test cases to TestTypeCheckingValidator as the feature
touches all the logical operators.
All unit tests pass. TestStreaming had 1 failure which was not reproducible
when it was run on a standalone basis.
> Casting does not work in certain cases with multiple loads
> ----------------------------------------------------------
>
> Key: PIG-335
> URL: https://issues.apache.org/jira/browse/PIG-335
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Santhosh Srinivasan
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG_335.patch
>
>
> Given a script like:
> A = load 'bla' as (x, y) using Loader1();
> B = load 'morebla' as (s, t) using Loader2();
> C = cogroup A by x, B by s;
> D = foreach C generate flatten(A), flatten(B);
> E = foreach D generate x, y, t + 1;
> In this case, in the last foreach, a cast will need to be added to t + 1 to
> allow t (a byte array) to be added to an integer. We use load functions to
> handle this late casting. The issue is that we do not currently have a way
> to know whether to use Loader1 or Loader2 to cast the data. We need to track
> the lineage of fields so that the cast operator can select the correct loader.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
