[
https://issues.apache.org/jira/browse/PIG-1434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883958#action_12883958
]
Thejas M Nair commented on PIG-1434:
------------------------------------
Should we have a special syntax when relational alias is used as a scalar ?
Something like "[C]" (instead of just "C") .
I think the special sytax is useful because -
1. Users might accidentally use relation-op alias in this scalar context and
not figure out the problem until runtime (when the evaluation of the relation
results in more than one row).
2. It will prevent surprises if a column with same name as alias is introduced
in a new version of input data . Assuming the load function implements
LoadMetadata.getSchema() and the load statement does not specify a new schema.
With current plan, if there is a new column by same name, the column gets used
instead of relation-op value. This will give different results, not what the
user expects.
> Allow casting relations to scalars
> ----------------------------------
>
> Key: PIG-1434
> URL: https://issues.apache.org/jira/browse/PIG-1434
> Project: Pig
> Issue Type: Improvement
> Reporter: Olga Natkovich
> Assignee: Aniket Mokashi
> Fix For: 0.8.0
>
> Attachments: scalarImpl.patch
>
>
> This jira is to implement a simplified version of the functionality described
> in https://issues.apache.org/jira/browse/PIG-801.
> The proposal is to allow casting relations to scalar types in foreach.
> Example:
> A = load 'data' as (x, y, z);
> B = group A all;
> C = foreach B generate COUNT(A);
> .....
> X = ....
> Y = foreach X generate $1/(long) C;
> Couple of additional comments:
> (1) You can only cast relations including a single value or an error will be
> reported
> (2) Name resolution is needed since relation X might have field named C in
> which case that field takes precedence.
> (3) Y will look for C closest to it.
> Implementation thoughts:
> The idea is to store C into a file and then convert it into scalar via a UDF.
> I believe we already have a UDF that Ben Reed contributed for this purpose.
> Most of the work would be to update the logical plan to
> (1) Store C
> (2) convert the cast to the UDF
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
