[
https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589513#action_12589513
]
Pi Song commented on PIG-161:
-----------------------------
Shravan:-
1) I don't mind if we want to call "Relational" or not ( My "Relational" is
from http://en.wikipedia.org/wiki/Relational_algebra ). What I want is to
signify the shared characteristic of these operators that they are only applied
to bags. This should simplify the way we think when dealing with these
operators. Plus, when we expose interfaces for UDFs, I would like to have
different interfaces like AggregateUDF, ArithmeticUDF, etc.
2) The MapReduce layer idea is interesting. By functionality, the Physical
layer is something that does real complex processing work when the MapReduce
bit is a work distribution mechanism. Possibly we can think of it as "Work
Distribution Layer" where we have local engine and MapReduce implementations ?
Most (or all?) distributed processing frameworks primarily give you work
distribution functionalty, then we should be able to reuse our POs in other
kinds of backends.
Alan:-
I might have missed something. I thought we're creating all the new POs
(particularly ExpressionOPs) to replace EvalFunc. For example in Add.java, we
already have the "add" logic here :-
{noformat}
public Result getNext(Double d) throws ExecException {
byte status;
Result res;
Double left = null, right = null;
res = lhs.getNext(left);
status = res.returnStatus;
if(status != POStatus.STATUS_OK) {
return res;
}
left = (Double) res.result;
res = rhs.getNext(right);
status = res.returnStatus;
if(status != POStatus.STATUS_OK) {
return res;
}
right = (Double) res.result;
res.result = new Double(left + right);
return res;
}
{noformat}
In terms of Type-Checking, I do it in Logical Layer where every LO has
GetType() and GetSchema() so that should be enough for me. I don't have to look
at any lower layer. Please explain if I'm wrong.
> Rework physical plan
> --------------------
>
> Key: PIG-161
> URL: https://issues.apache.org/jira/browse/PIG-161
> Project: Pig
> Issue Type: Sub-task
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch,
> incr4.patch, Phy_AbsClass.patch, pogenerate.patch, pogenerate.patch,
> pogenerate.patch
>
>
> This bug tracks work to rework all of the physical operators as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
