[
https://issues.apache.org/jira/browse/PIG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12606420#action_12606420
]
Alan Gates commented on PIG-162:
--------------------------------
Regarding making common builtin functions operators:
I still like the idea of fastpathing some common operations (count, sum, etc.)
as operators. I appreciate Ben's concern about the slippery slope (we don't
want to end up in the SQL world where half the words in the English language
are keywords). But in practice it's common to heavily optimize your most
commonly used features.
Once we add types, standard functions like SUM, etc. need to be type aware. If
people sum ints and get a double, they aren't going to be happy. And we just
don't have time to build a proper function overload resolution functionality.
So, I propose a compromise.
1. We don't add any keywords.
2. We add a translation table in the logical layer that, for a small set of
builtin functions, translates the generic name to a type specific name. We
then implement type specific versions of that function. So the table would
look something like this:
|| generic name || type || type specific name ||
| SUM | int | longSum |
| SUM | long | longSum |
| SUM | float | doubleSum |
| SUM | double | doubleSum |
| SUM | bytearray | doubleSum |
...
The functions I think we'd want to put in the table would be SUM, MIN, MAX,
AVG, concat (new function to concatenate chararrays or bytearrays), and size
(new function to give size of a tuple (replaces ARITY), bag, or map).
In the future, when we implement real function overloading, we can simply take
out this translation table, with no apparent changes to users.
Thoughts?
> Rework mapreduce submission and monitoring
> ------------------------------------------
>
> Key: PIG-162
> URL: https://issues.apache.org/jira/browse/PIG-162
> Project: Pig
> Issue Type: Sub-task
> Environment: This bug tracks works to rework the submission and
> monitoring interface to map reduce as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: changes.patch, changes1.patch, incr7.patch, incr8.patch,
> mapreduceJumbo.patch, mapreduceJumboWithComInc.patch, modJumbo.patch,
> newChanges.patch, newChanges1.patch, split.png,
> TEST-org.apache.pig.test.TestFilterOpString.txt,
> TEST-org.apache.pig.test.TestLogToPhyCompiler.txt,
> TEST-org.apache.pig.test.TestMapReduce.txt,
> TEST-org.apache.pig.test.TestMRCompiler.txt,
> TEST-org.apache.pig.test.TestMRCompiler.txt,
> TEST-org.apache.pig.test.TestUnion.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
