Logical Optimizer: Unique sort rule
-----------------------------------
Key: PIG-1326
URL: https://issues.apache.org/jira/browse/PIG-1326
Project: Pig
Issue Type: New Feature
Components: impl
Affects Versions: 0.6.0
Reporter: Daniel Dai
Assignee: Daniel Dai
We do not have unique sort in Pig Latin. If user want to do a unique sort, they
have to do a distinct followed by a sort. This is two map-reduce jobs. However,
we can identify some patterns in the script. If the user intend to do a unique
sort, we can merge two map-reduce jobs into one. This rule need physical layer
support to deal with unique sort.
Example:
B = distinct A;
C = sort B by *;
=> C = uniq sort A by *;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
