[
https://issues.apache.org/jira/browse/PIG-3279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Johnny Zhang updated PIG-3279:
------------------------------
Attachment: PIG-3279-3.patch.txt
Thanks a lot for your comments, [~daijy]! Appreciate. I changed
LogToPhyTranslationVisitor.java:
1. for RANK BY operation, only include POSort -> POCounter -> PORank ->
POForEach. The current physical plan looks like:
c: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-42
|
|---c: New For Each(true)[bag] - scope-41
| |
| RelationToExpressionProject[bag][*] - scope-32
| |
| |---New For Each(false,true)[tuple] - scope-40
| | |
| | Project[long][0] - scope-38
| | |
| | Project[bag][2] - scope-39
| |
| |---d: PORank[tuple] - scope-37
| | |
| | Project[int][0] - scope-34
| |
| |---d: POCounter[tuple] - scope-36
| | |
| | Project[int][0] - scope-34
| |
| |---d: POSort[tuple]() - scope-35
| | |
| | Project[int][0] - scope-34
| |
| |---Project[bag][1] - scope-33
|
|---b: Package[tuple]{chararray} - scope-29
|
|---b: Global Rearrange[tuple] - scope-28
|
|---b: Local Rearrange[tuple]{chararray}(false) - scope-30
| |
| Project[chararray][1] - scope-31
|
|---a: New For Each(false,false,false)[bag] - scope-27
| |
| Cast[chararray] - scope-19
| |
| |---Project[bytearray][0] - scope-18
| |
| Cast[chararray] - scope-22
| |
| |---Project[bytearray][1] - scope-21
| |
| Cast[int] - scope-25
| |
| |---Project[bytearray][2] - scope-24
|
|---a:
Load(file:///home/xiaoyuz/PIG-new/pig/input1:org.apache.pig.builtin.PigStorage)
- scope-17
2. for RANK operation, there is no difference between nested and non-nested
RANK. Since there is no POPackage, global rearrange for non-nested RANK anyway
However, I still got exception for RANK BY and RANK operations
{noformat}
Caused by: java.lang.RuntimeException: Unable to read counter
pig.counters.counter_2415405541993583480_-1
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PORank.addRank(PORank.java:165)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.PORank.getNextTuple(PORank.java:134)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:281)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNextTuple(POForEach.java:242)
... 13 more
{noformat}
thing get closer, but still not complete. Thanks.
> Support nested RANK
> -------------------
>
> Key: PIG-3279
> URL: https://issues.apache.org/jira/browse/PIG-3279
> Project: Pig
> Issue Type: Improvement
> Reporter: Gianmarco De Francisci Morales
> Assignee: Johnny Zhang
> Attachments: PIG-3279-1.patch.txt, PIG-3279-2.patch.txt,
> PIG-3279-3.patch.txt
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
