[
https://issues.apache.org/jira/browse/PIG-441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated PIG-441:
---------------------------
Attachment: PIG-441.patch
This patch does a couple of things:
1) add object comparators to the PigXRawComparator classes so that in places
where hadoop uses the object instead of null comparators the desc behavior can
still be handled.
2) Change FindQuantiles UDF to take in its constructor an array that indicates
ascending vs descending order so that the quantiles can be correclty computed.
> Order by desc does not work with more than one reducer
> ------------------------------------------------------
>
> Key: PIG-441
> URL: https://issues.apache.org/jira/browse/PIG-441
> Project: Pig
> Issue Type: Bug
> Components: impl
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Alan Gates
> Fix For: types_branch
>
> Attachments: PIG-441.patch
>
>
> Order by with desc works if you use a single reducer. But with more than one
> reducer the partitions are not correctly constructed. The SortParitioner
> uses the object comparators, not the raw the comparators. The object
> comparators have no concept of order by descending. So they build the
> partitions for ascending order. The end result is data sorted descending
> within the partition but not across partitions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
