[
https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chopra updated PIG-161:
-------------------------------
Attachment: posort.patch
Attached is the POSort implementation. It was made such that it can be used for
the local backend. The PhysicalToMapReduce compiler should take care of
implementing the jobs associated with sorting. This is because the distribution
of the input data is not known and some reducers might get most of the tuples
if the distribution is particularly skewed.
This way, the PhysicalToMapReduce compiler would be aware of the sort operator
in the plan and should be able to effectively convert it into 2 jobs. One to
find out the data distribution and the second to do the final sorting based on
the information obtained. This was also the reason converting Sort to other
basic physical operators didn't seem like a good idea. Comments??
> Rework physical plan
> --------------------
>
> Key: PIG-161
> URL: https://issues.apache.org/jira/browse/PIG-161
> Project: Pig
> Issue Type: Sub-task
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch,
> incr4.patch, Phy_AbsClass.patch, pogenerate.patch, pogenerate.patch,
> pogenerate.patch, posort.patch
>
>
> This bug tracks work to rework all of the physical operators as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
