Ashutosh Chauhan updated PIG-845:

    Attachment: merge-join-1.patch

Specification: http://wiki.apache.org/pig/PigMergeJoin

Updated patch with following enhancements:

a) Got completely rid of POForEach for generating joined output tuples.
b) Creating output tuple of required size and then doing set instead of append.
c) Caching of key as suggested by Pradeep in previous comment.
d) Creating new arraylist for holding buffered left tuples instead of clearing 
it, thus avoiding resizing of array.
a) Added typecasting for index keys, thus making join work when schemas are 
b) Refactored visit(LOJoin loj) in LogToPhyTranslationVisitor to avoid 
duplicate code.

Error Handling:
a) Better error handling at various places.
b) Added validateMergeJoin() in LogToPhyTranslationVisitor to generate 
exception where Merge Join cant be used.
c) Added more tests.

Merge Join doesn't work when there are splits, streaming and order-by in 
predecessors or streaming is present in successors.
Some of these are related to an issue outlined here: 
https://issues.apache.org/jira/browse/PIG-858 and requires work in MRCompiler.
Currently we detect these conditions in validateMergeJoin() and fail at compile 

> -----------------------
>                 Key: PIG-845
>                 URL: https://issues.apache.org/jira/browse/PIG-845
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Olga Natkovich
>         Attachments: merge-join-1.patch, merge-join-for-review.patch
> Thsi join would work if the data for both tables is sorted on the join key.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to