Pradeep Kamath updated PIG-651:

    Status: Patch Available  (was: Open)

Attached patch implements a simpler POForEachNoFlatten whenever no flattens are 
present in the POForEach (this is determined in LogToPhyTranslator and is also 
used in CombinerOptimizer for the map and combine plan foreachs). Initial tests 
show marginal speedup. (10 seconds out of 9 mins 50 secs in one particular 
group by query). 

> PERFORMANCE: Use specialized POForEachNoFlatten for cases where the foreach 
> has no flattens
> -------------------------------------------------------------------------------------------
>                 Key: PIG-651
>                 URL: https://issues.apache.org/jira/browse/PIG-651
>             Project: Pig
>          Issue Type: Improvement
>    Affects Versions: types_branch
>            Reporter: Pradeep Kamath
>            Assignee: Pradeep Kamath
>             Fix For: types_branch
>         Attachments: PIG-651.patch
> POForEach has lot of code to handle flattening (cross product) of the fields 
> in the generate. This is relevant only when atleast one field in the generate 
> needs to be flattened. If all fields in the generate do not need to be 
> flattened, a more simplified and hopefully more efficient POForEach can be 
> used.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to