[
https://issues.apache.org/jira/browse/PIG-262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated PIG-262:
---------------------------
Attachment: optimizer2.patch
It is true we should be careful when doing transformation, but I still belive
we need it because many optimizations depend on that. Let's define it more
clearly to alleviate the non-deterministic problem:
1. All matches are taken from the snapshot before any transformation
2. We use well-defined tree walker algorithm: depth-first, dependency-order, etc
3. For every node visited, check all possible matches start from the node
I provide an implementation optimizer2.patch for reference. This is an
incremental patch based on optimizer.patch.
> Pig needs an optimizer
> ----------------------
>
> Key: PIG-262
> URL: https://issues.apache.org/jira/browse/PIG-262
> Project: Pig
> Issue Type: Sub-task
> Components: impl
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: optimizer.patch, optimizer2.patch
>
>
> We need to add an optimizer to pig. This will enable us to do some
> traditional optimizations, such as filter and projection pushing, join order
> and execution choices, etc. It will also enable optimizations specific to
> map/reduce (such as using the combiner).
> The optimizer will need to operate at various levels, including the logical,
> physical, and possibly map/reduce plan levels.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
