[
https://issues.apache.org/jira/browse/PIG-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604915#action_12604915
]
Pi Song commented on PIG-262:
-----------------------------
A minute, I think transformer and validator being interleaved is logically
correct.
Let's think about the way we process our plans like data that goes through a
number of stages. In some stages, you might want to transform but in some
stages you might just want to validate. We just don't have it modeled as graph
right now but that doesn't matter.
Our plan processing is not complex (from what I can see it is very linear) We
just have to interleave transformer and validator in the right order manually
in the code!
> Pig needs an optimizer
> ----------------------
>
> Key: PIG-262
> URL: https://issues.apache.org/jira/browse/PIG-262
> Project: Pig
> Issue Type: Sub-task
> Components: impl
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: optimizer.patch
>
>
> We need to add an optimizer to pig. This will enable us to do some
> traditional optimizations, such as filter and projection pushing, join order
> and execution choices, etc. It will also enable optimizations specific to
> map/reduce (such as using the combiner).
> The optimizer will need to operate at various levels, including the logical,
> physical, and possibly map/reduce plan levels.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
