[
https://issues.apache.org/jira/browse/PIG-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12604553#action_12604553
]
Alan Gates commented on PIG-262:
--------------------------------
In response to Shravan's comment: "In the java doc you say that PlanOptimizer
need not be subclassed but you do so for the LogicalOptimizer. Wasn't just a
plan optimizer with LogicalTransformer sufficient?" I decided it was a little
cleaner to put all the rules in one place, so that I could instantiate them for
testing and the real code in the same way. Otherwise everytime I defined the
logical optimizer, I'd have to define all the rules. I should update the
javadoc comments.
In response to the type checker concerns. I went back and forth on this, and
chose to put them in the optimizer because it actually transforms the tree,
while the validators mostly check the tree. I could put this in the validators
in front of the type checker (that's where I originally had it). Since the
cast I'm inserting just mimics whatever the load in front of it does, hopefully
it won't get it wrong (ie we don't need to typecheck again). One way or
another the optimizer will be rearranging the tree, which will mean inserting
some casts, etc.
Also, I realized I neglected to put include tests that actually use this new
functionality. I'll be adding those before I submit this patch.
> Pig needs an optimizer
> ----------------------
>
> Key: PIG-262
> URL: https://issues.apache.org/jira/browse/PIG-262
> Project: Pig
> Issue Type: Sub-task
> Components: impl
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: optimizer.patch
>
>
> We need to add an optimizer to pig. This will enable us to do some
> traditional optimizations, such as filter and projection pushing, join order
> and execution choices, etc. It will also enable optimizations specific to
> map/reduce (such as using the combiner).
> The optimizer will need to operate at various levels, including the logical,
> physical, and possibly map/reduce plan levels.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
