[
https://issues.apache.org/jira/browse/PIG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12594646#action_12594646
]
Alan Gates commented on PIG-162:
--------------------------------
Why is there a separate class for map only jobs? The collection at the end of
the map is slightly different, but it would be good if we could find a way to
share the rest of the code between the two classes, which appears to be
identical. I like that because they're separate there isn't an if statement on
each record to decide which type of collector to pass it to. Perhaps they could
subclass a common class that contained all the same logic and then each of them
just handle the collection differently.
JobControlCompiler is walking the MR plan itself. Why doesn't it extend visitor
and depend on the visitor to do that for it instead? If it needs to walk the
tree in a distinctive way that could be encoded as another walker and used in
the visitor. It needs to be able to determine when it's found the top of the
list (there are not predecessors) so that it knows to create the JobControl
object and then start unrolling. But this can be done in a visitor.
Is Try1 a real test? It looks like it might have been a test you tried just to
make sure you could set up test stuff.
Let's put the files GoldenFiles and InputFiles in
test/org/apache/pig/test/data. This will match the location that test input and
output files are already being placed in the main branch.
Many people are still using older style junit. Many of your unit tests don't
extend TestCase. This causes them to fail when I run them, because my version
of junit doesn't recognize them as tests.
When I run the unit tests I get failures in TestUnion and TestMRCompiler. I'll
attach those results separately.
> Rework mapreduce submission and monitoring
> ------------------------------------------
>
> Key: PIG-162
> URL: https://issues.apache.org/jira/browse/PIG-162
> Project: Pig
> Issue Type: Sub-task
> Environment: This bug tracks works to rework the submission and
> monitoring interface to map reduce as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: mapreduceJumbo.patch, split.png
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
