[
https://issues.apache.org/jira/browse/PIG-162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shravan Matthur Narayanamurthy updated PIG-162:
-----------------------------------------------
Attachment: changes.patch
Includes some changes that have accumulated over time. Most of them are fixes
to get the end to end tests working. Some others are bugs I found while running
through the code and some are changes to the name() methods of the operators so
that we can debug better using the PlanPrinter. I have also modified POUserFunc
& some others which hold transient references to function specs more efficient
by removing the if(func==null) checks. I don't know why it did not strike me
then but the soln is simple. I have just overridden the readObject method of
the serialization framework and after calling a default read method, I
instantiate the func from the spec. So whenever other methods are called they
can be sure that they will get a non null func handle. To give an overview of
the changes here is a list:
1) Byte Comparison
2) Launcher separation
3) Make basic types, Cast & Sort serializable
4) Combiner plus Combiner support during JobCreation - Though I don't have test
cases for this one, I am pretty confident that it gets launched. I was trying
to get a hand crafted case working. But the limitations of the generate and
chained projects did not enable me to do so. However, I saw through manually at
least that the combiner gets called and if used appropriately, it shoud not
have any problems.
5) Changes to translator that I had done to get some of the test cases working
> Rework mapreduce submission and monitoring
> ------------------------------------------
>
> Key: PIG-162
> URL: https://issues.apache.org/jira/browse/PIG-162
> Project: Pig
> Issue Type: Sub-task
> Environment: This bug tracks works to rework the submission and
> monitoring interface to map reduce as described in
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: changes.patch, changes1.patch, incr7.patch, incr8.patch,
> mapreduceJumbo.patch, mapreduceJumboWithComInc.patch, modJumbo.patch,
> split.png, TEST-org.apache.pig.test.TestMapReduce.txt,
> TEST-org.apache.pig.test.TestMRCompiler.txt,
> TEST-org.apache.pig.test.TestMRCompiler.txt,
> TEST-org.apache.pig.test.TestUnion.txt
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
