[jira] Commented: (PIG-926) Merge-Join phase 2
[ https://issues.apache.org/jira/browse/PIG-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745510#action_12745510 ] Ashutosh Chauhan commented on PIG-926: -- Findbugs warning is about dummyTuple. A dummyTuple is used as an argument to call appropriate overloaded getNext() of physical operator. Since this is just a marker, it is initialized as null and never updated. Findbugs thinks that it will always be null, which is true, but it doesn't affect in any way. There is no workaround to get rid of this warning. Merge-Join phase 2 -- Key: PIG-926 URL: https://issues.apache.org/jira/browse/PIG-926 Project: Pig Issue Type: Improvement Components: impl Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor Attachments: mj_phase2_1.patch This jira is created to keep track of phase-2 work for MergeJoin. Various limitations exist in phase-1 for Merge Join which are listed on: http://wiki.apache.org/pig/PigMergeJoin Those will be addressed here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-926) Merge-Join phase 2
[ https://issues.apache.org/jira/browse/PIG-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745535#action_12745535 ] Pradeep Kamath commented on PIG-926: Patch committed - thanks for the contribution Ashutosh! Merge-Join phase 2 -- Key: PIG-926 URL: https://issues.apache.org/jira/browse/PIG-926 Project: Pig Issue Type: Improvement Components: impl Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor Attachments: mj_phase2_1.patch This jira is created to keep track of phase-2 work for MergeJoin. Various limitations exist in phase-1 for Merge Join which are listed on: http://wiki.apache.org/pig/PigMergeJoin Those will be addressed here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-926) Merge-Join phase 2
[
https://issues.apache.org/jira/browse/PIG-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745181#action_12745181
]
Pradeep Kamath commented on PIG-926:
In MRCompiler:
You should change:
{code}
indexerArgs[0] = rightLoader.getLFile().getFuncName();
to
indexerArgs[0] = rightLoader.getLFile().getFuncSpec().toString();
{code}
to handle the case where the loader may have constructor args (like
PigStorage(,) - PigStorage with comma as delim)
In the error message when the loader does not implement SamplableLoader, you
can change:
{noformat}
This loader doesn't implement it.
to
The loader specified in + indexerArgs[0] + doesn't implement it
{noformat}
Otherwise looks good.
Merge-Join phase 2
--
Key: PIG-926
URL: https://issues.apache.org/jira/browse/PIG-926
Project: Pig
Issue Type: Improvement
Components: impl
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Priority: Minor
Attachments: mj_phase2_1.patch
This jira is created to keep track of phase-2 work for MergeJoin. Various
limitations exist in phase-1 for Merge Join which are listed on:
http://wiki.apache.org/pig/PigMergeJoin Those will be addressed here.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-926) Merge-Join phase 2
[ https://issues.apache.org/jira/browse/PIG-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12745270#action_12745270 ] Hadoop QA commented on PIG-926: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12417053/mj_phase2_1.patch against trunk revision 805684. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/175/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/175/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/175/console This message is automatically generated. Merge-Join phase 2 -- Key: PIG-926 URL: https://issues.apache.org/jira/browse/PIG-926 Project: Pig Issue Type: Improvement Components: impl Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor Attachments: mj_phase2_1.patch This jira is created to keep track of phase-2 work for MergeJoin. Various limitations exist in phase-1 for Merge Join which are listed on: http://wiki.apache.org/pig/PigMergeJoin Those will be addressed here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-926) Merge-Join phase 2
[ https://issues.apache.org/jira/browse/PIG-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744705#action_12744705 ] Pradeep Kamath commented on PIG-926: Review commments Merge-Join phase 2 -- Key: PIG-926 URL: https://issues.apache.org/jira/browse/PIG-926 Project: Pig Issue Type: Improvement Components: impl Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor Attachments: mj_phase2_1.patch This jira is created to keep track of phase-2 work for MergeJoin. Various limitations exist in phase-1 for Merge Join which are listed on: http://wiki.apache.org/pig/PigMergeJoin Those will be addressed here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-926) Merge-Join phase 2
[ https://issues.apache.org/jira/browse/PIG-926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12744706#action_12744706 ] Pradeep Kamath commented on PIG-926: Review comments: In MergeJoinIndexer: Is there a reason MergeJoinIndexer is a subclass of RandomSampleLoader? It does not use numSamples or skipInterval and has its own getNext() - is there any benefit to inheritance? I see that readObject() method has been removed - is this no longer needed? Also for the case where right pipeline returns POStatus.STATUS_ERR, the code currently ignores it and continues. I am wondering in the worst case, if all iterations of running the right pipeline result in POStatus.STATUS_ERR, MergeJoinIndexer would return null for the join key. If this happened from all maps, we would get an index will all null entries, in this case, I think merge join succeeds with empty output which would be wrong (I am assuming POMergeJoin treats an index full of null entries as indicating that the right input is empty). If this would be the case, we should throw an ExecException on POStatus.STATUS_ERR. Merge-Join phase 2 -- Key: PIG-926 URL: https://issues.apache.org/jira/browse/PIG-926 Project: Pig Issue Type: Improvement Components: impl Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Priority: Minor Attachments: mj_phase2_1.patch This jira is created to keep track of phase-2 work for MergeJoin. Various limitations exist in phase-1 for Merge Join which are listed on: http://wiki.apache.org/pig/PigMergeJoin Those will be addressed here. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
