[jira] Updated: (PIG-882) log level not propogated to loggers
[ https://issues.apache.org/jira/browse/PIG-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-882: --- Fix Version/s: 0.4.0 Affects Version/s: 0.3.0 Status: Patch Available (was: In Progress) log level not propogated to loggers Key: PIG-882 URL: https://issues.apache.org/jira/browse/PIG-882 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.3.0 Reporter: Thejas M Nair Assignee: Daniel Dai Fix For: 0.4.0 Attachments: PIG-882-1.patch, PIG-882-2.patch, PIG-882-3.patch, PIG-882-4.patch, PIG-882-5.patch Pig accepts log level as a parameter. But the log level it captures is not set appropriately, so that loggers in different classes log at the specified level. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-882) log level not propogated to loggers
[ https://issues.apache.org/jira/browse/PIG-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned PIG-882: -- Assignee: Daniel Dai log level not propogated to loggers Key: PIG-882 URL: https://issues.apache.org/jira/browse/PIG-882 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.3.0 Reporter: Thejas M Nair Assignee: Daniel Dai Fix For: 0.4.0 Attachments: PIG-882-1.patch, PIG-882-2.patch, PIG-882-3.patch, PIG-882-4.patch, PIG-882-5.patch Pig accepts log level as a parameter. But the log level it captures is not set appropriately, so that loggers in different classes log at the specified level. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-513) PERFORMANCE: optimize some of the code in DefaultTuple
[
https://issues.apache.org/jira/browse/PIG-513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737024#action_12737024
]
Hadoop QA commented on PIG-513:
---
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12414853/pig-513_2.patch
against trunk revision 799141.
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no tests are needed for this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/146/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/146/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Console output:
http://hudson.zones.apache.org/hudson/job/Pig-Patch-minerva.apache.org/146/console
This message is automatically generated.
PERFORMANCE: optimize some of the code in DefaultTuple
--
Key: PIG-513
URL: https://issues.apache.org/jira/browse/PIG-513
Project: Pig
Issue Type: Bug
Affects Versions: 0.2.0
Reporter: Pradeep Kamath
Assignee: Pradeep Kamath
Attachments: PIG-513.patch, pig-513_2.patch
The following areas in DefaultTuple.java can be changed:
The member methods get(), set(), getType() and isNull() all call
checkBounds() which is redundant call since all these 4 functions throw
ExecException. Instead of doing a bounds check, we can catch the
IndexOutOfBounds exception in a try-catch and throw it as an ExecException
The write() method has the following unused object (d in the code below):
{code}
for (int i = 0; i sz; i++) {
try {
Object d = get(i);
} catch (ExecException ee) {
throw new RuntimeException(ee);
}
DataReaderWriter.writeDatum(out, mFields.get(i));
}
{code}
{noformat}
The get(i) call in the try should be replaced by the writeDatum call directly
since d is never used and there is an unncessary call to get()
{noformat}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (PIG-899) missing document review basics
missing document review basics Key: PIG-899 URL: https://issues.apache.org/jira/browse/PIG-899 Project: Pig Issue Type: Bug Components: documentation Affects Versions: site Reporter: solrize Priority: Minor http://hadoop.apache.org/pig/ getting started section has a link labelled review basics ( http://hadoop.apache.org/pig/docs/r0.3.0/quickstart.html ) which gives a 404. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santhosh Srinivasan updated PIG-880: Status: Open (was: Patch Available) Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Work started: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on PIG-880 started by Santhosh Srinivasan. Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santhosh Srinivasan updated PIG-880: Status: Patch Available (was: In Progress) Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-892) Make COUNT and AVG deal with nulls accordingly with SQL standar
[ https://issues.apache.org/jira/browse/PIG-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-892: --- Status: Open (was: Patch Available) Make COUNT and AVG deal with nulls accordingly with SQL standar --- Key: PIG-892 URL: https://issues.apache.org/jira/browse/PIG-892 Project: Pig Issue Type: Improvement Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Olga Natkovich Fix For: 0.4.0 Attachments: PIG-892.patch, PIG-892_v2.patch, PIG-892_v3.patch both COUNT and AVG need to ignore nulls. Also add COUNT_STAR to match COUNT(*) in SQL -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-892) Make COUNT and AVG deal with nulls accordingly with SQL standar
[ https://issues.apache.org/jira/browse/PIG-892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Olga Natkovich updated PIG-892: --- Status: Patch Available (was: Open) Make COUNT and AVG deal with nulls accordingly with SQL standar --- Key: PIG-892 URL: https://issues.apache.org/jira/browse/PIG-892 Project: Pig Issue Type: Improvement Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Olga Natkovich Fix For: 0.4.0 Attachments: PIG-892.patch, PIG-892_v2.patch, PIG-892_v3.patch both COUNT and AVG need to ignore nulls. Also add COUNT_STAR to match COUNT(*) in SQL -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-882) log level not propogated to loggers
[ https://issues.apache.org/jira/browse/PIG-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-882: --- Status: Open (was: Patch Available) log level not propogated to loggers Key: PIG-882 URL: https://issues.apache.org/jira/browse/PIG-882 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.3.0 Reporter: Thejas M Nair Assignee: Daniel Dai Fix For: 0.4.0 Attachments: PIG-882-1.patch, PIG-882-2.patch, PIG-882-3.patch, PIG-882-4.patch, PIG-882-5.patch Pig accepts log level as a parameter. But the log level it captures is not set appropriately, so that loggers in different classes log at the specified level. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-882) log level not propogated to loggers
[ https://issues.apache.org/jira/browse/PIG-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-882: --- Status: Patch Available (was: Open) Pick me up, Hudson! log level not propogated to loggers Key: PIG-882 URL: https://issues.apache.org/jira/browse/PIG-882 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.3.0 Reporter: Thejas M Nair Assignee: Daniel Dai Fix For: 0.4.0 Attachments: PIG-882-1.patch, PIG-882-2.patch, PIG-882-3.patch, PIG-882-4.patch, PIG-882-5.patch Pig accepts log level as a parameter. But the log level it captures is not set appropriately, so that loggers in different classes log at the specified level. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737310#action_12737310 ] Pradeep Kamath commented on PIG-880: +1 - looks good. Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-845) PERFORMANCE: Merge Join
[ https://issues.apache.org/jira/browse/PIG-845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated PIG-845: - Attachment: merge-join-for-review.patch Initial patch for review. PERFORMANCE: Merge Join --- Key: PIG-845 URL: https://issues.apache.org/jira/browse/PIG-845 Project: Pig Issue Type: Improvement Reporter: Olga Natkovich Attachments: merge-join-for-review.patch Thsi join would work if the data for both tables is sorted on the join key. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-885) New UDFs for piggybank (Bin, Decode, LookupInFiles, RegexExtract, RegexMatch, HashFVN, DiffDate)
[ https://issues.apache.org/jira/browse/PIG-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737315#action_12737315 ] Olga Natkovich commented on PIG-885: The latest patch looks good. Couple of comments: (1) RegexExtract - input.get(1).equals(mExpression)) - need to check for null return from get(1). The same for get(2) (2) RegexpMatch - the same Once they are addressed, please, commit the patch New UDFs for piggybank (Bin, Decode, LookupInFiles, RegexExtract, RegexMatch, HashFVN, DiffDate) Key: PIG-885 URL: https://issues.apache.org/jira/browse/PIG-885 Project: Pig Issue Type: New Feature Affects Versions: 0.3.0 Reporter: Daniel Dai Assignee: Daniel Dai Priority: Minor Fix For: 0.4.0 Attachments: PIG-885-2.patch, PIG-885-3.patch, PIG-885-4.patch, PIG-885-5.patch, PIG-885-6.patch, PIG-885-7.patch, PIG-885-8.patch, PIG-885.patch Bunch of UDFs: 1. Bin -- Converts a continuous value into discrete values 2. Decode -- Converts a given attribute or expression into another string value, based on the value of the source attribute 3. LookupInFiles -- Check for the existence of an expression in a serial of text files 4. RegexExtract and RegexMatch -- Similar to perl regexes 5. HashFNV -- An implementation of FNV hash 6. DiffDate -- Caculate the number of days in between -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-898) TextDataParser does not handle delimiters from one complex type in another
[ https://issues.apache.org/jira/browse/PIG-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737319#action_12737319 ] Santhosh Srinivasan commented on PIG-898: - In addition, empty bags, tuples and constants and nulls are not handled. TextDataParser does not handle delimiters from one complex type in another -- Key: PIG-898 URL: https://issues.apache.org/jira/browse/PIG-898 Project: Pig Issue Type: Bug Components: impl Affects Versions: 0.4.0 Reporter: Santhosh Srinivasan Priority: Minor Fix For: 0.4.0 Currently, TextDataParser does not handle delimiters of one complex type in another. An example of such a case is key1(#value1} will not be parsed correctly. The production for strings matches any sequence of character that do not contain any delimiters for the complex types. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santhosh Srinivasan updated PIG-880: Attachment: (was: PIG-880.patch) Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880_1.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Santhosh Srinivasan updated PIG-880: Status: Patch Available (was: In Progress) Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880_1.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-845) PERFORMANCE: Merge Join
[
https://issues.apache.org/jira/browse/PIG-845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737377#action_12737377
]
Pradeep Kamath commented on PIG-845:
Some initial comments on POMergeJoin.java:
If status is not OK - it shuld just be returned (no run time
exception like above) - similar comments for other places in POMergeJoin where
there is
a switch case on processInput() - once this change is made, the code in
if(processingFE) also
will need to change accordingly
{code}
if(firstTime){
// Do initial setup.
curLeftInp = processInput();
switch(curLeftInp.returnStatus){
case POStatus.STATUS_OK:
break;
case POStatus.STATUS_EOP: // Return because we want to fetch next
left tuple.
return curLeftInp;
default:
throw new RuntimeException(Unexpected Status);
}
{code}
All non RuntimeExceptions should follow error handling specification by using
the correct Exception created with error code, cause, message, src constructor.
http://wiki.apache.org/pig/PigErrorHandlingFunctionalSpecification#head-9f71d78d362c3307711f98ec9db3ee12b55e92f6
should be updated with new error code #
detachInput() is not required in POMergeJoin - processInput takes care of it
IN the code below, we could cache away the key to be used while processFE is
true as processFEKey and then we need not
extract key for each join
// Cant use the prevLeftKey, because we are reading ahead.
// Need key of current bag. Since we have just
finished doing the join
// bag must contain atleast one element.
res.returnStatus = POStatus.STATUS_OK;
res.result = leftTuples.get(0);
curLeftKey = extractKeysFromTuple(res, 0);
PERFORMANCE: Merge Join
---
Key: PIG-845
URL: https://issues.apache.org/jira/browse/PIG-845
Project: Pig
Issue Type: Improvement
Reporter: Olga Natkovich
Attachments: merge-join-for-review.patch
Thsi join would work if the data for both tables is sorted on the join key.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-880) Order by is borken with complex fields
[ https://issues.apache.org/jira/browse/PIG-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12737379#action_12737379 ] Pradeep Kamath commented on PIG-880: +1 to the new changes. Order by is borken with complex fields -- Key: PIG-880 URL: https://issues.apache.org/jira/browse/PIG-880 Project: Pig Issue Type: Bug Affects Versions: 0.3.0 Reporter: Olga Natkovich Assignee: Santhosh Srinivasan Fix For: 0.4.0 Attachments: PIG-880-bytearray-mapvalue-code-without-tests.patch, PIG-880_1.patch Pig script: a = load 'studentcomplextab10k' as (smap:map[],c2,c3); f = foreach a generate smap#'name, smap#'age', smap#'gpa' ; s = order f by $0; store s into 'sc.out' Stack: Caused by: java.lang.ArrayStoreException at java.lang.System.arraycopy(Native Method) at java.util.Arrays.copyOf(Arrays.java:2763) at java.util.ArrayList.toArray(ArrayList.java:305) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.convertToArray(WeightedRangePartitioner.java:154) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.WeightedRangePartitioner.configure(WeightedRangePartitioner.java:96) ... 5 more at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:230) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:179) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:204) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:265) at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:769) at org.apache.pig.PigServer.execute(PigServer.java:762) at org.apache.pig.PigServer.access$100(PigServer.java:91) at org.apache.pig.PigServer$Graph.execute(PigServer.java:933) at org.apache.pig.PigServer.executeBatch(PigServer.java:245) at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:112) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:168) at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:140) at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:88) at org.apache.pig.Main.main(Main.java:389) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-885) New UDFs for piggybank (Bin, Decode, LookupInFiles, RegexExtract, RegexMatch, HashFVN, DiffDate)
[ https://issues.apache.org/jira/browse/PIG-885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated PIG-885: --- Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed. New UDFs for piggybank (Bin, Decode, LookupInFiles, RegexExtract, RegexMatch, HashFVN, DiffDate) Key: PIG-885 URL: https://issues.apache.org/jira/browse/PIG-885 Project: Pig Issue Type: New Feature Affects Versions: 0.3.0 Reporter: Daniel Dai Assignee: Daniel Dai Priority: Minor Fix For: 0.4.0 Attachments: PIG-885-2.patch, PIG-885-3.patch, PIG-885-4.patch, PIG-885-5.patch, PIG-885-6.patch, PIG-885-7.patch, PIG-885-8.patch, PIG-885.patch Bunch of UDFs: 1. Bin -- Converts a continuous value into discrete values 2. Decode -- Converts a given attribute or expression into another string value, based on the value of the source attribute 3. LookupInFiles -- Check for the existence of an expression in a serial of text files 4. RegexExtract and RegexMatch -- Similar to perl regexes 5. HashFNV -- An implementation of FNV hash 6. DiffDate -- Caculate the number of days in between -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (PIG-526) Order of key, value pairs not preserved in MAP type.
[ https://issues.apache.org/jira/browse/PIG-526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Tang resolved PIG-526. --- Resolution: Won't Fix Order of key, value pairs not preserved in MAP type. -- Key: PIG-526 URL: https://issues.apache.org/jira/browse/PIG-526 Project: Pig Issue Type: Bug Components: data Affects Versions: 0.2.0 Reporter: Hong Tang PIG uses HashMap to deserialize the Pig MAP type which will not observe the order of key, value pairs. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
