Re: [ANNOUNCE] Welcome new Apache Pig Committers Rohini Palaniswamy
Congrats, Rohini! - Original Message - From: Daniel Dai da...@hortonworks.com To: dev@pig.apache.org; u...@pig.apache.org Cc: Sent: Friday, October 26, 2012 4:37 PM Subject: [ANNOUNCE] Welcome new Apache Pig Committers Rohini Palaniswamy Here is another Pig committer announcement today. Please welcome Rohini Palaniswamy to be a Pig committer! Thanks, Daniel
[jira] [Commented] (PIG-2990) the -secretDebugCmd shouldn't be a secret and should just be...a command
[ https://issues.apache.org/jira/browse/PIG-2990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485445#comment-13485445 ] Rohini Palaniswamy commented on PIG-2990: - +1. the -secretDebugCmd shouldn't be a secret and should just be...a command Key: PIG-2990 URL: https://issues.apache.org/jira/browse/PIG-2990 Project: Pig Issue Type: Improvement Reporter: Jonathan Coveney Assignee: Jonathan Coveney Priority: Minor Labels: newbie Attachments: PIG-2990-0.patch It's a useful command, and it's weird that it's not in -help -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2507) Semicolon in paramenters for UDF results in parsing error
[ https://issues.apache.org/jira/browse/PIG-2507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485446#comment-13485446 ] Rohini Palaniswamy commented on PIG-2507: - You should be able to add tests for grunt in http://svn.apache.org/repos/asf/pig/branches/branch-0.10/test/org/apache/pig/test/TestGrunt.java Semicolon in paramenters for UDF results in parsing error - Key: PIG-2507 URL: https://issues.apache.org/jira/browse/PIG-2507 Project: Pig Issue Type: Bug Affects Versions: 0.8.0, 0.9.1, 0.10.0 Reporter: Vivek Padmanabhan Assignee: Timothy Chen Attachments: PIG_2507.patch If I have a semicolon in the parameter passed to a udf, the script execution will fail with a parsing error. a = load 'i1' as (f1:chararray); c = foreach a generate REGEX_EXTRACT(f1, '.;' ,1); dump c; The above script fails with the below error [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1200: file test.pig, line 3, column 0 mismatched character 'EOF' expecting ''' Even replacing the semicolon with Unicode \u003B results in the same error. c = foreach a generate REGEX_EXTRACT(f1, '.\u003B',1); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2997) Provide a convenience constructor on PigServer that accepts Configuration
[ https://issues.apache.org/jira/browse/PIG-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13485450#comment-13485450 ] Rohini Palaniswamy commented on PIG-2997: - Good utility method. Could you also change unit tests to use it? Provide a convenience constructor on PigServer that accepts Configuration - Key: PIG-2997 URL: https://issues.apache.org/jira/browse/PIG-2997 Project: Pig Issue Type: Improvement Reporter: Prashant Kommireddi Assignee: Prashant Kommireddi Fix For: 0.11 Attachments: PIG-2997.patch PigServer currently has Properties based constructor. Hadoop in general deals with Configuration and it would be good to have a PigServer constructor that accepts the same. With this, user does not have to worry about creating Properties object out of conf and can simply invoke this new constructor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [ANNOUNCE] Welcome new Apache Pig Committers Rohini Palaniswamy
Wonderful news! On Oct 26, 2012, at 9:51 PM, Gianmarco De Francisci Morales g...@apache.org wrote: Congratulations Rohini! Welcome onboard :) -- Gianmarco On Fri, Oct 26, 2012 at 7:32 PM, Prasanth J buckeye.prasa...@gmail.com wrote: Congrats Rohini! Thanks -- Prasanth On Oct 26, 2012, at 10:21 PM, Santhosh Srinivasan santhosh_mut...@yahoo.com wrote: Congrats Rohini! Full speed ahead now :) On Oct 26, 2012, at 4:37 PM, Daniel Dai da...@hortonworks.com wrote: Here is another Pig committer announcement today. Please welcome Rohini Palaniswamy to be a Pig committer! Thanks, Daniel
[jira] Subscription: PIG patch available
Issue Subscription Filter: PIG patch available (40 issues) Subscriber: pigdaily Key Summary PIG-3010Allow UDF's to flatten themselves https://issues.apache.org/jira/browse/PIG-3010 PIG-3006Modernize a chunk of the tests https://issues.apache.org/jira/browse/PIG-3006 PIG-3001TestExecutableManager.testAddJobConfToEnv fails randomly https://issues.apache.org/jira/browse/PIG-3001 PIG-2990the -secretDebugCmd shouldn't be a secret and should just be...a command https://issues.apache.org/jira/browse/PIG-2990 PIG-2979Pig.jar doesn't work with hadoop-2.0.x https://issues.apache.org/jira/browse/PIG-2979 PIG-2978TestLoadStoreFuncLifeCycle fails with hadoop-2.0.x https://issues.apache.org/jira/browse/PIG-2978 PIG-2973TestStreaming test times out https://issues.apache.org/jira/browse/PIG-2973 PIG-2968ColumnMapKeyPrune fails to prune a subtree inside foreach https://issues.apache.org/jira/browse/PIG-2968 PIG-2960Increase the timeout for unit test https://issues.apache.org/jira/browse/PIG-2960 PIG-2959Add a pig.cmd for Pig to run under Windows https://issues.apache.org/jira/browse/PIG-2959 PIG-2957TetsScriptUDF fail due to volume prefix in jar https://issues.apache.org/jira/browse/PIG-2957 PIG-2956Invalid cache specification for some streaming statement https://issues.apache.org/jira/browse/PIG-2956 PIG-2955 Fix bunch of Pig e2e tests on Windows https://issues.apache.org/jira/browse/PIG-2955 PIG-2954 TestParamSubPreproc still depends on bash to run https://issues.apache.org/jira/browse/PIG-2954 PIG-2953which utility does not exist on Windows https://issues.apache.org/jira/browse/PIG-2953 PIG-2942DevTests, TestLoad has a false failure on Windows https://issues.apache.org/jira/browse/PIG-2942 PIG-2937generated field in nested foreach does not inherit the variable name as the field name https://issues.apache.org/jira/browse/PIG-2937 PIG-2924PigStats should not be assuming all Storage classes to be file-based storage https://issues.apache.org/jira/browse/PIG-2924 PIG-2898Parallel execution of e2e tests https://issues.apache.org/jira/browse/PIG-2898 PIG-2881Add SUBTRACT eval function https://issues.apache.org/jira/browse/PIG-2881 PIG-2873Converting bin/pig shell script to python https://issues.apache.org/jira/browse/PIG-2873 PIG-2834MultiStorage requires unused constructor argument https://issues.apache.org/jira/browse/PIG-2834 PIG-2824Pushing checking number of fields into LoadFunc https://issues.apache.org/jira/browse/PIG-2824 PIG-2801grunt sh command should invoke the shell implicitly instead of calling exec directly with the command tokens https://issues.apache.org/jira/browse/PIG-2801 PIG-2799Update pig streaming interface to run correctly on Windows without Cygwin https://issues.apache.org/jira/browse/PIG-2799 PIG-2798pig streaming tests assume interpreters are auto-resolved https://issues.apache.org/jira/browse/PIG-2798 PIG-2796Local temporary paths are not always valid HDFS path names. https://issues.apache.org/jira/browse/PIG-2796 PIG-2795Fix test cases that generate pig scripts with load + pathStr to encode \ in the path https://issues.apache.org/jira/browse/PIG-2795 PIG-2661Pig uses an extra job for loading data in Pigmix L9 https://issues.apache.org/jira/browse/PIG-2661 PIG-2657Print warning if using wrong jython version https://issues.apache.org/jira/browse/PIG-2657 PIG-2507Semicolon in paramenters for UDF results in parsing error https://issues.apache.org/jira/browse/PIG-2507 PIG-2495Using merge JOIN from a HBaseStorage produces an error https://issues.apache.org/jira/browse/PIG-2495 PIG-2433Jython import module not working if module path is in classpath https://issues.apache.org/jira/browse/PIG-2433 PIG-2417Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation. https://issues.apache.org/jira/browse/PIG-2417 PIG-2405svn tags/release-0.9.1: some unit test case failed with open JDK https://issues.apache.org/jira/browse/PIG-2405 PIG-2362Rework Ant build.xml to use macrodef instead of antcall https://issues.apache.org/jira/browse/PIG-2362 PIG-2312NPE when relation and column share the same name and used in Nested Foreach https://issues.apache.org/jira/browse/PIG-2312 PIG-1942script UDF (jython) should utilize the intended output schema to more directly convert Py objects to Pig objects https://issues.apache.org/jira/browse/PIG-1942 PIG-1431
Re: Review Request: Modernize a chunk of the tests
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7734/#review12859 --- Hi, Jonathan. It looks really good! I made a few minor comments inline. But my only complaint is about indentations. I know that we don't generate patches with white space changes, but I think that white space changes should be as part of the patch in this case because you are cleaning up a lot of old code. In particular, you are removing several try-catch blocks. But since indentations are not fixed, they look really odd. Could you upload a new patch that includes white space changes to the RB? I don't think that it's too hard to review it since the RB has an option to hide them. Please let me know what you think. Thanks! test/org/apache/pig/test/TestNewPlanOperatorPlan.java https://reviews.apache.org/r/7734/#comment27620 Please change to @Test(expected = FrontendException). test/org/apache/pig/test/TestPigScriptParser.java https://reviews.apache.org/r/7734/#comment27627 Please change to @Test(expected = FrontendException). test/org/apache/pig/test/TestPigScriptParser.java https://reviews.apache.org/r/7734/#comment27628 Please change the order of args in assertEquals to expected and output. test/org/apache/pig/test/TestPigScriptParser.java https://reviews.apache.org/r/7734/#comment27630 Please change the order of args in assertEquals to expected and output. test/org/apache/pig/test/TestPigSplit.java https://reviews.apache.org/r/7734/#comment27634 Isn't this a test case? If so, please change the method name to testXX and add @Test to it. test/org/apache/pig/test/TestPigSplit.java https://reviews.apache.org/r/7734/#comment27635 Please change this to assertFalse(iter.hasNext()). test/org/apache/pig/test/TestPigSplit.java https://reviews.apache.org/r/7734/#comment27636 Please change this to assertFalse(iter.hasNext()). test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27637 Please change the order of args in assertEquals to expected and output. test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27638 Please change the order of args in assertEquals to expected and output. test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27639 Please change the order of args in assertEquals to expected and output. test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27640 Please change the order of args in assertEquals to expected and output. test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27643 Please change this to assertNull(). test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27644 Please change this to assertNull(). test/org/apache/pig/test/TestPruneColumn.java https://reviews.apache.org/r/7734/#comment27645 Please change the order of args in assertEquals to expected to output. test/org/apache/pig/test/TestRelationToExprProject.java https://reviews.apache.org/r/7734/#comment27646 Please delete this line? - Cheolsoo Park On Oct. 25, 2012, 6:05 p.m., Jonathan Coveney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7734/ --- (Updated Oct. 25, 2012, 6:05 p.m.) Review request for pig and Julien Le Dem. Description --- A lot of the tests use antiquated patterns. My goal was to refactor them in a couple ways: - get rid of the annotation specifying Junit 4. All should use JUnit 4 (question: where is the Junit 3 dependency even being pulled?) - Nothing should extend TestCase. Everything should be annotation driven. - Properly use asserts. There was a lot of assertTrue(null==thing), so I replaced it with assertNull(thing), and so on. - Get rid of MiniCluster use in a handful of cases. This addresses bug PIG-3006. https://issues.apache.org/jira/browse/PIG-3006 Diffs - test/org/apache/pig/test/PigExecTestCase.java 32a502c test/org/apache/pig/test/TestAlgebraicEval.java 0bbd83d test/org/apache/pig/test/TestAlgebraicEvalLocal.java df4b76a test/org/apache/pig/test/TestBagFormat.java 09298d4 test/org/apache/pig/test/TestBatchAliases.java 6e952c7 test/org/apache/pig/test/TestCompressedFiles.java d54ffaa test/org/apache/pig/test/TestConversions.java 152ad5c test/org/apache/pig/test/TestDeleteOnFail.java 7070285 test/org/apache/pig/test/TestFilterOpNumeric.java 730e808 test/org/apache/pig/test/TestFilterOpString.java b65965f test/org/apache/pig/test/TestFilterSimplification.java ade97b6
