Re: Unit test classpath trouble

2013-05-13 Thread Konstantin Boudnik
If unit tests can't be run - which seems to be the case - it sound like a release blocker to me. Any plans to back port this fix into 0.11.2? 0.11.1 doesn't have it, right? Cos On Sun, May 12, 2013 at 09:12PM, Cheolsoo Park wrote: Hi Andrew, I'm trying HEAD of branch-0.11. You need

[jira] [Commented] (PIG-3302) JSONStorage throws NPE if map has null values

2013-05-13 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13655830#comment-13655830 ] Prashant Kommireddi commented on PIG-3302: -- Looks good to me, TestJsonLoaderStorage

Re: Unit test classpath trouble

2013-05-13 Thread Andrew Purtell
Cheolsoo, Thank you so much. That was the problem. On Mon, May 13, 2013 at 12:12 PM, Cheolsoo Park piaozhe...@gmail.comwrote: Hi Andrew, I'm trying HEAD of branch-0.11. You need PIG-3248: https://issues.apache.org/jira/browse/PIG-3248. It's not in branch-0.11 but in trunk. In

Re: Unit test classpath trouble

2013-05-13 Thread Cheolsoo Park
Hi Cos, I am happy to backport it to branch-011. But I am not entirely sure whether we should bump the Hadoop version in a minor release. As of now, branch-0.11 unit tests run fine with 2.0.0-alpha. Please let me know if anyone has an opinion on this. Thanks, Cheolsoo On Sun, May 12, 2013 at

[jira] [Updated] (PIG-3317) disable optimizations via pig properties

2013-05-13 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Graham updated PIG-3317: - Status: Open (was: Patch Available) Canceling patch since Travis and Julien identified issues with SET in

Re: Unit test classpath trouble

2013-05-13 Thread Johnny Zhang
I run unit test against 2.0.4-alpha in the trunk. The tests hit OutOfMemoryError: PermGen space in the middle, but so far no failures or errors. I re-kicked off the test. Johnny On Fri, May 10, 2013 at 6:49 PM, Johnny Zhang xiao...@cloudera.com wrote: Hi, Andrew: I just set up a job to run

[jira] [Created] (PIG-3320) AVRO: no empty field expressed when loading with AvroStorage using reader schema with extra field that has no default

2013-05-13 Thread Egil Sorensen (JIRA)
Egil Sorensen created PIG-3320: -- Summary: AVRO: no empty field expressed when loading with AvroStorage using reader schema with extra field that has no default Key: PIG-3320 URL:

[jira] [Updated] (PIG-3320) AVRO: no empty field expressed when loading with AvroStorage using reader schema with extra field that has no default

2013-05-13 Thread Egil Sorensen (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Egil Sorensen updated PIG-3320: --- Description: Somewhat different use case than PIG-3318: Loading with AvroStorage giving a loader

[jira] [Created] (PIG-3321) AVRO: Support user specified schema on load

2013-05-13 Thread Harvey Chong (JIRA)
Harvey Chong created PIG-3321: - Summary: AVRO: Support user specified schema on load Key: PIG-3321 URL: https://issues.apache.org/jira/browse/PIG-3321 Project: Pig Issue Type: Improvement

Re: Unit test classpath trouble

2013-05-13 Thread Rohini Palaniswamy
Johnny, You need to increase PermGen space for 2.x else TestGrunt will hang with the OOM. Run ant with -DdebugArgs=-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=128M Regards, Rohini On Mon, May 13, 2013 at 10:22 AM, Johnny Zhang xiao...@cloudera.com wrote: I run unit test against

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11032/ --- (Updated May 13, 2013, 8:35 p.m.) Review request for pig, Julien Le Dem, Bill

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11032/#review20501 --- Tested disabling this rule in three ways:

[jira] [Updated] (PIG-3317) disable optimizations via pig properties

2013-05-13 Thread Travis Crawford (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Crawford updated PIG-3317: - Attachment: PIG-3317_disable_opts.2.patch Updated patch; please see review for comments. Main

[jira] [Updated] (PIG-3317) disable optimizations via pig properties

2013-05-13 Thread Travis Crawford (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Travis Crawford updated PIG-3317: - Status: Patch Available (was: Open) disable optimizations via pig properties

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Julien Le Dem
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11032/#review20514 --- Overall, this looks good. Some comments bellow

[jira] [Updated] (PIG-3302) JSONStorage throws NPE if map has null values

2013-05-13 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3302: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Bill Graham
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11032/#review20516 --- src/docs/src/documentation/content/xdocs/perf.xml

[jira] [Created] (PIG-3322) AVRO: AvroStorage give NPE on reading file with union as top level schema

2013-05-13 Thread Egil Sorensen (JIRA)
Egil Sorensen created PIG-3322: -- Summary: AVRO: AvroStorage give NPE on reading file with union as top level schema Key: PIG-3322 URL: https://issues.apache.org/jira/browse/PIG-3322 Project: Pig

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
On May 13, 2013, 11:23 p.m., Julien Le Dem wrote: src/org/apache/pig/PigConstants.java, lines 47-51 https://reviews.apache.org/r/11032/diff/2/?file=290927#file290927line47 I would like to clearly separate keys for pig.properties from others in there. Can we either

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
On May 13, 2013, 11:23 p.m., Julien Le Dem wrote: src/org/apache/pig/newplan/logical/optimizer/LogicalPlanOptimizer.java, line 215 https://reviews.apache.org/r/11032/diff/2/?file=290929#file290929line215 would checkNotNull(ruleSet, ruleSet) give a better error message ? Travis

[jira] [Updated] (PIG-3322) AVRO: AvroStorage give NPE on reading file with union as top level schema

2013-05-13 Thread Egil Sorensen (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Egil Sorensen updated PIG-3322: --- Description: I am getting NPE when loading a file with AvroStorage a file that has schema like:

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
On May 13, 2013, 11:35 p.m., Bill Graham wrote: src/docs/src/documentation/content/xdocs/perf.xml, line 493 https://reviews.apache.org/r/11032/diff/2/?file=290925#file290925line493 Would you please specify that setting this value in both the pig properties file and the command

[jira] [Created] (PIG-3323) AVRO: default value not stored in file when given as paramter to AvroStorage

2013-05-13 Thread Egil Sorensen (JIRA)
Egil Sorensen created PIG-3323: -- Summary: AVRO: default value not stored in file when given as paramter to AvroStorage Key: PIG-3323 URL: https://issues.apache.org/jira/browse/PIG-3323 Project: Pig

[jira] [Updated] (PIG-3323) AVRO: default value not stored in file when given as paramter to AvroStorage

2013-05-13 Thread Egil Sorensen (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Egil Sorensen updated PIG-3323: --- Description: A pig script like the below succeeds, but inspecting the resulting file I find that the

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Bill Graham
On May 13, 2013, 11:35 p.m., Bill Graham wrote: src/docs/src/documentation/content/xdocs/perf.xml, line 493 https://reviews.apache.org/r/11032/diff/2/?file=290925#file290925line493 Would you please specify that setting this value in both the pig properties file and the command

[jira] Subscription: PIG patch available

2013-05-13 Thread jira
Issue Subscription Filter: PIG patch available (23 issues) Subscriber: pigdaily Key Summary PIG-3318AVRO: 'default value' not honored when merging schemas on load with AvroStorage https://issues.apache.org/jira/browse/PIG-3318 PIG-3317disable optimizations

[jira] [Created] (PIG-3324) STARTSWITH documentation

2013-05-13 Thread Bill Graham (JIRA)
Bill Graham created PIG-3324: Summary: STARTSWITH documentation Key: PIG-3324 URL: https://issues.apache.org/jira/browse/PIG-3324 Project: Pig Issue Type: Bug Reporter: Bill Graham

[jira] [Updated] (PIG-3324) STARTSWITH documentation

2013-05-13 Thread Bill Graham (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bill Graham updated PIG-3324: - Fix Version/s: 0.12 STARTSWITH documentation Key:

Review Request: Patch to address default values when schemas are merged in AvroStorage. It does this for Records containing primitive values

2013-05-13 Thread Viraj Bhat
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/11135/ --- Review request for pig. Description --- Default values are not honoured

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
On May 13, 2013, 11:35 p.m., Bill Graham wrote: src/docs/src/documentation/content/xdocs/perf.xml, line 493 https://reviews.apache.org/r/11032/diff/2/?file=290925#file290925line493 Would you please specify that setting this value in both the pig properties file and the command

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Bill Graham
On May 13, 2013, 11:35 p.m., Bill Graham wrote: src/docs/src/documentation/content/xdocs/perf.xml, line 493 https://reviews.apache.org/r/11032/diff/2/?file=290925#file290925line493 Would you please specify that setting this value in both the pig properties file and the command

Re: Review Request: disable optimizations via pig properties

2013-05-13 Thread Travis Crawford
On May 13, 2013, 11:35 p.m., Bill Graham wrote: src/docs/src/documentation/content/xdocs/perf.xml, line 493 https://reviews.apache.org/r/11032/diff/2/?file=290925#file290925line493 Would you please specify that setting this value in both the pig properties file and the command

Re: Unit test classpath trouble

2013-05-13 Thread Konstantin Boudnik
Cheolsoo, just to make sure I do not misinterpret your words: - there were a change in YARN packaging between 2.0.0 and 2.0.3 that move some APIs into separate artifact hadoop-yarn-client - adding hadoop-yarn-client is only relevant for 2.0.3 and later and won't work on 2.0.0-2.0.2