[jira] [Updated] (PIG-3019) Need a target in build.xml for source releases

2012-10-31 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-3019: Status: Patch Available (was: Open) Need a target in build.xml for source releases

[jira] [Commented] (PIG-3008) Fix whitespace in Pig code

2012-10-30 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487301#comment-13487301 ] Alan Gates commented on PIG-3008: - A couple of notes: We already have coding standards, see

Re: Adding new test cases to TestBuiltin.java

2012-10-30 Thread Alan Gates
On Oct 30, 2012, at 2:43 PM, Cheolsoo Park wrote: Hi all, While reviewing PIG-2881 (Add SUBTRACT eval func), I had 2 questions: 1) How do we decide whether an eval func be a built-in func? For example, should SUBTRACT be added to the o.a.pig.builtin or piggybank? The question here is

Re: Pig 0.11

2012-10-26 Thread Alan Gates
information in the release notes section so that we can quickly compile release documentation. Thanks for you help! Olga From: Alan Gates ga...@hortonworks.com To: dev@pig.apache.org Sent: Monday, October 15, 2012 11:55 AM Subject: Re: Pig 0.11

Re: Pig 0.11

2012-10-26 Thread Alan Gates
: Alan, Are there any blog posts or whatnot explaining the logic behind this? Just curious Jon 2012/10/25 Alan Gates ga...@hortonworks.com There's one other issue I believe we should resolve before we release 0.11. As part of my work with the Incubator I've learned that official

Re: Pig 0.11

2012-10-26 Thread Alan Gates
to be aggressive about mentioning the ant step to users all over the docs, as they may have never built software before. Russell Jurney twitter.com/rjurney On Oct 26, 2012, at 3:25 PM, Alan Gates ga...@hortonworks.com wrote: No blog posts, but a long and tortured email thread on incubator

Re: Welcome our newest committer Cheolsoo Park

2012-10-26 Thread Alan Gates
Welcome Cheolsoo, and well deserved. Alan. On Oct 26, 2012, at 2:54 PM, Julien Le Dem wrote: All, Please join me in welcoming Cheolsoo Park as our newest Pig committer. He's been contributing to Pig for a while now, helping fixing the build and improve Pig. We look forward to him being a

Re: Add patch to reviewboard?

2012-10-26 Thread Alan Gates
come back and bug the list every week if it just stays there? Tim On Thu, Oct 25, 2012 at 10:45 AM, Alan Gates ga...@hortonworks.com wrote: In Pig we leave the use of reviewboard up to the contributor and reviewer. If you find it helpful feel free to use it. A reviewer may also ask

Re: Add patch to reviewboard?

2012-10-25 Thread Alan Gates
In Pig we leave the use of reviewboard up to the contributor and reviewer. If you find it helpful feel free to use it. A reviewer may also ask for it, especially if the patch is large. But we do not require all patches be placed there. Alan. On Oct 25, 2012, at 1:31 PM, Timothy Chen wrote:

[jira] [Commented] (PIG-2795) Fix test cases that generate pig scripts with load + pathStr to encode \ in the path

2012-10-16 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477118#comment-13477118 ] Alan Gates commented on PIG-2795: - John, After applying this patch to we expect the tests

Re: Pig 0.11

2012-10-15 Thread Alan Gates
At this point no one has taken on release documentation for 0.11. Alan. On Oct 15, 2012, at 11:49 AM, Olga Natkovich wrote: Thanks! Are you talking about items 15 and 16 on the How To Release.Publish page? Also, who is doing release documentation these days? I can help with that as

Re: [jira] [Commented] (PIG-2963) Illustrate command and POPackageLite

2012-10-15 Thread Alan Gates
Send email to dev-unsubscr...@pig.apache.org Alan. On Oct 15, 2012, at 12:59 PM, Curtis Strite wrote: How do I remove myself from this distro? Thanks, Curtis On Mon, Oct 15, 2012 at 2:57 PM, Jonathan Coveney (JIRA) j...@apache.orgwrote: [

Re: Unit test failures

2012-10-15 Thread Alan Gates
I would like to push the fixes for Windows into 0.11, as they are mostly small bug fixes. There's already an umbrella JIRA for these, https://issues.apache.org/jira/browse/PIG-2793 Alan. On Oct 15, 2012, at 1:26 PM, Rohini Palaniswamy wrote: Me and Cheolsoo are kicking off a new run for the

[jira] [Updated] (PIG-2794) Pig test: add utils to simplify testing on Windows

2012-10-15 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2794: Resolution: Fixed Fix Version/s: 0.10.1 Status: Resolved (was: Patch Available) Patch

[jira] [Updated] (PIG-2689) JsonStorage fails to find schema when LimitAdjuster runs

2012-10-05 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2689: Status: Open (was: Patch Available) JsonStorage fails to find schema when LimitAdjuster runs

[jira] [Commented] (PIG-2689) JsonStorage fails to find schema when LimitAdjuster runs

2012-10-05 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470578#comment-13470578 ] Alan Gates commented on PIG-2689: - This patch no longer applies because PhysicalOperator

[jira] [Updated] (PIG-2932) Setting high default_parallel causes IOException in local mode

2012-10-05 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2932: Resolution: Fixed Fix Version/s: 0.11 Status: Resolved (was: Patch Available) Patch

[jira] [Resolved] (PIG-2277) Make Pig compile against Hadoop 0.22

2012-10-03 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved PIG-2277. - Resolution: Won't Fix Make Pig compile against Hadoop 0.22

[jira] [Commented] (PIG-2935) Catch NoSuchMethodError when StoreFuncInterface's new cleanupOnSuccess method isn't implemented.

2012-10-02 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467842#comment-13467842 ] Alan Gates commented on PIG-2935: - +1. Catch NoSuchMethodError when

[jira] [Updated] (PIG-2816) piggybank.jar not getting created with the current buil.xml

2012-10-02 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2816: Resolution: Won't Fix Status: Resolved (was: Patch Available) piggybank.jar not getting

[jira] [Commented] (PIG-2816) piggybank.jar not getting created with the current buil.xml

2012-10-02 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467844#comment-13467844 ] Alan Gates commented on PIG-2816: - Swathi, thanks for your work on this. If you do 'ant jar

[jira] [Commented] (PIG-2277) Make Pig compile against Hadoop 0.22

2012-10-02 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467974#comment-13467974 ] Alan Gates commented on PIG-2277: - Given that this patch can't be applied to 0.8

[jira] [Updated] (PIG-2277) Make Pig compile against Hadoop 0.22

2012-10-02 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2277: Status: Open (was: Patch Available) Make Pig compile against Hadoop 0.22

[jira] [Commented] (PIG-2923) Lazily register bags with SpillableMemoryManager

2012-09-28 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465793#comment-13465793 ] Alan Gates commented on PIG-2923: - +1. Lazily register bags

[jira] [Commented] (PIG-1891) Enable StoreFunc to make intelligent decision based on job success or failure

2012-09-27 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13464869#comment-13464869 ] Alan Gates commented on PIG-1891: - This is my bad. I figured most people extended StoreFunc

[jira] [Created] (PIG-2935) Catch NoSuchMethodError when StoreFuncInterface's new cleanupOnSuccess method isn't implemented.

2012-09-27 Thread Alan Gates (JIRA)
Alan Gates created PIG-2935: --- Summary: Catch NoSuchMethodError when StoreFuncInterface's new cleanupOnSuccess method isn't implemented. Key: PIG-2935 URL: https://issues.apache.org/jira/browse/PIG-2935

[jira] [Commented] (PIG-2923) Lazily register bags with SpillableMemoryManager

2012-09-20 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459751#comment-13459751 ] Alan Gates commented on PIG-2923: - In DefaultAbstractBag it would be good to have a comment

[jira] [Commented] (PIG-2712) Pig does not call OutputCommitter.abortJob() on the underlying OutputFormat

2012-09-19 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459023#comment-13459023 ] Alan Gates commented on PIG-2712: - Patch looks good. I'm running the tests and will commit

[jira] [Commented] (PIG-2712) Pig does not call OutputCommitter.abortJob() on the underlying OutputFormat

2012-09-19 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13459304#comment-13459304 ] Alan Gates commented on PIG-2712: - Committed on trunk. I'll test it on 0.10 next

[jira] [Updated] (PIG-2712) Pig does not call OutputCommitter.abortJob() on the underlying OutputFormat

2012-09-19 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2712: Resolution: Fixed Status: Resolved (was: Patch Available) branch10 patch checked into branch-10

[jira] [Commented] (PIG-2918) Avoid Spillable bag overhead where possible

2012-09-14 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13456125#comment-13456125 ] Alan Gates commented on PIG-2918: - Patch looks good. I'm running it through some of the e2e

[jira] [Commented] (PIG-2918) Avoid Spillable bag overhead where possible

2012-09-14 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13456269#comment-13456269 ] Alan Gates commented on PIG-2918: - +1, tests pass. Avoid Spillable bag

[jira] [Updated] (PIG-2909) Add a new option for ignoring corrupted files to AvroStorage load func

2012-09-13 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2909: Resolution: Fixed Fix Version/s: 0.11 Status: Resolved (was: Patch Available) Patch 2

Re: POCollectedGroup and LoadFunc indicator interface

2012-09-12 Thread Alan Gates
You are correct, this would be better named OrderedCollectableLoadFunc. I suspect the way this happened is that this is usually used on the output of MapReduce jobs. In that case (at least in MR1) the keys are sorted as well as guaranteed to be in a particular part file. Alan. On Sep 7,

[jira] [Commented] (PIG-2887) Macro cannot handle negative number

2012-09-12 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13454632#comment-13454632 ] Alan Gates commented on PIG-2887: - Code looks good. I'll run the relevant tests and check

[jira] [Commented] (PIG-2900) Streaming should provide conf settings in the environment

2012-09-12 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13454636#comment-13454636 ] Alan Gates commented on PIG-2900: - In general it looks good. I had a couple of questions

[jira] [Commented] (PIG-2909) Add a new option for ignoring corrupted files to AvroStorage load func

2012-09-12 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13454641#comment-13454641 ] Alan Gates commented on PIG-2909: - A couple of small comments posted on review board

[jira] [Resolved] (PIG-1891) Enable StoreFunc to make intelligent decision based on job success or failure

2012-09-07 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved PIG-1891. - Resolution: Fixed Fix Version/s: 0.11 Release Note: This adds a new method, cleanupOnSuccess

Re: Modifying databag on the fly

2012-09-05 Thread Alan Gates
You cannot modify a bag once it is written. The implementation is written around the assumption that bags are immutable after they are written. Creating a new bag should not create an OOM exception, as bags are built to spill when they grow too large. In fact it's this spilling feature that

Re: Modifying databag on the fly

2012-09-05 Thread Alan Gates
. Thanks -- Prasanth On Sep 5, 2012, at 9:24 PM, Alan Gates ga...@hortonworks.com wrote: You cannot modify a bag once it is written. The implementation is written around the assumption that bags are immutable after they are written. Creating a new bag should not create an OOM exception

Re: Current patch available' and open issues

2012-09-04 Thread Alan Gates
+1. I think we'll also need to work on training contributors to change the state to Patch Available, as I find a lot of JIRAs in the open state that are ready for review. Alan. On Sep 4, 2012, at 10:42 AM, Bill Graham wrote: HCatalog is having a similar discussion and has opened this JIRA

[jira] [Commented] (PIG-2846) Can we skip hcat related e2e when hcat is not installed?

2012-09-04 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13447895#comment-13447895 ] Alan Gates commented on PIG-2846: - Once I apply this patch it seems to skip these tests

[jira] [Updated] (PIG-2892) piggybank build failing on trunk

2012-08-29 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2892: Resolution: Duplicate Status: Resolved (was: Patch Available) Duplicates PIG-2893

[jira] [Resolved] (PIG-2893) fix DBStorage compile issue

2012-08-29 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved PIG-2893. - Resolution: Fixed Fix Version/s: 0.11 I went ahead and checked in the fix since it was breaking my

[jira] [Commented] (PIG-2893) fix DBStorage compile issue

2012-08-28 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443785#comment-13443785 ] Alan Gates commented on PIG-2893: - +1, patch looks good. fix DBStorage

[jira] [Commented] (PIG-2892) piggybank build failing on trunk

2012-08-28 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443786#comment-13443786 ] Alan Gates commented on PIG-2892: - Thejas filed a separate issue for this, PIG-2893. He's

[jira] [Commented] (PIG-2895) jodatime jar missing in pig-withouthadoop.jar

2012-08-28 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13443811#comment-13443811 ] Alan Gates commented on PIG-2895: - When I run the e2e tests I am still seeing an error, even

[jira] [Commented] (PIG-1891) Enable StoreFunc to make intelligent decision based on job success or failure

2012-08-27 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13442699#comment-13442699 ] Alan Gates commented on PIG-1891: - This adds a failure in TestLoadStoreFuncLifeCycle

[jira] [Updated] (PIG-1891) Enable StoreFunc to make intelligent decision based on job success or failure

2012-08-27 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-1891: Status: Open (was: Patch Available) Enable StoreFunc to make intelligent decision based on job success

[jira] [Commented] (PIG-1891) Enable StoreFunc to make intelligent decision based on job success or failure

2012-08-27 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13442760#comment-13442760 ] Alan Gates commented on PIG-1891: - Never mind on TestMacroExpansion. I see that is failing

[jira] [Created] (PIG-2892) piggybank build failing on trunk

2012-08-27 Thread Alan Gates (JIRA)
Alan Gates created PIG-2892: --- Summary: piggybank build failing on trunk Key: PIG-2892 URL: https://issues.apache.org/jira/browse/PIG-2892 Project: Pig Issue Type: Bug Components

[jira] [Commented] (PIG-2881) Add SUBTRACT eval function

2012-08-26 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13442240#comment-13442240 ] Alan Gates commented on PIG-2881: - Patch looks fine, except I don't see a reason

[jira] [Commented] (PIG-2889) HBaseAvroStorage UDF

2012-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440406#comment-13440406 ] Alan Gates commented on PIG-2889: - Hive has an AvroSerDe which I believe can read schema

[jira] [Commented] (PIG-2844) ant makepom is misconfigured

2012-08-23 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440707#comment-13440707 ] Alan Gates commented on PIG-2844: - Ok, +1 I guess then. ant makepom

Re: Number of mappers in MRCompiler

2012-08-23 Thread Alan Gates
Sorry for the very slow response, but here it is, hopefully better late than never. On Jul 25, 2012, at 4:28 PM, Prasanth J wrote: Thanks Alan. The requirement for me is that I want to load N number of samples based on the input file size and perform naive cube computation to determine the

[jira] [Commented] (PIG-2844) ant makepom is misconfigured

2012-08-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13439935#comment-13439935 ] Alan Gates commented on PIG-2844: - Patch looks fine to me. The generated pom is very

[jira] [Updated] (PIG-1332) NullPointerException in PigServer when invoking debugOff() method

2012-08-20 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-1332: Status: Patch Available (was: Open) NullPointerException in PigServer when invoking debugOff() method

Re: Fixing bad JIRAs

2012-08-15 Thread Alan Gates
I don't seem to have permission to do this as well. I do have the Administration tab but nothing labeled System pops up when I go there. Olga may be able to do it. If not, we'll need to ask someone with more JIRA power to do this. Alan. On Aug 12, 2012, at 7:45 PM, Bill Graham wrote:

[jira] [Commented] (PIG-1891) Enable StoreFunc to make intelligent decision based on job success or failure

2012-08-13 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433396#comment-13433396 ] Alan Gates commented on PIG-1891: - I don't see where cleanupOnSuccess is invoked

Re: Storing statistics of input dataset

2012-08-06 Thread Alan Gates
Pig does not have a metadata store, so it doesn't store statistics on data. However, through HCatalog it will have access to the same statistics that Hive stores. As far as using this data to optimize Pig operations, I'd like to rework the backend to start taking advantage of such

Re: Number of mappers in MRCompiler

2012-07-25 Thread Alan Gates
No. The number of mappers is determined by the InputFormat used by your load function (TextInputFormat if you're using the default PigStorage loader) when the Hadoop job is submitted. Pig doesn't have access to that info until it's handed the jobs off to MapReduce. Alan. On Jul 25, 2012, at

[jira] [Created] (PIG-2826) Training link on front page no longer points to Pig training

2012-07-18 Thread Alan Gates (JIRA)
Alan Gates created PIG-2826: --- Summary: Training link on front page no longer points to Pig training Key: PIG-2826 URL: https://issues.apache.org/jira/browse/PIG-2826 Project: Pig Issue Type: Bug

[jira] [Updated] (PIG-2826) Training link on front page no longer points to Pig training

2012-07-18 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2826: Attachment: PIG-2826.patch Training link on front page no longer points to Pig training

[jira] [Updated] (PIG-2826) Training link on front page no longer points to Pig training

2012-07-18 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2826: Status: Patch Available (was: Open) Training link on front page no longer points to Pig training

Re: Jenkins / Clover

2012-07-18 Thread Alan Gates
the nightly. 2012/7/17 Alan Gates ga...@hortonworks.com I'm fine with removing it from the nightly build. I don't see any reason to run that every day, especially since it slows down the tests. Let's not remove it from ant, as it's useful to run occasionally. Alan. On Jul 17, 2012

Re: Jenkins / Clover

2012-07-17 Thread Alan Gates
I'm fine with removing it from the nightly build. I don't see any reason to run that every day, especially since it slows down the tests. Let's not remove it from ant, as it's useful to run occasionally. Alan. On Jul 17, 2012, at 3:17 PM, Gianmarco De Francisci Morales wrote: Hi, Clover

Re: Including wonderdog in Pig contrib

2012-07-13 Thread Alan Gates
: Who's the author for Wonderdog? Can Russell or the author talk about it in our next hackthon? Also we need to discuss with the author about it. On Tue, Jul 10, 2012 at 9:23 AM, Alan Gates ga...@hortonworks.com wrote: From https://issues.apache.org/jira/browse/PIG-2803 posted yesterday

[jira] [Commented] (PIG-2812) Spill InternalCachedBag into only 1 file

2012-07-12 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13413301#comment-13413301 ] Alan Gates commented on PIG-2812: - Why not just change the {{clear}} method to delete

[jira] [Commented] (PIG-2803) Include Wonderdog (ElasticSearch Integration) in contrib/

2012-07-10 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13410484#comment-13410484 ] Alan Gates commented on PIG-2803: - I think this should be discussed as a proposal on the dev

Including wonderdog in Pig contrib

2012-07-10 Thread Alan Gates
From https://issues.apache.org/jira/browse/PIG-2803 posted yesterday by Russell. I'm copying it here because I think we need to discuss this and decide what we want to do: I propose to add Wonderdog to Pig contrib/ Wonderdog is an Apache 2.0 licensed project that adds Hadoop and Pig

Re: Pig for MongoDB

2012-07-07 Thread Alan Gates
There are mongo load and store functions for pig at https://github.com/mongodb/mongo-hadoop/ Is this what you were looking for or were you more asking if pig and mongo play well together? Alan. On Jul 7, 2012, at 2:56 PM, Russell Jurney wrote: I want Pig for MongoDB, for acting on smaller

[jira] [Commented] (PIG-2742) Rank Operator Syntax

2012-06-26 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13401537#comment-13401537 ] Alan Gates commented on PIG-2742: - Your suggested text in CHANGES would be fine. Or you can

[jira] [Assigned] (PIG-2766) Pig-HCat Usability

2012-06-25 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates reassigned PIG-2766: --- Assignee: Vikram Dixit K Pig-HCat Usability -- Key: PIG-2766

[jira] [Updated] (PIG-2764) Add a biginteger and bigdecimal type to pig

2012-06-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2764: Attachment: fixedpoint.patch First pass at a class to implement fixed point types. Add

[jira] [Commented] (PIG-2764) Add a biginteger and bigdecimal type to pig

2012-06-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399793#comment-13399793 ] Alan Gates commented on PIG-2764: - I don't know of any good libraries. And everything

[jira] [Commented] (PIG-2764) Add a biginteger and bigdecimal type to pig

2012-06-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13399822#comment-13399822 ] Alan Gates commented on PIG-2764: - I was hoping we could build something faster since

[jira] [Commented] (PIG-2764) Add a biginteger and bigdecimal type to pig

2012-06-21 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13398937#comment-13398937 ] Alan Gates commented on PIG-2764: - Do you really need biginteger and bigdecimal, or are you

Re: Pig blog?

2012-06-19 Thread Alan Gates
you add me as admin? Thanks, Julien On Mon, Jun 18, 2012 at 10:32 AM, Alan Gates ga...@hortonworks.com wrote: The blog has been created. If you have something to post you'll need to request a blog account as described below and then I can add you as an author. I'm happy to add any PMC

Re: Pig blog?

2012-06-15 Thread Alan Gates
Apache supports blogs for its projects, http://www.apache.org/dev/apache-blogs.html#askforblog I've filed an infra request to create a blog, https://issues.apache.org/jira/browse/INFRA-4923 Once it's created any PMC member or committer who wants to blog should file a request with infra as

[jira] [Updated] (PIG-2745) Pig e2e test RubyUDFs fails in MR mode when running from tarball

2012-06-15 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2745: Status: Patch Available (was: Open) Pig e2e test RubyUDFs fails in MR mode when running from tarball

[jira] [Updated] (PIG-2632) Create a SchemaTuple which generates efficient Tuples via code gen

2012-06-15 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2632: Status: Patch Available (was: Open) Create a SchemaTuple which generates efficient Tuples via code gen

[jira] [Commented] (PIG-2754) pig can't sum with inner mutiply

2012-06-15 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13295811#comment-13295811 ] Alan Gates commented on PIG-2754: - This is not a bug, it is the semantics of the language

[jira] [Resolved] (PIG-2754) pig can't sum with inner mutiply

2012-06-15 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved PIG-2754. - Resolution: Invalid pig can't sum with inner mutiply

Re: Handle NULL values in Cube dimensions

2012-06-08 Thread Alan Gates
Option 1 (throwing an error) is bad. It violates Pigs eat anything (see http://pig.apache.org/philosophy.html). Do we need to give users an ability to name this unknown column? Why not just label it unknown and be done? Alan. On Jun 6, 2012, at 2:24 PM, Prasanth J wrote: Hello everyone

Re: CUBE/ROLLUP/GROUPING SETS syntax

2012-05-30 Thread Alan Gates
Some thoughts on this: 1) +1 to what Dmitriy said on HAVING 2) We need to be clear about separating operators in the grammar versus logical plan versus physical plan. The choices you make in the grammar are totally independent of the other two. That is, you could choose the syntax: rel =

Re: CUBE/ROLLUP/GROUPING SETS syntax

2012-05-30 Thread Alan Gates
won't beat it any further... if people prefer a different syntax, that's fine. Just excited to have the features in Pig! +1, I can live with any of the 3 syntax choices (near SQL, original, and Jon's). Alan. Jon 2012/5/30 Alan Gates ga...@hortonworks.com Some thoughts on this: 1) +1 to what

Re: [jira] [Commented] (PIG-2732) Let's get rid of the deprecated Tuple methods

2012-05-30 Thread Alan Gates
Officially our policy is deprecated methods can be removed after they have been deprecated for at least one release. Unofficially we've tended to follow the path of least resistance and only remove deprecated things when we had a reason. This approach seems reasonable to me, as it inflicts

Re: Is there a good benchmark to evaluate the CPU time/space tradeoff in the shuffle stage of hadoop?

2012-05-22 Thread Alan Gates
You might post this same question to mapred-user@hadoop. I know Owen and Arun have done a lot of analysis of these kinds of things when optimizing the terasort. Others may have valuable feedback there as well. Alan. On May 22, 2012, at 12:23 PM, Jonathan Coveney wrote: I've been dealing

Re: About the Pig Latin Implementation

2012-05-21 Thread Alan Gates
http://infolab.stanford.edu/~olston/publications/vldb09.pdf contains details of how we built Pig. This paper is old and does not include many of the newer optimizations. But the basic approach has not changed. Alan. On May 21, 2012, at 4:30 AM, Li Shengmei wrote: Hi, all I am

[jira] [Commented] (PIG-2066) Accumulators should be able to early-terminate

2012-05-18 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279098#comment-13279098 ] Alan Gates commented on PIG-2066: - +1, lgtm Accumulators should be able

[jira] [Commented] (PIG-2694) Accumulator e2e tests don't work

2012-05-18 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279101#comment-13279101 ] Alan Gates commented on PIG-2694: - Do you mean they fail or abort? We haven't been seeing

[jira] [Commented] (PIG-2695) e2e test should thrown an error if an entry in nightly.conf is malformed

2012-05-18 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279108#comment-13279108 ] Alan Gates commented on PIG-2695: - I agree that the error messages in the e2e tests

[jira] [Commented] (PIG-2166) UDFs to flatten a bag

2012-05-18 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13279128#comment-13279128 ] Alan Gates commented on PIG-2166: - -1 to join. We already use that for another concept

[jira] [Updated] (PIG-2677) Add target to build.xml to generate clover summary reports

2012-04-30 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2677: Attachment: PIG-2677.patch This patch adds a generate-pdf-clover-reports target that generates a one page

[jira] [Created] (PIG-2677) Add target to build.xml to generate clover summary reports

2012-04-30 Thread Alan Gates (JIRA)
Alan Gates created PIG-2677: --- Summary: Add target to build.xml to generate clover summary reports Key: PIG-2677 URL: https://issues.apache.org/jira/browse/PIG-2677 Project: Pig Issue Type

[jira] [Updated] (PIG-2677) Add target to build.xml to generate clover summary reports

2012-04-30 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2677: Fix Version/s: 0.11 Status: Patch Available (was: Open) Add target to build.xml to generate

[jira] [Updated] (PIG-2677) Add target to build.xml to generate clover summary reports

2012-04-30 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2677: Resolution: Fixed Status: Resolved (was: Patch Available) Patch checked in. Add

Re: [jira] [Resolved] (PIG-2650) Convenience mock Loader and Storer to simplify unit testing of Pig scripts

2012-04-26 Thread Alan Gates
One other caveat I'd like to add, we should never ever check in interface changes on branches. You could argue that falls under disruptive changes, but I think they're worth calling out. I'm definitely +1 on checking this in though. In general I'd like to figure out how we can use mock

[jira] [Commented] (PIG-2663) Expose helpful ScriptState methods

2012-04-26 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13263231#comment-13263231 ] Alan Gates commented on PIG-2663: - +1 Expose helpful ScriptState methods

[jira] [Commented] (PIG-2660) PPNL should get notified of plan before it gets executed

2012-04-25 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13262117#comment-13262117 ] Alan Gates commented on PIG-2660: - Looks good. Do we have any idea how often people

<    1   2   3   4   5   6   7   8   >