Re: [VOTE] Release Pig 0.10.0 (candidate 0)

2012-04-24 Thread Alan Gates
Oddly enough that's how we've always done the version numbering in the build file, setting it up for the next version. I agree we should change it, but since we've done it in every release since 0.1, I don't think we should block 0.10 for it. The change is to change the directions in

[jira] [Commented] (PIG-2651) Provide a much easier to use accumulator interface

2012-04-17 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13256065#comment-13256065 ] Alan Gates commented on PIG-2651: - In general looks good. This will be great

Re: Java 7 and Pig, hijacked from PIG-2643

2012-04-16 Thread Alan Gates
There are some exciting new features in Java 7. However, realistically we can't start using it until Hadoop does. I don't recall any discussion on it on their list, though I may have missed it. But AFAIK they have no migration plans at this time. Alan. On Apr 12, 2012, at 11:55 AM,

Re: Scalar problem

2012-04-09 Thread Alan Gates
I'm +1 on this idea, since it's been a problem since the beginning. Why not use regular casting notation though, rather than develop another notation? That's what we discussed originally when we were deciding whether to require casting or do it silently. So instead of D-a or SCALAR(D) it

Re: Scalar problem

2012-04-09 Thread Alan Gates
this error case and throw during compilation. 3) Improve MR-side logging to make it clear that a relation is being loaded from the side, what the relation is, etc. I believe we have jiras open for all of these.. D On Mon, Apr 9, 2012 at 10:15 AM, Alan Gates ga...@hortonworks.com wrote: I'm

[jira] [Created] (PIG-2640) Usage message gives wrong information for Pig additional jars

2012-04-09 Thread Alan Gates (Created) (JIRA)
: 0.11 Reporter: Alan Gates Priority: Minor The usage statement in Main.java says: {code} pig.additional.jars=comma seperated list of jars. Used in place of register command. {code} But in PigServer.java it actually splits on ':', not ','. See the method

Re: What outstanding patches are must haves for 0.10? (path to a RC theater)

2012-04-04 Thread Alan Gates
at 11:54 AM, Alan Gates ga...@hortonworks.com wrote: Sure, that's fine. Alan. On Mar 27, 2012, at 11:46 AM, Jonathan Coveney wrote: I agree, Alan, though I think we should make the deadline Monday, 4/2. Most testing is going to happen during the week anyway, and I'd like to give true

[jira] [Updated] (PIG-2601) Additional document for 0.10

2012-04-03 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2601: Attachment: PIG-2601-2.patch Modified version of the patch, mostly with minor grammatical corrections

Re: Making git the repo of choice for Pig?

2012-04-03 Thread Alan Gates
, 2012, at 9:36 AM, Alan Gates ga...@hortonworks.com wrote: AFAIK Apache still does not support git as a primary repository. You can use the git mirror, which Pig does. If this has changed (or when it does), I'm +0 on changing, by which I mean I don't care which we use. Alan

Re: What outstanding patches are must haves for 0.10? (path to a RC theater)

2012-03-27 Thread Alan Gates
I propose we set a deadline and say patches need to be ready by that day, or they don't go in. Otherwise this time next week there will be another set of patches people would like to see. I'd say the deadline should be 3/30 (this Friday). That would mean we could address any final test

Re: What outstanding patches are must haves for 0.10? (path to a RC theater)

2012-03-27 Thread Alan Gates
. Sound ok? 2012/3/27 Alan Gates ga...@hortonworks.com I propose we set a deadline and say patches need to be ready by that day, or they don't go in. Otherwise this time next week there will be another set of patches people would like to see. I'd say the deadline should be 3/30

Re: Fixing a broken dependency // can we include a patched piece of JRuby source code in Pig?

2012-03-23 Thread Alan Gates
Won't a lot of people already have their version of JRuby and not want a special one? I'm fine with having a patched version on github and referring it in our release notes. I'm not wild about including a version of JRuby with Pig, for both licensing reasons and because our tar file is

Re: Making git the repo of choice for Pig?

2012-03-22 Thread Alan Gates
On Wed, Mar 21, 2012 at 12:47 PM, Russell Jurney russell.jur...@gmail.comwrote: I think this changed, and you are now allowed. Looking for the tweets. Russell Jurney http://datasyndrome.com On Mar 21, 2012, at 9:36 AM, Alan Gates ga...@hortonworks.com wrote: AFAIK Apache still

Re: Making git the repo of choice for Pig?

2012-03-21 Thread Alan Gates
AFAIK Apache still does not support git as a primary repository. You can use the git mirror, which Pig does. If this has changed (or when it does), I'm +0 on changing, by which I mean I don't care which we use. Alan. On Mar 20, 2012, at 11:22 PM, Jonathan Coveney wrote: Would anyone be

Re: [PIG-2226] Submitting a patch how to

2012-03-19 Thread Alan Gates
See Contributing your work in https://cwiki.apache.org/confluence/display/PIG/HowToContribute It talks about how to upload the patch to JIRA. Alan. On Mar 19, 2012, at 6:02 AM, Sachith Withana wrote: Hi folks, I prepared a patch for the PIG-2226 bug and I'd like to know how I can submit it.

Re: Pig User Group

2012-03-19 Thread Alan Gates
There will be a Pig meetup the day before Hadoop summit. But if you guys are willing to organize one sooner that's great. It would be cool to have a PUG every couple months, maybe rotating between SF and the valley. We just need someone with the time and desire to organize it. Alan. On Mar

Re: Pig User Group

2012-03-19 Thread Alan Gates
:) , and I am sure Salesforce campus could be used too, what other logistics would need to be worked out to arrange such a meet? Thanks, Prashant On Mon, Mar 19, 2012 at 5:23 PM, Alan Gates ga...@hortonworks.com wrote: There will be a Pig meetup the day before Hadoop summit. But if you guys

[jira] [Commented] (PIG-2598) Change source structure for sources in other languages

2012-03-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13231416#comment-13231416 ] Alan Gates commented on PIG-2598: - As said on the dev list, I vote for options 2 or 3 until

Re: Where do we want to put non-java source files?

2012-03-16 Thread Alan Gates
once it is in. 2012/3/16 Alan Gates ga...@hortonworks.com I vote we avoid the re-organization until there's a tangible benefit. I don't think there's any cost (beyond annoyance maybe) to putting ruby stuff in src-ruby. There isn't any benefit to moving to src/main/java/maven/demands

[jira] [Commented] (PIG-2182) Add more append support to DataByteArray

2012-03-14 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13229756#comment-13229756 ] Alan Gates commented on PIG-2182: - As long as the tests pass consistently, no objections

[jira] [Commented] (PIG-2528) Add setScriptPlan() method to PigProgressNotificationListener

2012-02-14 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13208078#comment-13208078 ] Alan Gates commented on PIG-2528: - I don't see the abstract implementation in the second

Re: Adding a method to PigProgressNotificationListener

2012-02-13 Thread Alan Gates
AFAIK the only user of this interface is Oozie. You might want to shoot a message to oozie-...@incubator.apache.org and let them know about the change. Alan. On Feb 12, 2012, at 5:12 PM, Dmitriy Ryaboy wrote: I created https://issues.apache.org/jira/browse/PIG-2528 to track this issue.

[jira] [Commented] (PIG-2482) Integrate HCat DDL command into Pig

2012-01-27 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195271#comment-13195271 ] Alan Gates commented on PIG-2482: - In general looks good. A couple of comments/questions

[jira] [Commented] (PIG-2482) Integrate HCat DDL command into Pig

2012-01-27 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13195319#comment-13195319 ] Alan Gates commented on PIG-2482: - +1, looks good. Integrate HCat DDL

[jira] [Commented] (PIG-2454) Make use of primitive tuples in builtin UDFs and operators

2012-01-20 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13190070#comment-13190070 ] Alan Gates commented on PIG-2454: - My run of the nightly e2e tests got aborts

[jira] [Updated] (PIG-2454) Make use of primitive tuples in builtin UDFs and operators

2012-01-20 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2454: Attachment: BugFix_1.log Types_6.log Cross_1.log The errors seemed to fall

[jira] [Created] (PIG-2488) Move Python unit tests to e2e tests

2012-01-20 Thread Alan Gates (Created) (JIRA)
Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.11 In keeping with our goal to shrink the unit test run time, and where appropriate move unit tests to end-to-end tests, there are a number of tests that could be removed from the TestScriptUDF module and placed in e2e

[jira] [Updated] (PIG-2488) Move Python unit tests to e2e tests

2012-01-20 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2488: Status: Patch Available (was: Open) Move Python unit tests to e2e tests

[jira] [Updated] (PIG-2488) Move Python unit tests to e2e tests

2012-01-20 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2488: Attachment: python_udfs.patch Patch that moves as many tests as possible from junit to e2e. Several tests

[jira] [Commented] (PIG-2454) Make use of primitive tuples in builtin UDFs and operators

2012-01-19 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189187#comment-13189187 ] Alan Gates commented on PIG-2454: - I haven't had time to review this yet, but since it's

[jira] [Commented] (PIG-2482) Integrate HCat DDL command into Pig

2012-01-19 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13189189#comment-13189189 ] Alan Gates commented on PIG-2482: - I think it's fine not to add commands for it to PigServer

Re: [VOTE] Release Pig 0.9.2 (candidate 1)

2012-01-18 Thread Alan Gates
+1. Checked the rat report, signatures and md5s on all packages. I installed the tar/src release and ran smoke tests in local and cluster mode. I also build the packages and ran the commit unit tests. Alan. On Jan 17, 2012, at 5:16 PM, Daniel Dai wrote: Hi, I have created a candidate

Re: [VOTE] Release Pig 0.9.2 (candidate 0)

2012-01-16 Thread Alan Gates
+1. I checked the keys and signature of .rpm, .deb, and source release. I ran smoke tests in both local and cluster mode on the source release. One issue that we should clear up but that I don't believe blocks the tests is that there are a bunch (~100) 0 length .java files. These are files

[jira] [Commented] (PIG-2359) Support more efficient Tuples when schemas are known

2012-01-13 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13186028#comment-13186028 ] Alan Gates commented on PIG-2359: - +1, latest patch looks good. Looking forward to Pig

[jira] [Commented] (PIG-2406) Improve double precision

2012-01-12 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13185349#comment-13185349 ] Alan Gates commented on PIG-2406: - I have no problem adding a fixed type decimal to Pig

Re: Request for contribution

2012-01-10 Thread Alan Gates
What's your ID on JIRA? I can make you a contributor so that you can assign JIRAs to yourself. Alan. On Jan 10, 2012, at 9:54 AM, Carl Frendo wrote: Hi, I would like to start contributing to the pig project starting by the newbie issues. I tried to assign a ticket in JIRA but I found

[jira] [Commented] (PIG-2457) JsonLoaderStorage tests is broken for e2e

2012-01-04 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13179871#comment-13179871 ] Alan Gates commented on PIG-2457: - +1, looks good. JsonLoaderStorage tests

[jira] [Updated] (PIG-2432) Eclipse .classpath file is out of date

2012-01-03 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2432: Resolution: Fixed Status: Resolved (was: Patch Available) Patch checked

[jira] [Commented] (PIG-2317) Ruby/Jruby UDFs

2012-01-03 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13179093#comment-13179093 ] Alan Gates commented on PIG-2317: - I've been reviewing and playing with this patch over

[jira] [Updated] (PIG-2317) Ruby/Jruby UDFs

2012-01-03 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2317: Attachment: jruby_scripting_7.patch Ruby/Jruby UDFs --- Key: PIG-2317

[jira] [Commented] (PIG-2431) Upgrade bundled hadoop version to 20.205

2012-01-03 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13179128#comment-13179128 ] Alan Gates commented on PIG-2431: - +1, changes look good. Upgrade bundled

Re: StoreMetadata.storeStatistics

2011-12-20 Thread Alan Gates
At the moment it is unused. It was placed there on the belief that someday storage functions like HCatStorer would want to record statistics from Pig when data was being generated. Alan. On Dec 19, 2011, at 11:04 PM, Vivek Padmanabhan wrote: Hi, Can someone tell what is the exact use case

Re: How do we feel about improving pigmix queries?

2011-12-20 Thread Alan Gates
On Dec 14, 2011, at 12:41 PM, Dmitriy Ryaboy wrote: Two questions relating to that: 1) we currently hardcode parallel 40 in pigmix. Since Pig can now automatically select parallelism, would it be better to let it do so? I agree the hard wiring is bad. But my take is that the auto-parallel

[jira] [Commented] (PIG-2359) Support more efficient Tuples when schemas are known

2011-12-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171315#comment-13171315 ] Alan Gates commented on PIG-2359: - bq. The use case isn't just internal, I started

[jira] [Commented] (PIG-2421) EvalFuncs need redesigned

2011-12-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171327#comment-13171327 ] Alan Gates commented on PIG-2421: - Responses to Julien's comments above: bq. Not to have

[jira] [Commented] (PIG-2421) EvalFuncs need redesigned

2011-12-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171353#comment-13171353 ] Alan Gates commented on PIG-2421: - BTW, thanks both Julien and Dmitriy for the feedback

[jira] [Commented] (PIG-2359) Support more efficient Tuples when schemas are known

2011-12-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171358#comment-13171358 ] Alan Gates commented on PIG-2359: - Use the aggregated warnings, PigLogger.warn rather than

[jira] [Created] (PIG-2432) Eclipse .classpath file is out of date

2011-12-16 Thread Alan Gates (Created) (JIRA)
Reporter: Alan Gates Assignee: Alan Gates Priority: Minor log4j and slf4j-log4j jars have changed versions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira

[jira] [Updated] (PIG-2432) Eclipse .classpath file is out of date

2011-12-16 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2432: Fix Version/s: 0.11 Status: Patch Available (was: Open) Eclipse .classpath file is out

[jira] [Commented] (PIG-2426) ProgressableReporter.progress(String msg) is an empty function

2011-12-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171406#comment-13171406 ] Alan Gates commented on PIG-2426: - When I run the tests, I get failures in TestAccumulator

[jira] [Commented] (PIG-2304) Give jython access to site-packages

2011-12-16 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13171410#comment-13171410 ] Alan Gates commented on PIG-2304: - Running test-patch gives {code} [exec] [exec

[jira] [Updated] (PIG-2304) Give jython access to site-packages

2011-12-16 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2304: Status: Open (was: Patch Available) Give jython access to site-packages

[jira] [Updated] (PIG-2304) Give jython access to site-packages

2011-12-16 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2304: Attachment: patchFindbugsWarnings.xml Search for JythonScriptEngine to find the issue that needs addressed

[jira] [Commented] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

2011-12-15 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13170589#comment-13170589 ] Alan Gates commented on PIG-2417: - Jeremy, For the patch to apply in an SVN environment

[jira] [Updated] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

2011-12-15 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2417: Status: Open (was: Patch Available) Streaming UDFs - allow users to easily write UDFs in scripting

[jira] [Updated] (PIG-2422) Add log messages for Jython schema definitions

2011-12-15 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2422: Resolution: Fixed Fix Version/s: (was: 0.9.1) 0.11 Status: Resolved

[jira] [Updated] (PIG-2426) ProgressableReporter.progress(String msg) is an empty function

2011-12-15 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2426: Status: Open (was: Patch Available) ProgressableReporter.progress(String msg) is an empty function

[jira] [Commented] (PIG-2403) Reduce code duplication in SUM, MAX, MIN udfs

2011-12-13 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13168881#comment-13168881 ] Alan Gates commented on PIG-2403: - Hmm, it works on my linux box but not my mac. How odd

[jira] [Commented] (PIG-2417) Streaming UDFs - allow users to easily write UDFs in scripting languages with no JVM implementation.

2011-12-10 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166889#comment-13166889 ] Alan Gates commented on PIG-2417: - This looks interesting. Some thoughts regarding the open

[jira] [Created] (PIG-2421) EvalFuncs need redesigned

2011-12-10 Thread Alan Gates (Created) (JIRA)
Gates Assignee: Alan Gates The current EvalFunc interface (and associated Algebraic and Accumulator interfaces) have grown unwieldy. In particular, people have noted the following issues: # Writing a UDF requires a lot of boiler plate code. # Since UDFs always pass a tuple, users

[jira] [Commented] (PIG-2406) Improve double precision

2011-12-09 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13166314#comment-13166314 ] Alan Gates commented on PIG-2406: - You could certainly contribute a UDF to Piggybank to do

Re: Adding a flatten that doesn't throw out the row if you have an empty bag?

2011-12-09 Thread Alan Gates
+1, it seems like people often ask for this. I also would prefer a new operator. OUTER_FLATTEN maybe? Alan. On Dec 9, 2011, at 5:49 PM, Jonathan Coveney wrote: I think this would be nice to have. We could either add a LEFTFLATTEN() operator, or add a flag to FLATTEN ie FLATTEN({()},true)

[jira] [Commented] (PIG-2384) Generic Invokers should use PigContext to resolve classes

2011-11-29 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13159454#comment-13159454 ] Alan Gates commented on PIG-2384: - +1, looks fine. Generic Invokers should

[jira] [Commented] (PIG-2358) JobStats.getHadoopCounters() is never set and always returns null

2011-11-23 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13156225#comment-13156225 ] Alan Gates commented on PIG-2358: - I see quite a few failures in TestPigRunner when I apply

[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never set and always returns null

2011-11-23 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2358: Attachment: TEST-org.apache.pig.test.TestPigRunner.txt Log file from failing test

[jira] [Updated] (PIG-2358) JobStats.getHadoopCounters() is never set and always returns null

2011-11-23 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2358: Status: Open (was: Patch Available) JobStats.getHadoopCounters() is never set and always returns null

[jira] [Updated] (PIG-2245) Add end to end test for tokenize

2011-11-23 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2245: Attachment: PIG-2245_2.patch Updated version of the patch that addresses my earlier comments

[jira] [Resolved] (PIG-2245) Add end to end test for tokenize

2011-11-23 Thread Alan Gates (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates resolved PIG-2245. - Resolution: Fixed Fix Version/s: 0.11 Patch 2 checked in. Add end to end test

[jira] [Commented] (PIG-2362) Rework Ant build.xml to use macrodef instead of antcall

2011-11-22 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13155302#comment-13155302 ] Alan Gates commented on PIG-2362: - +1, patch looks good. Runs fine with jar, jar

Re: What is the canonicalname field in a Schema object used for?

2011-11-18 Thread Alan Gates
Santosh is the best person to answer this, as he wrote that code. But, IIRC its purpose is to store the full name of a column after cogroups and joins. For example, A = load 'foo' as (u, v); B = load 'bar' as (x, y); C = join A by u, B by x; I believe the canonicalname will now hold A::u,

Re: Does the name of the tuple that a bag has to have matter?

2011-11-18 Thread Alan Gates
The name doesn't matter. We mostly left it there for backward compatibility, for both specifying schemas and for UDFs. I do think we should make sure we ignore it everywhere (including equality for schemas). Alan. On Nov 16, 2011, at 7:17 PM, Jonathan Coveney wrote: This is related to an

[jira] [Commented] (PIG-2355) ant clean does not clean e2e test build artifacts

2011-11-09 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13147304#comment-13147304 ] Alan Gates commented on PIG-2355: - +1 ant clean does not clean e2e test

[jira] [Updated] (PIG-2352) e2e test harness' use of environment variables causes unintended effects between tests

2011-11-08 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2352: Resolution: Fixed Status: Resolved (was: Patch Available) Patch checked into both 0.10 and trunk

[jira] [Commented] (PIG-2209) JsonMetadata fails to find schema for glob paths

2011-11-04 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13144225#comment-13144225 ] Alan Gates commented on PIG-2209: - I agree with Dmitriy that this will be very useful

[jira] [Commented] (PIG-2352) e2e test harness' use of environment variables causes unintended effects between tests

2011-11-04 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13144388#comment-13144388 ] Alan Gates commented on PIG-2352: - The issue is that the java parameters are stored

[jira] [Created] (PIG-2352) e2e test harness' use of environment variables causes unintended effects between tests

2011-11-04 Thread Alan Gates (Created) (JIRA)
: Pig Issue Type: Bug Components: site Affects Versions: 0.9.1 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.10, 0.11 If you run the new Bloom tests by themselves they pass. If you run them after running the Accumulator tests

[jira] [Updated] (PIG-2352) e2e test harness' use of environment variables causes unintended effects between tests

2011-11-04 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2352: Attachment: PIG-2352.patch I did cheat a bit. This patch also adds an hdfs temp path to the test harness

[jira] [Updated] (PIG-2352) e2e test harness' use of environment variables causes unintended effects between tests

2011-11-04 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2352: Status: Patch Available (was: Open) e2e test harness' use of environment variables causes unintended

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-11-03 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Resolution: Fixed Fix Version/s: 0.11 Status: Resolved (was: Patch Available) Checked

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-11-02 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Attachment: PIG-bloom-3.patch Add builtin UDFs for building and using bloom filters

[jira] [Commented] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-11-02 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142738#comment-13142738 ] Alan Gates commented on PIG-2328: - Uploaded new patch that follows Dmitriy's suggestion

[jira] [Commented] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-11-02 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13142747#comment-13142747 ] Alan Gates commented on PIG-2328: - Checked into trunk. I'll hold the JIRA open until I get

[jira] [Commented] (PIG-2332) JsonLoader/JsonStorage

2011-10-26 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13136644#comment-13136644 ] Alan Gates commented on PIG-2332: - One comment. I didn't see any tests that check

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-24 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Status: Open (was: Patch Available) Add builtin UDFs for building and using bloom filters

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-24 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Attachment: PIG-bloom-2.patch A new version of the patch that fixes the single bloom filter per query issue

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-24 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Status: Patch Available (was: Open) Add builtin UDFs for building and using bloom filters

[jira] [Commented] (PIG-2318) Push extra jars to distributed cache and use the classloader enxtension mechanism in PigContext to load them on the backend

2011-10-20 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132184#comment-13132184 ] Alan Gates commented on PIG-2318: - This patch does not create any new functionality, correct

[jira] [Commented] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-20 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132212#comment-13132212 ] Alan Gates commented on PIG-2328: - bq. Correct me if I am wrong, but this doesn't work

[jira] [Commented] (PIG-2318) Push extra jars to distributed cache and use the classloader enxtension mechanism in PigContext to load them on the backend

2011-10-20 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13132216#comment-13132216 ] Alan Gates commented on PIG-2318: - When running the following script (from e2e test

[jira] [Commented] (PIG-2182) Add more append support to DataByteArray

2011-10-19 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130772#comment-13130772 ] Alan Gates commented on PIG-2182: - Reviewing this patch it looks fine. But every time I run

[jira] [Created] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-19 Thread Alan Gates (Created) (JIRA)
: internal-udfs Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.10 Bloom filters are a common way to do select a limited set of records before moving data for a join or other heavy weight operation. Pig should add UDFs to support building and using bloom filters

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-19 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Attachment: PIG-bloom.patch Add builtin UDFs for building and using bloom filters

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-19 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Status: Patch Available (was: Open) Add builtin UDFs for building and using bloom filters

[jira] [Commented] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-19 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13130904#comment-13130904 ] Alan Gates commented on PIG-2328: - This patch includes two new UDFs, BuildBloom and Bloom

[jira] [Updated] (PIG-2328) Add builtin UDFs for building and using bloom filters

2011-10-19 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2328: Release Note: Bloom filters are a common way to select a limited set of records before moving data

[jira] [Commented] (PIG-2317) Ruby/Jruby UDFs

2011-10-11 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13125499#comment-13125499 ] Alan Gates commented on PIG-2317: - I haven't finished testing this, but it looks good after

[jira] [Commented] (PIG-2317) Ruby/Jruby UDFs

2011-10-10 Thread Alan Gates (Commented) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124619#comment-13124619 ] Alan Gates commented on PIG-2317: - I'm reviewing this patch. Ruby/Jruby

[jira] [Updated] (PIG-2287) add test cases for limit and sample that use expressions with constants only (no scalar variables)

2011-10-06 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2287: Resolution: Fixed Status: Resolved (was: Patch Available) Patch checked in. add

[jira] [Updated] (PIG-2114) Enhancements to PIG HBaseStorage Load Store Func with extra scan configurations

2011-10-06 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2114: Fix Version/s: (was: 0.10) Status: Open (was: Patch Available) Cancelling patch as there has

[jira] [Updated] (PIG-1777) LoadFunc in a scripting language

2011-10-06 Thread Alan Gates (Updated) (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-1777: Fix Version/s: (was: 0.10) LoadFunc in a scripting language

<    1   2   3   4   5   6   7   8   >