Re: PROPOSAL: how to handle release documentation going forward

2012-10-22 Thread Jonathan Coveney
As someone who chronically under-documents, I think that this is a good idea. +1 2012/10/22 Olga Natkovich onatkov...@yahoo.com Hi, Since we lost the dedicated document writer for Pig, would it make sense to require that going forward (0.12 and beyond) we require that documentation updates

Review Request: Modernize a chunk of the tests

2012-10-25 Thread Jonathan Coveney
(I made no changes to TestLargeFile that should affect whether it passed, it was small and cosmetic) Thanks, Jonathan Coveney

Re: Pig 0.11

2012-10-26 Thread Jonathan Coveney
Francisci Morales g...@apache.org wrote: We are missing some documentation on the RANK but I guess we could add that to the branch and trunk in parallel. All the patches I was keeping an eye on are in. So +1 for me. -- Gianmarco On Wed, Oct 10, 2012 at 5:31 PM, Jonathan Coveney

Re: Review Request: Modernize a chunk of the tests

2012-10-26 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/7734/#review12786 --- On Oct. 25, 2012, 6:05 p.m., Jonathan Coveney wrote

Re: Did anything interesting come out of the Strata Pig meetup, from a dev perspective?

2012-10-26 Thread Jonathan Coveney
to Kim Truong from Horton marketing) * Mortar Data will continue to organize NYC meetups, and we'll continue to sponsor those as well Russell Jurney twitter.com/rjurney On Oct 26, 2012, at 2:54 PM, Jonathan Coveney jcove...@gmail.com wrote: I am told that the turnout was really good

Re: Welcome our newest committer Cheolsoo Park

2012-10-26 Thread Jonathan Coveney
Now is when the real fun starts, Cheolsoo. Congrats :) 2012/10/26 Alan Gates ga...@hortonworks.com Welcome Cheolsoo, and well deserved. Alan. On Oct 26, 2012, at 2:54 PM, Julien Le Dem wrote: All, Please join me in welcoming Cheolsoo Park as our newest Pig committer. He's been

Re: Add patch to reviewboard?

2012-10-26 Thread Jonathan Coveney
Poking is integral to the apache process :) 2012/10/26 Alan Gates ga...@hortonworks.com Yeah, feel free to keep poking us until someone takes it up. Alan. On Oct 25, 2012, at 10:47 AM, Timothy Chen wrote: Gotcha, my patch is very small so I'll leave it in the JIRA issue then. I see a

Whitespace changes in patches (i.e. please comment on PIG-3008)

2012-10-28 Thread Jonathan Coveney
Howdy y'all, I was hoping we could get a discussion going around this JIRA: https://issues.apache.org/jira/browse/PIG-3008 And it has also come up in the RB for https://issues.apache.org/jira/browse/PIG-3006 (see Cheolsoo's overall comments here https://reviews.apache.org/r/7734/) I largely

Re: Review Request: Modernize a chunk of the tests

2012-10-29 Thread Jonathan Coveney
, Jonathan Coveney

Review Request: Modernize more tests

2012-10-29 Thread Jonathan Coveney
/TestUnion.java 8da2bb7 test/org/apache/pig/test/pigunit/TestPigTest.java 65cd163 Diff: https://reviews.apache.org/r/7771/diff/ Testing --- All tests pass on my local machine. Thanks, Jonathan Coveney

Re: Jenkins build is back to normal : Pig-trunk #1350

2012-10-30 Thread Jonathan Coveney
woah there party people! This is awesome! I'm sure that some transient failure will bite us in the butt but the build has been failing for ages. 2012/10/30 Apache Jenkins Server jenk...@builds.apache.org See https://builds.apache.org/job/Pig-trunk/1350/changes

Re: Our release process

2012-11-05 Thread Jonathan Coveney
This seems to make sense to me. People can always back-port features, and this encourages them to use the newer ones. It also means we will be more rigorous about stability, which is good as it is a big plus for Pig. I think for older branches, stability trumps features in a big way. 2012/11/5

Re: Our release process

2012-11-06 Thread Jonathan Coveney
, Jonathan Coveney wrote: This seems to make sense to me. People can always back-port features, and this encourages them to use the newer ones. It also means we will be more rigorous about stability, which is good as it is a big plus for Pig. I think for older branches, stability trumps features

Re: Jenkins / Clover

2012-11-08 Thread Jonathan Coveney
Morales wrote: What is the procedure to modify the nightly build? If everyone agrees (and somebody explains me how) I volunteer to fix it. Cheers, -- Gianmarco On Wed, Jul 18, 2012 at 8:25 AM, Jonathan Coveney jcove...@gmail.com wrote: +1 A while

Re: Join implementations

2012-11-13 Thread Jonathan Coveney
A key class that aids in understanding how the physical layer works is the LogToPhyTranslationVisitor. You can look at the visitor for the LOJoin logical operator and see what it does for different join types (FRJoin being the easier). The code around plan generation is IMHO some of the most

Re: PIG-2599

2012-11-23 Thread Jonathan Coveney
This would be very welcome. Feel free to revitalize that JIRA, and I'm sure you'll get tons of support. This is something Pig sorely needs. 2012/11/22 Russell Jurney russell.jur...@gmail.com Bless you! I have no advice, but this would be great. Thanks for having a go! Russell Jurney

Re: PIG Reference Project for Learning Purposes

2012-12-03 Thread Jonathan Coveney
Hey Adarsh! I would recommend going through the open, but uncompleted JIRA's for Pig. If one looks like something you can tackle, we're all willing to help out. Jon 2012/12/2 Adarsh Sastry adarsh.sas...@yahoo.com Hello: I am new to HADOOP. I had gone through and practiced various topics and

Re: Our release process

2012-12-13 Thread Jonathan Coveney
. If the backward compatibility is not preserved then I am -1 for having to run e2e for every commit to a released branch. Santhosh From: Jonathan Coveney jcove...@gmail.com To: dev@pig.apache.org dev@pig.apache.org

Can someone explain the purpose UID serves in the logical plan?

2012-12-13 Thread Jonathan Coveney
Howdy ya'll, I'm trying to fix the issue in this JIRA: https://issues.apache.org/jira/browse/PIG-3093 I got the plan at one point, as saw this: #--- # New Logical Plan: #--- D: (Name: LOForEach Schema:

Is it known that you can't use rank as a column name?

2012-12-18 Thread Jonathan Coveney
I am getting weird errors. I assume this is because of the rank operator, and it is a non-backwards compatible change that could break scripts (it is breaking some scripts on our end). Wanted to see what people knew before trying to make this work in the parser. I updated the JIRA, but wanted to

Review Request: Add BigInteger and BigDecimal to Pig

2013-01-17 Thread Jonathan Coveney
test/org/apache/pig/test/TestBestFitCast.java e3172d2 test/org/apache/pig/test/TestPOCast.java 72787b3 Diff: https://reviews.apache.org/r/9012/diff/ Testing --- Thanks, Jonathan Coveney

Review Request: Introduce a syntax to be able to easily refer to the previously defined relation

2013-01-18 Thread Jonathan Coveney
/org/apache/pig/parser/QueryParser.g 642884e test/org/apache/pig/test/TestCommit.java 7df409e test/org/apache/pig/test/TestPreviousRelation.java PRE-CREATION Diff: https://reviews.apache.org/r/9019/diff/ Testing --- Thanks, Jonathan Coveney

Re: Review Request: Add BigInteger and BigDecimal to Pig

2013-01-18 Thread Jonathan Coveney
test/org/apache/pig/test/TestAdd.java de9e90d test/org/apache/pig/test/TestBestFitCast.java e3172d2 test/org/apache/pig/test/TestPOCast.java 72787b3 Diff: https://reviews.apache.org/r/9012/diff/ Testing --- Thanks, Jonathan Coveney

Re: Custom Scripting Engine

2013-01-21 Thread Jonathan Coveney
Can you describe at a higher level what you have in mind? 2013/1/21 Connor Woodson cwoodson@gmail.com Is there a way to get Pig to use your custom scripting engine without having to modify ScriptingEngine.java and placing it in the enum? It looks like it's possible with enums, but what

Re: Review Request: Add BigInteger and BigDecimal to Pig

2013-01-22 Thread Jonathan Coveney
., Jonathan Coveney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9012/ --- (Updated Jan. 18, 2013, 10:11 p.m.) Review

Re: Review Request: Add BigInteger and BigDecimal to Pig

2013-01-22 Thread Jonathan Coveney
--- On Jan. 18, 2013, 10:11 p.m., Jonathan Coveney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9012/ --- (Updated

Re: Review Request: Add BigInteger and BigDecimal to Pig

2013-01-22 Thread Jonathan Coveney
test/org/apache/pig/test/TestAdd.java de9e90d test/org/apache/pig/test/TestBestFitCast.java e3172d2 test/org/apache/pig/test/TestPOCast.java 72787b3 Diff: https://reviews.apache.org/r/9012/diff/ Testing --- Thanks, Jonathan Coveney

Re: Custom Scripting Engine

2013-01-22 Thread Jonathan Coveney
engine). So is this second use possible, or (for now) can custom SE's only be used for UDFs? (I'll admit here that I don't understand what I meant in the end of my previous email; feel free to ignore it). Thanks, - Connor On Mon, Jan 21, 2013 at 5:04 PM, Jonathan Coveney jcove...@gmail.com

Review Request: Allow UDFs to flatten themselves

2013-01-22 Thread Jonathan Coveney
://reviews.apache.org/r/9060/diff/ Testing --- Thanks, Jonathan Coveney

Re: Run a job async

2013-01-25 Thread Jonathan Coveney
On Thu, Jan 24, 2013 at 11:18 PM, Ramakrishna Nalam nramakris...@gmail.comwrote: That clarifies it for me, thanks a lot. Regards, Rama. On Fri, Jan 25, 2013 at 10:09 AM, Jonathan Coveney jcove...@gmail.com wrote: Well, when I say that Pig is not multi-threaded, what I mean

Re: Failing unit tests in trunk

2013-02-07 Thread Jonathan Coveney
Thanks for doing this Cheolsoo! 2013/2/7 Cheolsoo Park cheol...@cloudera.com Hello, There was a discussion about failing unit tests in trunk at today's meet-up. I just wanted to let you know that we have jiras for all failing test cases in trunk: TestPigRunner PIG-3138 TestScriptUDF

Re: How do you pass a schema to the backend in a builtin UDF?

2013-02-12 Thread Jonathan Coveney
EvalFunc now has a getInputSchema method, so this is done for you. Unless I am misunderstanding why you want to do? 2013/2/13 Prashant Kommireddi prash1...@gmail.com Take a look at UDFContext. Sent from my iPhone On Feb 12, 2013, at 7:54 PM, Russell Jurney russell.jur...@gmail.com wrote:

Re: Review Request: Review for PIG-2641 - add ToJson builtin to Pig

2013-02-16 Thread Jonathan Coveney
and tearDown - Jonathan Coveney On Feb. 16, 2013, 11:56 p.m., Russell Jurney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9481

Re: Review Request: Review for PIG-2641 - add ToJson builtin to Pig

2013-02-16 Thread Jonathan Coveney
On Feb. 17, 2013, 1:03 a.m., Jonathan Coveney wrote: This looks useful. Have some stuff to address, but nothing too huge. - Jonathan --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9481

Re: Review Request: Review for PIG-2641 - add ToJson builtin to Pig

2013-02-18 Thread Jonathan Coveney
/ --- (Updated Feb. 17, 2013, 9:47 p.m.) Review request for pig, Ashutosh Chauhan, Dmitriy Ryaboy, Alan Gates, Jonathan Coveney, and Bill Graham. Description --- This patch is for JIRA, https://issues.apache.org/jira/browse/PIG-2641PIG-2641

Re: Pig 0.12 3 months after Pig 0.11?

2013-02-18 Thread Jonathan Coveney
I mean, I think everyone agrees in theory that more frequent releases is a good thing...it's just a matter of getting effort behind that :) that said, I think the first step is being a bit harsher on features that fall behind. The biggest thing that drove back was just one more feature, I'm almost

Re: How to make ivy import lucene-core and lucene-analyzers-common

2013-02-18 Thread Jonathan Coveney
you need conf=compile-master after the rev I believe 2013/2/18 Russell Jurney russell.jur...@gmail.com I am working on a LuceneTokenize command, and need to import some stuff from lucene. I add this to ivy.xml: dependency org=org.apache.lucene name=lucene-core rev=4.1.0/ dependency

Re: Low hanging fruit

2013-02-18 Thread Jonathan Coveney
Thanks Russell! I will review. 2013/2/18 Russell Jurney russell.jur...@gmail.com I knocked out some of the documentation ones. On Sun, Feb 17, 2013 at 2:06 PM, Russell Jurney russell.jur...@gmail.com wrote: Thanks! On Sat, Feb 16, 2013 at 6:02 PM, Jonathan Coveney jcove

Review Request: Introduce a syntax making declared aliases optional

2013-02-18 Thread Jonathan Coveney
test/org/apache/pig/test/TestBuiltin.java 9c5d408 Diff: https://reviews.apache.org/r/9496/diff/ Testing --- ant -Dtestcase=TestBuiltin test, and ant test-commit I made them TestBuiltin use it Thanks, Jonathan Coveney

Re: Review Request: Allow UDFs to flatten themselves

2013-02-18 Thread Jonathan Coveney
/TestExampleGenerator.java 4246b65 Diff: https://reviews.apache.org/r/9060/diff/ Testing --- Thanks, Jonathan Coveney

Re: What do we need to change site documentation?

2013-02-19 Thread Jonathan Coveney
to web is now automatic. It all works fine with the default Java on my mac. Alan. On Feb 19, 2013, at 4:39 AM, Jonathan Coveney wrote: I know we need forrest, but do we still need java 5?

Re: What do we need to change site documentation?

2013-02-19 Thread Jonathan Coveney
I am such a scrub :) thanks Bill! 2013/2/19 Bill Graham billgra...@gmail.com It's the '~'. Swap that out for $HOME. On Tue, Feb 19, 2013 at 7:15 AM, Jonathan Coveney jcove...@gmail.com wrote: Hm, that's what I thought. Not sure why I'm having an issue then...I tried to build

Re: pig 0.11 candidate 2 feedback: Several problems

2013-02-20 Thread Jonathan Coveney
I agree that we shouldn't have to support old versions forever. That said, I also don't think we should be too blase about supporting older versions where it is not odious to do so. We have a lot of competition in the language space and the broader the versions we can support, the better (assuming

Re: [ANNOUNCE] Welcome Bill Graham to join Pig PMC

2013-02-20 Thread Jonathan Coveney
congrats :) 2013/2/20 Jarek Jarcec Cecho jar...@apache.org Congratulations Bill, good job! Jarcec On Tue, Feb 19, 2013 at 01:48:18PM -0800, Daniel Dai wrote: Please welcome Bill Graham as our latest Pig PMC member. Congrats Bill!

When making a change to pig.apache.org, do we attach just the patch for the changes to author, or to the post-forrest changes to publish as well?

2013-02-20 Thread Jonathan Coveney
I believe that this is how it works, but it has been a while and I want to make sure...

Re: Review Request: Allow UDFs to flatten themselves

2013-02-21 Thread Jonathan Coveney
/TestExampleGenerator.java 4246b65 Diff: https://reviews.apache.org/r/9060/diff/ Testing --- Thanks, Jonathan Coveney

Re: Review Request: Allow UDF's to flatten themselves (no ws)

2013-02-21 Thread Jonathan Coveney
/LogicalPlanGenerator.g 01dc570 test/org/apache/pig/test/OptimizeLimitPlanPrinter.java eadc1d5 Diff: https://reviews.apache.org/r/9529/diff/ Testing --- Thanks, Jonathan Coveney

Re: Replicated join: is there a setting to make this better?

2013-02-22 Thread Jonathan Coveney
One quick way to vastly improve the memory efficiency is to utilize the SchemaTuple addition. https://issues.apache.org/jira/browse/PIG-2359 This should cut memory use in half, at least. 2013/2/22 Aniket Mokashi aniket...@gmail.com Interesting, I found this in 0.11 documentation: Fragment

Review Request: Let users use any function from PigType - PigType as if it were builtlin

2013-02-22 Thread Jonathan Coveney
-CREATION Diff: https://reviews.apache.org/r/9559/diff/ Testing --- Thanks, Jonathan Coveney

Re: ON_ERROR command

2013-02-22 Thread Jonathan Coveney
To just add my enthusiasm for this idea to the ring, it would be awesome for Pig. If you're interested in making an impact on a key tool in big data, you'll find a lot of people will be keep to help you out in this endeavor. 2013/2/22 Dmitriy Ryaboy dvrya...@gmail.com Adam it would be *great*

Re: How do we post to the apache pig blog?

2013-02-22 Thread Jonathan Coveney
The pig blog exists: http://blogs.apache.org/pig/ so I think someone has that right? I thought we went through that process a bit ago... 2013/2/22 Bill Graham billgra...@gmail.com Looks like an infra jira is needed to get a PMC member admin rights: Creating new Project Blog users The

Re: Review Request: PIG-3215 [piggybank] Add LTSVLoader to load LTSV files

2013-03-01 Thread Jonathan Coveney
/TestLTSVLoader.java https://reviews.apache.org/r/9685/#comment36643 you should be able to just do new PigServer(ExecType.LOCAL) - Jonathan Coveney On Feb. 28, 2013, 10:53 p.m., Taku Miyakawa wrote: --- This is an automatically generated e-mail

Re: Review Request: Introduce a syntax making declared aliases optional

2013-03-02 Thread Jonathan Coveney
/9496/#review17244 --- On March 1, 2013, 10:32 a.m., Jonathan Coveney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9496

Re: Can we commit PIG-3015 (Rewrite of AvroStorage) to trunk?

2013-03-19 Thread Jonathan Coveney
I'll take a look. I think it should be part of the core of Pig. I will comment in the JIRA. Thanks, Cheolsoo 2013/3/19 Cheolsoo Park cheol...@apache.org Hello, Thanks to Joseph Adler's contribution, we have a new AvroStorage ready. Although there are additional requests that we would like

Re: Review Request: PIG-3015 Rewrite of AvroStorage

2013-03-19 Thread Jonathan Coveney
/builtin/avro/code/pig/directory_test.pig https://reviews.apache.org/r/8104/#comment38188 trailing whitespace :) - Jonathan Coveney On Jan. 4, 2013, 7:22 p.m., Joseph Adler wrote: --- This is an automatically generated e-mail

Re: A major addition to Pig. Working with spatial data

2013-05-04 Thread Jonathan Coveney
I agree that this is cool, and if other projects are using JTS it is worth talking them to see how. I also agree that licensing is very frustrating. In the short term, however, while it is annoying to have to manage the serialization and deserialization yourself, you can have the geometry type be

Re: A major addition to Pig. Working with spatial data

2013-05-06 Thread Jonathan Coveney
issues and start working on them but I also need to keep the work grouped in some sense just for organization. Thanks Ahmed Best regards, Ahmed Eldawy On Sat, May 4, 2013 at 4:47 PM, Jonathan Coveney jcove...@gmail.com wrote: I agree that this is cool, and if other projects are using

Re: A major addition to Pig. Working with spatial data

2013-05-06 Thread Jonathan Coveney
to host it on github and have all the issues there. Let me know if you have any suggestions or comments. Thanks Ahmed Best regards, Ahmed Eldawy On Mon, May 6, 2013 at 9:53 AM, Jonathan Coveney jcove...@gmail.com wrote: You can give them all the same label or tag and filter

Re: Improving macros

2013-06-19 Thread Jonathan Coveney
This sounds excellent! Would love to see this in trunk. As far as #3, this is probably because pig does essentially reparse everything with each new line. I know there is a ticket where Rohini dealt with this in some cases where HDFS was being hit multiple times because of load statements getting

Re: Welcome new Pig Committer - Koji Noguchi

2013-09-13 Thread Jonathan Coveney
Very well deserved!! 2013/9/12 Thejas Nair the...@hortonworks.com Congrats Koji! Very well deserved! On Wed, Sep 11, 2013 at 9:49 AM, Daniel Dai da...@hortonworks.com wrote: Congratulation! You are well deserved. On Wed, Sep 11, 2013 at 6:33 AM, Miguel Angel Martin junquera

Adding a flatten that doesn't throw out the row if you have an empty bag?

2011-12-09 Thread Jonathan Coveney
I think this would be nice to have. We could either add a LEFTFLATTEN() operator, or add a flag to FLATTEN ie FLATTEN({()},true) or something. I personally prefer a new operator, but know that flatten is baked into pig so there's that. I'm willing to do the work, but would like some guidance on

Looking for guidance on adding an OUTER_FLATTEN operator

2011-12-12 Thread Jonathan Coveney
I made a ticket here https://issues.apache.org/jira/browse/PIG-2424 and would like to give it a shot, but I wanted to get opinions on the implementation. Extending the grammar is pretty simple, the question is how to actually implement the operation. Right now, it looks like there is protected

Adding the line of the alias to pig.alias?

2011-12-21 Thread Jonathan Coveney
Right now, there is pig.alias, which is super useful. It has one shortcoming, however... often, users will reuse relation names at many points in a script, so when you go to a dying MR job and see pig.alias = grouped, group_all, grouped_regrouped, it doesn't tell you much. However, if it told you

Re: problems with @hortonworks.com email and apache mailing lists?

2012-01-09 Thread Jonathan Coveney
I got the exact same email with my @gmail account, so I think it might be something else. 2012/1/9 Thejas Nair the...@hortonworks.com This is the 2nd apache user group that has reported that emails to my @ hortonworks.com address are bouncing. Is anybody else seeing this ? Any way to fix it

Re: Prepare for 0.9.2 release

2012-01-09 Thread Jonathan Coveney
I'd like to get this in: https://issues.apache.org/jira/browse/PIG-2428 I mean, the patch is there, I just haven't been able to test the test I wrote for it because of the issue raised in Bill's thread. I'm pro 0.9.2/ 2012/1/9 Daniel Dai da...@hortonworks.com Hi, Pig developers, We have

Where does the m in mBagFactory and mTupleFactory come from?

2012-02-13 Thread Jonathan Coveney
This came up today and I've always been curious as well. What is it supposed to convey?

Is this desirable: relation.projection as sugar for foreach relation generate projection

2012-02-21 Thread Jonathan Coveney
While thinking and talking through https://issues.apache.org/jira/browse/PIG-2536, something came up: should this idea, that relation.projection works in distincts, work in any case where a projection is present? In the patch I linked to, it allows you to do b = distinct a.$0; It accomplishes

Getting e2e test to work with JRuby -- how do I get it to see a library?

2012-02-23 Thread Jonathan Coveney
I'm getting an error when running the following: ant -Dharness.hadoop.home=. -Dharness.old.pig=. -Dtests.to.run=-t RubyUDFs_1 test-e2e-local The error is that the ruby scripts can't find pigudf.rb, which is the library they need to function. But I am not sure how to make this happen within the

Re: Getting e2e test to work with JRuby -- how do I get it to see a library?

2012-02-24 Thread Jonathan Coveney
to include ruby udfs. Thanks, Daniel On Thu, Feb 23, 2012 at 4:17 PM, Jonathan Coveney jcove...@gmail.com wrote: I'm getting an error when running the following: ant -Dharness.hadoop.home=. -Dharness.old.pig=. -Dtests.to.run=-t RubyUDFs_1 test-e2e-local The error is that the ruby

Re: Getting e2e test to work with JRuby -- how do I get it to see a library?

2012-02-24 Thread Jonathan Coveney
are not run from the same directory where the udf's are, when you do the require, it looks in that directory and doesn't find it...but there's no way to ship it to that directly. 2012/2/24 Jonathan Coveney jcove...@gmail.com Thanks Daniel! 2012/2/24 Daniel Dai da...@hortonworks.com Hi, Jonathan

Review Request: JRuby UDFs in Pig

2012-02-28 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091/ --- Review request for pig, Julien Le Dem and Alan Gates. Summary --- This is

Re: [jira] [Created] (PIG-2575) underline symbol is not supported in foreach block

2012-03-07 Thread Jonathan Coveney
Is region_modified_account used elsewhere? There is a known bug where within a foreach block you can't make a relation that shares a name externally. 2012/3/7 Dmitriy Ryaboy dvrya...@gmail.com More likely, its a variable name collision. Could you post the whole script?

Where do we want to put non-java source files?

2012-03-15 Thread Jonathan Coveney
So with the jruby addition (which I'm putting a cherry on top of as we speak!), there's going to be some source files in ruby. Given that we don't currently have (afaik) any code in languages other than java, there isn't a clear place to put this. The files are such that they can be packaged in

Re: Better map support?

2012-03-15 Thread Jonathan Coveney
a JIRA against this. Thanks, Prashant On Wed, Feb 29, 2012 at 5:47 PM, Jonathan Coveney jcove...@gmail.com wrote: Hey all! In the last couple of weeks I've found myself wanting for better map support in pig. I'd be willing to do the work, just wanted to get a sense of what people

Re: Interested in PIG - Gsoc 2012

2012-03-17 Thread Jonathan Coveney
Harsha, This would be really great for the Pig project. Reach out to the listserv with any questions, but it would be a really great way to have an impact on Pig and help facilitate development! Jon 2012/3/17 Daniel Dai da...@hortonworks.com That will be great! Do you have any maven

Re: Pig User Group

2012-03-19 Thread Jonathan Coveney
I'd love for there to be a PUG, and equally would love to know if there is still a contributor meeting... 2012/3/19 Russell Jurney russell.jur...@gmail.com Are the contributor meetings still happening? Russell Jurney http://datasyndrome.com On Mar 19, 2012, at 3:23 PM, Prashant Kommireddi

Re: Pig User Group

2012-03-19 Thread Jonathan Coveney
here for San Francisco :) On Mon, Mar 19, 2012 at 4:20 PM, Dmitriy Ryaboy dvrya...@gmail.com wrote: Twitter can host :) D On Mon, Mar 19, 2012 at 3:44 PM, Jonathan Coveney jcove...@gmail.com wrote: I'd love for there to be a PUG, and equally would love to know

Re: Review Request: JRuby UDFs in Pig

2012-03-20 Thread Jonathan Coveney
--- On 2012-02-28 22:02:52, Jonathan Coveney wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091

Re: Review Request: JRuby UDFs in Pig

2012-03-20 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091/ --- (Updated 2012-03-20 20:42:48.606378) Review request for pig, Julien Le Dem and

Making git the repo of choice for Pig?

2012-03-21 Thread Jonathan Coveney
Would anyone be opposed to doing this? I think you can apply to do this via Apache, no? It would be nice to make git the first class citizen of choice... forcing developers to use svn to work with pig is just another barrier, imho. I thought this had been bandied about on Twitter and the like, but

Re: Making git the repo of choice for Pig?

2012-03-22 Thread Jonathan Coveney
the git mirror, which Pig does. If this has changed (or when it does), I'm +0 on changing, by which I mean I don't care which we use. Alan. On Mar 20, 2012, at 11:22 PM, Jonathan Coveney wrote: Would anyone be opposed to doing this? I think you can apply to do this via Apache

Fixing a broken dependency // can we include a patched piece of JRuby source code in Pig?

2012-03-23 Thread Jonathan Coveney
First off: JRuby patch is almost done. It's passing tests, I have some more to add, but I think the definitive version to work off will be out today (assuming we can reconcile what follows :) I hit a bug in JRuby that is pretty impossible to avoid (it's a bug in the way files were found on the

Re: Fixing a broken dependency // can we include a patched piece of JRuby source code in Pig?

2012-03-23 Thread Jonathan Coveney
, I don't think we shall include a JRuby patch in Pig. Daniel On Fri, Mar 23, 2012 at 10:01 AM, Jonathan Coveney jcove...@gmail.com wrote: First off: JRuby patch is almost done. It's passing tests, I have some more to add, but I think the definitive version to work off will be out today

Re: Fixing a broken dependency // can we include a patched piece of JRuby source code in Pig?

2012-03-23 Thread Jonathan Coveney
require pig.jar. Did we change anything in PIG-2317-9.patch? Daniel On Fri, Mar 23, 2012 at 2:50 PM, Jonathan Coveney jcove...@gmail.com wrote: Alan, My idea wasn't to have a patched version included with Pig, just one specific file that needed to be patched, that way it would be earlier

Re: Review Request: JRuby UDFs in Pig

2012-03-23 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091/ --- (Updated 2012-03-24 01:33:02.389407) Review request for pig, Julien Le Dem and

Re: Review Request: JRuby UDFs in Pig

2012-03-26 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091/ --- (Updated 2012-03-26 17:28:16.350869) Review request for pig, Julien Le Dem and

Re: Review Request: JRuby UDFs in Pig

2012-03-26 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091/ --- (Updated 2012-03-27 00:11:05.270583) Review request for pig, Julien Le Dem and

What outstanding patches are must haves for 0.10? (path to a RC theater)

2012-03-26 Thread Jonathan Coveney
It's time to get serious about rolling an RC. 0.10 seems pretty strong... are there any pressing JIRAs that people want to try and get in? Looking at the open JIRAs with a fix labeled of 0.10 (which is admittedly possibly incomplete), it looks like there isn't anything terribly seriously left...

Re: What outstanding patches are must haves for 0.10? (path to a RC theater)

2012-03-26 Thread Jonathan Coveney
with this. Daniel On Mon, Mar 26, 2012 at 10:29 PM, Jonathan Coveney jcove...@gmail.com wrote: It's time to get serious about rolling an RC. 0.10 seems pretty strong... are there any pressing JIRAs that people want to try and get in? Looking at the open JIRAs with a fix labeled of 0.10

Re: What outstanding patches are must haves for 0.10? (path to a RC theater)

2012-03-27 Thread Jonathan Coveney
RE the CI/e2e, we should see if we can't get the e2e tests running on a CI server. 2012/3/27 Jonathan Coveney jcove...@gmail.com Bill, Thanks for going through those tickets. I marked them all as 0.10/0.11... 2616 seems like the only critical one, and the others are nice to haves (I imagine

Re: Review Request: JRuby UDFs in Pig

2012-03-27 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/4091/ --- (Updated 2012-03-27 07:11:17.441864) Review request for pig, Julien Le Dem and

Re: Build failed in Jenkins: Pig-trunk #1217

2012-03-28 Thread Jonathan Coveney
This failure is beginning to get annoying... what steps do we (and I'm willing to take them) in order to rectify this? Does apache need to renew the license? Do we need to turn off clover in the CI builds? 2012/3/28 Apache Jenkins Server jenk...@builds.apache.org See

Re: Pig test package structure

2012-03-28 Thread Jonathan Coveney
been replaced by e2e tests, haven't they? Cheers, -- Gianmarco On Tue, Mar 27, 2012 at 20:55, Jonathan Coveney jcove...@gmail.com wrote: Backwards compatibility is really important for patches etc, but I would like to set a line in the sand (even far out), where we give

Is it in fact safe to use append and/or return the input Tuple in an EvalFunc

2012-04-03 Thread Jonathan Coveney
This has come up before, and I thought ti wasn't, but I'm looking at POUserFunc and it looks like it should be. Context: I want to be able to do: public Tuple exec(Tuple input) throws IOException { input.append(1); return input; } I was told this wasn't safe, however, looking at

Re: Making git the repo of choice for Pig?

2012-04-03 Thread Jonathan Coveney
the infrastructure problem. Daniel On Thu, Mar 22, 2012 at 12:02 PM, Jonathan Coveney jcove...@gmail.com wrote: If people don't know git or svn, they are going to have to learn one or the other if they want to work with open source projects. It might as well

Re: Cross Product of Two Tuples?

2012-04-04 Thread Jonathan Coveney
)] return b def tupify(tup): if isinstance(tup, tuple): return tup return (tup,) I'll add that into Python PiggyBank as soon as I get a chance to finish that stuff up. Eli On 4/4/12 2:43 PM, Jonathan Coveney wrote: FLATTEN(TOBAG(t1)), FLATTEN(TOBAG(t2)) should give you

Re: Schema mismatch for files with changing avro schemas

2012-04-05 Thread Jonathan Coveney
For the Avro people, does AvroStorage support column pruning? 2012/4/5 Stan Rosenberg stan.rosenb...@gmail.com AFAIK, by default AvroStorage enforces that all input files have exactly the same schema. I've submitted a patch to improve this somewhat by allowing different input schemas so long

Re: Welcome Pig's newest committer, Bill Graham!

2012-04-05 Thread Jonathan Coveney
Awesome! Well deserved. I'm excited at the energy on the project these days. Let's keep up that momentum! 2012/4/5 Aniket Mokashi aniket...@gmail.com Congrats Bill... On Thu, Apr 5, 2012 at 3:04 PM, Prashant Kommireddi prash1...@gmail.com wrote: Congrats Bill. Sent from my iPhone

Is there any reason why the private instance methods of BinInterSedes shouldn't be made protected static?

2012-04-06 Thread Jonathan Coveney
The vast majority don't use any state, and could be potentially useful to other classes that want to do serialization. I guess the other way to take it is that the logic of those classes should be put into BinInterSedes, but this isn't always reasonable or desirable. Thoughts? Any pitfalls I am

Re: Review Request: SchemaTuple in Pig

2012-04-06 Thread Jonathan Coveney
it: TupleFactory.geInstanceForSchema().newTuple() Jonathan Coveney wrote: I think it's worth thinking about how we want to do this. Dmitriy implemented newTupleForSchema for the PrimitiveTuple work, and he suggested I use that for this... however I agree, I think there should be a SchemaTupleFactory

  1   2   3   4   5   6   7   8   9   10   >