How do we post to the apache pig blog?

2013-02-22 Thread Dmitriy Ryaboy
I prepared a detailed post going over the pig 0.11 release, and realized I don't know how to post to the apache pig blog. Does anyone have a pointer?

Re: How do we post to the apache pig blog?

2013-02-22 Thread Aniket Mokashi
Just a guess- http://www.apache.org/dev/project-blogs On Fri, Feb 22, 2013 at 12:07 AM, Dmitriy Ryaboy dvrya...@gmail.com wrote: I prepared a detailed post going over the pig 0.11 release, and realized I don't know how to post to the apache pig blog. Does anyone have a pointer? --

Re: Replicated join: is there a setting to make this better?

2013-02-22 Thread Aniket Mokashi
Interesting, I found this in 0.11 documentation: Fragment replicate joins are experimental; we don't have a strong sense of how small the small relation must be to fit into memory. In our tests with a simple query that involves just a JOIN, a relation of up to 100 M can be used if the process

Re: Replicated join: is there a setting to make this better?

2013-02-22 Thread Jonathan Coveney
One quick way to vastly improve the memory efficiency is to utilize the SchemaTuple addition. https://issues.apache.org/jira/browse/PIG-2359 This should cut memory use in half, at least. 2013/2/22 Aniket Mokashi aniket...@gmail.com Interesting, I found this in 0.11 documentation: Fragment

[jira] [Created] (PIG-3212) Race Conditions in POSort and (Internal)SortedBag during Proactive Spill.

2013-02-22 Thread Kai Londenberg (JIRA)
Kai Londenberg created PIG-3212: --- Summary: Race Conditions in POSort and (Internal)SortedBag during Proactive Spill. Key: PIG-3212 URL: https://issues.apache.org/jira/browse/PIG-3212 Project: Pig

[jira] [Updated] (PIG-3212) Race Conditions in POSort and (Internal)SortedBag during Proactive Spill.

2013-02-22 Thread Kai Londenberg (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Londenberg updated PIG-3212: Description: The following bug exists in the latest release of Pig 0.11.0 While running some

[jira] [Updated] (PIG-3198) Let users use any function from PigType - PigType as if it were builtlin

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney updated PIG-3198: -- Attachment: PIG-3198-0.patch So I actually implemented this. You can check TestBuilinInvoker for

Review Request: Let users use any function from PigType - PigType as if it were builtlin

2013-02-22 Thread Jonathan Coveney
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/9559/ --- Review request for pig. Description ---

[jira] [Updated] (PIG-3198) Let users use any function from PigType - PigType as if it were builtlin

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney updated PIG-3198: -- Assignee: Jonathan Coveney Status: Patch Available (was: Open) Let users use any

[jira] [Commented] (PIG-3198) Let users use any function from PigType - PigType as if it were builtlin

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584296#comment-13584296 ] Jonathan Coveney commented on PIG-3198: --- https://reviews.apache.org/r/9559/

[jira] [Updated] (PIG-3212) Race Conditions in POSort and (Internal)SortedBag during Proactive Spill.

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney updated PIG-3212: -- Fix Version/s: 0.12 Race Conditions in POSort and (Internal)SortedBag during Proactive

[jira] [Commented] (PIG-3212) Race Conditions in POSort and (Internal)SortedBag during Proactive Spill.

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584300#comment-13584300 ] Jonathan Coveney commented on PIG-3212: --- Good job tracking down these threading

[jira] [Updated] (PIG-3212) Race Conditions in POSort and (Internal)SortedBag during Proactive Spill.

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney updated PIG-3212: -- Fix Version/s: 0.11.1 Race Conditions in POSort and (Internal)SortedBag during Proactive

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584304#comment-13584304 ] Jonathan Coveney commented on PIG-3199: --- Given that a logical plan is only available

[jira] [Commented] (PIG-3183) rm or rmf commands should respect globbing/regex of path

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584305#comment-13584305 ] Jonathan Coveney commented on PIG-3183: --- So obviously this isn't a huge change and it

[jira] [Commented] (PIG-3190) Add LuceneTokenizer and SnowballTokenizer to Pig - useful text tokenization

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584312#comment-13584312 ] Jonathan Coveney commented on PIG-3190: --- Can you throw this in RB? Either way, some

[jira] [Commented] (PIG-3197) Add the Stanford Tokenizer to the list of Tokenizers in Pig: StanfordTokenize

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584322#comment-13584322 ] Jonathan Coveney commented on PIG-3197: --- Perhaps, given that there are a lot of

[jira] [Updated] (PIG-3055) Make it possible to register new script engines

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney updated PIG-3055: -- Fix Version/s: 0.12 Assignee: Greg Bowyer Make it possible to register new script

[jira] [Commented] (PIG-3141) Giving CSVExcelStorage an option to handle header rows

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584340#comment-13584340 ] Jonathan Coveney commented on PIG-3141: --- Can you put this into RB?

[jira] [Assigned] (PIG-3141) Giving CSVExcelStorage an option to handle header rows

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney reassigned PIG-3141: - Assignee: Jonathan Packer Giving CSVExcelStorage an option to handle header rows

[jira] [Assigned] (PIG-3143) Enable TOKENIZE to use any configurable Lucene Tokenizer, if a config parameter is set and the JARs included

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Coveney reassigned PIG-3143: - Assignee: (was: Jonathan Coveney) Enable TOKENIZE to use any configurable Lucene

Re: How do we post to the apache pig blog?

2013-02-22 Thread Bill Graham
Looks like an infra jira is needed to get a PMC member admin rights: Creating new Project Blog users The blogs.apache.org backend is not currently connected to Apache LDAP services, so blog users need to be created by our infrastructure team. To get a username, create anINFRA

Re: Question about loader and storer

2013-02-22 Thread Jeff Yuan
Thanks to Johnny, Aniket, and Prashant for your help! On Thu, Feb 21, 2013 at 7:05 PM, Prashant Kommireddi prash1...@gmail.com wrote: I have opened a JIRA https://issues.apache.org/jira/browse/PIG-3211 On Thu, Feb 21, 2013 at 6:29 PM, Prashant Kommireddi prash1...@gmail.comwrote: I agree.

[jira] [Commented] (PIG-3183) rm or rmf commands should respect globbing/regex of path

2013-02-22 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584512#comment-13584512 ] Prashant Kommireddi commented on PIG-3183: -- Thanks for the review, Jonathan. I have

[jira] [Commented] (PIG-3183) rm or rmf commands should respect globbing/regex of path

2013-02-22 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584518#comment-13584518 ] Prashant Kommireddi commented on PIG-3183: -- The example above got jumbled up (was

[jira] [Commented] (PIG-3200) MiniCluster should delete hadoop-site.xml on shutDown

2013-02-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584568#comment-13584568 ] Cheolsoo Park commented on PIG-3200: +1. Unit tests pass in both 20 and 23. Please let

[jira] [Commented] (PIG-3141) Giving CSVExcelStorage an option to handle header rows

2013-02-22 Thread Kai Londenberg (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584607#comment-13584607 ] Kai Londenberg commented on PIG-3141: - Why remove the choice of delimiter ? While the

ON_ERROR command

2013-02-22 Thread Adam Silberstein
Hi, I'm interested in custom-error handling in Pig. I came across this wiki: http://wiki.apache.org/pig/PigErrorHandlingInScripts which introduces an 'ON_ERROR' command. But it is about 2 years old and I haven't seen anything like it appear in Pig. Is there any work still going on along

Re: ON_ERROR command

2013-02-22 Thread Alan Gates
AFAIK no one has picked it up to work on it. I still believe this would be a very valuable feature. If you want to pick it up and drive it that would be really cool. Alan. On Feb 22, 2013, at 12:10 PM, Adam Silberstein wrote: Hi, I'm interested in custom-error handling in Pig. I came

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584751#comment-13584751 ] Alan Gates commented on PIG-3199: - When you say now that [the logical plan] is public do you

Re: ON_ERROR command

2013-02-22 Thread Dmitriy Ryaboy
Adam it would be *great* if someone worked on this. I would love to hear your thoughts on the design, it was written a while ago and could probably be improved (though it's pretty viable still, I think). You guys are using Pig at trifacta? On Fri, Feb 22, 2013 at 12:10 PM, Adam Silberstein

[jira] [Created] (PIG-3213) [zebra] Remove local TFile source code - use hadoop-supplied version instead

2013-02-22 Thread Eugene Koontz (JIRA)
Eugene Koontz created PIG-3213: -- Summary: [zebra] Remove local TFile source code - use hadoop-supplied version instead Key: PIG-3213 URL: https://issues.apache.org/jira/browse/PIG-3213 Project: Pig

[jira] [Updated] (PIG-3203) ROLLUP not documented in user docs

2013-02-22 Thread Prasanth J (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated PIG-3203: Assignee: Prasanth J ROLLUP not documented in user docs --

[jira] [Updated] (PIG-3202) CUBE operator not documented in user docs

2013-02-22 Thread Prasanth J (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated PIG-3202: Assignee: Prasanth J CUBE operator not documented in user docs

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Jonathan Coveney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584772#comment-13584772 ] Jonathan Coveney commented on PIG-3199: --- Alan, This patch almost exclusively exists

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584776#comment-13584776 ] Prashant Kommireddi commented on PIG-3199: -- It's not exposed at the moment but a

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584784#comment-13584784 ] Alan Gates commented on PIG-3199: - Even making Operator public is dangerous. These are

[jira] [Updated] (PIG-3213) [zebra] Remove local TFile source code - use hadoop-supplied version instead

2013-02-22 Thread Eugene Koontz (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koontz updated PIG-3213: --- Description: It seems like we shouldn't need to duplicate the existing TFile code since it's already

[jira] [Commented] (PIG-3213) [zebra] Remove local TFile source code - use hadoop-supplied version instead

2013-02-22 Thread Eugene Koontz (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584787#comment-13584787 ] Eugene Koontz commented on PIG-3213: Just read the comments on PIG-1077 - looks like

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584795#comment-13584795 ] Prashant Kommireddi commented on PIG-3199: -- Ok, I can understand the concerns and

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584808#comment-13584808 ] Alan Gates commented on PIG-3199: - Take a look at

Re: ON_ERROR command

2013-02-22 Thread Jonathan Coveney
To just add my enthusiasm for this idea to the ring, it would be awesome for Pig. If you're interested in making an impact on a key tool in big data, you'll find a lot of people will be keep to help you out in this endeavor. 2013/2/22 Dmitriy Ryaboy dvrya...@gmail.com Adam it would be *great*

Re: How do we post to the apache pig blog?

2013-02-22 Thread Jonathan Coveney
The pig blog exists: http://blogs.apache.org/pig/ so I think someone has that right? I thought we went through that process a bit ago... 2013/2/22 Bill Graham billgra...@gmail.com Looks like an infra jira is needed to get a PMC member admin rights: Creating new Project Blog users The

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584821#comment-13584821 ] Prashant Kommireddi commented on PIG-3199: -- What are your thoughts on exposing the

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584825#comment-13584825 ] Alan Gates commented on PIG-3199: - If the PigNotificationListener doesn't work for you I

Re: How do we post to the apache pig blog?

2013-02-22 Thread Aniket Mokashi
Quick check: https://issues.apache.org/jira/issues/?jql=project%20%3D%20INFRA%20AND%20text%20~%20%22pig%20blog%22 Alan seems to have the admin right. https://issues.apache.org/jira/browse/INFRA-4923 On Fri, Feb 22, 2013 at 3:26 PM, Jonathan Coveney jcove...@gmail.comwrote: The pig blog

Re: How do we post to the apache pig blog?

2013-02-22 Thread Alan Gates
I tried to send Dmitriy an invitation to be an author on the blog, but it told me Error creating user invitation. I'm happy to post the content and make it clear you're the author. I've also filed https://issues.apache.org/jira/browse/INFRA-5894 to try to fix the issue. Alan. On Feb 22,

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584851#comment-13584851 ] Prashant Kommireddi commented on PIG-3199: -- That makes sense, I will work on

[jira] [Commented] (PIG-3199) Expose LogicalPlan via PigServer API

2013-02-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584853#comment-13584853 ] Alan Gates commented on PIG-3199: - Keep this one, that way the history of the discussion is

[jira] [Updated] (PIG-3202) CUBE operator not documented in user docs

2013-02-22 Thread Prasanth J (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth J updated PIG-3202: Attachment: PIG-3202.1.git.patch CUBE operator not documented in user docs

[jira] [Commented] (PIG-3202) CUBE operator not documented in user docs

2013-02-22 Thread Prasanth J (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13584865#comment-13584865 ] Prasanth J commented on PIG-3202: - Hi [~billgraham] The attached patch adds user

[jira] Subscription: PIG patch available

2013-02-22 Thread jira
Issue Subscription Filter: PIG patch available (34 issues) Subscriber: pigdaily Key Summary PIG-3205Passing arguments to python script does not work with -f option https://issues.apache.org/jira/browse/PIG-3205 PIG-3200MiniCluster should delete hadoop-site.xml

[jira] [Updated] (PIG-3174) Remove rpm and deb artifacts from build.xml

2013-02-22 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-3174: Resolution: Fixed Hadoop Flags: Incompatible change Status: Resolved (was: Patch Available)

Pig 0.11: new features and improvements

2013-02-22 Thread Dmitriy Ryaboy
I pulled together some of the highlights of the pig 0.11 release on the Apache Pig blog (which now officially exists!): https://blogs.apache.org/pig/ D