Re: Pig and Storm

2013-07-24 Thread Russell Jurney
I think a Storm backend for Pig would be AWESOME. Btw, check out HStreaming. It's not FOSS, but shows there is demand. http://www.hstreaming.com/products/community/ Russell Jurney http://datasyndrome.com On Jul 23, 2013, at 9:53 AM, Pradeep Gollakota pradeep...@gmail.com wrote: Hi Pig

Re: Pig and Storm

2013-07-24 Thread Aniket Mokashi
Following projects might interest you: Pig and Spark: https://github.com/twitter/pig/tree/spork Storm and Hadoop: https://speakerdeck.com/sritchie/summingbird-streaming-mapreduce-at-twitter Thanks, Aniket On Tue, Jul 23, 2013 at 11:18 PM, Russell Jurney russell.jur...@gmail.comwrote: I think

[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit

2013-07-24 Thread Ruslan Al-Fakikh (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718179#comment-13718179 ] Ruslan Al-Fakikh commented on PIG-3114: --- Sajid Raza: Can you please paste the code for

[jira] [Updated] (PIG-3182) Pig currently lacks functions to trim the whitespace only on one side

2013-07-24 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3182: --- Component/s: internal-udfs Summary: Pig currently lacks functions to trim the whitespace only

[jira] [Created] (PIG-3392) Document ENDSWITH UDF

2013-07-24 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created PIG-3392: -- Summary: Document ENDSWITH UDF Key: PIG-3392 URL: https://issues.apache.org/jira/browse/PIG-3392 Project: Pig Issue Type: Improvement Components:

[jira] [Updated] (PIG-3392) Document STARTSWITH and ENDSWITH UDFs

2013-07-24 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3392: --- Description: PIG-2879 and PIG-3163 added new built-in udfs STARTSWITH and ENDSWITH, but documentation

[jira] [Updated] (PIG-3163) Pig current releases lack a UDF endsWith.This UDF tests if a given string ends with the specified suffix.

2013-07-24 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3163: --- Release Note: Pig now includes a ENDSWITH built-in UDF that checks for presence of a given suffix in a

[jira] [Created] (PIG-3393) STARTSWITH udf doesn't override outputSchema method

2013-07-24 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created PIG-3393: -- Summary: STARTSWITH udf doesn't override outputSchema method Key: PIG-3393 URL: https://issues.apache.org/jira/browse/PIG-3393 Project: Pig Issue Type: Bug

Re: Pig and Storm

2013-07-24 Thread Alan Gates
This sounds exciting. The next question is how do you plan to do it? Would a physical plan be translated to a Storm job (or jobs)? Would it need a different physical plan? Or would you just have the connection at the language layer and all the planning separate? Do you envision needing

[jira] [Updated] (PIG-2248) Pig parser does not detect when a macro name masks a UDF name

2013-07-24 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-2248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated PIG-2248: Status: Open (was: Patch Available) Canceling patch as discussion is still on-going as to best approach

[jira] [Commented] (PIG-3182) Pig currently lacks functions to trim the whitespace only on one side

2013-07-24 Thread Kousuke Saruta (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718879#comment-13718879 ] Kousuke Saruta commented on PIG-3182: - Thank you for updating the title and description

[jira] [Commented] (PIG-3389) Set job.name does not work with dump command

2013-07-24 Thread Alan Gates (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718904#comment-13718904 ] Alan Gates commented on PIG-3389: - +1 Set job.name does not work with dump

[jira] [Commented] (PIG-3114) Duplicated macro name error when using pigunit

2013-07-24 Thread Sajid Raza (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719060#comment-13719060 ] Sajid Raza commented on PIG-3114: - Attached the workaround to this JIRA.

[jira] [Updated] (PIG-3114) Duplicated macro name error when using pigunit

2013-07-24 Thread Sajid Raza (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sajid Raza updated PIG-3114: Attachment: PatchedPigTest.java Temporary workaround in end-user code to get PigTest to work.

[jira] Subscription: PIG patch available

2013-07-24 Thread jira
Issue Subscription Filter: PIG patch available (15 issues) Subscriber: pigdaily Key Summary PIG-3389Set job.name does not work with dump command https://issues.apache.org/jira/browse/PIG-3389 PIG-3374CASE and IN fail when expression includes dereferencing

Re: Pig and Storm

2013-07-24 Thread Pradeep Gollakota
I've added a wiki page for a Pig on Storm Proposal at https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal I've included a primer on Storm (and Trident) as well as some of the challenges I foresee. Please read though my proposal and let me know what your thoughts are. On Wed,