[jira] [Updated] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

2013-08-22 Thread Pavel Fedyakov (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Fedyakov updated PIG-3434: Description: According to docs, for bincond operator If a Boolean subexpression results in null

Re: Slow Group By operator

2013-08-22 Thread Benjamin Jakobus
Hi Cheolsoo, Thanks - I will try this now and get back to you. Out of interest; could you explain (or point me towards resources that would) why the combiner would be a problem? Also, could the fact that Pig builds an intermediary data structure (?) whilst Hive just performs a sort then the

[jira] [Commented] (PIG-1420) Make CONCAT act on all fields of a tuple, instead of just the first two fields of a tuple

2013-08-22 Thread Ido Hadanny (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13747531#comment-13747531 ] Ido Hadanny commented on PIG-1420: -- So, is this fixed or not? I couldn't get it to work in

[jira] [Commented] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13747564#comment-13747564 ] Cheolsoo Park commented on PIG-3419: Attached test_failures.txt.

Re: Slow Group By operator

2013-08-22 Thread Cheolsoo Park
Hi Benjamin, To answer your question, how the Hadoop combiner works is that 1) mappers write outputs to disk and 2) combiners read them, combine and write them again. So you're paying extra disk I/O as well as serialization/deserialization. This will pay off if combiners significantly reduce the

[jira] [Updated] (PIG-3424) Package import list should consider class name as is first even if -Dudf.import.list is passed

2013-08-22 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3424: Attachment: PIG-3424-fixtest.patch When running the full test suite, encountered a test

[jira] [Commented] (PIG-3424) Package import list should consider class name as is first even if -Dudf.import.list is passed

2013-08-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13747603#comment-13747603 ] Cheolsoo Park commented on PIG-3424: Thanks for fixing it. +1. Package

Re: Slow Group By operator

2013-08-22 Thread Alan Gates
When data comes out of a map task, Hadoop serializes it so that it can know its exact size as it writes it into the output buffer. To run it through the combiner it needs to deserialize it again, and then re-serialize it when it comes out. So each pass through the combiner costs a

[jira] [Updated] (PIG-3168) TestMultiQueryBasic.testMultiQueryWithSplitInMapAndMultiMerge fails in trunk

2013-08-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3168: --- Attachment: PIG-3168-3.patch I noticed that TestShortcuts is broken with my changes to explain in

[jira] [Commented] (PIG-3168) TestMultiQueryBasic.testMultiQueryWithSplitInMapAndMultiMerge fails in trunk

2013-08-22 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13747745#comment-13747745 ] Rohini Palaniswamy commented on PIG-3168: - +1

[jira] [Assigned] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

2013-08-22 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner reassigned PIG-3434: Assignee: Mark Wagner Null subexpression in bincond nullifies outer tuple (or bag)

[jira] [Commented] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

2013-08-22 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13747789#comment-13747789 ] Mark Wagner commented on PIG-3434: -- I was able to reproduce this. It looks like POUserFunc

[jira] [Updated] (PIG-3168) TestMultiQueryBasic.testMultiQueryWithSplitInMapAndMultiMerge fails in trunk

2013-08-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3168: --- Resolution: Fixed Status: Resolved (was: Patch Available) Thanks Rohini. Committed to trunk.

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: (was: finalpatch.patch) Pluggable Execution Engine ---

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: (was: finalpatch.patch) Pluggable Execution Engine ---

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: updated-8-22-2013-exec-engine.patch Pluggable Execution Engine

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: finalpatch.patch Pluggable Execution Engine ---

[jira] [Commented] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13747968#comment-13747968 ] Achal Soni commented on PIG-3419: - Here is the ReviewBoard for the new patch :

[jira] [Updated] (PIG-3435) Custom Partitioner not working with MultiQueryOptimizer

2013-08-22 Thread Koji Noguchi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Noguchi updated PIG-3435: -- Attachment: pig-3435-v01.patch Looking at the multi-query optimization code and documents. I chickened

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: (was: updated-8-22-2013-exec-engine.patch) Pluggable Execution Engine

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: updated-8-22-2013-exec-engine.patch Pluggable Execution Engine

[jira] [Commented] (PIG-1420) Make CONCAT act on all fields of a tuple, instead of just the first two fields of a tuple

2013-08-22 Thread Russell Jurney (JIRA)
[ https://issues.apache.org/jira/browse/PIG-1420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748057#comment-13748057 ] Russell Jurney commented on PIG-1420: - This JIRA is not fixed. I don't know how to

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: (was: updated-8-22-2013-exec-engine.patch) Pluggable Execution Engine

[jira] [Updated] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Achal Soni (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Achal Soni updated PIG-3419: Attachment: updated-8-22-2013-exec-engine.patch Pluggable Execution Engine

[jira] [Commented] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748137#comment-13748137 ] Cheolsoo Park commented on PIG-3419: I will kick off the unit tests with the new patch

[jira] Subscription: PIG patch available

2013-08-22 Thread jira
Issue Subscription Filter: PIG patch available (18 issues) Subscriber: pigdaily Key Summary PIG-3431Return more information for parsing related exceptions. https://issues.apache.org/jira/browse/PIG-3431 PIG-3430Add xml format for explaining MapReduce Plan.

[jira] [Commented] (PIG-3419) Pluggable Execution Engine

2013-08-22 Thread Julien Le Dem (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748183#comment-13748183 ] Julien Le Dem commented on PIG-3419: +1 LGTM If test-commit passes I think we can commit

[jira] [Updated] (PIG-3434) Null subexpression in bincond nullifies outer tuple (or bag)

2013-08-22 Thread Mark Wagner (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Wagner updated PIG-3434: - Attachment: PIG-3434.1.patch Fixed null handling in POUserFunc. It seems like POStatus.STATUS_NULL isn't