Review Request 18203: [PIG-3767] Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18203/ --- Review request for pig, Cheolsoo Park and Daniel Dai. Bugs: PIG-3767

[jira] [Updated] (PIG-3767) Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3767: Attachment: PIG-3767-2.patch Work with TEZ-668 which allows starting and closing of inputs

[jira] [Updated] (PIG-3767) Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3767: Status: Patch Available (was: Open) Work with TEZ-668 which allows starting and closing of

[jira] [Commented] (PIG-3767) Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13903873#comment-13903873 ] Rohini Palaniswamy commented on PIG-3767: - TEZ-688 is committed to tez trunk.

Re: Review Request 18203: [PIG-3767] Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Daniel Dai
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18203/#review34683 --- Ship it! Otherwise +1, feel free to commit.

[jira] [Commented] (PIG-3629) Implement STREAM in Tez

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13903892#comment-13903892 ] Daniel Dai commented on PIG-3629: - Seems there is regression introduced by

[jira] [Created] (PIG-3768) TezCompiler.visitUnion() doesn't add compiled TezOp to phyToTezOpMap

2014-02-18 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created PIG-3768: -- Summary: TezCompiler.visitUnion() doesn't add compiled TezOp to phyToTezOpMap Key: PIG-3768 URL: https://issues.apache.org/jira/browse/PIG-3768 Project: Pig

[jira] [Updated] (PIG-3768) TezCompiler.visitUnion() doesn't add compiled TezOp to phyToTezOpMap

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3768: --- Attachment: PIG-3768-1.patch A trivial fix- adding the compiled union operator to phyToTezOpMap at the

[jira] [Updated] (PIG-3768) TezCompiler.visitUnion() doesn't add compiled TezOp to phyToTezOpMap

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3768: --- Status: Patch Available (was: Open) TezCompiler.visitUnion() doesn't add compiled TezOp to

Re: Review Request 18203: [PIG-3767] Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Cheolsoo Park
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18203/#review34688 --- Can we hold on please? If we commit this, can I still compile and

Re: Review Request 18203: [PIG-3767] Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Cheolsoo Park
On Feb. 18, 2014, 12:21 p.m., Cheolsoo Park wrote: Can we hold on please? If we commit this, can I still compile and run Pig with Tez that is prior to TEZ-688? Due to the Guava conflict (TEZ-787), I cannot upgrade Tez to the head. So I'd like to avoid any incompatible changes

[jira] [Created] (PIG-3769) Scripting UDF is broken after PIG-3629

2014-02-18 Thread Cheolsoo Park (JIRA)
Cheolsoo Park created PIG-3769: -- Summary: Scripting UDF is broken after PIG-3629 Key: PIG-3769 URL: https://issues.apache.org/jira/browse/PIG-3769 Project: Pig Issue Type: Sub-task

[jira] [Commented] (PIG-3629) Implement STREAM in Tez

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904017#comment-13904017 ] Cheolsoo Park commented on PIG-3629: I also ran into the issue while running a

[jira] [Updated] (PIG-3768) TezCompiler.visitUnion() doesn't add compiled TezOp to phyToTezOpMap

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3768: --- Attachment: PIG-3768-2.patch While running a benchmark, I also realized that union doesn't honor

[jira] [Updated] (PIG-3746) NPE is thrown if Pig fails before PigStats is intialized

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3746: --- Status: Patch Available (was: Open) NPE is thrown if Pig fails before PigStats is intialized

[jira] [Updated] (PIG-3746) NPE is thrown if Pig fails before PigStats is intialized

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3746: --- Attachment: PIG-3746-1.patch PigStats is not initialized means PigServer hasn't started yet. So

[jira] [Updated] (PIG-3747) Update skewed join documentation

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3747: --- Status: Patch Available (was: Open) Update skewed join documentation

[jira] [Updated] (PIG-3747) Update skewed join documentation

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3747: --- Attachment: PIG-3747-1.patch Uploading a patch. Update skewed join documentation

[jira] [Updated] (PIG-3648) Make the sample size for RandomSampleLoader configurable

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3648: --- Status: Patch Available (was: Open) Actually, this turned out to be helpful. Since I am not change

[jira] [Updated] (PIG-3767) Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3767: Attachment: PIG-3767-3.patch Attaching final patch that changes guava-tez.version to

[jira] [Updated] (PIG-3767) Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy updated PIG-3767: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch

Re: Review Request 18203: [PIG-3767] Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Rohini Palaniswamy
On Feb. 18, 2014, 12:21 p.m., Cheolsoo Park wrote: Can we hold on please? If we commit this, can I still compile and run Pig with Tez that is prior to TEZ-688? Due to the Guava conflict (TEZ-787), I cannot upgrade Tez to the head. So I'd like to avoid any incompatible changes

[jira] [Commented] (PIG-3747) Update skewed join documentation

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904248#comment-13904248 ] Daniel Dai commented on PIG-3747: - I think the right table name massive should also be

[jira] [Commented] (PIG-3746) NPE is thrown if Pig fails before PigStats is intialized

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904263#comment-13904263 ] Daniel Dai commented on PIG-3746: - +1 NPE is thrown if Pig fails before PigStats is

[jira] [Updated] (PIG-3747) Update skewed join documentation

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3747: --- Attachment: PIG-3747-2.patch Thanks Daniel. I changed the table names in the example query- {code} A =

[jira] [Commented] (PIG-3747) Update skewed join documentation

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904389#comment-13904389 ] Daniel Dai commented on PIG-3747: - +1 Update skewed join documentation

[jira] [Updated] (PIG-3648) Make the sample size for RandomSampleLoader configurable

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3648: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thank you

[jira] [Updated] (PIG-3747) Update skewed join documentation

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3747: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thank you

[jira] [Updated] (PIG-3746) NPE is thrown if Pig fails before PigStats is intialized

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3746: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Daniel!

Re: Review Request 18203: [PIG-3767] Work with TEZ-668 which allows starting and closing of inputs and outputs

2014-02-18 Thread Cheolsoo Park
On Feb. 18, 2014, 12:21 p.m., Cheolsoo Park wrote: Can we hold on please? If we commit this, can I still compile and run Pig with Tez that is prior to TEZ-688? Due to the Guava conflict (TEZ-787), I cannot upgrade Tez to the head. So I'd like to avoid any incompatible changes

[jira] [Updated] (PIG-3768) TezCompiler.visitUnion() doesn't add compiled TezOp to phyToTezOpMap

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3768: --- Resolution: Fixed Status: Resolved (was: Patch Available) Committed to tez branch. Thanks

[jira] [Commented] (PIG-3657) New partition filter extractor fails with NPE

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904461#comment-13904461 ] Cheolsoo Park commented on PIG-3657: I think we should also commit this into 0.12.1 to

[jira] [Commented] (PIG-3657) New partition filter extractor fails with NPE

2014-02-18 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904474#comment-13904474 ] Aniket Mokashi commented on PIG-3657: - Yes, please commit on 0.12.1. +1. New partition

[jira] [Updated] (PIG-3463) Pig should use hadoop local mode for small jobs

2014-02-18 Thread Sen Fang (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sen Fang updated PIG-3463: -- Attachment: JobControlCompiler.java.patch This patch moves the local mode conversion block around to work around

[jira] [Commented] (PIG-3463) Pig should use hadoop local mode for small jobs

2014-02-18 Thread Sen Fang (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904486#comment-13904486 ] Sen Fang commented on PIG-3463: --- Aniket, thanks a lot for making this wonderful feature!

[jira] [Updated] (PIG-3657) New partition filter extractor fails with NPE

2014-02-18 Thread Cheolsoo Park (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheolsoo Park updated PIG-3657: --- Fix Version/s: 0.12.1 Done. It's committed to 0.12.1 too. Thanks Aniket for the quick response. New

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904574#comment-13904574 ] Dmitriy V. Ryaboy commented on PIG-3558: I am pro adding ORC support in Pig, but

[jira] [Comment Edited] (PIG-3558) ORC support for Pig

2014-02-18 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904574#comment-13904574 ] Dmitriy V. Ryaboy edited comment on PIG-3558 at 2/18/14 8:55 PM:

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904590#comment-13904590 ] Dmitriy V. Ryaboy commented on PIG-3558: So that's a -1. I would +1 this if it was

[jira] [Commented] (PIG-3648) Make the sample size for RandomSampleLoader configurable

2014-02-18 Thread Aniket Mokashi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904596#comment-13904596 ] Aniket Mokashi commented on PIG-3648: - Thanks [~cheolsoo], it would be great if you can

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904598#comment-13904598 ] Daniel Dai commented on PIG-3558: - [~dvryaboy], this is only compile time dependency. We

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904600#comment-13904600 ] Daniel Dai commented on PIG-3558: - I am fine to unlink from 0.13 release though. ORC

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904627#comment-13904627 ] Dmitriy V. Ryaboy commented on PIG-3558: [~daijy] not quite: {code} -

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904641#comment-13904641 ] Daniel Dai commented on PIG-3558: - This is actually a downgrade. test dependency implies

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Dmitriy V. Ryaboy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904663#comment-13904663 ] Dmitriy V. Ryaboy commented on PIG-3558: Help me understand this. My understanding

[jira] [Comment Edited] (PIG-3558) ORC support for Pig

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904641#comment-13904641 ] Daniel Dai edited comment on PIG-3558 at 2/18/14 9:55 PM: -- This is

[jira] [Commented] (PIG-3558) ORC support for Pig

2014-02-18 Thread Daniel Dai (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13904684#comment-13904684 ] Daniel Dai commented on PIG-3558: - Please ignore my previous comment. It is actually an

[jira] [Updated] (PIG-3740) Document direct fetch optimization

2014-02-18 Thread Lorand Bendig (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lorand Bendig updated PIG-3740: --- Attachment: PIG-3740.patch Document direct fetch optimization --

[jira] [Updated] (PIG-3740) Document direct fetch optimization

2014-02-18 Thread Lorand Bendig (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lorand Bendig updated PIG-3740: --- Status: Patch Available (was: Open) Document direct fetch optimization

[jira] [Updated] (PIG-3765) Ability to disable Pig commands and operators

2014-02-18 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Kommireddi updated PIG-3765: - Attachment: PIG-3765_2.patch Includes test cases Ability to disable Pig commands and

[jira] [Updated] (PIG-3765) Ability to disable Pig commands and operators

2014-02-18 Thread Prashant Kommireddi (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Kommireddi updated PIG-3765: - Patch Info: Patch Available Ability to disable Pig commands and operators

[jira] [Created] (PIG-3770) Enhance DBStorage to make it more flexible (should batch statements?, rollback on job failure, support command line arguments etc.)

2014-02-18 Thread Nezih Yigitbasi (JIRA)
Nezih Yigitbasi created PIG-3770: Summary: Enhance DBStorage to make it more flexible (should batch statements?, rollback on job failure, support command line arguments etc.) Key: PIG-3770 URL:

[jira] Subscription: PIG patch available

2014-02-18 Thread jira
Issue Subscription Filter: PIG patch available (12 issues) Subscriber: pigdaily Key Summary PIG-3761Outer join fail on tez https://issues.apache.org/jira/browse/PIG-3761 PIG-3757Make scalar work https://issues.apache.org/jira/browse/PIG-3757 PIG-3740

[jira] [Assigned] (PIG-3769) Scripting UDF is broken after PIG-3629

2014-02-18 Thread Rohini Palaniswamy (JIRA)
[ https://issues.apache.org/jira/browse/PIG-3769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohini Palaniswamy reassigned PIG-3769: --- Assignee: Rohini Palaniswamy Scripting UDF is broken after PIG-3629