Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

done. Thanks, Rohini Palaniswamy

Re: Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

unit tests and e2e tests good. Thanks, Rohini Palaniswamy

Re: Branching Pig 0.14

Please mark jira Patch available if ready for review and needs to go into 0.14. I miss the Patch available notifications. Not sure when INFRA will fix them. I will see if I can set up one myself and do a mail fwd to dev@ from my mail id. On Tue, Sep 23, 2014 at 3:32 PM, Prashant Kommireddi

Re: Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

Diff: https://reviews.apache.org/r/25912/diff/ Testing --- test-tez unit tests and e2e tests good. Thanks, Rohini Palaniswamy

Re: Review Request 25912: PIG-4162: Intermediate reducer parallelism in Tez should be higher

., Rohini Palaniswamy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/25912/ --- (Updated Sept. 24, 2014, 3:27 p.m.) Review

Re: need use ant jar -Dhadoopversion=23 to build, then e2e tests works?

No. Seems like a bug introduced after breaking down the fat jar. Please file a jira. On Thu, Sep 25, 2014 at 10:56 PM, Zhang, Liyun liyun.zh...@intel.com wrote: Hi all, I'm using e2e test of pig( https://cwiki.apache.org/confluence/display/PIG/HowToTest#HowToTest-HowtoRune2eTests). My

[RESULT][VOTE] Drop support for JDK 6 from Pig 0.14

...@hortonworks.com wrote: +1 On Wed, Sep 17, 2014 at 8:43 AM, Cheolsoo Park piaozhe...@gmail.com wrote: +1 On Wed, Sep 17, 2014 at 6:52 AM, Alan Gates ga...@hortonworks.com wrote: +1. Alan. Rohini Palaniswamy rohini.adi...@gmail.com September 16, 2014 at 21

Re: [RESULT] [VOTE] Drop support for Hadoop 0.20 from Pig 0.14

AM, Xuefu Zhang xzh...@cloudera.com wrote: +1 On Wed, Sep 17, 2014 at 7:04 AM, Julien Le Dem jul...@ledem.net wrote: +1 Julien -Original Message- From: Rohini Palaniswamy [mailto:rohini.adi...@gmail.com] Sent: Wednesday, September

Re: Pig on Spark - Suggestions in handling code changes out of Spark

Do you want to submit all the changes in one single separate Jira? Will be easier for us to review. On Tue, Sep 30, 2014 at 7:01 AM, Praveen R prav...@sigmoidanalytics.com wrote: *Hi Everyone,* Earlier we have made some changes on https://github.com/sigmoidanalytics/spork/tree/spork-pig-12

Moving jiras to 0.15

Daniel, Looks like we have just been moving a lot of feature or enhancement jiras from 0.12-0.13-0.14 and now 0.15 with no one working on them. Could we just remove target version for those jiras and just mark 0.15 for those that we actually know will be done in 0.15 or that are important or

Re: Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record

Diff: https://reviews.apache.org/r/17876/diff/ Testing --- Full suite of unit tests pass Thanks, Rohini Palaniswamy

Re: Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record

--- On Oct. 21, 2014, 3:58 p.m., Rohini Palaniswamy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17876

Re: Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record

://reviews.apache.org/r/17876/diff/ Testing --- Full suite of unit tests pass Thanks, Rohini Palaniswamy

Re: Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record

. 23, 2014, 9:02 p.m., Rohini Palaniswamy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/17876/ --- (Updated Oct. 23, 2014, 9

Review Request 27169: [PIG-3979] - group all performance, garbage collection, and incremental aggregation

behaviour. Thanks, Rohini Palaniswamy

Re: Review Request 27169: [PIG-3979] - group all performance, garbage collection, and incremental aggregation

://reviews.apache.org/r/27169/diff/ Testing --- Unit tests added to TestPOPartialAgg. Ran couple of manual e2e tests to check behaviour. Thanks, Rohini Palaniswamy

Review Request 27429: [PIG-4259] Fix few issues related to Union, CROSS and auto parallelism in Tez

and e2e tests. In the process of running full suite of unit and e2e tests. Thanks, Rohini Palaniswamy

Re: Review Request 27429: [PIG-4259] Fix few issues related to Union, CROSS and auto parallelism in Tez

/apache/pig/tez/TestTezAutoParallelism.java 1635795 Diff: https://reviews.apache.org/r/27429/diff/ Testing --- Added unit and e2e tests. In the process of running full suite of unit and e2e tests. Thanks, Rohini Palaniswamy

Review Request 27708: [PIG-3977] - Get TezStats working for Oozie

--- Had run full suite of unit and e2e once. But have made more changes after. In the process of running them again Manually tested with Oozie and verified the oozie features. Thanks, Rohini Palaniswamy

Review Request 27792: [PIg-4318] Make PigConfiguration naming consistent

http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1637728 Diff: https://reviews.apache.org/r/27792/diff/ Testing --- Thanks, Rohini Palaniswamy

Re: Review Request 27938: Documentation for 0.14

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27938/#review61194 --- Ship it! Ship It! - Rohini Palaniswamy On Nov. 13, 2014, 3:02

Re: [VOTE] Release Pig 0.14.0 (candidate 0)

I ran full suite of unit tests and e2e on the branch with Hadoop 2.x and they are fine. But running bin/pig -x tez in the tar ball, I get the below error 14/11/16 15:58:54 ERROR pig.Main: ERROR 2998: Unhandled internal error. org.apache.pig.ExecType: Provider

Re: [VOTE] Release Pig 0.14.0 (candidate 1)

+1. Verified that the issue in rc0 is fixed and able to run pig -x tez now. On Sun, Nov 16, 2014 at 7:17 PM, Daniel Dai da...@hortonworks.com wrote: Hi, I have created a candidate build for Pig 0.14.0. Keys used to sign the release are available at

Re: [ANNOUNCE] Apache Pig 0.14.0 released

Thanks Daniel for the hard work, wrapping up so many loose ends and driving it to a very stable release. Thanks to all the other contributors as well without whom the release would not have been possible. There are lot of new contributors in this release and it is very nice to see the Pig

Review Request 28827: PIG-4338: Fix test failures with JDK8

/tez/TestTezJobControlCompiler.java 1643944 Diff: https://reviews.apache.org/r/28827/diff/ Testing --- Failing tests now pass against jdk7 and jdk8 Thanks, Rohini Palaniswamy

Re: Review Request 28827: PIG-4338: Fix test failures with JDK8

., Rohini Palaniswamy wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/28827/ --- (Updated Dec. 8, 2014, 10:48 p.m.) Review

Re: Review Request 28827: PIG-4338: Fix test failures with JDK8

, I think we can go directly to 1.7. Rohini Palaniswamy wrote: I haven't run with 1.6 in a while. But kept it at 1.6 as 1.5 is deprecated and you wanted to keep support for 1.6 for sometime - https://issues.apache.org/jira/browse/PIG-4210?focusedCommentId=14171627page

Re: New Contributor

Welcome Karuna. You can find the required information in https://cwiki.apache.org/confluence/display/PIG/HowToContribute. Regards, Rohini On Fri, Dec 12, 2014 at 2:31 AM, Karuna Devanagavi karuna.devanag...@gmail.com wrote: Hello, I am new to the contribution team and I would like to

Re: Is there any way to guarantee the sequence of “group” field as the input when using “group” operator in pig

Usually I have been fixing these kinds of tests by adding an order by when I added new tests for Union for Tez. In this case you can add order by after the distinct in the nested foreach. Daniel, Any better suggestions? Regards, Rohini On Wed, Dec 17, 2014 at 10:38 PM, Zhang, Liyun

Re: Is there any way to guarantee the sequence of “group” field as the input when using “group” operator in pig

I see that the jira is for unit tests and not e2e test. Please use Util.checkQueryOutputsAfterSort(iter, expectedResults); -Rohini On Mon, Dec 22, 2014 at 6:39 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Usually I have been fixing these kinds of tests by adding an order by when I

Re: Pig 0.14. and Hadoop 2.0.0

java.lang.NoSuchMethodError: org.apache.hadoop.mapred. jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/ jobcontrol/Job;)Ljava/lang/String; This error usually happens when you use Pig compiled with -Dhadoopversion=23 with Hadoop 1.x mapreduce or -Dhadoopversion=20 with Hadoop 2.x

Re: A question regarding Pig versions

https://issues.apache.org/jira/browse/PIG-2167 is the parent jira. https://issues.apache.org/jira/browse/PIG-2710 is the current naive implementation available in pig distributions. PIG-2831 hasn't been worked on in a while. It mostly needs reviewing and cleanup to be tracked to completion and to

Re: Review Request 29644: Port local mode tests to Tez - part4

://reviews.apache.org/r/29644/#comment113029 @Ignore trunk/test/tez-local-tests https://reviews.apache.org/r/29644/#comment113030 TestStore should be in tez-tests right? - Rohini Palaniswamy On Jan. 16, 2015, 11:58 p.m., Daniel Dai wrote

Re: Review Request 29644: Port local mode tests to Tez - part4

On Jan. 13, 2015, 10:52 p.m., Rohini Palaniswamy wrote: trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POStore.java, line 338 https://reviews.apache.org/r/29644/diff/1/?file=808161#file808161line338 Why do we need this? Daniel Dai

Re: Review Request 29645: Port local mode tests to Tez - part5

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29645/#review68695 --- Ship it! Ship It! - Rohini Palaniswamy On Jan. 17, 2015, 2:21

Re: Review Request 29644: Port local mode tests to Tez - part4

On Jan. 13, 2015, 10:52 p.m., Rohini Palaniswamy wrote: trunk/test/org/apache/pig/test/TestMultiQueryLocal.java, line 447 https://reviews.apache.org/r/29644/diff/1/?file=808163#file808163line447 If the location is different why do we have to pass two diffent keys for two

Re: Review Request 29644: Port local mode tests to Tez - part4

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29644/#review68808 --- Ship it! Ship It! - Rohini Palaniswamy On Jan. 20, 2015, 10:08

Re: Review Request 29645: Port local mode tests to Tez - part5

-local-tests https://reviews.apache.org/r/29645/#comment112648 Can we add tests in the order they will be executed? Find it easy to comment the already run ones and re-run if I find a failure in the middle. - Rohini Palaniswamy On Jan. 6, 2015, 11:49 p.m., Daniel Dai wrote

Re: how to test NativeMapReduceOper?

nightly.conf - Native group tests. On Sat, Feb 14, 2015 at 10:02 PM, Zhang, Liyun liyun.zh...@intel.com wrote: Hi all I want to ask a question about how to write a pig script to test NativeMapReduceOper? I approciate if someone can provide a pig script for me. Best regards Zhang,Liyun

Review Request 29628: PIG-4337: Split Types and MultiQuery e2e tests into multiple groups

/multiquery.conf 1649829 http://svn.apache.org/repos/asf/pig/trunk/test/e2e/pig/tests/nightly.conf 1649829 Diff: https://reviews.apache.org/r/29628/diff/ Testing --- Yes. Thanks, Rohini Palaniswamy

Re: Hbase and Pig compatibility

Hbase 0.95 and up. If you need pig 0.14 to work with hbase 0.94, you need to compile the pig code with -Dhbaseversion=94 property. Regards, Rohini On Fri, Feb 13, 2015 at 2:10 AM, ssvinarc...@cybervisiontech.com wrote: Hello, What Hbase versions are compatible with Pig 14? Thanks, Sergey

Pig on Spark

Folks, For many of the Pig on Spark jars, I find that they do not have a parent and Fix Version. Could one of you working on Pig on Spark please update the parent to the umbrella jira PIG-4059 and Fix Version to spark-branch for all the spark jiras? You can search and bulk update. Regards,

Review Request 31798: PIG-4443: Write inputsplits in Tez to disk if the size is huge and option to compress pig input splits

/TestTezAutoParallelism.java 1664412 Diff: https://reviews.apache.org/r/31798/diff/ Testing --- Yes Thanks, Rohini Palaniswamy

Re: Optimization opportunity for group by followed by join on the same key ?

Jeff, There is already a JIRA - https://issues.apache.org/jira/browse/PIG-3849. You can update it with the details/diagrams. Regards, Rohini On Thu, Mar 5, 2015 at 9:41 AM, Daniel Dai da...@hortonworks.com wrote: Thanks Jeff. I think mailing list does not allow attachment, but I get your

Re: Pig on Spark

Thanks Praveen for fixing them. Regards, Rohini On Wed, Feb 18, 2015 at 3:54 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Folks, For many of the Pig on Spark jars, I find that they do not have a parent and Fix Version. Could one of you working on Pig on Spark please update

Re: Pig on Spark

once that is done as well. Regards, Praveen On Tue, Feb 24, 2015 at 11:08 AM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Thanks Praveen for fixing them. Regards, Rohini On Wed, Feb 18, 2015 at 3:54 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Folks

Re: pig script works with mapreduce, fails with tez

Please create a jira with these details and also a pig script with which we can reproduce the issue. Thanks, Rohini On Fri, Mar 27, 2015 at 8:01 AM, Michael Howard mhow...@podiumdata.com wrote: I am a newbie. Isn't clear to me if I should post this type of issue to dev or user, so I decided

Re: Review Request 29644: Port local mode tests to Tez - part4

or to the beginning of the tests? - Rohini Palaniswamy On Jan. 6, 2015, 11:47 p.m., Daniel Dai wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29644

Re: Welcome our new Pig PMC chair Rohini Palaniswamy

: Hi all, Now it's official that Rohini Palaniswamy is our new Pig PMC chair. Please join me in congratulating Rohini for her new role. Congrats! Thanks! Cheolsoo -- Best Regards Jeff Zhang

Re: Branching Pig 0.15

+1 for frequent releases. On Fri, Apr 17, 2015 at 8:38 PM, Daniel Dai da...@hortonworks.com wrote: It's almost 5 months since Pig 0.14.0 released, and we added Hive UDF, tez grace parallelism, numerous tez fixes and quite a few other patches. I would like to branch 0.15 by next week

Re: Review requested for PIG-4526 and PIG-4530

Niel, I committed PIG-4530. Unit tests breaking with hadoop20 was unintentional. We have been testing only in Hadoop 2.x mode mostly and commit builds also run for that version, so this got missed. Just FYI. https://builds.apache.org/job/Pig-trunk-commit/ (Full unit test suite of Pig on

Re: Review requested for PIG-4526 and PIG-4530

*PIG-4526Make setting up the build environment easier* We are busy reviewing patches and wrap up work for 0.15 this week. Will get to this later next week. Regards, Rohini On Mon, May 4, 2015 at 11:29 AM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: Niel, I committed PIG

Re: HowTo build pig

Updated the compilation and unit tests section in https://cwiki.apache.org/confluence/display/PIG/HowToContribute. Hope that answers all the questions. Let me know if something is still not clear. Regards, Rohini On Fri, May 15, 2015 at 11:01 AM, Michael Howard mhow...@podiumdata.com wrote:

Re: Review Request 32492: PIG-4434: Improve auto-parallelism for tez

/test/Util.java https://reviews.apache.org/r/32492/#comment128904 Can we get rid of this method and replace this with the new method added? It is only used in two test classes. - Rohini Palaniswamy On April 12, 2015, 3:51 p.m., Daniel Dai wrote

Re: Review Request 32492: PIG-4434: Improve auto-parallelism for tez

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32492/#review79879 --- Ship it! Ship It! - Rohini Palaniswamy On April 13, 2015, 7:09

Re: Review Request 32868: PIG-4495: Better multi-query planning in case of multiple edges

/TestTezCompiler.java 1671263 Diff: https://reviews.apache.org/r/32868/diff/ Testing (updated) --- New tests added. Running full test suite now Thanks, Rohini Palaniswamy

Review Request 32868: PIG-4495: Better multi-query planning in case of multiple edges

, Rohini Palaniswamy

Re: Review Request 32868: PIG-4495: Better multi-query planning in case of multiple edges

-Union-9.gld 1671263 http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1671263 Diff: https://reviews.apache.org/r/32868/diff/ Testing (updated) --- New tests added. Ran full unit and e2e test suite Thanks, Rohini Palaniswamy

Re: A problem about implicit POSPLIT found in mr mode

Liyun, I was out and could not reply soon. Please send questions like this to dev@ so that other committers can also look into and answer. There is no bug here and behavior is as expected. C = group B by id; D = foreach C { E = order B by f desc; F = E.f; generate group,

Re: [VOTE] Release Pig 0.15.0 (candidate 1)

people.apache.org is down. Don't see anything on https://twitter.com/infrabot. On Mon, Jun 1, 2015 at 5:14 PM, Alan Gates alanfga...@gmail.com wrote: +1. Checked the keys and signature. Looked for any binary files in the source. Made sure there were no snapshot dependencies. Ran

Re: Review Request 35491: PIG-4574: Eliminate identity vertex for order by and skewed join right after LOAD

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/35491/#review89225 --- On June 16, 2015, 7:19 a.m., Rohini Palaniswamy wrote

Re: Review Request 34537: PIG-4365 TOP udf should implement Accumulator interface

://reviews.apache.org/r/34537/#comment140155 Instead of this, can you change to use Util.checkQueryOutputsAfterSort(iter, expectedResults); Refer TestBinaryExpressionOps.testArithmeticOperators - Rohini Palaniswamy On May 21, 2015, 10:38 a.m., Eyal Allweil wrote

Review Request 35491: PIG-4574: Eliminate identity vertex for order by and skewed join right after LOAD

FILE_BYTES_WRITTEN=2016036957653 HDFS_BYTES_READ=2023066976790 HDFS_BYTES_WRITTEN=1010554380555 Thanks, Rohini Palaniswamy

Re: [VOTE] Release Pig 0.15.0 (candidate 1)

, tez_local and tez execution modes. Also ran them on a secure Hadoop 2.6 cluster. Regards, Rohini On Tue, Jun 2, 2015 at 1:24 PM, Rohini Palaniswamy rohini.adi...@gmail.com wrote: people.apache.org is down. Don't see anything on https://twitter.com/infrabot. On Mon, Jun 1, 2015 at 5:14 PM, Alan

Re: HowTo build pig

What error are you getting on command line. I have run with jdk8 and never had any issues. Regards, Rohini On Mon, Jun 1, 2015 at 1:51 PM, Serega Sheypak serega.shey...@gmail.com wrote: Okay... will downgrade, maybe it's the root cause. But right now I don't understand the reason for that.

Review Request 34635: PIG-4564: Pig can deadlock in POPartialAgg if there is a bag

/impl/util/SpillableMemoryManager.java 1681368 Diff: https://reviews.apache.org/r/34635/diff/ Testing --- Manually tested for a large job with big bags and lot of spills. Thanks, Rohini Palaniswamy

Re: Running piggybank unit tests?

//Compile pig and tests first ant clean compile-test -Dhadoopversion=23 // Run test for piggybank cd contrib/piggybank/java ant clean test -Dtestcase=TestAvroStorage -Dhadoopversion=23 On Fri, Jul 31, 2015 at 2:49 PM, Niels Basjes ni...@basjes.nl wrote: Hi, How do I correctly run a single

Re: Review Request 39226: PIG-4680 [Pig workflows can checkpoint the state and can resume from the last successful node]

reviews.apache.org/r/39226/#comment161410> You should be going instead with pig.logical.plan.signature instead of creating a hash of the script. Most of the scripts are parameterized and hash of the script is not useful to differentiate when run with different parameters.

Re: Dependency version on Kryo

That should be fine. We wanted to get rid of the kryo dependency in ORC and use the shaded one that hive uses. But that is in hive-exec jar which is huge and has too many other jars packed in and did not want to add that as dependency to Pig. On Mon, Oct 12, 2015 at 8:54 PM, Xuefu Zhang

Re: Problem when running our code with tez

A possible solution is to use conf.get(³mapreduce.workflow.id²) + conf.get(³mapreduce.workflow.node.name²) Daniel, currently they are set only in vertex conf and will not be available for MROutput. Shiri, Can you tell your actual usecase which lead to implementing a RecordWriter which

Re: group all Reverse order

You will have to use a ORDER BY inside nested foreach after the GROUP statement. On Sat, Sep 12, 2015 at 8:49 PM, 李运田 wrote: > when I use group all,I find that the order reverses. is there a setting I > can use not to Reverse order. > data: > (1,a) > (2,b) > (all,{(2,b),(1,a)})

Review Request 42089: PIG-4411: Support for vertex level configuration like speculative execution

/hadoop/executionengine/tez/util/MRToTezHelper.java 1723016 Diff: https://reviews.apache.org/r/42089/diff/ Testing --- Thanks, Rohini Palaniswamy

Re: Review Request 42195: [PIG-4737] Check and fix clone implementation for all classes extending PhysicalOperator

/GoldenFiles/tez/TEZC-Union-2-OPTOFF.gld 1724173 http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/data/GoldenFiles/tez/TEZC-Union-2.gld 1724173 Diff: https://reviews.apache.org/r/42195/diff/ Testing --- Thanks, Rohini Palaniswamy

Re: Review Request 42195: [PIG-4737] Check and fix clone implementation for all classes extending PhysicalOperator

er reuse as it is declared in one place. - Rohini --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42195/#review114072 ------- On

Re: Review Request 42195: [PIG-4737] Check and fix clone implementation for all classes extending PhysicalOperator

/apache/pig/test/data/GoldenFiles/tez/TEZC-Union-2.gld 1724173 Diff: https://reviews.apache.org/r/42195/diff/ Testing --- Thanks, Rohini Palaniswamy

Review Request 42195: [PIG-4737] Check and fix clone implementation for all classes extending PhysicalOperator

.gld 1724173 Diff: https://reviews.apache.org/r/42195/diff/ Testing --- Thanks, Rohini Palaniswamy

Re: Review Request 42089: PIG-4411: Support for vertex level configuration like speculative execution

/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java 1724028 http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/util/MRToTezHelper.java 1724028 Diff: https://reviews.apache.org/r/42089/diff/ Testing --- Thanks, Rohini

Review Request 43498: PIG-4759 Fix TestMultiQueryLocal.testStoreOrder failure from the initial change

did not get to the bottom of it. So ended up doing two different golden files. Will track that down sometime later to see if it can be changed to make explain output same for JDK7 and JDK8. Thanks, Rohini Palaniswamy

Re: Review Request 43498: PIG-4759 Fix TestMultiQueryLocal.testStoreOrder failure from the initial patch

to make explain output same for JDK7 and JDK8. Thanks, Rohini Palaniswamy

Welcome to our new Pig PMC member Xuefu Zhang

It is my pleasure to announce that Xuefu Zhang is our newest addition to the Pig PMC. Xuefu is a long time committer of Pig and has been actively involved in driving the Pig on Spark effort for the past year. Please join me in congratulating Xuefu !!! Regards, Rohini

Re: Review Request 42709: PIG-4690 Union with self replicate join will fail in Tez

http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1726579 Diff: https://reviews.apache.org/r/42709/diff/ Testing --- Thanks, Rohini Palaniswamy

Review Request 42709: PIG-4690 Union with self replicate join will fail in Tez

/ Testing --- Thanks, Rohini Palaniswamy

Review Request 45127: PIG-4840: Do not turn off UnionOptimizer for unsupported storefuncs in case of no vertex groups

--- Thanks, Rohini Palaniswamy

Welcome our new Pig PMC chair Daniel Dai

Hi folks, I am very happy to announce that we elected Daniel Dai as our new Pig PMC Chair and it is official now. Please join me in congratulating Daniel. Regards, Rohini

Re: Apache Pig GSoC 2016

Sandeep, We will most likely be skipping GSoC this year. I will be the one who will have to work you on that jira. But Q2 is going to be very tight for me and don't think I will be able to do justice and spare enough time for the mentoring. Regards, Rohini On Thu, Mar 3, 2016 at 12:27 PM,

Re: Review Request 45667: Support Pig On Spark

s://reviews.apache.org/r/45667/#comment199050> Use Assume.assumeTrue test/org/apache/pig/test/TestStoreBase.java (line 155) <https://reviews.apache.org/r/45667/#comment199047> Second store s

Branching for Pig 0.16

Folks, Majority of the work on Pig on Spark is done and code is up for review to merge into trunk. So we need to branch for Pig 0.16 shortly so that release for rest of the fixes and enhancements that have gone in so far can be released while trunk is used for further stabilization of Pig on

Work on Pig 0.17 release

nch is ready to merge to > trunk? If use spark branch code, I will update newest patch to review > board and community will review new commits. Appreciate to get your > suggestion. > > > > > > Best Regards > Kelly Zhang/Zhang,Liyun > > -Original Message- &

Re: Preparing for Pig 0.16.1 release

release before it. So if you need any issues that has to go in the Pig 0.16.1 patch release, please raise it now. If there are no issues raised in the next couple of days, will start the RC process end of this week. Regards, Rohini On Thu, Dec 15, 2016 at 3:05 PM, Rohini Palaniswamy <rohini.

Re: Preparing for Pig 0.16.1 release

ve temporary HDFS files in > some cases ), PIG-4957 (See "Received kill signal" message for a normal run > after PIG-4921) and PIG-4921 (Kill running jobs on InterruptedException) to > 0.16.1. > > Thanks, > Daniel > > > > On 1/31/17, 1:21 PM, "Rohin

Re: Review Request 55681: [PIG-4963] Add a Bloom join

-CREATION http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1780416 Diff: https://reviews.apache.org/r/55681/diff/ Testing --- Unit and e2e tests added for many different scenarios. Thanks, Rohini Palaniswamy

Re: Review Request 56043: PIG-5118: Script fails with Invalid dag containing 0 vertices

he/pig/tez/TestTezCompiler.java 1780959 Diff: https://reviews.apache.org/r/56043/diff/ Testing --- e2e tests added Thanks, Rohini Palaniswamy

Review Request 56043: PIG-5118: Script fails with Invalid dag containing 0 vertices

56043/diff/ Testing --- e2e tests added Thanks, Rohini Palaniswamy

Re: Review Request 55681: [PIG-4963] Add a Bloom join

-BloomJoin-7.gld PRE-CREATION http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1779665 Diff: https://reviews.apache.org/r/55681/diff/ Testing --- Unit and e2e tests added for many different scenarios. Thanks, Rohini Palaniswamy

Re: Review Request 55681: [PIG-4963] Add a Bloom join

/GoldenFiles/tez/TEZC-BloomJoin-7.gld PRE-CREATION http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1779665 Diff: https://reviews.apache.org/r/55681/diff/ Testing --- Unit and e2e tests added for many different scenarios. Thanks, Rohini Palaniswamy

Re: Review Request 55681: [PIG-4963] Add a Bloom join

for many different scenarios. Thanks, Rohini Palaniswamy

Re: Review Request 56043: PIG-5118: Script fails with Invalid dag containing 0 vertices

e/pig/test/data/GoldenFiles/tez/TEZC-LoadStore-5.gld PRE-CREATION http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1780431 Diff: https://reviews.apache.org/r/56043/diff/ Testing --- e2e tests added Thanks, Rohini Palaniswamy

Re: Review Request 55681: [PIG-4963] Add a Bloom join

/apache/pig/tez/TestTezCompiler.java 1779329 Diff: https://reviews.apache.org/r/55681/diff/ Testing --- Unit and e2e tests added for many different scenarios. Thanks, Rohini Palaniswamy

Review Request 55681: [PIG-4963] Add a Bloom join

/GoldenFiles/tez/TEZC-BloomJoin-7.gld PRE-CREATION http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/tez/TestTezCompiler.java 1779329 Diff: https://reviews.apache.org/r/55681/diff/ Testing --- Unit and e2e tests added for many different scenarios. Thanks, Rohini Palaniswamy

Re: How to test the efficiency of multiple join

Limit 4 would make processing of join stop after 4 records. It is not a good idea to add it if you are testing performance of join. On Tue, Dec 6, 2016 at 8:13 PM mingda li wrote: > Thanks for your quick reply. If so, I can use the limit operator to compare > > good and

<    1   2   3   4   5   6   7   8   9   10   >