Regarding e2e Native tests, they only fail in source tarball. 1) The source tarball doesn't include the following jars-
test/e2e/pig/lib/hadoop-0.23.0-streaming.jar test/e2e/pig/lib/hadoop-examples.jar test/e2e/pig/lib/hadoop-streaming.jar 2) Due to #1, the following line throws NPE- https://github.com/apache/pig/blob/branch-0.13/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/JobControlCompiler.java#L1707 It seems trivial to fix #2, but not sure whether #1 is a regression or not. On Tue, Jun 17, 2014 at 11:52 AM, Daniel Dai <[email protected]> wrote: > I will take a look of TestHBaseStorage today. > > Also run e2e tests on Hadoop 2.0.3. All clear for me. > > On Tue, Jun 17, 2014 at 11:31 AM, Rohini Palaniswamy > <[email protected]> wrote: > > I will look at the e2e test failures I reported. Can someone pick up > > TestHBaseStorage? > > > > > > On Mon, Jun 16, 2014 at 2:59 PM, Aniket Mokashi <[email protected]> > wrote: > > > >> Any updates on this? Please let me know if you need me to look into any > of > >> these. > >> > >> > >> On Fri, Jun 13, 2014 at 9:08 AM, Cheolsoo Park <[email protected]> > >> wrote: > >> > >> > In my e2e on Hadoop 1 run, I found Native_1,2,3 fail. I'll file a > jira to > >> > fix them. > >> > > >> > Caused by: java.lang.NullPointerException > >> > at > >> > > >> > > >> > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.shipToHDFS(JobControlCompiler.java:1707) > >> > > >> > > >> > In my unit test run with jdk7, I found 4 failures. However, I only see > >> > TestHBaseStorage fails in my jenkins build, so the rest could be due > to > >> my > >> > environment. Nevertheless, I'll look into them- > >> > > >> > cheolsoop@cheolsoop-gateway[502]source-tarball: grep FAILED > unittest.log > >> > [junit] Test org.apache.pig.test.TestHBaseStorage FAILED > >> > [junit] Test org.apache.pig.test.TestMRCompiler FAILED > >> > [junit] Test org.apache.pig.test.TestNativeMapReduce FAILED > >> > [junit] Test org.apache.pig.test.TestPigRunner FAILED > >> > > >> > > >> > On Thu, Jun 12, 2014 at 12:55 PM, Rohini Palaniswamy < > >> > [email protected]> wrote: > >> > > >> > > I ran e2e on Hadoop 0.23 > >> > > BigData_4 failed with OOM - This certainly needs investigation > >> > > JobManagement_3, CmdErrors_3 failed with > >> java.lang.NoClassDefFoundError: > >> > > org/apache/zookeeper/Shell > >> > > > >> > > Unit tests except for HBaseStorage was good. piggybank unit tests > are > >> > good. > >> > > > >> > > Regards, > >> > > Rohini > >> > > > >> > > > >> > > On Wed, Jun 11, 2014 at 1:05 PM, Daniel Dai <[email protected]> > >> > wrote: > >> > > > >> > > > I will run e2e on Hadoop 2. > >> > > > > >> > > > On Wed, Jun 11, 2014 at 12:36 PM, Cheolsoo Park < > >> [email protected]> > >> > > > wrote: > >> > > > > I am able to run e2e on Hadoop 1 cluster. > >> > > > > > >> > > > > > >> > > > > On Wed, Jun 11, 2014 at 12:28 PM, Rohini Palaniswamy < > >> > > > > [email protected]> wrote: > >> > > > > > >> > > > >> I have kicked off a unit test run as well. Did someone run e2e > >> > tests? > >> > > > >> > >> > > > >> > >> > > > >> On Wed, Jun 11, 2014 at 12:26 PM, Cheolsoo Park < > >> > [email protected] > >> > > > > >> > > > >> wrote: > >> > > > >> > >> > > > >> > >> Do we publish another RC? > >> > > > >> > > >> > > > >> > Unfortunately, I think we need another RC. Can you wait > before > >> > > rolling > >> > > > >> out > >> > > > >> > a new RC until we collect as many problems as possible? I'll > >> > report > >> > > my > >> > > > >> unit > >> > > > >> > test results by tomorrow. > >> > > > >> > > >> > > > >> > On Wed, Jun 11, 2014 at 12:09 PM, Rohini Palaniswamy < > >> > > > >> > [email protected]> wrote: > >> > > > >> > > >> > > > >> > > I am also seeing that TestHBaseStorage has 6 failures. > >> > > > >> > > > >> > > > >> > > Regards, > >> > > > >> > > Rohini > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > On Wed, Jun 11, 2014 at 11:37 AM, Aniket Mokashi < > >> > > > [email protected]> > >> > > > >> > > wrote: > >> > > > >> > > > >> > > > >> > > > Thanks Cheolsoo, thanks for catching this. > >> > > > >> > > > > >> > > > >> > > > Do you know what's the best way of fixing this? Do we > >> publish > >> > > > another > >> > > > >> > RC? > >> > > > >> > > > I will make sure to clean working directory before I > package > >> > > next > >> > > > >> time. > >> > > > >> > > > > >> > > > >> > > > Thanks, > >> > > > >> > > > Aniket > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > On Wed, Jun 11, 2014 at 8:38 AM, Cheolsoo Park < > >> > > > [email protected] > >> > > > >> > > >> > > > >> > > > wrote: > >> > > > >> > > > > >> > > > >> > > > > Hi Aniket, > >> > > > >> > > > > > >> > > > >> > > > > Thank you so much for the hard work! > >> > > > >> > > > > > >> > > > >> > > > > I discovered that several temporary files are included > in > >> > the > >> > > > >> source > >> > > > >> > > > > tarball. We should remove them from the tarball. > >> > > > >> > > > > > >> > > > >> > > > > Here is the diff between the source tarball and the > >> > > repository- > >> > > > >> > > > > > >> > > > >> > > > > Only in repo/contrib/piggybank/java/lib: .gitignore > >> > > > >> > > > > Only in source-tarball/contrib/piggybank/java: > >> > > > >> > > > > testPathPartitioner-testGetKeys-1402332960101 > >> > > > >> > > > > Only in repo: .eclipse.templates > >> > > > >> > > > > Only in repo: .git > >> > > > >> > > > > Only in repo: .gitignore > >> > > > >> > > > > Only in source-tarball: script3.txt > >> > > > >> > > > > Only in > >> > > source-tarball/src/docs/src/documentation/content/xdocs: > >> > > > >> > > > > func.xml.orig > >> > > > >> > > > > Only in repo/test/e2e/pig/lib: > hadoop-0.23.0-streaming.jar > >> > > > >> > > > > Only in repo/test/e2e/pig/lib: hadoop-examples.jar > >> > > > >> > > > > Only in repo/test/e2e/pig/lib: hadoop-streaming.jar > >> > > > >> > > > > Only in > >> > source-tarball/test/org/apache/pig/builtin/avro/data: > >> > > > avro > >> > > > >> > > > > Only in > >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: > >> > > > >> > > > > evenFileNameTestDirectoryCounts.json > >> > > > >> > > > > Only in > >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: > >> > > > >> > > > > simpleRecordsTrevni.json > >> > > > >> > > > > Only in > >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: > >> > > > >> > > > > testdirectory > >> > > > >> > > > > Only in > >> > > > source-tarball/test/org/apache/pig/builtin/avro/data/json: > >> > > > >> > > > > testDirectoryCounts.json > >> > > > >> > > > > Only in > >> > source-tarball/test/org/apache/pig/builtin/avro/data: > >> > > > >> trevni > >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data: > >> > pigtestloader.jar > >> > > > >> > > > > Only in repo/test/org/apache/pig/test/data: > >> > TestWordCount.jar > >> > > > >> > > > > Only in repo/test/perf/pigmix/lib: sdsuLibJKD12.jar > >> > > > >> > > > > Only in source-tarball: > >> > > > >> > > > > > >> > > > >> > > > >> > > > >> > >> > > > >> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.BinStorage.txt > >> > > > >> > > > > Only in source-tarball: > >> > > > >> > > > > > >> > > > >> > > > >> > > > >> > >> > > > >> TestDataBagAccess-testBagStoreLoad-org.apache.pig.builtin.PigStorage.txt > >> > > > >> > > > > > >> > > > >> > > > > Thanks, > >> > > > >> > > > > Cheolsoo > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > On Mon, Jun 9, 2014 at 11:59 AM, Aniket Mokashi < > >> > > > >> [email protected] > >> > > > >> > > > >> > > > >> > > > > wrote: > >> > > > >> > > > > > >> > > > >> > > > > > Hi, > >> > > > >> > > > > > > >> > > > >> > > > > > I have created a candidate build for Pig 0.13.0. > >> > > > >> > > > > > > >> > > > >> > > > > > Keys used to sign the release are available at > >> > > > >> > > > > > > http://svn.apache.org/viewvc/pig/trunk/KEYS?view=markup > >> . > >> > > > >> > > > > > > >> > > > >> > > > > > Please download, test, and try it out: > >> > > > >> > > > > > http://people.apache.org/~aniket486/pig-0.13.0-rc0 > >> > > > >> > > > > > > >> > > > >> > > > > > Release notes and the rat report are available at the > >> same > >> > > > >> > location. > >> > > > >> > > > > > > >> > > > >> > > > > > Should we release this? Vote closes on Tuesday, June > >> 17th > >> > > > 2014. > >> > > > >> > > > > > > >> > > > >> > > > > > Thanks, > >> > > > >> > > > > > Aniket > >> > > > >> > > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > > >> > > > >> > > > -- > >> > > > >> > > > "...:::Aniket:::... Quetzalco@tl" > >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > > >> > > > -- > >> > > > CONFIDENTIALITY NOTICE > >> > > > NOTICE: This message is intended for the use of the individual or > >> > entity > >> > > to > >> > > > which it is addressed and may contain information that is > >> confidential, > >> > > > privileged and exempt from disclosure under applicable law. If the > >> > reader > >> > > > of this message is not the intended recipient, you are hereby > >> notified > >> > > that > >> > > > any printing, copying, dissemination, distribution, disclosure or > >> > > > forwarding of this communication is strictly prohibited. If you > have > >> > > > received this communication in error, please contact the sender > >> > > immediately > >> > > > and delete it from your system. Thank You. > >> > > > > >> > > > >> > > >> > >> > >> > >> -- > >> "...:::Aniket:::... Quetzalco@tl" > >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
