It took a deep debugging session to find the reason why my test failed. After two hours of tracing step by step through the insides of Pig itself I noticed it was going to the "cleanupOnFailure" method. It turns out that the entire job failed in the unit test and the error was hidden to me.
The error was: Unexpected System Error Occured: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.setupUdfEnvAndStores(PigOutputFormat.java:243) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:190) Sigh ... So it's the Hadoop 1.x vs 2.x problem again. So why was the message hidden? Turns out I forgot to put in a log4j.properties file and even though the junit errors are put on the screen; this error wasn't. The fix was easy: Change the pig dependency in the pom.xml from this <dependency> <groupId>org.apache.pig</groupId> <artifactId>pig</artifactId> <version>0.13.0</version> </dependency> into this <dependency> <groupId>org.apache.pig</groupId> <artifactId>pig</artifactId> <version>0.13.0</version> <classifier>h2</classifier> </dependency> Niels Basjes On Sun, Nov 23, 2014 at 8:29 PM, Niels Basjes <ni...@basjes.nl> wrote: > I did some additional testing myself and I now have the "smallest > possible" way to reproduce the effect I see. > > I was able to reproduce the problem I'm facing by simply trying to run an > existing unit test from pig without modification of the code at all. > > https://github.com/nielsbasjes/pig-loader-testing > > So what I did: > > Create a very simple pom.xml > Download > https://raw.githubusercontent.com/apache/pig/branch-0.13/test/org/apache/pig/builtin/mock/TestMockStorage.java > and put it in the right directory. > > Now when you run > > mvn test > > > I see: > > java.lang.RuntimeException: No data for location 'bar' >> at org.apache.pig.builtin.mock.Storage$Data.get(Storage.java:327) >> at >> org.apache.pig.builtin.mock.TestMockStorage.testMockStoreAndLoad(TestMockStorage.java:55) > > > and > > junit.framework.AssertionFailedError: >> Expected :{a: chararray,b: chararray} >> Actual :null > > > and > > java.lang.RuntimeException: No data for location 'output' >> at org.apache.pig.builtin.mock.Storage$Data.get(Storage.java:327) >> at >> org.apache.pig.builtin.mock.TestMockStorage.testMockStoreUnion(TestMockStorage.java:106) > > > Any help suggestions to what I'm doing wrong is very welcome. > > Niels Basjes > > > On Sun, Nov 23, 2014 at 12:29 AM, Niels Basjes <ni...@basjes.nl> wrote: > >> Hi, >> >> I have written a Pig Loader and I want to make unit tests for it that >> actually use it from Pig. >> >> It's a Maven project and as a first attempt I created a testing class and >> tried to create a java based unit test by doing something similar to this: >> >> https://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/builtin/mock/TestMockStorage.java >> >> I couldn't get it to work so I copied the exact method >> testMockStoreAndLoad and I still get this error: >> *java.lang.RuntimeException: No data for location 'bar'* >> >> Does anyone have a suggestion on how to proceed? >> >> -- >> Best regards >> >> Niels Basjes >> > > > > -- > Best regards / Met vriendelijke groeten, > > Niels Basjes > -- Best regards / Met vriendelijke groeten, Niels Basjes