This is a good point.. the DataGenerator on the Wiki page needs updating. In Hadoop 0.20 the -libjars comes *after* the jar file.
It should say: hadoop jar $datagenjar org.apache.pig.test.utils.datagen.DataGenerator</pig/DataGenerator> -libjars $zipfjar -conf $conf_file [options] colspec... Iman - Could you try doing this and let us know? Thanks, Rob Stewart 2009/12/11 Iman E <[email protected]> > Hi all, > Thanks for sharing PigMix with us. > I do have a related question to this thread. The generate_data.sh > references test.jar in datagenjar=$PIG_HOME/build/test/classes/test.jar. > This jar is never generated for me. I generated one out of the test class > files and hope that it was it. > My other question, I tried to run the script but I keep getting this error: > Exception in thread "main" java.io.IOException: Error opening job jar: > -libjars > at org.apache.hadoop.util.RunJar.main(RunJar.java:90) > Caused by: java.util.zip.ZipException: error in opening zip file > at java.util.zip.ZipFile.open(Native Method) > at java.util.zip.ZipFile.<init>(ZipFile.java:114) > at java.util.jar.JarFile.<init>(JarFile.java:133) > at java.util.jar.JarFile.<init>(JarFile.java:70) > at org.apache.hadoop.util.RunJar.main(RunJar.java:88) > > I made sure that the sdsuLibJKD12.jar is in place and I was able to compile > the tests using it, but still the script will not work! Any suggestions. > Thanks > > > > > ________________________________ > From: Ashutosh Chauhan <[email protected]> > To: [email protected] > Sent: Mon, November 2, 2009 12:01:00 PM > Subject: Re: DataGenerator Location > > > > > > > I Have searched through the jar's in both the Pig 0.4.0 and 0.5.0 and > > cannot find any such classpath. Where can I expect to find this > > DataGenerator class? > > > > As Alan wrote earlier DataGenerator is in patch > https://issues.apache.org/jira/browse/PIG-200 You need to download the > patch > and build it. > > > > > Finally, that Wiki page refers to specifying the path to $zipfjar: > > sdsuLibJKD12.jar . Where can I find this jar file? > > > > http://www.eli.sdsu.edu/java-SDSU/ > > Hope it helps, > Ashutosh > > > >
