Hello, I tried several versions to generate data for pigmix queries: *- Hadoop apache 0.20.204 with pig 0.7* *==>* java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils... *[Full error at the very bottom]* * * *- Hadoop apache 0.20.204 with pig 0.9* ==> I get an error while patching the pixmix2.patch (on build.xml: Reversed (or previously applied) patch detected! ) I didnapplied the patch up to that error and when generating the data: Exception in thread "main" org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol.. org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch.
*- CDH3 with pig 0.7* *==>* java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils... *- CDH3 with CDH3-pig (which I downloaded from http://nightly.cloudera.com/cdh/3/ )* I applied pigmix2.patch, and used pig.jar and pigperf.jar (which I couldnt recompile locally for an internal reason), and got the same error: *==>* java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils... Bottom line, none of these configurations worked. Keren [kouaknine@dataland_oss:node020062 scripts]$ ./generate_data.sh Generating pages50m Using seed 1315606991650 Generate data in hadoop mode. Generating column config file in hdfs://node020062.boca.local:54310/user/kouak ne/tmp/tmp1129210882/tmp-1545723655 Generating mapping file for column s:20:1600000:z:7 into hdfs://node020062.boc local:54310/user/kouaknine/tmp/tmp1129210882/tmp-163757285 processed 18%. processed 37%. processed 56%. processed 75%. processed 93%. processed 99%. Generating mapping file for column s:10:1800000:z:20 into hdfs:// node020062.bo .local:54310/user/kouaknine/tmp/tmp1129210882/tmp-1525412142 processed 16%. processed 33%. processed 50%. processed 66%. processed 83%. processed 99%. Generating mapping file for column d:1:100000:z:5 into hdfs://node020062.boca. cal:54310/user/kouaknine/tmp/tmp1129210882/tmp-738880094 processed 99%. Generating input files into hdfs://node020062.boca.local:54310/user/kouaknine/ p/tmp1129210882/tmp-1696754417 Submit hadoop job... 11/09/09 18:23:38 INFO mapred.FileInputFormat: Total input paths to process : 11/09/09 18:23:39 INFO mapred.JobClient: Running job: job_201109091527_0005 11/09/09 18:23:40 INFO mapred.JobClient: map 0% reduce 0% *11/09/09 18:24:45 INFO mapred.JobClient: Task Id : attempt_201109091527_0005_m* *00000_0, Status : FAILED* *java.lang.RuntimeException: Error in configuring object* * at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.j* *a:93)* at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java 4) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils. va:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:386) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInfor tion.java:1115) at org.apache.hadoop.mapred.Child.main(Child.java:262) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImp java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcc sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.j a:88) ... 9 more On Fri, Sep 9, 2011 at 1:12 PM, Alan Gates <[email protected]> wrote: > If you're going to run with Apache Pig 0.8.1 or 0.9, you should use Apache > Hadoop 0.20.2. If you want to use CDH, you should stick with their versions > of Hadoop and Pig. > > Alan. > > On Sep 9, 2011, at 6:59 AM, Keren Ouaknine wrote: > > > Hello, > > > > What is the latest version of pig supporting the pigmix queries? The jira > > latest update mentions pig 0.7 only: > > Assuming its 0.8 or 0.9, can I use hadoop cdh3 or should I switch to > > apache's version and which one? > > > > == > > * > > * > > *1. Download pig 0.7 release > > 2. Apply the patch > > 3. copy http://www.eli.sdsu.edu/java-SDSU/sdsuLibJKD12.jar to lib > > 4. ant jar pigperf > > 5. You will use pig.jar, pigperf.jar. Scripts is in > > test/utils/pigmix/scripts. To generate data, use generate_data.sh. To run > > PigMix2, use runpigmix-adhoc.pl.* > > > > Thanks, > > Keren > > > > -- > > Keren Ouaknine > > Cell: +972 54 2565404 > > Web: www.kereno.com > > -- Keren Ouaknine Cell: +972 54 2565404 Web: www.kereno.com
