I kept seening Failed to create DataStroage error when try to run pig $ java -cp pig-0.7.0-core.jar:$HADOOP_CONF_DIR org.apache.pig.Main -x mapreduce 10/12/09 20:35:31 INFO pig.Main: Logging error messages to: /home/testpig/pig-0.7.0/pig_1291944931735.log 2010-12-09 20:35:31,997 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:8020 2010-12-09 20:35:32,333 [main] ERROR org.apache.pig.Main - ERROR 2999: Unexpected internal error. Failed to create DataStorage
$ cat pig_1291944931735.log Error before Pig is launched ---------------------------- ERROR 2999: Unexpected internal error. Failed to create DataStorage java.lang.RuntimeException: Failed to create DataStorage at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75) at org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:216) at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:126) at org.apache.pig.impl.PigContext.connect(PigContext.java:184) at org.apache.pig.PigServer.<init>(PigServer.java:184) at org.apache.pig.PigServer.<init>(PigServer.java:173) at org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:54) at org.apache.pig.Main.main(Main.java:354) Caused by: java.io.IOException: Call to localhost/127.0.0.1:8020 failed on local exception: java.io.EOFException at org.apache.hadoop.ipc.Client.wrapException(Client.java:775) at org.apache.hadoop.ipc.Client.call(Client.java:743) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) at $Proxy0.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:82) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:95) at org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:72) ... 8 more Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:375) at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) if I ran java -cp pig-0.7.0-core.jar org.apache.pig.Main -x mapreduce command, I can atleast see the grunt shell. However, when using hadoop commands $ hadoop fs -ls Found 1 items -rw-r--r-- 1 testpig supergroup 454557 2010-12-09 19:31 /user/testpig/access_log.2010-08-30-23-01.lzo everything seems to be fine connecting to hdfs. My environment have the following settings PIG_HOME=/home/testpig/pig-0.7.0 HADOOP_HOME=/usr/lib/hadoop-0.20 (cloudera distribution) HADOOP_CONF_DIR=/usr/lib/hadoop-0.20/conf JAVA_HOME=/usr/java/default pig-env.sh have the following setting export PIG_OPTS="$PIG_OPTS -Djava.library.path=$HADOOP_HOME/lib/native/Linux-amd64-64" export PIG_CLASSPATH=$PIG_CLASSPATH:/home/testpig/hadoop-lzo.jar:/home/testpig/elephant-bird.jar:/home/testpig/elephant-bird/lib/* export PIG_HADOOP_VERSION=20 What is going on there? Thanks a lot. Felix
