I can connect to hdfs while running in script mode but am having trouble doing
so in embedded mode(local mode is fine)
I've tried all possible export settings, could someone please point out what
I'm doing wrong in trying to connect to HDFS
Do I have to do anything else in the java file other than pigServer = new
PigServer("mapreduce")
which hadoop
/usr/local/software/hadoop-0.20.2/bin/hadoop
pig -x mapreduce try1.pig is fine ... Connecting to hadoop
file system at: hdfs://master-hadoop:54310
java -cp pig.jar:. test.java is fine too (pigServer = new
PigServer("local"))
echo $HADOOPDIR gives /usr/local/software/hadoop-0.20.2/conf
however
java -cp pig.jar:.:$HADOOPDIR test is not OK (pigServer = new
PigServer("mapreduce") Connecting to hadoop file system at: file:/// instead
of hdfs://)
Also it shows: ERROR mapReduceLayer.MapReduceLauncher: java.io.IOException:
sample2.txt does not exist
I have sample2.txt in local dir as well as in hdfs
ls -l sample2.txt
-rw-r--r-- 1 hadoop hadoop 225 2011-01-13 17:47 sample2.txt
hadoop dfs -ls sample2.txt
Found 1 items
-rw-r--r-- 3 hadoop supergroup 225 2011-01-20 18:34
/user/hadoop/sample2.txt
-----
export JAVA_HOME=/usr/local/jdk1.6.0_18
HADOOP_HOME=/usr/local/software/hadoop-0.20.2
PIG_HOME=/home/hadoop/code/pig-0.7.0/pig-0.7.0
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$PIG_HOME/bin
export HADOOPDIR=$HADOOP_HOME/conf
export PIGDIR=$PIG_HOME
export PIG_HADOOP_VERSION=20
export PIG_CLASSPATH=$PIG_HOME/pig.jar:.:$HADOOPDIR
thanks a lot!
