2008/06/21 from Boris Starchev [EMAIL PROTECTED]
Problem using Pseudo-Distributed Operation under Windows XP - bin/hadoop
dfs -put conf input
I need some help!
I have installed cygwin and hadoop-0.17.0 and have done 3 steps:
1)add JAVA_HOME in hadoop-env.sh
2)create hadoop-site.xml
3)execute commands:
cd /cygdrive/c/hadoop-0.17.0
bin/start-all.sh
bin/hadoop dfs -rmr input
bin/hadoop dfs -put conf input NOT WORKING!!!!
bin/hadoop dfs -ls
bin/stop-all.sh
the results are:
cd /cygdrive/c/hadoop-0.17.0
bin/start-all.sh
bin/hadoop dfs -rmr input
bin/hadoop dfs -put conf input NOT WORKING!!!!
bin/hadoop dfs -ls
bin/stop-all.sh
[EMAIL PROTECTED] ~
$ ./make-hadoop-site.sh
[EMAIL PROTECTED] ~
$ cd /cygdrive/c/hadoop-0.17.0
[EMAIL PROTECTED] /cygdrive/c/hadoop-0.17.0
$ bin/start-all.sh
starting namenode, logging to
/cygdrive/c/hadoop-0.17.0/bin/../logs/hadoop-bstar
chev-namenode-bis-laptop.out
: no address associated with name localhost
: no address associated with name localhost
starting jobtracker, logging to
/cygdrive/c/hadoop-0.17.0/bin/../logs/hadoop-bst
archev-jobtracker-bis-laptop.out
: no address associated with name localhost
[EMAIL PROTECTED] /cygdrive/c/hadoop-0.17.0
$ bin/hadoop dfs -rmr input
Deleted /user/bstarchev/input
[EMAIL PROTECTED] /cygdrive/c/hadoop-0.17.0
$ bin/hadoop dfs -put conf input NOT WORKING!!!!
08/06/21 19:19:55 INFO dfs.DFSClient:
org.apache.hadoop.ipc.RemoteException: jav
a.io.IOException: File /user/bstarchev/input/commons-logging.properties
could on
ly be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.ja
va:1145)
at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
at org.apache.hadoop.ipc.Client.call(Client.java:557)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryI
nvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocat
ionHandler.java:59)
at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(
DFSClient.java:2334)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream
(DFSClient.java:2219)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient
.java:1702)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSC
lient.java:1842)
08/06/21 19:19:55 WARN dfs.DFSClient: NotReplicatedYetException sleeping
/user/b
starchev/input/commons-logging.properties retries left 4
08/06/21 19:19:56 INFO dfs.DFSClient:
org.apache.hadoop.ipc.RemoteException: jav
a.io.IOException: File /user/bstarchev/input/commons-logging.properties
could on
ly be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.ja
va:1145)
at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
at org.apache.hadoop.ipc.Client.call(Client.java:557)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryI
nvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocat
ionHandler.java:59)
at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(
DFSClient.java:2334)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream
(DFSClient.java:2219)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient
.java:1702)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSC
lient.java:1842)
08/06/21 19:19:56 WARN dfs.DFSClient: NotReplicatedYetException sleeping
/user/b
starchev/input/commons-logging.properties retries left 3
08/06/21 19:19:56 INFO dfs.DFSClient:
org.apache.hadoop.ipc.RemoteException: jav
a.io.IOException: File /user/bstarchev/input/commons-logging.properties
could on
ly be replicated to 0 nodes, instead of 1
at
org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.ja
va:1145)
at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:300)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
at org.apache.hadoop.ipc.Client.call(Client.java:557)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.
java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
sorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryI
nvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocat
ionHandler.java:59)
at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(
DFSClient.java:2334)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream
(DFSClient.java:2219)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1700(DFSClient
.java:1702)
at
org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSC
lient.java:1842)
08/06/21 19:19:56 WARN dfs.DFSClient: NotReplicatedYetException sleeping
/user/b
starchev/input/commons-logging.properties retries left 2
.............................................................
[EMAIL PROTECTED] /cygdrive/c/hadoop-0.17.0
$ bin/hadoop dfs -ls
Found 1 items
/user/bstarchev/input <dir> 2008-06-21 18:48 rwxr-xr-x
bstarchev supergroup
[EMAIL PROTECTED] /cygdrive/c/hadoop-0.17.0
$ bin/stop-all.sh
no jobtracker to stop
: no address associated with name localhost
no namenode to stop
: no address associated with name localhost
: no address associated with name localhost
[EMAIL PROTECTED] /cygdrive/c/hadoop-0.17.0
$
1)in C:\cygwin\home\bstarchev change hadoop-env.sh and copy in
C:\hadoop-0.17.0\conf
echo 'export JAVA_HOME=/cygdrive/c/"Program Files"/Java/jdk1.5.0_12' >>
hadoop-env.sh
2)in C:\cygwin\home\bstarchev create hadoop-site.xml and copy in
C:\hadoop-0.17.0\conf
echo '<?xml version="1.0"?>' >
hadoop-site.xml
echo '<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>' >>
hadoop-site.xml
echo '<!-- Put site-specific property overrides in this file. -->' >>
hadoop-site.xml
echo " <configuration>" >>
hadoop-site.xml
echo " <property>" >>
hadoop-site.xml
echo " <name>fs.default.name</name>" >>
hadoop-site.xml
echo " <value>hdfs://localhost:9000</value>" >>
hadoop-site.xml
echo " </property>" >>
hadoop-site.xml
echo " <property>" >>
hadoop-site.xml
echo " <name>mapred.job.tracker</name>" >>
hadoop-site.xml
echo " <value>localhost:9001</value>" >>
hadoop-site.xml
echo " </property>" >>
hadoop-site.xml
echo " <property>" >>
hadoop-site.xml
echo " <name>dfs.replication</name>" >>
hadoop-site.xml
echo " <value>1</value>" >>
hadoop-site.xml
echo " </property>" >>
hadoop-site.xml
echo " </configuration>" >>
hadoop-site.xml
