i checked out the latest mapred branch version from svn
and it seems to work!
java -cp ./:./classes/:./conf org.apache.nutch.ndfs.NameNode &
java -cp ./:./classes/:./conf org.apache.nutch.ndfs.DataNode &
java -cp ./:./classes/:./conf org.apache.nutch.fs.TestClient -put
somebigfile /testfile
java -cp ./:./classes/:./conf org.apache.nutch.fs.TestClient -report
cool.
ud
Mr. Udatny wrote:
i use the nutch 0.7.1 distribution
starting namenode seems to be ok (as i can see from the logs)
051122 161211 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
051122 161212 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161212 Server listener on port 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
051122 161212 Server handler on 8009: starting
when trying to start ndfs datanode i get the following error:
(i changed the code to show the exception stacktrace)
051122 161220 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml
051122 161220 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161220 Using [/tmp/nutch/ndfs/data] directory for data storage.
051122 161220 Opened server at 7000
051122 161220 Lost connection to namenode [/10.100.100.181:8009].
Retrying...
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
051122 161225 Lost connection to namenode [/10.100.100.181:8009].
Retrying...
java.util.NoSuchElementException
at java.util.StringTokenizer.nextToken(StringTokenizer.java:332)
at org.apache.nutch.ndfs.DF.<init>(DF.java:51)
at org.apache.nutch.ndfs.FSDataset.getCapacity(FSDataset.java:204)
at org.apache.nutch.ndfs.NDFS$DataNode.offerService(NDFS.java:476)
at org.apache.nutch.ndfs.NDFS$DataNode.main(NDFS.java:922)
i could connect to the namenode by just telnetting there:
telnet 10.100.100.181 8009
Trying 10.100.100.181...
Connected to 10.100.100.181 (10.100.100.181).
Escape character is '^]'.
...
further i can use the testclient to create directories and list the
contents of the ndfs. if it comes to a "-put" another exception arises:
[EMAIL PROTECTED] nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
-du /
051122 161956 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
161956 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 161956 No FS indicated, using default:10.100.100.181:8009
051122 161956 Client connection to 10.100.100.181:8009: starting
Found 1 items
/test 0
[EMAIL PROTECTED] nutch-0.7.1]$ ./bin/nutch org.apache.nutch.fs.TestClient
-put ./nutch-0.7.1.jar /testfile
051122 162013 parsing
file:/home/rude/workspace/nutch-0.7.1/conf/nutch-default.xml051122
162013 parsing file:/home/rude/workspace/nutch-0.7.1/conf/nutch-site.xml
051122 162013 No FS indicated, using default:10.100.100.181:8009
051122 162013 Client connection to 10.100.100.181:8009: starting
Exception in thread "main" java.io.IOException: Could not obtain new
output block for file /testfile
at
org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921)
at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616)
at
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597)
at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85)
at
org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76)
at
org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71)
at
org.apache.nutch.fs.NDFSFileSystem.doFromLocalFile(NDFSFileSystem.java:183)
at
org.apache.nutch.fs.NDFSFileSystem.copyFromLocalFile(NDFSFileSystem.java:158)
at
org.apache.nutch.fs.TestClient.copyFromLocal(TestClient.java:46)
at org.apache.nutch.fs.TestClient.main(TestClient.java:234)
[EMAIL PROTECTED] nutch-0.7.1]$
who tells me what i do wrong?
regards,
ud
