Hello, Doug! I try with "mapred" branch, but anyway get errors like this: $./nutch ndfs -put ./test.txt /test.txt ===================== 050831 055936 Client connection to 192.168.0.170:9000: starting 050831 060245 Waiting to find target node ===================== On namenode I see : 050831 055936 Server connection on port 9000 from 192.168.0.170: starting
At the same time $./nutch ndfs -report works fine: ===================== Total effective bytes: 0 (0.0 k) Effective replication multiplier: Infinity ------------------------------------------------- Datanodes available: 1 Name: server.domain.com:7000 Total raw bytes: 75487932416 (70.30 Gb) Used raw bytes: 7289752863 (6.78 Gb) % used: 9.65% Last contact with namenode: Wed Aug 31 06:08:32 CDT 2005 ===================== What also I can try? I really interested in NDFS... Thanks for any help. Tuesday, August 30, 2005, 10:51:10 PM, you wrote: Doug Cutting> It sounds like you're using a nightly Doug Cutting> build of trunk. The NDFS code in Doug Cutting> trunk is old. The NDFS code is currently Doug Cutting> maintained in a branch named Doug Cutting> "mapred". Please check out the mapred branch and retry. Doug Cutting> svn co Doug Cutting> https://svn.apache.org/repos/asf/lucene/nutch/branches/mapred/ Doug Cutting> Doug Doug Cutting> Egor Chernodarov wrote: >> Hello! >> >> I want to test NDFS on my nutch installation, but I have some problem. >> I have started from wiki, where is quick demo for NDFS: >> http://wiki.apache.org/nutch/NutchDistributedFileSystem >> >> On "$ nutch ndfs -put local_file /test/testfile"(or ./nutch admin db >> -create and etc.) I always have exception "Could not obtain new output >> block": >> ======================================================================= >> 050830 061956 Waiting to find target node >> Exception in thread "main" java.io.IOException: Could not obtain new >> output block for file /test/testfile >> at >> org.apache.nutch.ndfs.NDFSClient$NameNodeCaller.getNewOutputBlock(NDFSClient.java:921) >> at >> org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:616) >> at >> org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.<init>(NDFSClient.java:597) >> at >> org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:85) >> at >> org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76) >> at >> org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71) >> at >> org.apache.nutch.io.SequenceFile$Writer.<init>(SequenceFile.java:80) >> at >> org.apache.nutch.io.MapFile$Writer.<init>(MapFile.java:94) >> at >> org.apache.nutch.db.WebDBWriter.<init>(WebDBWriter.java:1507) >> at >> org.apache.nutch.db.WebDBWriter.createWebDB(WebDBWriter.java:1438) >> at >> org.apache.nutch.tools.WebDBAdminTool.main(WebDBAdminTool.java:172) >> ======================================================================= >> >> On namenode I see something like this: >> ======================================================================= >> 050830 061445 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061447 Renewed lease [Lease. Holder: >> NDFSClient_-1094164187, heldlocks: 1, pendingcreates: 1] >> 050830 061448 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061451 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061454 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061455 Renewed lease [Lease. Holder: >> NDFSClient_-1094164187, heldlocks: 1, pendingcreates: 1] >> 050830 061457 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061500 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061503 Pending transfer from server.domain.com:7000 to 3 destinations >> 050830 061503 Renewed lease [Lease. Holder: >> NDFSClient_-1094164187, heldlocks: 1, pendingcreates: 1] >> ======================================================================= >> >> But if I run datanode and namenode on the same server - all's ok! >> >> On "$ nutch ndfs -report" I see list of my datanodes, but these >> datanodes defined by external hostname. I think that namenode tries >> to connect to datanodes by this NOT LOCAL hostnames. It is impossible >> because firewall not allow incoming connection from external network >> interfaces to this port(7000). >> >> It's right? The error can be generated in this case? >> >> So, can you tell me, please, what I can make to define namenode for use >> local interfaces for data transfer? I can't reconfigure firewall.. >> >> Red hat ES3.0, nutch-2005-08-25 (>nutch-0.7). >> $ java -version >> java version "1.4.2-01" >> Java(TM) 2 Runtime Environment, Standard Edition (build Blackdown-1.4.2-01) >> Java HotSpot(TM) 64-Bit Server VM (build Blackdown-1.4.2-01, mixed mode) >> >> >> Thanks for your time! >> >> -- Best regards, Chernodarov Egor
