On Fri, 2005-11-04 at 13:43 -0800, Doug Cutting wrote:
> Rod Taylor wrote:
> > Every segment that I fetch seems to be missing a part when stored on the
> > filesystem. The stranger thing is it is always the same part (very
> > reproducible).
>
> This sounds strange. Are the datanode errors always on the same host?
> How many hosts are you running this on?
I lied earlier. It still happens with smaller segments, just not as
frequently.
Found this in the namenode log file:
051104 200412 Server connection on port 5466 from 192.168.100.11:
exiting
051104 200438 Server connection on port 5466 from 192.168.100.11:
starting
051104 200438 Cannot start file because pendingCreates is non-null
051104 200438 Server handler on 5466 call error: java.io.IOException:
Cannot create file /opt/sitesell/sbider
_data/nutch/segments/20051104185259/20051104185300/crawl_fetch/part-00011/data
java.io.IOException: Cannot create
file /opt/sitesell/sbider_data/nutch/segments/20051104185259/2005110418530
0/crawl_fetch/part-00011/data
at org.apache.nutch.ndfs.NameNode.create(NameNode.java:98)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.nutch.ipc.RPC$1.call(RPC.java:186)
at org.apache.nutch.ipc.Server$Handler.run(Server.java:198)
051104 200440 Server connection on port 5466 from 192.168.100.11:
exiting
051104 200504 Server connection on port 5466 from 192.168.100.11:
starting
051104 200504 Cannot start file because pendingCreates is non-null
051104 200504 Server handler on 5466 call error: java.io.IOException:
Cannot create
file
/opt/sitesell/sbider_data/nutch/segments/20051104185259/20051104185300/crawl_fetch/part-00011/data
java.io.IOException: Cannot create
file
/opt/sitesell/sbider_data/nutch/segments/20051104185259/20051104185300/crawl_fetch/part-00011/data
at org.apache.nutch.ndfs.NameNode.create(NameNode.java:98)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.nutch.ipc.RPC$1.call(RPC.java:186)
at org.apache.nutch.ipc.Server$Handler.run(Server.java:198)
051104 200505 Server connection on port 5466 from 192.168.100.11:
exiting
051104 200506 Removing lease [Lease. Holder: NDFSClient_1755346663,
heldlocks: 0, pendingcreates: 0], leases remaining: 1
051104 200529 Server connection on port 5466 from 192.168.100.11:
starting
051104 201807 Server connection on port 5466 from 192.168.100.11:
exiting
051104 201812 Server connection on port 5466 from 192.168.100.15:
exiting
051104 201823 Server connection on port 5466 from 192.168.100.15:
starting
--
Rod Taylor <[EMAIL PROTECTED]>
-------------------------------------------------------
SF.Net email is sponsored by:
Tame your development challenges with Apache's Geronimo App Server. Download
it for free - -and be entered to win a 42" plasma tv or your very own
Sony(tm)PSP. Click here to play: http://sourceforge.net/geronimo.php
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
