[
http://issues.apache.org/jira/browse/NUTCH-211?page=comments#action_12366533 ]
Stefan Groschupf commented on NUTCH-211:
----------------------------------------
I'm already in process of creating the patch, however I spend some more time to
find the real problem source. It is just a guess, but could it be that the
problem source is that the NutchBean.getDetail method is concurently invoked by
different threads, what is throwing a
java.nio.channels.ClosedByInterruptException? Shouldn't be the access to a nio
channel syncronized? Means shouldn't be the
LocalFileSystem$LocalNFSFileInputStream.seek method syncronized, or is that
nonsense? May this by now more a hadoop releated question, but it is somehow
part of this problem. Any hints?
Caused by: java.nio.channels.ClosedByInterruptException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:289)
at
org.apache.nutch.fs.LocalFileSystem$LocalNFSFileInputStream.seek(LocalFileSystem.java:83)
at
org.apache.nutch.fs.NFSDataInputStream$Checker.seek(NFSDataInputStream.java:66)
at
org.apache.nutch.fs.NFSDataInputStream$PositionCache.seek(NFSDataInputStream.java:162)
at
org.apache.nutch.fs.NFSDataInputStream$Buffer.seek(NFSDataInputStream.java:191)
at
org.apache.nutch.fs.NFSDataInputStream.seek(NFSDataInputStream.java:241)
at org.apache.nutch.io.SequenceFile$Reader.seek(SequenceFile.java:403)
at org.apache.nutch.io.MapFile$Reader.seek(MapFile.java:329)
at org.apache.nutch.io.MapFile$Reader.get(MapFile.java:374)
at
org.apache.nutch.mapred.MapFileOutputFormat.getEntry(MapFileOutputFormat.java:76)
at
org.apache.nutch.searcher.FetchedSegments$Segment.getEntry(FetchedSegments.java:93)
at
org.apache.nutch.searcher.FetchedSegments$Segment.getParseText(FetchedSegments.java:84)
at
org.apache.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:147)
at org.apache.nutch.searcher.NutchBean.getSummary(NutchBean.java:321)
at de.ingrid.iplug.se.NutchSearcher.getDetail(NutchSearcher.java:219)
> FetchedSegments leave readers open
> ----------------------------------
>
> Key: NUTCH-211
> URL: http://issues.apache.org/jira/browse/NUTCH-211
> Project: Nutch
> Type: Bug
> Versions: 0.8-dev
> Reporter: Stefan Groschupf
> Assignee: Stefan Groschupf
> Priority: Critical
> Fix For: 0.8-dev
>
> I have a case here where the NutchBean is instantiated more than once,
> however I do cache the nutch bean, but in some situations the bean needs to
> re created. The problem is the FetchedSegments leaves open all reads it
> uses. So a nio Exception is thrown as soon I try to create the NutchBean
> again.
> I would suggest to add a close method to FetchedSegments and all involved
> objects to be able cleanly shutting down the NutchBean.
> Any comments? Would a patch be welcome?
> Caused by: java.nio.channels.ClosedChannelException
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:89)
> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:272)
> at
> org.apache.nutch.fs.LocalFileSystem$LocalNFSFileInputStream.seek(LocalFileSystem.java:83)
> at
> org.apache.nutch.fs.NFSDataInputStream$Checker.seek(NFSDataInputStream.java:66)
> at
> org.apache.nutch.fs.NFSDataInputStream$PositionCache.seek(NFSDataInputStream.java:162)
> at
> org.apache.nutch.fs.NFSDataInputStream$Buffer.seek(NFSDataInputStream.java:191)
> at org.apache.nutch.fs.NFSDataInputStream.seek(NFSDataInputStream.java:241)
> at org.apache.nutch.io.SequenceFile$Reader.seek(SequenceFile.java:403)
> at org.apache.nutch.io.MapFile$Reader.seek(MapFile.java:329)
> at org.apache.nutch.io.MapFile$Reader.get(MapFile.java:374)
> at
> org.apache.nutch.mapred.MapFileOutputFormat.getEntry(MapFileOutputFormat.java:76)
> at
> org.apache.nutch.searcher.FetchedSegments$Segment.getEntry(FetchedSegments.java:93)
> at
> org.apache.nutch.searcher.FetchedSegments$Segment.getParseText(FetchedSegments.java:84)
> at
> org.apache.nutch.searcher.FetchedSegments.getSummary(FetchedSegments.java:147)
> at org.apache.nutch.searcher.NutchBean.getSummary(NutchBean.java:321)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Nutch-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-developers
