[
https://issues.apache.org/jira/browse/NUTCH-258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12467887
]
Chris A. Mattmann commented on NUTCH-258:
-----------------------------------------
Guys,
From recent conversations on the mailing list where Doug mentioned that this
issue may now be rendered resolved by recent changes to Hadoop, I'm wondering
if we can close this issue? It's currently listed as a "critical" priority bug,
and there are currently 3 watchers of the issue. I've asked this several times
recently in the last few months about whether people are still experiencing
this issue. So, the question is, are they? If not, I'd like to close out the
issue as I'm trying to get things organized here in JIRA, so that developers
and contributors can have a good idea of what issues are out there, that really
need some attention. With the recent lack of developer resources, I think
closing out issues that are not reproduceable, issues that people are no longer
experiencing, or issues resolved by recent changes in Hadoop/etc. is an
important thing to do in this process.
Thus, I'm opening this issue up to any objections for closing/resolving it. If
I don't hear any objections in the next week, I will close this issue out.
Thanks!
Cheers,
Chris
> Once Nutch logs a SEVERE log item, Nutch fails forevermore
> ----------------------------------------------------------
>
> Key: NUTCH-258
> URL: https://issues.apache.org/jira/browse/NUTCH-258
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.8
> Environment: All
> Reporter: Scott Ganyo
> Assigned To: Chris A. Mattmann
> Priority: Critical
> Fix For: 0.9.0
>
> Attachments: dumbfix.patch, NUTCH-258.Mattmann.060906.patch.txt,
> NUTCH-258.Mattmann.080406.patch.txt
>
>
> Once a SEVERE log item is written, Nutch shuts down any fetching forevermore.
> This is from the run() method in Fetcher.java:
> public void run() {
> synchronized (Fetcher.this) {activeThreads++;} // count threads
>
> try {
> UTF8 key = new UTF8();
> CrawlDatum datum = new CrawlDatum();
>
> while (true) {
> if (LogFormatter.hasLoggedSevere()) // something bad happened
> break; // exit
>
> Notice the last 2 lines. This will prevent Nutch from ever Fetching again
> once this is hit as LogFormatter is storing this data as a static.
> (Also note that "LogFormatter.hasLoggedSevere()" is also checked in
> org.apache.nutch.net.URLFilterChecker and will disable this class as well.)
> This must be fixed or Nutch cannot be run as any kind of long-running
> service. Furthermore, I believe it is a poor decision to rely on a logging
> event to determine the state of the application - this could have any number
> of side-effects that would be extremely difficult to track down. (As it has
> already for me.)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
