: threads get stuck in spinwaiting
maybe the problem is not in the fetcher but rather in the generate fetch
list phase where it should take care in not sticking all URLs to the same
domain together.
-Ray-
2009/5/27 Larsson85
You're probably right that it has something to do
the threads get stuck in
som
spinwaiting mode. At first the crawl goes as planned, and I
couldnt
be
happier. But after som time, it starts reporting more of these
spinwaiting
messages.
I print a log here to show you what it looks like. As you can
see it
gets
stuck
- Original Message
From: Raymond Balmès raymond.bal...@gmail.com
To: nutch-user@lucene.apache.org
Sent: Tuesday, May 26, 2009 4:11:27 PM
Subject: Re: threads get stuck in spinwaiting
Observing what my crawls do, I believe Ken must be right.
Towards the end of the crawl (when
this behavior... no
idea why.
-Raymond-
2009/5/26 Larsson85 kristian1...@hotmail.com
When I try to do my crawl it seems like the threads get stuck in som
spinwaiting mode. At first the crawl goes as planned, and I couldnt be
happier. But after som time, it starts reporting more
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Raymond Balmès raymond.bal...@gmail.com
To: nutch-user@lucene.apache.org
Sent: Tuesday, May 26, 2009 4:11:27 PM
Subject: Re: threads get stuck in spinwaiting
Observing what my crawls do
in multiple-threads.
I noted that if one does a recrawl you don't get this behavior...
no
idea why.
-Raymond-
2009/5/26 Larsson85 kristian1...@hotmail.com
When I try to do my crawl it seems like the threads get stuck in som
spinwaiting mode. At first the crawl goes as planned, and I
it seems like the threads get stuck in som
spinwaiting mode. At first the crawl goes as planned, and I couldnt
be
happier. But after som time, it starts reporting more of these
spinwaiting
messages.
I print a log here to show you what it looks like. As you can see it
gets
, spinWaiting=1000,
fetchQueues.totalSize=2523
--
View this message in context:
http://www.nabble.com/threads-get-stuck-in-spinwaiting-tp23723825p23723825.html
Sent from the Nutch - User mailing list archive at Nabble.com.
--
View this message
Subject: Re: threads get stuck in spinwaiting
maybe the problem is not in the fetcher but rather in the generate fetch
list phase where it should take care in not sticking all URLs to the same
domain together.
-Ray-
2009/5/27 Larsson85
You're probably right that it has something
:43:02 AM
Subject: Re: threads get stuck in spinwaiting
maybe the problem is not in the fetcher but rather in the generate fetch
list phase where it should take care in not sticking all URLs to the same
domain together.
-Ray-
2009/5/27 Larsson85
You're probably right that it has
://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Raymond Balmès raymond.bal...@gmail.com
To: nutch-user@lucene.apache.org
Sent: Wednesday, May 27, 2009 9:40:34 AM
Subject: Re: threads get stuck in spinwaiting
@otis
or did you mean nutch/host. I have only one server
on this?
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
- Original Message
From: Otis Gospodnetic ogjunk-nu...@yahoo.com
To: nutch-user@lucene.apache.org
Sent: Wednesday, May 27, 2009 11:38:48 PM
Subject: Re: threads get stuck in spinwaiting
Ray,
I don't think
this behavior... no idea
why.
-Raymond-
2009/5/26 Larsson85 kristian1...@hotmail.com
When I try to do my crawl it seems like the threads get stuck in som
spinwaiting mode. At first the crawl goes as planned, and I couldnt be
happier. But after som time, it starts reporting more
a recrawl you don't get this behavior... no
idea why.
-Raymond-
2009/5/26 Larsson85 kristian1...@hotmail.com
When I try to do my crawl it seems like the threads get stuck in som
spinwaiting mode. At first the crawl goes as planned, and I couldnt be
happier. But after som time, it starts
14 matches
Mail list logo
