Hi,
I generated batchid 1111 with topN 100, in Hbase I have 100 rows with
generate mark with the same value, but fetch is not working, after run
nutch* fetch 1111* I got no progres for this generated urls, in database is
no row with fetch mark and pages are unfetched.
I write debug prints in nutch source codes and build it. FetcherMapper works
fine, if generate mark is 1111 writes it in cotext (/context.write(new
IntWritable(random.nextInt(65536)),
new
FetchEntry(context.getConfiguration(), key, page));)/, but in
FetcherReducer is nothing /*fit = fetchQueues.getFetchItem(); */ produces
null /(org.apache.nutch.fetcher.FetcherReducer$FetcherThread getFetchItem :
null/
)
It looks like the queues are empty, but why? What could be problem here?
Thanks in advance
/FetcherJob: starting
FetcherJob: batchId: 1111
FetcherJob: threads: 10
FetcherJob: parsing: false
FetcherJob: resuming: false
FetcherJob : timelimit set for : 1395683736716
Using queue mode : byHost
Fetcher: threads: 10
QueueFeeder finished: total 0 records. Hit by time limit :100
-finishing thread FetcherThread0, activeThreads=0
-finishing thread FetcherThread2, activeThreads=1
-finishing thread FetcherThread1, activeThreads=0
-finishing thread FetcherThread4, activeThreads=0
-finishing thread FetcherThread3, activeThreads=0
-finishing thread FetcherThread5, activeThreads=0
-finishing thread FetcherThread6, activeThreads=0
-finishing thread FetcherThread7, activeThreads=0
-finishing thread FetcherThread8, activeThreads=0
Fetcher: throughput threshold: -1
-finishing thread FetcherThread9, activeThreads=0
Fetcher: throughput threshold sequence: 5
0/0 spinwaiting/active, 0 pages, 0 errors, 0.0 0 pages/s, 0 0 kb/s, 0 URLs
in 0 queues
-activeThreads=0
FetcherJob: done/
HBase shell: return 100 rows only for generate mark
scan 'webpage_webpage', {COLUMNS => ['mk:_gnmrk_','mk:_ftcmrk_']}
--
View this message in context:
http://lucene.472066.n3.nabble.com/Nutch-2-1-fetching-is-not-working-maybe-broken-generate-tp4123813p4126652.html
Sent from the Nutch - User mailing list archive at Nabble.com.
