I started going through the source code debugging this problem.. The
extra thread comes from FetchedSegments(Configuration conf, Path
segmentsDir) constructor. This constructor creates a segmentUpdater and
then starts it.. This is the thread that I am talking about..
Does anyone know how to cleanly shut down this segment updater?
>From FetchedSegments constructor....
this.segUpdater = new SegmentUpdater();
if (segmentDirs != null) {
for (final Path segmentDir : segmentDirs) {
segments.put(segmentDir.getName(),
new Segment(this.fs, segmentDir, this.conf));
}
}
this.segUpdater.start(); <-- this is the line I am talking about..
Any ideas, has anyone run into this ?
-----Original Message-----
From: Lukas, Ray [mailto:[email protected]]
Sent: Thursday, April 23, 2009 4:36 PM
To: [email protected]
Subject: Using nutchBean
Is this correct..
NativeCrawler nativeCrawler = null;
NutchBean nutchBean = null;
Query nutchQuery = null;
Hits nutchHits = null;
for (int index=0; index<10; index++) {
nativeCrawler = new
NativeCrawler("www.ajpm.com", "ajpm-index", 2, 5);
int maxHits = 1000;
(*) nutchBean = new NutchBean(
nativeCrawler.getConfig(),
nativeCrawler.getIndexPath());
nutchQuery = Query.parse("gold",
nativeCrawler.getConfig());
nutchHits = nutchBean.search(nutchQuery,
maxHits);
nutchBean.close();
System.out.println("gold nutchHits: " +
nutchHits.getLength());
}
nutchQuery = null;
nutchBean = null;
System.out.println("credit nutchHits: " +
nutchHits.getLength());
Everytime I execute this (*) line a new thread is started and never
ends. At the end of this I have ten threads. This loop in real life
might execute 5000 times..
How does a person close off, shutdown a nutchBean object. I call close
when, and as soon as, I am done with it.
NutchCrawler is my code that basically points me to the nutch index
directory.
Thank you for the help
ray
