I have currently 32 segments on one computer.
What would be a good amount of segments to have on one computer for best performance?
What is more important than the number of segments is the total number of documents. For optimal performance on a search node you should merge the desired set of segments into a single index (after first performing duplicate elimination). You can then look at the size of the .f* files in the index to see how many documents are in the index.
Then it depends on how much traffic you expect. If your peak traffic is only around 1 query/second, then you can probably stand to have 20 or more million pages per node. But if your peak traffic is higher, then things will be more cost-effective if you can fit the major index strutures in RAM. These require around 2kB/document, so a search node with 4GB of RAM can handle around 2M documents at around 20 queries/second.
Do I have to delete old segments after some time?
If you're fetching regularly, segments older than the db.default.fetch.interval can be deleted, as their pages should have been refetched. This is 30 days by default.
Doug
------------------------------------------------------- This SF.Net email is sponsored by: IBM Linux Tutorials Free Linux tutorial presented by Daniel Robbins, President and CEO of GenToo technologies. Learn everything from fundamentals to system administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
