Author: toad
Date: 2007-09-06 16:40:36 +0000 (Thu, 06 Sep 2007)
New Revision: 14988
Modified:
trunk/plugins/XMLSpider/XMLSpider.java
Log:
Make sub-indexes much bigger.
We can't rely on grouping them together in containers, because:
- mostly words which are near to each other in the index are not closely related
- we'd need multiple container support and we don't have it
- the containers would be big chunks to fetch and often wouldn't be reused on a
big index
So it makes sense to just use huge sub-indexes.
Long term we want sub-indexes to be split by size rather than number of entries.
Modified: trunk/plugins/XMLSpider/XMLSpider.java
===================================================================
--- trunk/plugins/XMLSpider/XMLSpider.java 2007-09-06 16:38:01 UTC (rev
14987)
+++ trunk/plugins/XMLSpider/XMLSpider.java 2007-09-06 16:40:36 UTC (rev
14988)
@@ -138,7 +138,7 @@
* Lists the allowed mime types of the fetched page.
*/
public Set allowedMIMETypes;
- private static final int MAX_ENTRIES = 20;
+ private static final int MAX_ENTRIES = 200;
private static int version = 7;
private static final String pluginName = "XML spider "+version;
/**
