I've put stopword support in Library, I think it behaves ideally in deciding when to alter a search containing stop words or when to reject it, except for not notifying the user if a stopword was removed from their query.
For the moment I have only used 6 english stopwords : the, and, that, have, not, with Any suggestions on how many stop words should be used and stop words in other languages? I've put hard coded stop words in XMLSpider too, but it's on my fork git://github.com/platy/plugin-XMLSpider-staging.git there are a lot of commits on there which haven't been pushed to git://github.com/freenet/plugin-XMLSpider-staging.git ?I didn't push it as it will be incompatible with older versions due to changes in the database, should I push this stuff? That NPE is now fixed, and i've implemented the simpler versions of intersect() and phrase() that you suggested they seem to be working better. 2009/8/15 Matthew Toseland <toad at amphibian.dyndns.org>: > [16:03:36] <toad_> java.lang.NullPointerException > [16:03:36] <toad_> -- > plugins.Library.ui.MainPage.processGetRequest(MainPage.java:83) > [16:03:36] <toad_> -- > plugins.Library.ui.MainPageToadlet.handleGet(MainPageToadlet.java:53) > [16:03:48] <toad_> mikeb_: when i go to a page and then come back > [16:03:57] <toad_> mikeb_: also i'm suspicious of the phrase search > [16:04:13] <toad_> "test subject" gives http://127.0.0.1:8888/SSK at > aYWBb6zo2AM13XCNhsmmRKMANEx6PG~C15CWjdZziKA,X1pAG4EIqR1gAiyGFVZ1iiw-uTlh460~rFACJ7ZHQXk,AQACAAE/fms-106/ > which doesn't seem to include that prhase > [16:04:37] * toad_ searches for "encrypted message" which definitely exists > on that page > [16:05:47] <mikeb_> toad_ : i'll have to deal with this later i need to pop > out > > "encrypted message" searched for with both indexes stalls: > > "encrypted message"@bookmark:freenetindex bookmark:wanna : Busy > "encrypted message"@bookmark:freenetindex : Busy > ? ? ? ?encrypted ? ? ? Fetching Subindex > ? ? ? ?46% > ? ? ? ?message ? ? ? ? Done > ? ? ? ?"encrypted message"@bookmark:wanna ? ? ?Done >
