On Sun, 1 Feb 2004 13:12:32 -0800 (PST) Otis Gospodnetic <[EMAIL PROTECTED]> wrote:
> Looking forward to the contribution. Sorry for the delay, but I've had quite some workload lately, and then I moved between apartments. I'm back and I'm ready to spend some time. I gave up detecting the language of a query. It is very possbile indeed and I got great results with Weka, but takes too much time: 5-50 seconds on my Pentium M. However, I'm still working on the "autoanalytic stemmer", atleast in my head. I've started to feed my index with docuemnts tagged with the language in a field, and thought it should analyze (still the n-gram approach) all words of a specific language to find stemming rules for each and every language. The output can be used per language stemming, BUT hopefully I'll be able to use this data to create my generic stemmer. The language models and inflectional form extraction should be based on the index content, but I can't seem to find out how to access the terms of a specific set of documents. Of course, I could just query my index and start working on the data, building my own trie-pattern, but I'm sure I don't have to. I've been browsing the list archives and API for several days without finding out how to iterate the (distinct/unique) terms of the index or a specific set of documents. How do I do that? -- karl --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
