Jan =?UTF-8?Q?Urba=C5=84ski?= <wulc...@wulczer.org> writes: >> I think the only relevance of stopwords to the current problem is that >> *if* stopwords have been removed, we would see a Zipfian distribution >> with the first few entries removed, and I'm not sure if it's still >> really Zipfian afterwards.
> That's why I was proposing to take s = 0.07 / (MCE-count + 10). But that > probably doesn't matter much. Oh, now I get the point of that. Yeah, it is probably a good idea. If the input doesn't have stopwords removed, the worst that will happen is we'll collect stats for an extra 10 or so lexemes, which will then get thrown away when they don't fit into the MCE list. +1. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers