On Mon, Dec 05, 2011 at 01:38:14PM -0800, Marvin Humphrey wrote: > Awesome stuff coming through on the new Lucy::Analysis::StandardTokenizer!
Following up on this: I've now reviewed all the StandardTokenizer commits that landed today. Very impressive work! Aside from the PolyAnalyzer issue, all I could come up with was some bikeshedding (which I'll just keep to myself). :) I didn't grok everything that was being done in the compressed table lookup scheme, but your code is as well-documented and easy to follow as anything that does that much bit-twiddling possibly could be, and I feel like I could dive in and work on it if the need arose. Marvin Humphrey
