On 8/29/06, David Balmain <[EMAIL PROTECTED]> wrote: > On 8/26/06, Florent Solt <[EMAIL PROTECTED]> wrote: > > Perhaps, I found where is my problem (during a big import). > > Why this silly (really silly :)) example crash ? > > > > http://pastie.caboo.se/10357 > > > > /usr/lib/ruby/site_ruby/1.8/ferret/index.rb:211:in `add_document': IO > > Error occured at <except.c>:79 in xraise (IOError) > > Error occured in fs_store.c:225 - fso_flush_i > > flushing src of length -2 > > > > from /usr/lib/ruby/site_ruby/1.8/ferret/index.rb:211:in `<<' > > from /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' > > from /usr/lib/ruby/site_ruby/1.8/ferret/index.rb:186:in `<<' > > from test.rb:13 > > from test.rb:8 > > Hi Florent, > > This is something that I still need to work on. The Locale sensitive > analyzers aren't as robust as they could be. Try using the > AsciiStandardAnalyzer instead. Or better yet, don't index binary data. > You can store binary data but indexing it doesn't usually make a lot > of sense. At least not without a custom analyzer. Having said that, I > will try and fix this. > > Cheers, > Dave >
Just an update on this issue. I've now made the StandardAnalyzer more robust so it won't crash as easily (hopefully not at all) with bad data. In the process of fixing this I also added a fix so that the StandardTokenizer will now tokenize negative numbers. ie it will parse "-23" as "-23" instead of just "23". Cheers, Dave _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
