On 5/26/06, Marcus Andersson <[EMAIL PROTECTED]> wrote:
> More testing:
>
> This document (with several fields in it) took 15 seconds to index:
> Field: new item
> Field: Presentationsmaterial
> Field: Ppt-presentationer
> Field:
> Field: new item
> Field: new item
> Field: new item
>
> A bit long for that little content if you ask me. I have several similar
> documents that take a lot of time ("new item" is an ugly default value
> that all content items get from the beginning, don't ask me why, does it
> affect indexing speed when a lot of documents contains similar tokens?).
>
> But, I don't know. I'm using the Ruby version. That is supposed to be
> slow. Maybe the super fast C implementation should take 150ms to handle
> a document of this size? What affects indexing speed?
Hi Marcus,
I just tested this here;
require 'lib/rferret.rb'
include Ferret
include Ferret::Document
include Ferret::Index
doc = Document.new
doc << Field.new(:field, "new item")
doc << Field.new(:field, "Presentationsmaterial")
doc << Field.new(:field, "Ppt-presentationer")
doc << Field.new(:field, " ")
doc << Field.new(:field, "new item")
doc << Field.new(:field, "new item")
doc << Field.new(:field, "new item")
i = Index.new(:path => "index_dir")
i << doc
i.close
[EMAIL PROTECTED]:~/workspace/ferret $ time ruby test.rb
real 0m0.147s
user 0m0.125s
sys 0m0.022s
This is with the pure ruby version. If this document is taking 15
seconds then something is going wrong. Similarly the bad data should
hurt indexing speed considerably although it will make your index
larger than usual and merging will take a little longer. Could you
post a simple testcase that takes a long time for you?
Cheers,
Dave
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
