I don't know too much about the internals of ferret. But I'm not too much surprised that ferret is choking on this 'content'. As all fulltext search engines ferret will presume that it's human readable language that is going to be indexed. It would be only because of coincidence that tests of the stemming, analyzing (and so on) algorithms won't fail, which results in lengthy parsings at least.
Is it only because of problems to get 'real world' test content? You'll find loads of content on http://www.gutenberg.org/ for example...
Regards
Jan
On 5/25/06, Marcus Andersson <[EMAIL PROTECTED]> wrote:
Jan Prill wrote:
>
> Regardless what I've said before: What performance are you experiencing
> with
> your pure ruby installation? How much datasets do you need to index
> initially? When (after how much datasets) are you experiencing the
> bottleneck?
>
After doing quite a bit more of testing it seems that speed seems to be
content dependant. The content is ugly test content it seems where
someone have just made random key strokes.
Content that it shokes on is down at the end.
/Marcus
Each document is built this way (documents may contain UTF-8 chars but I
ignore that for now):
class Node < ActiveRecord::Base
acts_as_ferret ...
end
class PageNode < Node
def to_doc
doc = super
page.content_items.each { |item| item.to_doc(doc) if
item.searchable? } if page
doc
end
end
class ContentItem
def to_doc(doc)
doc << Ferret::Document::Field.new(
'content_item', self.content,
Ferret::Document::Field::Store::NO,
Ferret::Document::Field::Index::TOKENIZED)
end
end
Content:
<h1>Huvudrubrik
svart</h1>ldfkgjdflkgjdflkgjdflgkdflgkdflgkjdflkgj<br><br><h2>Huvudrubrik
orange</h2>sdlkfjsdfkljsdlfksjdflsjflskfjslkfjslkdfsd<br>fsd
fsdfsd<br>fsdfsdfsddfdsdfsdf<br><h3>Underrubrik
svart</h3><p>dfgfgdfgdfgdfgdfgdfgdfgdf<br>gdfgdgdfgkjhdfkjghdkjgh dkjghd
kgjhd kgfjh d<br></p><h4>Underrubrik
orange</h4>lkdfjgldfkgjdlfkgjdlfkgjdflkgdfg<br>dfgdfgdfgdfgdfgdfg<br><br><h5>Styckerubrik
svart</h5>fghfhfghfhkfjglhkjfglhkfjhlkfjghlfkhjflkgh
jflgkhjflgkhf<br>ghfghfgh<br>fgh<br>fghfghfghgfh<br><br><h6>Styckerubrik
orange</h6>fghkfgjhlfgkjhflkghj flghkjfgl hkjfg lhkfgjhlfgkhfghfgh<br>
--
Posted via http://www.ruby-forum.com/.
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
_______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
