Hello Peter
we have multiple ways to support FTS within OWLIM. One of them is based on
proprietary FTI implementation and another one is based on Lucene. We are
not making much noise about them, because they are still not properly
documented, we are also working to further speed up both the indexing and
the queries. Still, Ivan can help you to try the current implementation, if
this feature is so important for you
Regards
Naso
----------------------------------------------------------
Atanas Kiryakov
Executive Director of Ontotext AD, http://www.ontotext.com
Sirma Group, http://www.sirma.bg
Phone: (+359 2) 974 61 44; Fax: 975 3226
----------------------------------------------------------
Fortes fortuna adiuvat.
----- Original Message -----
From: "Ing. Peter Kostelník PhD." <[email protected]>
To: <[email protected]>
Sent: Tuesday, January 05, 2010 3:24 PM
Subject: Re: [Owlim-discussion] BigOWLIM issues
hi there,
to the lucene topic: now we are using the lucene sail, but we were forced
to hack it to be able to put it above the bigowlim .. it is not
implemented too well, it has problems with handling the transactions
dealing with the big amounts of data (when building the imports, it holds
everything in the memory, so it, logically, has to go down on the heap
space) .. we also found serious problems in handling the index for
multi-value properties, context data, but also in the implementation of
query evaluation .. so, if we wanted to avoid of building our own external
full-text index, integration of lucene sail required many hacks and
workarounds directly in the lucene sail code ..
and this are, in our oppinion, the most important issues, which should be
taken into account .. for sure, with performance optimizations :)
by the way, don't you plan to use the new release of lucene 3.0.0, which
has the really many usefull optimizations?
cheers,
Peter K.
Hi Peter,
Both issues are currently in progress. Lucene integration is currently
only
experimental and is not really flexible or stable to use in production
(it
is
not even documented). Our goal is to provide enough flexibility so that
e.g.
custom analyzers and result rankings are easily pluggable by the engine
user.
It will be of great value to us if you could summarize the full-text
search
flexibility you will need.
The "when" question is a lot harder to answer. I can't give you any
concrete
due dates currently, but this is something on the table now and we should
be
able to deliver results within the next couple of months. I hope I'm not
too
wrong about that... :)
Cheers and have a happy new year!
Ivan
On Friday 18 December 2009 13:01:51 Ing. Peter Kostelník PhD. wrote:
hi there,
we're planning to use the BigOWLIM as the production backend, so I've
got
just the few questions regarding the further BigOWLIM developement ..
1. I've noticed, that in 3.2.6 snapshot, there is the direct dependency
to
Lucene 2.9 core .. so, I assume, you're planning to integrate the lucene
as the fulltext index/search engine .. pls, would there be the support
for
configuring the lucene? I mean the essential issues, such as adding
custom
analysers/tokenizers, fuzzy search support, custom query parsers, etc. ?
.. when do you plan to integrate the lucene?
2. is there some possibility to force BigOWLIM to perform logging in
some
reasonable way? .. now everything is flushed into (I guess)
System.out/err
.. and, well, this is not so suitable for production backend ..
thanks in advance,
best wishes and merry christmas,
Peter K.
_______________________________________________
OWLIM-discussion mailing list
[email protected]
http://ontotext.com/mailman/listinfo/owlim-discussion
_______________________________________________
OWLIM-discussion mailing list
[email protected]
http://ontotext.com/mailman/listinfo/owlim-discussion
_______________________________________________
OWLIM-discussion mailing list
[email protected]
http://ontotext.com/mailman/listinfo/owlim-discussion
