Hi Nathan,

At the time we wrote search, this functionality didn't exist. However, it's
still too limited for our needs.

First of all, not all databases that we support are supported by the full
text search, meaning that some users would be left out.

It makes development harder, since we develop and run nightly tests against
sqlite, and doing any development/testing with search would mean we'd have
to change our development/test infrastructure to use a database backend that
supports it.

Also, our search supports a number of search prefixes (file:, author:, etc),
which we can't fully implement with the full text seach. The reason for this
is that some of these are actually using pre-processed data that doesn't
exist in the database as something easily indexable. We want all results to
be tied to a particular review request. This is difficult in the cases of,
say, file searching and bug number search.

By default, we search a blob consisting of the summary, description, testing
done, bugs (in a normalized form), submitter name (joined with the full
name), and a normalized list of files. If we use full text search, we need
to now parse the search string, do possibly expensive searches based on all
those prefixes that we find in the string, and do a union of all those
results with anything from that blob (which requires searching all those
fields individually). This gets pretty hairy, fast.

What we really want to do is pre-process all the results into a search index
and search from that. PyLucene gives us this. Yes, it's a pain to install,
and I hate it as much as anyone else who has tried to install it, but it
does give us the functionality we need.

I'm hoping to find a better solution in the long term, but I don't believe
that the native database full text search is it. Now, maybe if we developed
a custom solution that provided tables for storing the pre-processed search
indexes and then searched from that using full text search. If we go this
route, what I'd like to do is split off the search functionality into its
own django app in our tree (reviewboard.search) and add a backends directory
with different search backends (fulltextdb.py, pylucene.py). We'd then need
a nice API that wraps the backends. I'd be all for a solution like this.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Tue, Oct 20, 2009 at 1:07 PM, Nathan Auch [Sybase] <na...@sybase.com>wrote:

>
> Hi Christian,
>
> Out of curiosity, have you already tried using Django's interface to
> database-based full text search?
>
> http://docs.djangoproject.com/en/dev/ref/models/querysets/#search
>
> I ask because we are running (a slightly modified) Review Board on top
> of a Sybase SQL Anywhere database. This thread reminded me that I had
> wanted to modify the search to use SQL Anywhere's full text indexes. Is
> there was any specific reason you opted against this approach?
>
> Thanks!
>
> Nathan Auch
> Sybase iAnywhere
>
> Christian Hammond wrote:
> > No, but good luck with it :)
> >
> > I really would like to find a good replacement for PyLucene. It's a
> > royal pain to install.
> >
> > Christian
> >
> > --
> > Christian Hammond - chip...@chipx86.com <mailto:chip...@chipx86.com>
> > Review Board - http://www.review-board.org
> > VMware, Inc. - http://www.vmware.com
> >
> >
> > On Wed, Oct 14, 2009 at 5:35 AM, Thilo-Alexander Ginkel
> > <th...@ginkel.com <mailto:th...@ginkel.com>> wrote:
> >
> >
> >     Hello everyone,
> >
> >     I was thinking about adding search support to our Review Board
> >     installation, which would require the installation of PyLucene. Has
> >     somebody succeeded in getting this working on a Windows-based
> >     platform?
> >
> >     Thanks,
> >     Thilo
> >
> >     To unsubscribe from this group, send email to
> >     
> > reviewboard+unsubscr...@googlegroups.com<reviewboard%2bunsubscr...@googlegroups.com>
> >     
> > <mailto:reviewboard%2bunsubscr...@googlegroups.com<reviewboard%252bunsubscr...@googlegroups.com>
> >
> >     For more options, visit this group at
> >     http://groups.google.com/group/reviewboard?hl=en
> >     -~----------~----~----~----~------~----~------~--~---
> >
> >
> >
> > > To unsubscribe from this group, send email to
> > reviewboard+unsubscr...@googlegroups.com<reviewboard%2bunsubscr...@googlegroups.com>
> > For more options, visit this group at
> > http://groups.google.com/group/reviewboard?hl=en
> > -~----------~----~----~----~------~----~------~--~---
> >
>
>
> > To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com<reviewboard%2bunsubscr...@googlegroups.com>
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en
> -~----------~----~----~----~------~----~------~--~---
>
>

--~--~---------~--~----~------------~-------~--~----~
Want to help the Review Board project? Donate today at
http://www.reviewboard.org/donate/
Happy user? Let us know at
http://www.reviewboard.org/users/
-~----------~----~----~----~------~----~------~--~---
To unsubscribe from this group, send email to
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/reviewboard?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to