Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-28 Thread Dave Page
Steinar H. Gunderson wrote: On Tue, Feb 27, 2007 at 01:33:47PM +, Dave Page wrote: When we outgrow PostgreSQL Tsearch2, then, well, we'll need to stop pretending to be Google... Just for the record: Google has been known to sponsor sites in need with Google Minis and such earlier -- I

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-28 Thread Oleg Bartunov
On Wed, 28 Feb 2007, Dave Page wrote: Steinar H. Gunderson wrote: On Tue, Feb 27, 2007 at 01:33:47PM +, Dave Page wrote: When we outgrow PostgreSQL Tsearch2, then, well, we'll need to stop pretending to be Google... Just for the record: Google has been known to sponsor sites in need

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-28 Thread Dave Page
Oleg Bartunov wrote: Guys, current tsearch2 should works with millions of documents. ... Search itself is incredibly fast ! Oh, I know - you and Teodor have done a wonderful job. Regards, Dave. ---(end of broadcast)--- TIP 3: Have you

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-27 Thread Magnus Hagander
On Mon, Feb 26, 2007 at 04:24:12PM -0500, Charles Sprickman wrote: On Mon, 26 Feb 2007, Madison Kelly wrote: Hi all, I'd really like to come up with a more intelligent search engine that doesn't take two minutes to return results. :) I know, in the end good indexes and underlying

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-27 Thread Dave Page
Magnus Hagander wrote: Just as a datapoint, we did try to use mnogosearch for the postgresql.org website+archives search, and it fell over completely. Indexing took way too long, and we had search times several thousand times longer than with tsearch2. That said, I'm sure there are cases

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-27 Thread Chris
Madison Kelly wrote: Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-27 Thread Charles Sprickman
On Tue, 27 Feb 2007, Dave Page wrote: Magnus Hagander wrote: Just as a datapoint, we did try to use mnogosearch for the postgresql.org website+archives search, and it fell over completely. Indexing took way too long, and we had search times several thousand times longer than with tsearch2.

[PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Madison Kelly
Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the query string being used

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Joshua D. Drake
Madison Kelly wrote: Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Mark Stosberg
Joshua D. Drake wrote: Madison Kelly wrote: Hi all, I am asking in this list because, at the end of the day, this is a performance question. I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Joshua D. Drake
So I am hoping some of you guys and gals might be able to point me towards some resources or offer some tips or gotcha's before I get started on this. I'd really like to come up with a more intelligent search engine that doesn't take two minutes to return results. :) I know, in the end

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Mark Stosberg
Madison Kelly wrote: I think the more direct question I was trying to get at is How do you build a 'relavence' search engine? One where results are returned/sorted by relevance of some sort?. At this point, the best I can think of, would be to perform multiple queries; first matching the

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Madison Kelly
Mark Stosberg wrote: Madison Kelly wrote: I think the more direct question I was trying to get at is How do you build a 'relavence' search engine? One where results are returned/sorted by relevance of some sort?. At this point, the best I can think of, would be to perform multiple queries;

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Mark Stosberg
Now see, this is exactly the kind of sagely advice I was hoping for! :) I'll look into tsearch2, and failing that for some reason, I love the keyword table idea. For example keyword search code, you can try this package:

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Jeff Davis
On Mon, 2007-02-26 at 11:29 -0500, Madison Kelly wrote: I am looking at writing a search engine of sorts for my database. I have only ever written very simple search engines before which amounted to not much more that the query string being used with ILIKE on a pile of columns. This was

Re: [PERFORM] Writting a search engine for a pgsql DB

2007-02-26 Thread Charles Sprickman
On Mon, 26 Feb 2007, Madison Kelly wrote: Hi all, I'd really like to come up with a more intelligent search engine that doesn't take two minutes to return results. :) I know, in the end good indexes and underlying hardware will be important, but a sane as possible query structure helps to