Re: [HACKERS] Simplifying Text Search

2008-03-08 Thread Bruce Momjian
Added to TODO: * Consider a function-based API for '@@' full text searches http://archives.postgresql.org/pgsql-hackers/2007-11/msg00511.php --- Simon Riggs wrote: > Something Tom Dunstan just mentioned has made me ask

Re: [HACKERS] Simplifying Text Search

2007-12-04 Thread Decibel!
On Dec 4, 2007, at 4:48 PM, Joshua D. Drake wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 4 Dec 2007 16:41:52 -0600 Decibel! <[EMAIL PROTECTED]> wrote: On Nov 15, 2007, at 4:41 AM, Gregory Stark wrote: Ask me about EnterpriseDB's Slony Replication support! Hah, wtf is that

Re: [HACKERS] Simplifying Text Search

2007-12-04 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 4 Dec 2007 16:41:52 -0600 Decibel! <[EMAIL PROTECTED]> wrote: > On Nov 15, 2007, at 4:41 AM, Gregory Stark wrote: > > Ask me about EnterpriseDB's Slony Replication support! > > > Hah, wtf is that all about? :) Stealth marketing :)... Jos

Re: [HACKERS] Simplifying Text Search

2007-12-04 Thread Decibel!
On Nov 15, 2007, at 4:41 AM, Gregory Stark wrote: Ask me about EnterpriseDB's Slony Replication support! Hah, wtf is that all about? :) BTW, looked at londiste? -- Decibel!, aka Jim C. Nasby, Database Architect [EMAIL PROTECTED] Give your computer some brain candy! www.distributed.net Team

Re: [HACKERS] Simplifying Text Search

2007-11-20 Thread Bruce Momjian
Simon Riggs wrote: > On Tue, 2007-11-20 at 14:25 -0500, Bruce Momjian wrote: > > This has been saved for the 8.4 release: > > > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold > > > > It isn't a patch, so isn't being held for later review, nor have you > added it to the TODO list, so

Re: [HACKERS] Simplifying Text Search

2007-11-20 Thread Simon Riggs
On Tue, 2007-11-20 at 14:25 -0500, Bruce Momjian wrote: > This has been saved for the 8.4 release: > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold > It isn't a patch, so isn't being held for later review, nor have you added it to the TODO list, so I'm not sure what this means. Wo

Re: [HACKERS] Simplifying Text Search

2007-11-20 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: > Something Tom Dunstan just mentioned has made me ask the question "Why > does our full text sear

Re: [HACKERS] Simplifying Text Search

2007-11-17 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > Magnus Hagander <[EMAIL PROTECTED]> writes: >> On Wed, Nov 14, 2007 at 07:46:58AM +, Gregory Stark wrote: >>> Have you yet given any advantages of contains over @@ ? > >> Familiarity for users of SQL Server that are migrating? ;-) >> (http://msdn2.micro

Re: [HACKERS] Simplifying Text Search

2007-11-15 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Mittwoch, 14. November 2007 schrieb Trevor Talbot: >> But that coversion itself is fundamentally flawed, is the problem. > I know it's incorrect, but with a different parser and/or dictionary you > could > make it work. No, I don't think so. Tr

Re: [HACKERS] Simplifying Text Search

2007-11-15 Thread Trevor Talbot
On 11/15/07, Peter Eisentraut <[EMAIL PROTECTED]> wrote: > In practice, the search pattern will mostly be provided dynamically from some > user input, so you could conceivably be able to modify the search patterns > more readily than the entire queries in your application. Anyway, it's just > an

Re: [HACKERS] Simplifying Text Search

2007-11-15 Thread Peter Eisentraut
Am Mittwoch, 14. November 2007 schrieb Trevor Talbot: > But that coversion itself is fundamentally flawed, is the problem. I know it's incorrect, but with a different parser and/or dictionary you could make it work. In practice, the search pattern will mostly be provided dynamically from some u

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Tom Lane
Magnus Hagander <[EMAIL PROTECTED]> writes: > On Wed, Nov 14, 2007 at 07:46:58AM +, Gregory Stark wrote: >> Have you yet given any advantages of contains over @@ ? > Familiarity for users of SQL Server that are migrating? ;-) > (http://msdn2.microsoft.com/en-us/library/ms187787.aspx) That arg

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > [ replace LIKE with this: ] > CREATE FUNCTION textlike_ts(text, text) RETURNS boolean > RETURNS NULL ON NULL INPUT IMMUTABLE > LANGUAGE SQL > AS $$ SELECT $1 @@ likepattern_to_tsquery($2) AND $1 #~~# $2; $$; Cute trick, but as-is this wil

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Trevor Talbot
On 11/14/07, Peter Eisentraut <[EMAIL PROTECTED]> wrote: > I wrote: > > What we'd need is a way to convert a LIKE pattern into a tsquery > > ('%foo%bar%' => 'foo & bar'). Then you might even be able to sneak > > index-optimized text search into existing applications. Might be worth a > > try. > >

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Oleg Bartunov
On Wed, 14 Nov 2007, Tom Lane wrote: Bruce Momjian <[EMAIL PROTECTED]> writes: Yes, this the same problem we had months ago trying to improve the syntax, that there was no easy syntax that covered all common use cases. The thing that we keep coming up against is that we'd like queries to be a

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Yes, this the same problem we had months ago trying to improve the > syntax, that there was no easy syntax that covered all common use cases. The thing that we keep coming up against is that we'd like queries to be able to depend on default_text_search_c

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Better idea: > > > create function > > contains(sourceText text, searchText text, config text) returns boolean > > as $$ > > to_tsvector(config, sourceText) @@ to_tsquery(config, searchText); > > $$ language sql; > > I think you have c

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Better idea: > create function > contains(sourceText text, searchText text, config text) returns boolean > as $$ > to_tsvector(config, sourceText) @@ to_tsquery(config, searchText); > $$ language sql; I think you have confused "simple query syntax" with "

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Peter Eisentraut
I wrote: > What we'd need is a way to convert a LIKE pattern into a tsquery > ('%foo%bar%' => 'foo & bar'). Then you might even be able to sneak > index-optimized text search into existing applications. Might be worth a > try. Here is how this could work: CREATE FUNCTION likepattern_to_tsquery(

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Bruce Momjian
Simon Riggs wrote: > Better idea: > > in-linable function called > > create function > contains(sourceText text, searchText text, config text) returns boolean > as $$ > to_tsvector(config, sourceText) @@ to_tsquery(config, searchText); > $$ language sql; > > so that > > SELECT title > FROM pgw

Re: [HACKERS] Simplifying Text Search

2007-11-14 Thread Magnus Hagander
On Wed, Nov 14, 2007 at 07:46:58AM +, Gregory Stark wrote: > "Simon Riggs" <[EMAIL PROTECTED]> writes: > > > Proposed changes: > > 1. Add function contains() > > 2. Alter docs to show use of contains() > > > > All other @@ features still the same > > Have you yet given any advantages of conta

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Gregory Stark
"Simon Riggs" <[EMAIL PROTECTED]> writes: > Proposed changes: > 1. Add function contains() > 2. Alter docs to show use of contains() > > All other @@ features still the same Have you yet given any advantages of contains over @@ ? -- Gregory Stark EnterpriseDB http://www.enterprised

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Simon Riggs
On Tue, 2007-11-13 at 08:58 +0100, Pavel Stehule wrote: > On 13/11/2007, Simon Riggs <[EMAIL PROTECTED]> wrote: > > I'm thinking we can have an inlinable function > > > > contains(text, text) returns int > > > > Return values limited to just 0 or 1 or NULL, as with SQL/MM. > > It's close to SQL/MM,

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Webb Sprague
> ... Therefore, we have to tell people > to use some other API anyway. The existing tsearch2 API at least has > the virtue of having been proven in the field over several years. I can only speak as a moderately sophisticated end user, but ... I think the tsearch2 API has been "proven" to alienat

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > The fact that we are having this discussion at all demonstrates to me > that we should leave well alone - any use of LIKE in this context is > just about guaranteed to cause massive confusion. Also, the semantics of LIKE are perfectly clearly specifie

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Andrew Dunstan
Trevor Talbot wrote: On 11/13/07, Peter Eisentraut <[EMAIL PROTECTED]> wrote: Am Dienstag, 13. November 2007 schrieb Gregory Stark: "Peter Eisentraut" <[EMAIL PROTECTED]> writes: What we'd need is a way to convert a LIKE pattern into a tsquery ('%foo%bar%' => 'foo & bar')

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Trevor Talbot
On 11/13/07, Peter Eisentraut <[EMAIL PROTECTED]> wrote: > Am Dienstag, 13. November 2007 schrieb Gregory Stark: > > "Peter Eisentraut" <[EMAIL PROTECTED]> writes: > > > What we'd need is a way to convert a LIKE pattern into a tsquery > > > ('%foo%bar%' => 'foo & bar'). Then you might even be abl

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Simon Riggs
On Tue, 2007-11-13 at 13:32 +0100, Peter Eisentraut wrote: > Am Dienstag, 13. November 2007 schrieb Gregory Stark: > > "Peter Eisentraut" <[EMAIL PROTECTED]> writes: > > > What we'd need is a way to convert a LIKE pattern into a tsquery > > > ('%foo%bar%' => 'foo & bar'). Then you might even be ab

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Gregory Stark
"Bruce Momjian" <[EMAIL PROTECTED]> writes: > I realized this when editing the documentation but not clearly. I > noticed that: > > > http://momjian.us/main/writings/pgsql/sgml/textsearch-intro.html#TEXTSEARCH-MATCHING > > tsvector @@ tsquery > tsquery @@ tsvector > text

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Gregory Stark
"Peter Eisentraut" <[EMAIL PROTECTED]> writes: > Am Dienstag, 13. November 2007 schrieb Gregory Stark: >> "Peter Eisentraut" <[EMAIL PROTECTED]> writes: >> > What we'd need is a way to convert a LIKE pattern into a tsquery >> > ('%foo%bar%' => 'foo & bar'). Then you might even be able to sneak >>

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Peter Eisentraut
Am Dienstag, 13. November 2007 schrieb Gregory Stark: > "Peter Eisentraut" <[EMAIL PROTECTED]> writes: > > What we'd need is a way to convert a LIKE pattern into a tsquery > > ('%foo%bar%' => 'foo & bar'). Then you might even be able to sneak > > index-optimized text search into existing applicati

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Gregory Stark
"Peter Eisentraut" <[EMAIL PROTECTED]> writes: > What we'd need is a way to convert a LIKE pattern into a tsquery ('%foo%bar%' > => 'foo & bar'). Then you might even be able to sneak index-optimized text > search into existing applications. Might be worth a try. I don't think that's the righ

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Peter Eisentraut
Am Montag, 12. November 2007 schrieb Pavel Stehule: > > Huh, what version of the standard is this? My copy (the typical 2003 > > draft) doesn't have SQL/MM AFAICS. > > I found > > http://jtc1sc32.org/doc/N0751-0800/32N0771T.pdf > http://www.sigmod.org/record/issues/0112/standards.pdf > http://dbs.

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Peter Eisentraut
Am Montag, 12. November 2007 schrieb Martijn van Oosterhout: > On Mon, Nov 12, 2007 at 03:44:18PM -0500, Aidan Van Dyk wrote: > > Can LIKE be easily overloaded in the parser? So: > > text LIKE text > > works in it's current form, and > > tsvector LIKE tsquery > > also works like the @@? O

Re: [HACKERS] Simplifying Text Search

2007-11-13 Thread Simon Riggs
On Tue, 2007-11-13 at 08:58 +0100, Pavel Stehule wrote: > On 13/11/2007, Simon Riggs <[EMAIL PROTECTED]> wrote: > > I'm thinking we can have an inlinable function > > > > contains(text, text) returns int > > > > Return values limited to just 0 or 1 or NULL, as with SQL/MM. > > It's close to SQL/MM

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Pavel Stehule
On 13/11/2007, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Mon, 2007-11-12 at 23:03 -0500, Bruce Momjian wrote: > > Simon Riggs wrote: > > > On Mon, 2007-11-12 at 11:56 -0500, Tom Lane wrote: > > > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > > > So we end up with a normal sounding function that

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Simon Riggs
On Mon, 2007-11-12 at 23:03 -0500, Bruce Momjian wrote: > Simon Riggs wrote: > > On Mon, 2007-11-12 at 11:56 -0500, Tom Lane wrote: > > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > > So we end up with a normal sounding function that is overloaded to > > > > provide all of the various goodies. >

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Bruce Momjian
Simon Riggs wrote: > On Mon, 2007-11-12 at 11:56 -0500, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > So we end up with a normal sounding function that is overloaded to > > > provide all of the various goodies. > > > > As best I can tell, @@ does exactly this already. This is

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Pavel Stehule
On 12/11/2007, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > Pavel Stehule escribió: > > Hello > > > > look to standard, please. SQL/MM has part - full text. > > Huh, what version of the standard is this? My copy (the typical 2003 > draft) doesn't have SQL/MM AFAICS. > > I found http://jtc1sc32.or

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Alvaro Herrera
Pavel Stehule escribió: > Hello > > look to standard, please. SQL/MM has part - full text. Huh, what version of the standard is this? My copy (the typical 2003 draft) doesn't have SQL/MM AFAICS. -- Alvaro Herrera http://www.amazon.com/gp/registry/CTMLCN8V17R4 "La espina, desde

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Simon Riggs
On Mon, 2007-11-12 at 21:59 +0100, Pavel Stehule wrote: > SELECT docno > FROM information > WHERE document.CONTAINS > ('STEMMED FORM OF "standard" > IN SAME PARAGRAPH AS > SOUNDS LIKE "sequel"') = 1 > > it's little bit baroque, It's sample of method. Seems thats the way Oracle does it too.

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Martijn van Oosterhout
On Mon, Nov 12, 2007 at 03:44:18PM -0500, Aidan Van Dyk wrote: > Can LIKE be easily overloaded in the parser? So: > text LIKE text > works in it's current form, and > tsvector LIKE tsquery > also works like the @@? Or have I gotten all the ts* types all mixed up > again... AIUI LIKE

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Joshua D. Drake
Simon Riggs wrote: On Mon, 2007-11-12 at 20:17 +, Heikki Linnakangas wrote: Granted, @@ is a bit awkward until you get used to it. "x LIKE y" would read out better, but unfortunately that's already taken ;-). Remember, I'm not suggesting we get rid of @@ In any case, it's way too late.

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Pavel Stehule
Hello look to standard, please. SQL/MM has part - full text. SELECT docno FROM information WHERE document.CONTAINS ('STEMMED FORM OF "standard" IN SAME PARAGRAPH AS SOUNDS LIKE "sequel"') = 1 it's little bit baroque, It's sample of method. So,it can be: SELECT .. FROM x.contains(y); It

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Simon Riggs
On Mon, 2007-11-12 at 20:17 +, Heikki Linnakangas wrote: > Granted, @@ is a bit awkward until you get used to it. "x LIKE y" would > read out better, but unfortunately that's already taken ;-). Remember, I'm not suggesting we get rid of @@ > In any case, it's way too late. I'm suggesting w

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Aidan Van Dyk
* Heikki Linnakangas <[EMAIL PROTECTED]> [071112 15:18]: > Simon Riggs wrote: > >Right now, Full Text Search SQL looks like complete gibberish and it > >dissuades many people from using what is an awesome set of features. I > >just want to add a little sugar to help people get started. > Granted,

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Granted, @@ is a bit awkward until you get used to it. "x LIKE y" would > read out better, but unfortunately that's already taken ;-). Actually LIKE does not make much sense when you have 'hay & needle'. Probably MATCHES would be a better term ... but then, MySQL defi

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Andrew Sullivan
On Mon, Nov 12, 2007 at 08:09:48PM +, Simon Riggs wrote: > > @@ would still exist, so no problems. These additions are for new users, > not old ones. Given that this is all sugar on top of tsearch anyway, why not put it in pgfoundry as the tsearch_sugar project? Then packagers could include

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Heikki Linnakangas
Simon Riggs wrote: On Mon, 2007-11-12 at 11:56 -0500, Tom Lane wrote: Simon Riggs <[EMAIL PROTECTED]> writes: So we end up with a normal sounding function that is overloaded to provide all of the various goodies. As best I can tell, @@ does exactly this already. This is just a different spell

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Simon Riggs
On Mon, 2007-11-12 at 11:56 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > So we end up with a normal sounding function that is overloaded to > > provide all of the various goodies. > > As best I can tell, @@ does exactly this already. This is just a > different spelling of

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > Simon Riggs <[EMAIL PROTECTED]> writes: >> So we end up with a normal sounding function that is overloaded to >> provide all of the various goodies. > > As best I can tell, @@ does exactly this already. This is just a > different spelling of the same capab

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > So we end up with a normal sounding function that is overloaded to > provide all of the various goodies. As best I can tell, @@ does exactly this already. This is just a different spelling of the same capability, and I don't actually find it better. Why

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Heikki Linnakangas
Simon Riggs wrote: Something Tom Dunstan just mentioned has made me ask the question "Why does our full text search feature look so strange?". It's the operator-laden syntax that causes the problem. By any stretch, this query is difficult for most people to understand: SELECT * FROM text_table

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread David Fetter
On Mon, Nov 12, 2007 at 03:48:20PM +, Simon Riggs wrote: > On Mon, 2007-11-12 at 16:28 +0100, Martijn van Oosterhout wrote: > > On Mon, Nov 12, 2007 at 03:00:36PM +, Simon Riggs wrote: > > > Something Tom Dunstan just mentioned has made me ask the question "Why > > > does our full text sear

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Simon Riggs
On Mon, 2007-11-12 at 16:28 +0100, Martijn van Oosterhout wrote: > On Mon, Nov 12, 2007 at 03:00:36PM +, Simon Riggs wrote: > > Something Tom Dunstan just mentioned has made me ask the question "Why > > does our full text search feature look so strange?". It's the > > operator-laden syntax that

Re: [HACKERS] Simplifying Text Search

2007-11-12 Thread Martijn van Oosterhout
On Mon, Nov 12, 2007 at 03:00:36PM +, Simon Riggs wrote: > Something Tom Dunstan just mentioned has made me ask the question "Why > does our full text search feature look so strange?". It's the > operator-laden syntax that causes the problem. > > By any stretch, this query is difficult for mos

[HACKERS] Simplifying Text Search

2007-11-12 Thread Simon Riggs
Something Tom Dunstan just mentioned has made me ask the question "Why does our full text search feature look so strange?". It's the operator-laden syntax that causes the problem. By any stretch, this query is difficult for most people to understand: SELECT * FROM text_table WHERE to_tsvector('fa