Alexander Korotkov writes:
> I found you committed GiST index implementation. That's cool.
> I found an easy way to optimize it. We can also use trigramsMatchGraph for
> signatures. Attached patch contains implementation.
Good idea, committed.
regards, tom lane
--
Sent
On Mon, Apr 15, 2013 at 05:53:41PM +0400, Alexander Korotkov wrote:
> I found you committed GiST index implementation. That's cool.
> I found an easy way to optimize it. We can also use trigramsMatchGraph for
> signatures. Attached patch contains implementation.
> Simple example in order to demonst
I found you committed GiST index implementation. That's cool.
I found an easy way to optimize it. We can also use trigramsMatchGraph for
signatures. Attached patch contains implementation.
Simple example in order to demonstrate it:
Before the patch:
test=# explain (analyze, buffers) select * from
On Tue, Apr 9, 2013 at 9:15 AM, Tom Lane wrote:
> Alexander Korotkov writes:
> > On Mon, Apr 8, 2013 at 9:28 AM, Tom Lane wrote:
> >> I spent the weekend hacking on this, making a number of bug fixes and a
> >> whole lot of cosmetic changes. I think there are large parts of this
> >> that are
Alexander Korotkov writes:
> On Mon, Apr 8, 2013 at 9:28 AM, Tom Lane wrote:
>> I spent the weekend hacking on this, making a number of bug fixes and a
>> whole lot of cosmetic changes. I think there are large parts of this
>> that are in committable shape now, but I still find the actual graph
On Mon, Apr 8, 2013 at 9:28 AM, Tom Lane wrote:
> Alexander Korotkov writes:
> > [ trgm-regexp-0.15.patch.gz ]
>
> I spent the weekend hacking on this, making a number of bug fixes and a
> whole lot of cosmetic changes. I think there are large parts of this
> that are in committable shape now,
Alexander Korotkov writes:
> [ trgm-regexp-0.15.patch.gz ]
I spent the weekend hacking on this, making a number of bug fixes and a
whole lot of cosmetic changes. I think there are large parts of this
that are in committable shape now, but I still find the actual graph
transformation logic to be
On Wed, Apr 3, 2013 at 11:10 AM, Erikjan Rijkers wrote:
> On Tue, April 2, 2013 23:54, Alexander Korotkov wrote:
>
> > [trgm-regexp-0.15.patch.gz]
>
> Yes, it does look good now; Attached a list of measurements. Most of the
> searches that I put in
> that test-program are now speeded up very much
On Tue, April 2, 2013 23:54, Alexander Korotkov wrote:
> [trgm-regexp-0.15.patch.gz]
Yes, it does look good now; Attached a list of measurements. Most of the
searches that I put in
that test-program are now speeded up very much.
There still are a few regressions, for example:
HEAD azj
On Wed, Apr 3, 2013 at 12:36 AM, Erikjan Rijkers wrote:
> On Mon, April 1, 2013 23:15, Alexander Korotkov wrote:
>
> [trgm-regexp-0.14.patch.gz]
>
> Hi Alexander,
>
Hi Erik!
> Something went wrong in this version of the patch: many (most) queries
> that were earlier
> spectacularly fast have b
On Mon, April 1, 2013 23:15, Alexander Korotkov wrote:
[trgm-regexp-0.14.patch.gz]
Hi Alexander,
Something went wrong in this version of the patch: many (most) queries that
were earlier
spectacularly fast have become slow, often slower than a seqscan or only
marginally faster. See
the attached
On Mon, Mar 25, 2013 at 2:38 AM, Tom Lane wrote:
> Alexander Korotkov writes:
> > On Mon, Mar 25, 2013 at 1:50 AM, Tom Lane wrote:
> >> Similarly, pushing PG-specific declarations like RE_compile_and_cache()
> >> into regex/regex.h is completely not the right thing for preserving a
> >> clear l
Alexander Korotkov writes:
> On Mon, Mar 25, 2013 at 1:50 AM, Tom Lane wrote:
>> Similarly, pushing PG-specific declarations like RE_compile_and_cache()
>> into regex/regex.h is completely not the right thing for preserving a
>> clear library boundary (even positing that we want to expose that
>>
On Mon, Mar 25, 2013 at 1:50 AM, Tom Lane wrote:
> Alexander Korotkov writes:
> > Now I have working implemetation of this API. Comments still need rework.
> > Could you give me any feedback?
>
> I looked at this a little bit, but it's not very far along at all
> towards resolving my API worries
Alexander Korotkov writes:
> Now I have working implemetation of this API. Comments still need rework.
> Could you give me any feedback?
I looked at this a little bit, but it's not very far along at all
towards resolving my API worries. The basic point that I'm concerned
about is that we would l
On Thu, Mar 14, 2013 at 9:40 PM, Alexander Korotkov wrote:
> On Wed, Jan 23, 2013 at 7:29 PM, Tom Lane wrote:
>
>> Heikki Linnakangas writes:
>> > On 23.01.2013 09:36, Alexander Korotkov wrote:
>> >> On Wed, Jan 23, 2013 at 6:08 AM, Tom Lane wrote:
>> >>> The biggest problem is that I really do
On Wed, Jan 23, 2013 at 7:29 PM, Tom Lane wrote:
> Heikki Linnakangas writes:
> > On 23.01.2013 09:36, Alexander Korotkov wrote:
> >> On Wed, Jan 23, 2013 at 6:08 AM, Tom Lane wrote:
> >>> The biggest problem is that I really don't care for the idea of
> >>> contrib/pg_trgm being this cozy with
* Alexander Korotkov (aekorot...@gmail.com) wrote:
> Now, we probably don't have enough of time before 9.3 to solve an API
> problem :(. It's likely we have to choose either commit to 9.3 without
> clean API factorization or postpone it to 9.4.
As much as I'd like this to get in, I don't think the
On Wed, Jan 23, 2013 at 7:29 PM, Tom Lane wrote:
> Heikki Linnakangas writes:
> > On 23.01.2013 09:36, Alexander Korotkov wrote:
> >> On Wed, Jan 23, 2013 at 6:08 AM, Tom Lane wrote:
> >>> The biggest problem is that I really don't care for the idea of
> >>> contrib/pg_trgm being this cozy with
On Sun, Jan 27, 2013 at 10:40 PM, Alexander Korotkov
wrote:
> Now I'm working on additional comments.
Some comments were added for addKey and addArc(s). I hope they clarify
something.
--
With best regards,
Alexander Korotkov.
trgm-regexp-0.12.patch.gz
Description: GNU Zip compressed data
On Fri, Jan 25, 2013 at 11:47 AM, Erik Rijkers wrote:
> On Wed, January 23, 2013 08:36, Alexander Korotkov wrote:
> > Hi!
> >
> > Some quick answers to the part of notes/issues. I will provide rest of
> > answers soon.
> >
> [...]
> > trgm-regexp-0.10.patch.gz27 k
>
> Trying to build this I g
On Wed, January 23, 2013 08:36, Alexander Korotkov wrote:
> Hi!
>
> Some quick answers to the part of notes/issues. I will provide rest of
> answers soon.
>
[...]
> trgm-regexp-0.10.patch.gz27 k
Trying to build this I get, after 'make install' in contrib/ :
/usr/bin/install: cannot stat `./pg
Heikki Linnakangas writes:
> On 23.01.2013 09:36, Alexander Korotkov wrote:
>> On Wed, Jan 23, 2013 at 6:08 AM, Tom Lane wrote:
>>> The biggest problem is that I really don't care for the idea of
>>> contrib/pg_trgm being this cozy with the innards of regex_t.
>> The only option I see now is to
On 23.01.2013 09:36, Alexander Korotkov wrote:
Hi!
Some quick answers to the part of notes/issues. I will provide rest of
answers soon.
On Wed, Jan 23, 2013 at 6:08 AM, Tom Lane wrote:
The biggest problem is that I really don't care for the idea of
contrib/pg_trgm being this cozy with the in
Hi!
Some quick answers to the part of notes/issues. I will provide rest of
answers soon.
On Wed, Jan 23, 2013 at 6:08 AM, Tom Lane wrote:
> The biggest problem is that I really don't care for the idea of
> contrib/pg_trgm being this cozy with the innards of regex_t. Sooner
> or later we are go
Heikki Linnakangas writes:
> I finally got around to look at this. I like this new version, without
> the path matrix, much better.
I looked through this version too. I have some notes/issues:
The biggest problem is that I really don't care for the idea of
contrib/pg_trgm being this cozy with
On Tue, Dec 18, 2012 at 12:51 PM, Erik Rijkers wrote:
> On Tue, December 18, 2012 09:45, Alexander Korotkov wrote:
> >
> > You should use {0,n} to express from 0 to n occurences.
> >
>
>
> Thanks, but I know that of course. It's a testing program; and in the end
> robustness with
> unexpected or
On Tue, December 18, 2012 09:45, Alexander Korotkov wrote:
>
> You should use {0,n} to express from 0 to n occurences.
>
Thanks, but I know that of course. It's a testing program; and in the end
robustness with
unexpected or even wrong input is as important as performance. (to put it
bluntly,
On Tue, Dec 18, 2012 at 11:45 AM, Erik Rijkers wrote:
> On Tue, December 18, 2012 08:04, Alexander Korotkov wrote:
> I ran the same test again: HEAD versus trgm_regex v6, 7 and 9. In v9
> there is some gain but also
> some regression.
>
> It remains a difficult problem...
>
> If I get some time
On Tue, December 18, 2012 08:04, Alexander Korotkov wrote:
> trgm-regexp-0.9.patch.gz 22 k
Hi.
I ran the same test again: HEAD versus trgm_regex v6, 7 and 9. In v9 there is
some gain but also
some regression.
It remains a difficult problem...
If I get some time in the holidays I'll try to
On Mon, Dec 17, 2012 at 1:16 PM, Alexander Korotkov wrote:
> Didn't reproduce it yet. Can you retry it with this line uncommented:
> #define TRGM_REGEXP_DEBUG
> Then we can see which stage it fails.
>
Bug is found and fixed in attached patch.
--
With best regards,
Alexander Korotkov.
trgm-
Hi!
On Mon, Dec 17, 2012 at 12:54 PM, Erik Rijkers wrote:
> On Sun, December 16, 2012 22:25, Alexander Korotkov wrote:
>
> > trgm-regexp-0.8.patch.gz 22 k
>
> Hi Alexander,
>
> I gave this a quick try; the patch works when compiled for DEBUG, but
> crashes as a
> 'speed'-compiled binary:
>
> C
On Sun, December 16, 2012 22:25, Alexander Korotkov wrote:
> trgm-regexp-0.8.patch.gz 22 k
Hi Alexander,
I gave this a quick try; the patch works when compiled for DEBUG, but crashes
as a
'speed'-compiled binary:
Compile for speed:
$ pg_config --configure
'--prefix=/home/aardvark/pg_stuff/p
On Fri, Dec 14, 2012 at 1:34 AM, Alexander Korotkov wrote:
> On Mon, Dec 3, 2012 at 4:31 PM, Alexander Korotkov
> wrote:
>
>> Actually, I generally dislike path matrix for same reasons. But:
>> 1) Output graphs could contain trigrams which are completely useless for
>> search. For example, for re
On Mon, Dec 3, 2012 at 4:31 PM, Alexander Korotkov wrote:
> Actually, I generally dislike path matrix for same reasons. But:
> 1) Output graphs could contain trigrams which are completely useless for
> search. For example, for regex /(abcdefgh)*ijk/ we need only "ijk" trigram
> while graph would c
On Mon, Dec 3, 2012 at 2:05 PM, Heikki Linnakangas
wrote:
> On 02.12.2012 20:19, Tom Lane wrote:
>
>> Alexander Korotkov writes:
>>
>>> Nice idea to delay expanding colors to characters! Obviously, we should
>>> delay expanding inly alphanumerical characters. Because
>>> non-alphanumberical
>>> c
On 02.12.2012 20:19, Tom Lane wrote:
Alexander Korotkov writes:
Nice idea to delay expanding colors to characters! Obviously, we should
delay expanding inly alphanumerical characters. Because non-alphanumberical
characters influence graph structure. Trying to implement...
Uh, why would that b
On Sun, December 2, 2012 19:07, Alexander Korotkov wrote:
>>
>> I've attached a similar test re-run that compares HEAD with patch versions
>> 0.6, and 0.7.
>>
>
> Thanks! Did you write scripts for automated testing? I would be nice if you
> share them.
>
Sure, here they are.
The perl program does
Alexander Korotkov writes:
> Nice idea to delay expanding colors to characters! Obviously, we should
> delay expanding inly alphanumerical characters. Because non-alphanumberical
> characters influence graph structure. Trying to implement...
Uh, why would that be? Colors are colors. The regexp
On Sat, Dec 1, 2012 at 3:22 PM, Erik Rijkers wrote:
> On Fri, November 30, 2012 12:22, Alexander Korotkov wrote:
> > Hi!
> >
> > On Thu, Nov 29, 2012 at 12:58 PM, er wrote:
> >
> >> On Mon, November 26, 2012 20:49, Alexander Korotkov wrote:
> >>
> >>
> >> I ran the simple-minded tests against ge
On Fri, Nov 30, 2012 at 6:23 PM, Heikki Linnakangas wrote:
> On 30.11.2012 13:20, Alexander Korotkov wrote:
>
>> On Thu, Nov 29, 2012 at 5:25 PM, Heikki Linnakangas> vmware.com
>>
>>> wrote:
>>>
>>
>> Would it be safe to simply stop short the depth-first search on overflow,
>>> and proceed with
On Fri, November 30, 2012 12:22, Alexander Korotkov wrote:
> Hi!
>
> On Thu, Nov 29, 2012 at 12:58 PM, er wrote:
>
>> On Mon, November 26, 2012 20:49, Alexander Korotkov wrote:
>>
>>
>> I ran the simple-minded tests against generated data (similar to the ones
>> I did in January 2012).
>> The prob
On 30.11.2012 13:20, Alexander Korotkov wrote:
On Thu, Nov 29, 2012 at 5:25 PM, Heikki Linnakangas
wrote:
Would it be safe to simply stop short the depth-first search on overflow,
and proceed with the graph that was constructed up to that point?
For depth-first it's not. But your proposal na
On Fri, Nov 30, 2012 at 3:20 PM, Alexander Korotkov wrote:
> For depth-first it's not.
>
Oh, I didn't explained it.
In order to stop graph processing we need to be sure that we put all
outgoing arcs from state or assume that state to be final. In DFS we can be
in the final part of graph producing
Hi!
On Thu, Nov 29, 2012 at 12:58 PM, er wrote:
> On Mon, November 26, 2012 20:49, Alexander Korotkov wrote:
>
> > trgm-regexp-0.6.patch.gz
>
> I ran the simple-minded tests against generated data (similar to the ones
> I did in January 2012).
> The problems of that older version seem pretty muc
On Thu, Nov 29, 2012 at 5:25 PM, Heikki Linnakangas wrote:
> One thing that bothers me with this algoritm is that the overflow
> mechanism is all-or-nothing. In many cases, even when there is a huge
> number of states in the diagram, you could still extract at least a few
> trigrams that must be
One thing that bothers me with this algoritm is that the overflow
mechanism is all-or-nothing. In many cases, even when there is a huge
number of states in the diagram, you could still extract at least a few
trigrams that must be present in any matching string, with little
effort. At least, it
On Mon, November 26, 2012 20:49, Alexander Korotkov wrote:
> trgm-regexp-0.6.patch.gz
I ran the simple-minded tests against generated data (similar to the ones I did
in January 2012).
The problems of that older version seem pretty much all removed. (although I
didn't do much work
on it -- just
On Mon, Nov 26, 2012 at 4:55 PM, Heikki Linnakangas wrote:
>
> Great, that top-level comment helped tremendously! I feel enlightened.
>
> I fixed some spelling, formatting etc. trivial stuff while reading through
> the patch, see attached. Below is some feedback on the details:
>
> * I don't like
On 25.11.2012 22:55, Alexander Korotkov wrote:
On Tue, Nov 20, 2012 at 1:43 PM, Heikki Linnakangas
wrote:
Glad to see this patch hasn't been totally forgotten. Being able to use
indexes for regular expressions would be really cool!
Back in January, I asked for some high-level description of h
Hi!
On Wed, Nov 21, 2012 at 12:51 AM, Pavel Stehule wrote:
> do you plan to support GiST?
>
At first, I would note that pg_trgm GiST opclass is quite ridiculous for
support regex search (and, actually for LIKE/ILIKE search which is already
implemented too). Because in GiST opclass we store set o
On Tue, Nov 20, 2012 at 1:43 PM, Heikki Linnakangas wrote:
> Glad to see this patch hasn't been totally forgotten. Being able to use
> indexes for regular expressions would be really cool!
>
> Back in January, I asked for some high-level description of how the
> algorithm works (http://archives.p
hello
do you plan to support GiST?
Regards
Pavel
2012/11/20 Alexander Korotkov :
> On Tue, Nov 20, 2012 at 3:02 AM, Tomas Vondra wrote:
>>
>> 2) It's common to use upper-case names for macros, but trgm.h defines
>>macro "iswordchr" - I see it's moved from trgm_op.c but maybe we
>>could
On Tue, Nov 20, 2012 at 3:02 AM, Tomas Vondra wrote:
> 2) It's common to use upper-case names for macros, but trgm.h defines
>macro "iswordchr" - I see it's moved from trgm_op.c but maybe we
>could make it a bit more correct?
>
> 3) I see there are two '#ifdef KEEPONLYALNUM" blocks right
Glad to see this patch hasn't been totally forgotten. Being able to use
indexes for regular expressions would be really cool!
Back in January, I asked for some high-level description of how the
algorithm works
(http://archives.postgresql.org/message-id/4f187d5c.30...@enterprisedb.com).
That's
Some quick comments.
On Tue, Nov 20, 2012 at 3:02 AM, Tomas Vondra wrote:
> 6) It does not compile - I do get a bunch of errors like this
>
Fixed.
7) Once fixed, it seems to work
>
> CREATE EXTENSION pg_trgm ;
> CREATE TABLE TEST (val TEXT);
> INSERT INTO test
>SELECT md5(i::text) FROM
On 19.11.2012 22:58, Alexander Korotkov wrote:
> Hi!
>
> New version of patch is attached. Changes are following:
> 1) Right way to convert from pg_wchar to multibyte.
> 2) Optimization of producing CFNA-like graph on trigrams (produce
> smaller, but equivalent, graphs in less time).
> 3) Comments
On Mon, November 19, 2012 22:58, Alexander Korotkov wrote:
> New version of patch is attached.
Hi Alexander,
I get some compile-errors:
(Centos 6.3, Linux 2.6.32-279.14.1.el6.x86_64 GNU/Linux, gcc (GCC) 4.7.2)
make contrib
trgm_regexp.c:73:2: error: unknown type name TrgmStateKey
make[1]: **
Hi!
New version of patch is attached. Changes are following:
1) Right way to convert from pg_wchar to multibyte.
2) Optimization of producing CFNA-like graph on trigrams (produce smaller,
but equivalent, graphs in less time).
3) Comments and refactoring.
--
With best regards,
Alexander Korotk
On Sat, January 21, 2012 06:26, Alexander Korotkov wrote:
> Hi!
>
> Thank you for your feedback!
>
> On Fri, Jan 20, 2012 at 3:33 AM, Erik Rijkers wrote:
>
>> The patch yields spectacular speedups with small, simple-enough regexen.
>> But it does not do a
>> good enough job when guessing where to
On Fri, Jan 20, 2012 at 12:54 AM, Alexander Korotkov
wrote:
> On Fri, Jan 20, 2012 at 12:30 AM, Heikki Linnakangas <
> heikki.linnakan...@enterprisedb.com> wrote:
>
>> Apart from that, the multibyte issue seems like the big one. Any way
>>> around that?
>>
>> Conversion of pg_wchar to multibyte ch
On Fri, Jan 20, 2012 at 8:45 PM, Marti Raudsepp wrote:
> On Fri, Jan 20, 2012 at 01:33, Erik Rijkers wrote:
> > Btw, it seems impossible to Ctrl-C out of a search once it is submitted;
> I suppose this is
> > normally necessary for perfomance reasons, but it would be useful te be
> able to compi
Hi!
Thank you for your feedback!
On Fri, Jan 20, 2012 at 3:33 AM, Erik Rijkers wrote:
> The patch yields spectacular speedups with small, simple-enough regexen.
> But it does not do a
> good enough job when guessing where to use the index and where fall back
> to Seq Scan. This can
> lead to
On Fri, Jan 20, 2012 at 01:33, Erik Rijkers wrote:
> Btw, it seems impossible to Ctrl-C out of a search once it is submitted; I
> suppose this is
> normally necessary for perfomance reasons, but it would be useful te be able
> to compile a test
> version that allows it.
I believe being interrup
On Thu, January 19, 2012 21:30, Heikki Linnakangas wrote:
> On 22.11.2011 21:38, Alexander Korotkov wrote:
>> WIP patch with index support for regexp search for pg_trgm contrib is
>> attached.
>> In spite of techniques which extracts continuous text parts from regexp,
>> this patch presents techniq
On Fri, Jan 20, 2012 at 1:07 AM, Alexander Korotkov wrote:
> What does last 7 zeros in the first column means? No conversion to
> pg_wchar is possible from these encodings?
>
Uh, I see. These encodings is not supported as server encodings.
--
With best regards,
Alexander Korotkov.
I also have a question about pg_wchar.
/*
*---
* encoding info table
* XXX must be sorted by the same order as enum pg_enc (in mb/pg_wchar.h)
*---
*/
pg_wchar_tbl pg
On Fri, Jan 20, 2012 at 12:30 AM, Heikki Linnakangas <
heikki.linnakan...@enterprisedb.com> wrote:
> The code badly needs comments. There is no explanation of how the trigram
>> extraction code in trgm_regexp.c works.
>
> Sure. I hoped to find a time for comments before commitfest starts.
Unfortun
On 22.11.2011 21:38, Alexander Korotkov wrote:
WIP patch with index support for regexp search for pg_trgm contrib is
attached.
In spite of techniques which extracts continuous text parts from regexp,
this patch presents technique of automatum transformation. That allows more
comprehensive trigram
On Thu, Dec 1, 2011 at 12:29 AM, Robert Haas wrote:
> Please add this patch here so it does not get lost in the shuffle:
>
> https://commitfest.postgresql.org/action/commitfest_view/open
Done.
--
With best regards,
Alexander Korotkov.
On Tue, Nov 22, 2011 at 2:38 PM, Alexander Korotkov
wrote:
> WIP patch with index support for regexp search for pg_trgm contrib is
> attached.
> In spite of techniques which extracts continuous text parts from regexp,
> this patch presents technique of automatum transformation. That allows more
>
Hackers,
WIP patch with index support for regexp search for pg_trgm contrib is
attached.
In spite of techniques which extracts continuous text parts from regexp,
this patch presents technique of automatum transformation. That allows more
comprehensive trigrams extraction.
A little example of poss
72 matches
Mail list logo
