Re: WIP: BRIN multi-range indexes

2021-03-27 Thread Tomas Vondra
On 3/27/21 7:09 PM, Alvaro Herrera wrote: > On 2021-Mar-26, Tomas Vondra wrote: > >> Hi, >> >> I've pushed both the bloom and minmax-multi indexes today. > > One thing I've been wondering all along is how useful are these > BRIN-backed bloom indexes compared to contrib-supplied bloom indexes. > M

Re: WIP: BRIN multi-range indexes

2021-03-27 Thread Alvaro Herrera
On 2021-Mar-26, Tomas Vondra wrote: > Hi, > > I've pushed both the bloom and minmax-multi indexes today. One thing I've been wondering all along is how useful are these BRIN-backed bloom indexes compared to contrib-supplied bloom indexes. My guess is that the BRIN implementation has some advanta

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tomas Vondra
On 3/26/21 3:45 PM, Tomas Vondra wrote: > On 3/26/21 3:04 PM, Tomas Vondra wrote: >> On 3/26/21 2:55 PM, Tom Lane wrote: >>> Tomas Vondra writes: I recall seeing "bus error" on sparc with other patches because of alignment issues, so I wonder if this is what's happening here. >>> >>> Try

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tomas Vondra
On 3/26/21 3:04 PM, Tomas Vondra wrote: > On 3/26/21 2:55 PM, Tom Lane wrote: >> Tomas Vondra writes: >>> I recall seeing "bus error" on sparc with other patches because of >>> alignment issues, so I wonder if this is what's happening here. >> >> Try compiling with the address sanitizer enabled.

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tomas Vondra
On 3/26/21 2:55 PM, Tom Lane wrote: > Tomas Vondra writes: >> I recall seeing "bus error" on sparc with other patches because of >> alignment issues, so I wonder if this is what's happening here. > > Try compiling with the address sanitizer enabled. Per c.h, > > * Testing can be done with "-fs

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tom Lane
Tomas Vondra writes: > I recall seeing "bus error" on sparc with other patches because of > alignment issues, so I wonder if this is what's happening here. Try compiling with the address sanitizer enabled. Per c.h, * Testing can be done with "-fsanitize=alignment -fsanitize-trap=alignment" *

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tomas Vondra
On 3/26/21 2:08 PM, Tomas Vondra wrote: > Hi, > > I've pushed both the bloom and minmax-multi indexes today. > Hmmm, I see a couple buildfarm animals are upset about the minmax-multi patch. It does pass for me both on x86_64 and 32-bit ARM (rpi4), so I'm unable to reproduce this. But most of

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Alvaro Herrera
On 2021-Mar-26, Tomas Vondra wrote: > I've pushed both the bloom and minmax-multi indexes today. Congratulations! I think this reimplementation of the minmax opclass infrastructure makes BRIN much more useful (read: actually usable). -- Álvaro Herrera Valdivia, Chile

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tomas Vondra
Hi, I've pushed both the bloom and minmax-multi indexes today. Based on the feedback and limitations described before I decided to keep them in core (i.e. not move them to contrib), but it was very useful experiment as it uncovered a couple issues - both in existing code, and in the definition of

Re: WIP: BRIN multi-range indexes

2021-03-26 Thread Tomas Vondra
On 3/23/21 7:28 PM, Alvaro Herrera wrote: > On 2021-Mar-23, Tomas Vondra wrote: > >> FWIW there's yet another difference between the current BRIN opclass >> definition, compared to what CREATE OPERATOR CLASS would do. Or more >> precisely, how we'd define opfamily for the cross-type cases (inte

Re: WIP: BRIN multi-range indexes

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-23, Tomas Vondra wrote: > FWIW there's yet another difference between the current BRIN opclass > definition, compared to what CREATE OPERATOR CLASS would do. Or more > precisely, how we'd define opfamily for the cross-type cases (integer, > float and timestamp cases). > > AFAICS we do

Re: WIP: BRIN multi-range indexes

2021-03-23 Thread Tomas Vondra
On 3/23/21 2:36 PM, Alvaro Herrera wrote: > On 2021-Mar-22, Tomas Vondra wrote: > >> I don't know what's the right fix, but it seems like this patch has >> nothing to do with it. If we want to move the opclasses into an >> extension, we can comment out that one (cidr/inet) case for now. > > I d

Re: WIP: BRIN multi-range indexes

2021-03-23 Thread Alvaro Herrera
On 2021-Mar-22, Tomas Vondra wrote: > I don't know what's the right fix, but it seems like this patch has > nothing to do with it. If we want to move the opclasses into an > extension, we can comment out that one (cidr/inet) case for now. I don't know what would be a good reason to define the opc

Re: WIP: BRIN multi-range indexes

2021-03-22 Thread Tomas Vondra
On 3/22/21 6:27 AM, Tomas Vondra wrote: > > ... > > All the regression tests work fine, with the exception of minmax-multi > on a CIDR column. I don't know why, but the CREATE INDEX then fails like > this: > > ERROR: missing operator 1(650,650) in opfamily 16463 > > 650 is cidr, so this essen

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread John Naylor
On Wed, Mar 17, 2021 at 3:16 PM Tomas Vondra wrote: > Ummm, no attachment ;-) Oops, here it is. -- John Naylor EDB: http://www.enterprisedb.com jcn-costing-test.sql Description: Binary data

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread Tomas Vondra
On 3/17/21 7:59 PM, John Naylor wrote: > On Thu, Mar 11, 2021 at 12:26 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: >> >> Hi, >> >> Here is an updated version of the patch series. >> >> It fixes the assert failure (just remove the multiplication from it) and >> adds a simple

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread John Naylor
On Thu, Mar 11, 2021 at 12:26 PM Tomas Vondra wrote: > > Hi, > > Here is an updated version of the patch series. > > It fixes the assert failure (just remove the multiplication from it) and > adds a simple regression test that exercises this. > > Based on the discussion so far, I've decided to kee

Re: WIP: BRIN multi-range indexes

2021-03-09 Thread Tomas Vondra
On 3/9/21 9:51 PM, John Naylor wrote: > > On Sun, Mar 7, 2021 at 8:53 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: > [v20210308b] > > I managed to trap an assertion that somehow doesn't happen during the > regression tests. The callers of fill_expanded_ranges() do math lik

Re: WIP: BRIN multi-range indexes

2021-03-09 Thread John Naylor
On Sun, Mar 7, 2021 at 8:53 PM Tomas Vondra wrote: [v20210308b] I managed to trap an assertion that somehow doesn't happen during the regression tests. The callers of fill_expanded_ranges() do math like this: /* both ranges and points are expanded into a separate element */ neranges = ranges->nr

Re: WIP: BRIN multi-range indexes

2021-03-04 Thread Tomas Vondra
On 3/4/21 5:30 PM, John Naylor wrote: > > On Tue, Mar 2, 2021 at 7:32 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: >> Ummm, in brin_minmax_multi_distance_timetz()? I don't think timetz can >> be infinite, no? I think brin_minmax_multi_distance_date you meant? > > In timestamp

Re: WIP: BRIN multi-range indexes

2021-03-04 Thread John Naylor
On Tue, Mar 2, 2021 at 7:32 PM Tomas Vondra wrote: > Ummm, in brin_minmax_multi_distance_timetz()? I don't think timetz can > be infinite, no? I think brin_minmax_multi_distance_date you meant? In timestamp.c there are plenty of checks for TIMESTAMP_NOT_FINITE for TimestampTz types and I assumed

Re: WIP: BRIN multi-range indexes

2021-03-02 Thread Tomas Vondra
On 3/3/21 1:17 AM, Alvaro Herrera wrote: > On 2021-Mar-03, Tomas Vondra wrote: > >> That's kinda my point - I agree the size of the patch is not the >> primary concern, but it makes the minmax/inclusion code a bit more >> complicated (because they now have to loop over the keys), with >> very l

Re: WIP: BRIN multi-range indexes

2021-03-02 Thread Alvaro Herrera
On 2021-Mar-03, Tomas Vondra wrote: > That's kinda my point - I agree the size of the patch is not the primary > concern, but it makes the minmax/inclusion code a bit more complicated > (because they now have to loop over the keys), with very little benefit > (there might be some speedup, but IMO

Re: WIP: BRIN multi-range indexes

2021-03-02 Thread Tomas Vondra
On 3/3/21 12:44 AM, Alvaro Herrera wrote: > On 2021-Mar-03, Tomas Vondra wrote: > >> 1) The 0001 patch allows passing of all scan keys to BRIN opclasses, >> which is needed for the minmax-multi to work. But it also modifies the >> existing opclasses (minmax and inclusion) to do this - but for thos

Re: WIP: BRIN multi-range indexes

2021-03-02 Thread Alvaro Herrera
On 2021-Mar-03, Tomas Vondra wrote: > 1) The 0001 patch allows passing of all scan keys to BRIN opclasses, > which is needed for the minmax-multi to work. But it also modifies the > existing opclasses (minmax and inclusion) to do this - but for those > opclasses it does not make much difference, I

Re: WIP: BRIN multi-range indexes

2021-02-22 Thread John Naylor
On Mon, Feb 15, 2021 at 10:37 AM Tomas Vondra wrote: > [v20210215] Hi Tomas, This time I only looked at the cumulative changes in the multiminmax opclass, since I'm pretty sure all the bloom issues have been addressed. * XXX CombineRange name seems a bit weird. Consider renaming, perhaps to *

Re: WIP: BRIN multi-range indexes

2021-02-09 Thread Tomas Vondra
On 2/9/21 3:46 PM, John Naylor wrote: On Wed, Feb 3, 2021 at 7:54 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > > [v-20210203] Hi Tomas, I have some random comments from reading the patch, but haven't gone into detail in the newer aspects. I'll do so in the near future

Re: WIP: BRIN multi-range indexes

2021-02-09 Thread John Naylor
On Wed, Feb 3, 2021 at 7:54 PM Tomas Vondra wrote: > > [v-20210203] Hi Tomas, I have some random comments from reading the patch, but haven't gone into detail in the newer aspects. I'll do so in the near future. The cfbot seems to crash on this patch during make check, but it doesn't crash for

Re: WIP: BRIN multi-range indexes

2021-02-03 Thread Zhihong Yu
Hi, bq. Not sure why would that be an issue Moving the (start > end) check is up to your discretion. But the midpoint computation should follow text book :-) Cheers On Wed, Feb 3, 2021 at 4:59 PM Tomas Vondra wrote: > > > On 2/4/21 1:49 AM, Zhihong Yu wrote: > > Hi, > > For 0007-Remove-the-sp

Re: WIP: BRIN multi-range indexes

2021-02-03 Thread Tomas Vondra
On 2/4/21 1:49 AM, Zhihong Yu wrote: > Hi, > For 0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch : > > +       /* same as preceding value, so store it */ > +       if (compare_values(&range->values[start + i - 1], > +                          &range->values[start + i], > +      

Re: WIP: BRIN multi-range indexes

2021-02-03 Thread Zhihong Yu
Hi, For 0007-Remove-the-special-batch-mode-use-a-larger--20210203.patch : + /* same as preceding value, so store it */ + if (compare_values(&range->values[start + i - 1], + &range->values[start + i], + (void *) &cxt) == 0) + c

Re: WIP: BRIN multi-range indexes

2021-01-30 Thread John Naylor
On Tue, Jan 26, 2021 at 6:59 PM Tomas Vondra wrote: > > > > On 1/26/21 7:52 PM, John Naylor wrote: > > On Fri, Jan 22, 2021 at 10:59 PM Tomas Vondra > > mailto:tomas.von...@enterprisedb.com>> > > wrote: > > > Hmm. I think Alvaro also mentioned he'd like to use this as a drop-in > > > replacement

Re: WIP: BRIN multi-range indexes

2021-01-26 Thread Tomas Vondra
On 1/26/21 7:52 PM, John Naylor wrote: On Fri, Jan 22, 2021 at 10:59 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > > > On 1/23/21 12:27 AM, John Naylor wrote: > > Still, it would be great if multi-minmax can be a drop in replacement. I > > know there was a sticking

Re: WIP: BRIN multi-range indexes

2021-01-26 Thread John Naylor
On Fri, Jan 22, 2021 at 10:59 PM Tomas Vondra wrote: > > > On 1/23/21 12:27 AM, John Naylor wrote: > > Still, it would be great if multi-minmax can be a drop in replacement. I > > know there was a sticking point of a distance function not being > > available on all types, but I wonder if that can

Re: WIP: BRIN multi-range indexes

2021-01-26 Thread John Naylor
Hi Tomas, I took another look through the Bloom opclass portion (0004) with sorted_mode omitted, and it looks good to me code-wise. I think this part is close to commit-ready. I also did one more proofreading pass for minor details. + rows per block). The default values is -0.1, and + gre

Re: WIP: BRIN multi-range indexes

2021-01-22 Thread Tomas Vondra
On 1/23/21 12:27 AM, John Naylor wrote: On Thu, Jan 21, 2021 at 9:06 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > [wip optimizations] > The pathological case (monotonic-asc) is now 4x faster, roughly equal to > regular minmax index build. The opposite (monotonic-desc) i

Re: WIP: BRIN multi-range indexes

2021-01-22 Thread John Naylor
On Thu, Jan 21, 2021 at 9:06 PM Tomas Vondra wrote: > [wip optimizations] > The pathological case (monotonic-asc) is now 4x faster, roughly equal to > regular minmax index build. The opposite (monotonic-desc) is about 33% > faster, roughly in line with btree. Those numbers look good. I get simil

Re: WIP: BRIN multi-range indexes

2021-01-19 Thread Tomas Vondra
On 1/19/21 9:44 PM, John Naylor wrote: On Tue, Jan 12, 2021 at 1:42 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > I suspect it'd due to minmax having to decide which "ranges" to merge, > which requires repeated sorting, etc. I certainly don't dare to claim > the current alg

Re: WIP: BRIN multi-range indexes

2021-01-19 Thread John Naylor
On Tue, Jan 12, 2021 at 1:42 PM Tomas Vondra wrote: > I suspect it'd due to minmax having to decide which "ranges" to merge, > which requires repeated sorting, etc. I certainly don't dare to claim > the current algorithm is perfect. I wouldn't have expected such big > difference, though - so defin

Re: WIP: BRIN multi-range indexes

2021-01-12 Thread Tomas Vondra
On 1/12/21 6:28 PM, John Naylor wrote: On Sat, Dec 19, 2020 at 8:15 PM Tomas Vondra mailto:tomas.von...@enterprisedb.com>> wrote: > [12-20 version] Hi Tomas, The measurements look good. In case it fell through the cracks, my earlier review comments for Bloom BRIN indexes regarding minor

Re: WIP: BRIN multi-range indexes

2021-01-12 Thread John Naylor
On Sat, Dec 19, 2020 at 8:15 PM Tomas Vondra wrote: > [12-20 version] Hi Tomas, The measurements look good. In case it fell through the cracks, my earlier review comments for Bloom BRIN indexes regarding minor details don't seem to have been addressed in this version. I'll point to earlier discu

Re: WIP: BRIN multi-range indexes

2021-01-02 Thread Tomas Vondra
Hi, On 1/2/21 7:42 AM, Thomas Munro wrote: > On Sun, Dec 20, 2020 at 1:16 PM Tomas Vondra > wrote: >> Attached is an updated version of the patch series, rebased on current >> master, and results for benchmark comparing the various bloom variants. > > Perhaps src/include/utils/inet.h needs to in

Re: WIP: BRIN multi-range indexes

2021-01-01 Thread Thomas Munro
On Sun, Dec 20, 2020 at 1:16 PM Tomas Vondra wrote: > Attached is an updated version of the patch series, rebased on current > master, and results for benchmark comparing the various bloom variants. Perhaps src/include/utils/inet.h needs to include , because FreeBSD says: brin_minmax_multi.c:169

Re: WIP: BRIN multi-range indexes

2020-11-09 Thread John Naylor
On Mon, Nov 9, 2020 at 1:39 PM Tomas Vondra wrote: > > > While investigating the failures, I've tried increasing the values a > lot, without observing any measurable increase in runtime. IIRC I've > even used (10 * target_partlen) or something like that. That tells me > it's not very sensitive par

Re: WIP: BRIN multi-range indexes

2020-11-09 Thread Tomas Vondra
On 11/9/20 3:29 PM, John Naylor wrote: > On Sat, Nov 7, 2020 at 4:38 PM Tomas Vondra > mailto:tomas.von...@enterprisedb.com>> > wrote: > >> Overall, I think there's very little difference, particularly in the >> "match" cases when we're searching for a value that we know is in the >> table. The

Re: WIP: BRIN multi-range indexes

2020-11-09 Thread John Naylor
On Sat, Nov 7, 2020 at 4:38 PM Tomas Vondra wrote: > Overall, I think there's very little difference, particularly in the > "match" cases when we're searching for a value that we know is in the > table. The one-hash variant seems to perform a bit better, but the > difference is fairly small. > >

Re: WIP: BRIN multi-range indexes

2020-11-02 Thread Anastasia Lubennikova
Status update for a commitfest entry. According to cfbot the patch no longer compiles. Tomas, can you send an update, please? I also see that a few last messages mention a data corruption bug. Sounds pretty serious. Alvaro, have you had a chance to look at it? I don't see anything committed yet

Re: WIP: BRIN multi-range indexes

2020-10-01 Thread Alvaro Herrera
On 2020-Oct-01, Tomas Vondra wrote: > OK, so this seems like a data corruption bug in BRIN, actually. Oh crap. You're right -- the data needs to be detoasted before being put in the index. I'll have a look at how this can be fixed.

Re: WIP: BRIN multi-range indexes

2020-10-01 Thread Tomas Vondra
On Wed, Sep 30, 2020 at 07:57:19AM -0400, John Naylor wrote: On Mon, Sep 28, 2020 at 10:12 PM Tomas Vondra wrote: Is it actually all that different from the existing BRIN indexes? Consider this example: create table x (a text, b text, c text); create index on x using brin (a,b,c); create or

Re: WIP: BRIN multi-range indexes

2020-09-30 Thread John Naylor
On Mon, Sep 28, 2020 at 10:12 PM Tomas Vondra wrote: > Is it actually all that different from the existing BRIN indexes? > Consider this example: > > create table x (a text, b text, c text); > > create index on x using brin (a,b,c); > > create or replace function random_str(p_len int) returns tex

Re: WIP: BRIN multi-range indexes

2020-09-28 Thread Tomas Vondra
On Mon, Sep 28, 2020 at 04:42:39PM -0400, John Naylor wrote: On Thu, Sep 24, 2020 at 7:50 PM Tomas Vondra wrote: On Thu, Sep 24, 2020 at 05:18:03PM -0400, John Naylor wrote: >Hmm, how ugly would it be to change the default range size depending >on the opclass? > Not sure. What would happen

Re: WIP: BRIN multi-range indexes

2020-09-28 Thread John Naylor
On Thu, Sep 24, 2020 at 7:50 PM Tomas Vondra wrote: > > On Thu, Sep 24, 2020 at 05:18:03PM -0400, John Naylor wrote: > >Hmm, how ugly would it be to change the default range size depending > >on the opclass? > > > > Not sure. What would happen for multi-column BRIN indexes with different > opclas

Re: WIP: BRIN multi-range indexes

2020-09-24 Thread Tomas Vondra
On Thu, Sep 24, 2020 at 05:18:03PM -0400, John Naylor wrote: On Mon, Sep 21, 2020 at 3:56 PM Tomas Vondra wrote: On Mon, Sep 21, 2020 at 01:42:34PM -0400, John Naylor wrote: >While playing around with the numbers I had an epiphany: At the >defaults, the filter already takes up ~4.3kB, over

Re: WIP: BRIN multi-range indexes

2020-09-24 Thread John Naylor
On Mon, Sep 21, 2020 at 3:56 PM Tomas Vondra wrote: > > On Mon, Sep 21, 2020 at 01:42:34PM -0400, John Naylor wrote: > >While playing around with the numbers I had an epiphany: At the > >defaults, the filter already takes up ~4.3kB, over half the page. > >There is no room for another tuple, so if

Re: WIP: BRIN multi-range indexes

2020-09-21 Thread Tomas Vondra
On Mon, Sep 21, 2020 at 01:42:34PM -0400, John Naylor wrote: On Fri, Sep 18, 2020 at 6:27 PM Tomas Vondra wrote: But maybe we could still use this scheme by actually computing h1 = hash_uint32_extended(value, seed1); h2 = hash_uint32_extended(value, seed2); and then use this as the i

Re: WIP: BRIN multi-range indexes

2020-09-21 Thread John Naylor
On Fri, Sep 18, 2020 at 6:27 PM Tomas Vondra wrote: > But maybe we could still use this scheme by actually computing > > h1 = hash_uint32_extended(value, seed1); > h2 = hash_uint32_extended(value, seed2); > > and then use this as the independent hash functions. I think that would > meet t

Re: WIP: BRIN multi-range indexes

2020-09-18 Thread Tomas Vondra
On Fri, Sep 18, 2020 at 05:06:49PM -0400, John Naylor wrote: On Fri, Sep 18, 2020 at 7:40 AM Tomas Vondra wrote: On Thu, Sep 17, 2020 at 06:48:11PM -0400, John Naylor wrote: >I wrote: > >> Hmm, I came across that paper while doing background reading. Okay, >> now I get that "% (filter->nbits -

Re: WIP: BRIN multi-range indexes

2020-09-18 Thread John Naylor
On Fri, Sep 18, 2020 at 7:40 AM Tomas Vondra wrote: > > On Thu, Sep 17, 2020 at 06:48:11PM -0400, John Naylor wrote: > >I wrote: > > > >> Hmm, I came across that paper while doing background reading. Okay, > >> now I get that "% (filter->nbits - 1)" is the second hash function in > >> that scheme.

Re: WIP: BRIN multi-range indexes

2020-09-18 Thread Tomas Vondra
On Thu, Sep 17, 2020 at 06:48:11PM -0400, John Naylor wrote: I wrote: Hmm, I came across that paper while doing background reading. Okay, now I get that "% (filter->nbits - 1)" is the second hash function in that scheme. But now I wonder if that second function should actually act on the passed

Re: WIP: BRIN multi-range indexes

2020-09-18 Thread Tomas Vondra
On Thu, Sep 17, 2020 at 05:42:59PM -0400, John Naylor wrote: On Thu, Sep 17, 2020 at 12:34 PM Tomas Vondra wrote: On Thu, Sep 17, 2020 at 10:33:06AM -0400, John Naylor wrote: >On Sun, Sep 13, 2020 at 12:40 PM Tomas Vondra > wrote: >> <20200913 patch set> But those are opclass parameters, so th

Re: WIP: BRIN multi-range indexes

2020-09-17 Thread John Naylor
I wrote: > Hmm, I came across that paper while doing background reading. Okay, > now I get that "% (filter->nbits - 1)" is the second hash function in > that scheme. But now I wonder if that second function should actually > act on the passed "value" (the original hash), so that they are > actuall

Re: WIP: BRIN multi-range indexes

2020-09-17 Thread John Naylor
On Thu, Sep 17, 2020 at 12:34 PM Tomas Vondra wrote: > > On Thu, Sep 17, 2020 at 10:33:06AM -0400, John Naylor wrote: > >On Sun, Sep 13, 2020 at 12:40 PM Tomas Vondra > > wrote: > >> <20200913 patch set> > But those are opclass parameters, so the parameters are not specified in > WITH clause, but

Re: WIP: BRIN multi-range indexes

2020-09-17 Thread John Naylor
On Sun, Sep 13, 2020 at 12:40 PM Tomas Vondra wrote: > <20200913 patch set> Hi Tomas, The cfbot fails to apply this, but with 0001 from 0912 it works on my end, so going with that. One problem I have is I don't get success with the new reloptions: create index cd_multi on iot using brin(create

Re: WIP: BRIN multi-range indexes

2020-09-11 Thread John Naylor
On Fri, Sep 11, 2020 at 2:05 PM Tomas Vondra wrote: > that bad. But yeah, testing and benchmarking it would be nice. Do you > plan to test just the minmax-multi opclass, or will you look at the > bloom one too? Yeah, I'll start looking at bloom next week, and I'll include it when I do perf testin

Re: WIP: BRIN multi-range indexes

2020-09-11 Thread John Naylor
On Fri, Sep 11, 2020 at 6:14 AM Tomas Vondra wrote: > I understand. I just feel a bit uneasy about replacing an index with > something that may or may not be better for a certain use case. I mean, > if you have data set for which regular minmax works fine, wouldn't you > be annoyed if we just swi

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread John Naylor
Ok, here's an attempt at a somewhat more natural test, to see what happens after bulk updates and deletes, followed by more inserts. The short version is that multi-minmax is resilient to a change that causes a 4x degradation for simple minmax. shared_buffers = 1GB random_page_cost = 1.1 effectiv

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread Alvaro Herrera
On 2020-Sep-10, Tomas Vondra wrote: > I've spent a bit of time experimenting with this. My idea was to allow > keeping an "expanded" version of the summary somewhere. As the addValue > function only receives BrinValues I guess one option would be to just > add bv_mem_values field. Or do you have a

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread Tomas Vondra
On Wed, Sep 09, 2020 at 10:26:00PM +0200, Tomas Vondra wrote: On Wed, Sep 09, 2020 at 04:53:30PM -0300, Alvaro Herrera wrote: On 2020-Sep-09, Tomas Vondra wrote: There are some minor optimizations possible - for example I noticed we call minmax_multi_get_strategy_procinfo often because it happ

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread Tomas Vondra
On Wed, Sep 09, 2020 at 04:53:30PM -0300, Alvaro Herrera wrote: On 2020-Sep-09, Tomas Vondra wrote: There are some minor optimizations possible - for example I noticed we call minmax_multi_get_strategy_procinfo often because it happens in a loop, and we could easily do it just once. But that sa

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread Alvaro Herrera
On 2020-Sep-09, Tomas Vondra wrote: > There are some minor optimizations possible - for example I noticed we > call minmax_multi_get_strategy_procinfo often because it happens in a > loop, and we could easily do it just once. But that saves only about 10% > or so, it's not a ground-breaking optimi

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread Tomas Vondra
On Wed, Sep 09, 2020 at 03:40:41PM -0300, Alvaro Herrera wrote: On 2020-Sep-09, John Naylor wrote: create index on t using brin (a); CREATE INDEX Time: 1631.452 ms (00:01.631) create index on t using brin (a int8_minmax_multi_ops); CREATE INDEX Time: 6521.026 ms (00:06.521) It seems strang

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread Alvaro Herrera
On 2020-Sep-09, John Naylor wrote: > create index on t using brin (a); > CREATE INDEX > Time: 1631.452 ms (00:01.631) > create index on t using brin (a int8_minmax_multi_ops); > CREATE INDEX > Time: 6521.026 ms (00:06.521) It seems strange that the multi-minmax index takes so much longer to buil

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread Tomas Vondra
On Wed, Sep 09, 2020 at 12:04:28PM -0400, John Naylor wrote: On Sat, Sep 5, 2020 at 7:21 PM Tomas Vondra wrote: OK, here is a rebased version. Most of the breakage was due to changes to the BRIN sgml docs. Hi Tomas, I plan on trying some different queries on different data distributions to

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread John Naylor
On Sat, Sep 5, 2020 at 7:21 PM Tomas Vondra wrote: > > OK, here is a rebased version. Most of the breakage was due to changes > to the BRIN sgml docs. Hi Tomas, I plan on trying some different queries on different data distributions to get a sense of when the planner chooses a multi-minmax index

Re: WIP: BRIN multi-range indexes

2020-09-04 Thread Michael Paquier
On Fri, Aug 07, 2020 at 06:27:01PM +0200, Tomas Vondra wrote: > Attached is an updated version of the patch series, implementing this. > Adding the extra data types was fairly simple, because both bloom and > minmax-multi indexes already used "struct as varlena" approach, so all > that needed was a

Re: WIP: BRIN multi-range indexes

2020-08-04 Thread Tomas Vondra
On Tue, Aug 04, 2020 at 05:36:51PM +0300, Alexander Korotkov wrote: Hi, Tomas! Sorry for the late reply. On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra wrote: I think there's a number of weak points in this approach. Firstly, it assumes the summaries can be represented as arrays of built-in ty

Re: WIP: BRIN multi-range indexes

2020-08-04 Thread Alexander Korotkov
Hi, Tomas! Sorry for the late reply. On Sun, Jul 19, 2020 at 6:19 PM Tomas Vondra wrote: > I think there's a number of weak points in this approach. > > Firstly, it assumes the summaries can be represented as arrays of > built-in types, which I'm not really sure about. It clearly is not true > f

Re: WIP: BRIN multi-range indexes

2020-07-19 Thread Tomas Vondra
On Wed, Jul 15, 2020 at 05:34:05AM +0300, Alexander Korotkov wrote: On Mon, Jul 13, 2020 at 5:59 PM Tomas Vondra wrote: >> > If we really want to print something nicer, I'd say it needs to be a >> > special function in the BRIN opclass. >> >> If that can be done, then +1. We just need to ensur

Re: WIP: BRIN multi-range indexes

2020-07-14 Thread Alexander Korotkov
On Mon, Jul 13, 2020 at 5:59 PM Tomas Vondra wrote: > >> > If we really want to print something nicer, I'd say it needs to be a > >> > special function in the BRIN opclass. > >> > >> If that can be done, then +1. We just need to ensure that the function > >> knows and can verify the type of index

Re: WIP: BRIN multi-range indexes

2020-07-13 Thread Tomas Vondra
On Mon, Jul 13, 2020 at 02:54:56PM +0900, Masahiko Sawada wrote: On Mon, 13 Jul 2020 at 09:33, Alvaro Herrera wrote: On 2020-Jul-13, Tomas Vondra wrote: > On Sun, Jul 12, 2020 at 07:58:54PM -0400, Alvaro Herrera wrote: > > Maybe we can try to handle this with some other function that interpr

Re: WIP: BRIN multi-range indexes

2020-07-12 Thread Masahiko Sawada
On Mon, 13 Jul 2020 at 09:33, Alvaro Herrera wrote: > > On 2020-Jul-13, Tomas Vondra wrote: > > > On Sun, Jul 12, 2020 at 07:58:54PM -0400, Alvaro Herrera wrote: > > > > Maybe we can try to handle this with some other function that interprets > > > the bytea in 'value' and returns a user-readable

Re: WIP: BRIN multi-range indexes

2020-07-12 Thread Alvaro Herrera
On 2020-Jul-13, Tomas Vondra wrote: > On Sun, Jul 12, 2020 at 07:58:54PM -0400, Alvaro Herrera wrote: > > Maybe we can try to handle this with some other function that interprets > > the bytea in 'value' and returns a user-readable text. I think it'd > > have to be a superuser-only function, bec

Re: WIP: BRIN multi-range indexes

2020-07-12 Thread Tomas Vondra
On Sun, Jul 12, 2020 at 07:58:54PM -0400, Alvaro Herrera wrote: On 2020-Jul-10, Tomas Vondra wrote: > postgres(1:12801)=# select * from brin_page_items(get_raw_page('mul', > 2), 'mul'); > -[ RECORD 1 ]-

Re: WIP: BRIN multi-range indexes

2020-07-12 Thread Alvaro Herrera
On 2020-Jul-10, Tomas Vondra wrote: > > postgres(1:12801)=# select * from brin_page_items(get_raw_page('mul', > > 2), 'mul'); > > -[ RECORD 1 > > ]-- > > ---

Re: WIP: BRIN multi-range indexes

2020-07-12 Thread Sascha Kuhl
Thanks, I see there is some understanding, though. Tomas Vondra schrieb am So., 12. Juli 2020, 01:30: > On Sat, Jul 11, 2020 at 03:32:43PM +0200, Sascha Kuhl wrote: > >Tomas Vondra schrieb am Sa., 11. Juli > 2020, > >13:24: > > > >> On Fri, Jul 10, 2020 at 04:44:41PM +0200, Sascha Kuhl wrote: >

Re: WIP: BRIN multi-range indexes

2020-07-11 Thread Tomas Vondra
On Sat, Jul 11, 2020 at 03:32:43PM +0200, Sascha Kuhl wrote: Tomas Vondra schrieb am Sa., 11. Juli 2020, 13:24: On Fri, Jul 10, 2020 at 04:44:41PM +0200, Sascha Kuhl wrote: >Tomas Vondra schrieb am Fr., 10. Juli 2020, >14:09: > >> On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrot

Re: WIP: BRIN multi-range indexes

2020-07-11 Thread Sascha Kuhl
Sorry, my topic is different Sascha Kuhl schrieb am Sa., 11. Juli 2020, 15:32: > > > > Tomas Vondra schrieb am Sa., 11. Juli > 2020, 13:24: > >> On Fri, Jul 10, 2020 at 04:44:41PM +0200, Sascha Kuhl wrote: >> >Tomas Vondra schrieb am Fr., 10. Juli >> 2020, >> >14:09: >> > >> >> On Fri, Jul 10,

Re: WIP: BRIN multi-range indexes

2020-07-11 Thread Sascha Kuhl
Tomas Vondra schrieb am Sa., 11. Juli 2020, 13:24: > On Fri, Jul 10, 2020 at 04:44:41PM +0200, Sascha Kuhl wrote: > >Tomas Vondra schrieb am Fr., 10. Juli > 2020, > >14:09: > > > >> On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrote: > >> >On Fri, 3 Jul 2020 at 09:58, Tomas Vondra <

Re: WIP: BRIN multi-range indexes

2020-07-11 Thread Tomas Vondra
On Fri, Jul 10, 2020 at 04:44:41PM +0200, Sascha Kuhl wrote: Tomas Vondra schrieb am Fr., 10. Juli 2020, 14:09: On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrote: >On Fri, 3 Jul 2020 at 09:58, Tomas Vondra wrote: >> >> On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov

Re: WIP: BRIN multi-range indexes

2020-07-10 Thread Sascha Kuhl
Tomas Vondra schrieb am Fr., 10. Juli 2020, 14:09: > On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrote: > >On Fri, 3 Jul 2020 at 09:58, Tomas Vondra > wrote: > >> > >> On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov wrote: > >> >On Sun, Apr 5, 2020 at 8:00 PM Tomas Von

Re: WIP: BRIN multi-range indexes

2020-07-10 Thread Tomas Vondra
On Fri, Jul 10, 2020 at 06:01:58PM +0900, Masahiko Sawada wrote: On Fri, 3 Jul 2020 at 09:58, Tomas Vondra wrote: On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov wrote: >On Sun, Apr 5, 2020 at 8:00 PM Tomas Vondra > wrote: ... >> > >> >Assuming we're not going to get 0001-0003 int

Re: WIP: BRIN multi-range indexes

2020-07-10 Thread Masahiko Sawada
On Fri, 3 Jul 2020 at 09:58, Tomas Vondra wrote: > > On Sun, Apr 05, 2020 at 08:01:50PM +0300, Alexander Korotkov wrote: > >On Sun, Apr 5, 2020 at 8:00 PM Tomas Vondra > > wrote: > ... > >> > > >> >Assuming we're not going to get 0001-0003 into v13, I'm not so > >> >inclined to rush on these three

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Alexander Korotkov
On Sun, Apr 5, 2020 at 8:00 PM Tomas Vondra wrote: > On Sun, Apr 05, 2020 at 07:33:40PM +0300, Alexander Korotkov wrote: > >On Sun, Apr 5, 2020 at 6:53 PM Tomas Vondra > > wrote: > >> On Sun, Apr 05, 2020 at 06:29:15PM +0300, Alexander Korotkov wrote: > >> >On Thu, Apr 2, 2020 at 5:29 AM Tomas Von

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Tomas Vondra
On Sun, Apr 05, 2020 at 07:33:40PM +0300, Alexander Korotkov wrote: On Sun, Apr 5, 2020 at 6:53 PM Tomas Vondra wrote: On Sun, Apr 05, 2020 at 06:29:15PM +0300, Alexander Korotkov wrote: >On Thu, Apr 2, 2020 at 5:29 AM Tomas Vondra > wrote: >> On Sun, Dec 01, 2019 at 10:55:02AM +0900, Michael P

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Alexander Korotkov
On Sun, Apr 5, 2020 at 6:53 PM Tomas Vondra wrote: > On Sun, Apr 05, 2020 at 06:29:15PM +0300, Alexander Korotkov wrote: > >On Thu, Apr 2, 2020 at 5:29 AM Tomas Vondra > > wrote: > >> On Sun, Dec 01, 2019 at 10:55:02AM +0900, Michael Paquier wrote: > >> >On Thu, Sep 26, 2019 at 09:01:48PM +0200, T

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Alexander Korotkov
On Sun, Apr 5, 2020 at 6:51 PM Tom Lane wrote: > Alexander Korotkov writes: > > I'd like to give this patchset a chance for v13. I'm going to make > > another pass trough this patchset. If I wouldn't find serious issues, > > I'm going to commit it. Any objections? > > I think it is way too lat

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Tomas Vondra
On Sun, Apr 05, 2020 at 06:29:15PM +0300, Alexander Korotkov wrote: Hi! On Thu, Apr 2, 2020 at 5:29 AM Tomas Vondra wrote: On Sun, Dec 01, 2019 at 10:55:02AM +0900, Michael Paquier wrote: >On Thu, Sep 26, 2019 at 09:01:48PM +0200, Tomas Vondra wrote: >> Yeah, the opclass params patches got bro

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Tom Lane
Alexander Korotkov writes: > I'd like to give this patchset a chance for v13. I'm going to make > another pass trough this patchset. If I wouldn't find serious issues, > I'm going to commit it. Any objections? I think it is way too late to be reviving major features that nobody has been lookin

Re: WIP: BRIN multi-range indexes

2020-04-05 Thread Alexander Korotkov
Hi! On Thu, Apr 2, 2020 at 5:29 AM Tomas Vondra wrote: > On Sun, Dec 01, 2019 at 10:55:02AM +0900, Michael Paquier wrote: > >On Thu, Sep 26, 2019 at 09:01:48PM +0200, Tomas Vondra wrote: > >> Yeah, the opclass params patches got broken by 773df883e adding enum > >> reloptions. The breakage is som

Re: WIP: BRIN multi-range indexes

2019-11-30 Thread Michael Paquier
On Thu, Sep 26, 2019 at 09:01:48PM +0200, Tomas Vondra wrote: > Yeah, the opclass params patches got broken by 773df883e adding enum > reloptions. The breakage is somewhat extensive so I'll leave it up to > Nikita to fix it in [1]. Until that happens, apply the patches on > top of caba97a9d9 for re

  1   2   >