Re: Yet another fast GiST build

> 17 нояб. 2021 г., в 16:33, Daniel Gustafsson написал(а): > >> On 5 Jul 2021, at 08:27, Emre Hasegeli wrote: > >> ... >> >> I couldn't understand patch number 2 "Remove DEBUG1 verification". It >> seems like something rather useful. Emre, thanks for the review! And sorry for this delay.

Re: Yet another fast GiST build

> On 5 Jul 2021, at 08:27, Emre Hasegeli wrote: > ... > > I couldn't understand patch number 2 "Remove DEBUG1 verification". It > seems like something rather useful. These questions have gone unanswered since July, and the patch fails to apply anymore. Is there an updated version on the way?

Re: Yet another fast GiST build

I tried reviewing the remaining patches. It seems to work correctly, and passes the tests on my laptop. > In this pattern I flipped PointerGetDatum(a) to PointerGetDatum(ra.lower), > because it seems to me correct. I've followed rule of thumb: every sort > function must extract and use "lower"

Re: Yet another fast GiST build

On 07/04/2021 22:41, Andrey Borodin wrote: I see there is a problem with "SET client_min_messages = DEBUG1;" on buildfarm. I think these checks were necessary to make sure test paths are triggered. When we know that code paths are tested, maybe we can omit checks? Yeah. We don't have very

Re: Yet another fast GiST build

> 7 апр. 2021 г., в 16:18, Heikki Linnakangas написал(а): > I see there is a problem with "SET client_min_messages = DEBUG1;" on buildfarm. I think these checks were necessary to make sure test paths are triggered. When we know that code paths are tested, maybe we can omit checks? Best

Re: Yet another fast GiST build

On 07/04/2021 15:12, Andrey Borodin wrote: 7 апр. 2021 г., в 14:56, Heikki Linnakangas написал(а): Ok, I think I understand that now. In btree_gist, the *_cmp() function operates on non-leaf values, and *_lt(), *_gt() et al operate on leaf values. For all other datatypes, the leaf and non-leaf

Re: Yet another fast GiST build

> 7 апр. 2021 г., в 13:23, Heikki Linnakangas написал(а): > > Committed with small fixes. Thanks! > 7 апр. 2021 г., в 14:56, Heikki Linnakangas написал(а): > > Ok, I think I understand that now. In btree_gist, the *_cmp() function > operates on non-leaf values, and *_lt(), *_gt() et al

Re: Yet another fast GiST build

On 07/04/2021 09:00, Heikki Linnakangas wrote: On 08/03/2021 19:06, Andrey Borodin wrote: There were numerous GiST-build-related patches in this thread. Yet uncommitted is a patch with sortsupport routines for btree_gist contrib module. Here's its version which needs review. Reviewing this

Re: Yet another fast GiST build

On 07/04/2021 09:00, Heikki Linnakangas wrote: On 08/03/2021 19:06, Andrey Borodin wrote: There were numerous GiST-build-related patches in this thread. Yet uncommitted is a patch with sortsupport routines for btree_gist contrib module. Here's its version which needs review. Committed with

Re: Yet another fast GiST build

On 08/03/2021 19:06, Andrey Borodin wrote: There were numerous GiST-build-related patches in this thread. Yet uncommitted is a patch with sortsupport routines for btree_gist contrib module. Here's its version which needs review. Reviewing this now again. One thing caught my eye: +static int

Re: Yet another fast GiST build

Thanks, Ibrar! > 8 марта 2021 г., в 21:15, Ibrar Ahmed написал(а): > > > > On Mon, Mar 8, 2021 at 8:59 PM Peter Geoghegan wrote: > On Mon, Mar 8, 2021 at 6:41 AM Ibrar Ahmed wrote: > > The patch (0001-Add-bool-column-for-LP_DEAF-flag-to-GiST-pageinspect.patch) > > does not apply

Re: Yet another fast GiST build

On Mon, Mar 8, 2021 at 8:59 PM Peter Geoghegan wrote: > On Mon, Mar 8, 2021 at 6:41 AM Ibrar Ahmed wrote: > > The patch > (0001-Add-bool-column-for-LP_DEAF-flag-to-GiST-pageinspect.patch) > > does not apply successfully and has multiple hanks failed. > > That's because it was committed. > >

Re: Yet another fast GiST build

On Mon, Mar 8, 2021 at 6:41 AM Ibrar Ahmed wrote: > The patch (0001-Add-bool-column-for-LP_DEAF-flag-to-GiST-pageinspect.patch) > does not apply successfully and has multiple hanks failed. That's because it was committed. -- Peter Geoghegan

Re: Yet another fast GiST build

On Mon, Jan 18, 2021 at 3:52 AM Heikki Linnakangas wrote: > On 18/01/2021 00:35, Peter Geoghegan wrote: > > On Sun, Jan 17, 2021 at 12:50 PM Tom Lane wrote: > >> I noticed that gist_page_items() thinks it can hold inter_call_data->rel > >> open across a series of calls. That's completely

Re: Yet another fast GiST build

On 18/01/2021 01:10, Peter Geoghegan wrote: On Sun, Jan 17, 2021 at 3:04 PM Peter Geoghegan wrote: I personally agree with you - it's not like there aren't other ways for superusers to crash the server (most of which seem very similar to this gist_page_items() issue, in fact). I just think

Re: Yet another fast GiST build

On Sun, Jan 17, 2021 at 3:04 PM Peter Geoghegan wrote: > I personally agree with you - it's not like there aren't other ways > for superusers to crash the server (most of which seem very similar to > this gist_page_items() issue, in fact). I just think that it's worth > being clear about that

Re: Yet another fast GiST build

On Sun, Jan 17, 2021 at 2:52 PM Heikki Linnakangas wrote: > I'm not sure I understand. It's true that the raw page image can contain > data from a different index, or any garbage really. And the function > will behave badly if you do that. That's an accepted risk with > pageinspect functions,

Re: Yet another fast GiST build

On 18/01/2021 00:35, Peter Geoghegan wrote: On Sun, Jan 17, 2021 at 12:50 PM Tom Lane wrote: I noticed that gist_page_items() thinks it can hold inter_call_data->rel open across a series of calls. That's completely unsafe: the executor might not run the call series to completion (see LIMIT),

Re: Yet another fast GiST build

On Sun, Jan 17, 2021 at 12:50 PM Tom Lane wrote: > I noticed that gist_page_items() thinks it can hold inter_call_data->rel > open across a series of calls. That's completely unsafe: the executor > might not run the call series to completion (see LIMIT), resulting in > relcache leak complaints.

Re: Yet another fast GiST build

Heikki Linnakangas writes: > On 12/01/2021 18:19, Andrey Borodin wrote: >> Thanks! Looks good to me. > Pushed, thanks! I noticed that gist_page_items() thinks it can hold inter_call_data->rel open across a series of calls. That's completely unsafe: the executor might not run the call series to

Re: Yet another fast GiST build

> 15 янв. 2021 г., в 10:24, Peter Eisentraut > написал(а): > > I noticed this patch while working on another patch for pageinspect [0], and > this one appears to introduce a problem similar to the one the other patch > attempts to fix: The "itemlen" output parameters are declared to be of

Re: Yet another fast GiST build

On 2021-01-12 14:49, Heikki Linnakangas wrote: I suggest calling BuildIndexValueDescription() from your own custom debug instrumentation code. Thanks for the hint, Peter! This function does exactly what I want to do. But I have no Relation inside gist_page_items(bytea) function... probably,

Re: Yet another fast GiST build

> 14 янв. 2021 г., в 04:47, Peter Geoghegan написал(а): > > On Tue, Jan 12, 2021 at 5:49 AM Heikki Linnakangas wrote: >> I did a bit of cleanup on the function signature. The .sql script >> claimed that gist_page_items() took bytea as argument, but in reality it >> was a relation name, as

Re: Yet another fast GiST build

On Tue, Jan 12, 2021 at 5:49 AM Heikki Linnakangas wrote: > I did a bit of cleanup on the function signature. The .sql script > claimed that gist_page_items() took bytea as argument, but in reality it > was a relation name, as text. I changed it so that it takes a page image > as argument,

Re: Yet another fast GiST build

On 13 January 2021 13:53:39 EET, Heikki Linnakangas wrote: >Buildfarm animal thorntail is still not happy: > >> --- >> /home/nm/farm/sparc64_deb10_gcc_64_ubsan/HEAD/pgsql.build/../pgsql/contrib/pageinspect/expected/gist.out >> 2021-01-13 13:38:09.721752365 +0300 >> +++ >>

Re: Yet another fast GiST build

On 13 January 2021 20:04:10 EET, Heikki Linnakangas wrote: > > >On 13 January 2021 13:53:39 EET, Heikki Linnakangas wrote: >>Looks like the LSN on the page is not set to GistBuildLSN as expected. >>Weird. >> >>Thorntail is a sparc64 system, so little-endian, but the other >>little-endian

Re: Yet another fast GiST build

Heikki Linnakangas writes: > On 13/01/2021 11:46, Andrey Borodin wrote: >> Maybe we can just omit key_data from tests? > Make sense, fixed it that way. Thanks! thorntail, at least, is still unhappy. regards, tom lane

Re: Yet another fast GiST build

On 13/01/2021 12:34, Heikki Linnakangas wrote: On 13/01/2021 11:46, Andrey Borodin wrote: 13 янв. 2021 г., в 13:41, Heikki Linnakangas написал(а): One more question: will bytea tests run correctly on 32bit\different-endian systems? Good question. Somehow I thought we were printing

Re: Yet another fast GiST build

On 13/01/2021 11:46, Andrey Borodin wrote: 13 янв. 2021 г., в 13:41, Heikki Linnakangas написал(а): One more question: will bytea tests run correctly on 32bit\different-endian systems? Good question. Somehow I thought we were printing esseantilly text values as bytea. But they are Points,

Re: Yet another fast GiST build

> 13 янв. 2021 г., в 13:41, Heikki Linnakangas написал(а): > >> One more question: will bytea tests run correctly on >> 32bit\different-endian systems? > > Good question. Somehow I thought we were printing esseantilly text values as > bytea. But they are Points, which consists of float8's.

Re: Yet another fast GiST build

On 12/01/2021 18:19, Andrey Borodin wrote: 12 янв. 2021 г., в 18:49, Heikki Linnakangas написал(а): Fixed the docs accordingly, and ran pgindent. New patch version attached. Thanks! Looks good to me. Pushed, thanks! One more question: will bytea tests run correctly on

Re: Yet another fast GiST build

> 12 янв. 2021 г., в 18:49, Heikki Linnakangas написал(а): > >> PFA patch with implementation. > > I did a bit of cleanup on the function signature. The .sql script claimed > that gist_page_items() took bytea as argument, but in reality it was a > relation name, as text. I changed it so

Re: Yet another fast GiST build

On 10/12/2020 12:16, Andrey Borodin wrote: 9 дек. 2020 г., в 14:47, Andrey Borodin написал(а): 7 дек. 2020 г., в 23:56, Peter Geoghegan написал(а): On Mon, Dec 7, 2020 at 2:05 AM Andrey Borodin wrote: Here's version with tests and docs. I still have no idea how to print some useful

Re: Yet another fast GiST build

> 9 дек. 2020 г., в 14:47, Andrey Borodin написал(а): > > > >> 7 дек. 2020 г., в 23:56, Peter Geoghegan написал(а): >> >> On Mon, Dec 7, 2020 at 2:05 AM Andrey Borodin wrote: >>> Here's version with tests and docs. I still have no idea how to print some >>> useful information about

Re: Yet another fast GiST build

> 7 дек. 2020 г., в 23:56, Peter Geoghegan написал(а): > > On Mon, Dec 7, 2020 at 2:05 AM Andrey Borodin wrote: >> Here's version with tests and docs. I still have no idea how to print some >> useful information about tuples keys. > > I suggest calling BuildIndexValueDescription() from

Re: Yet another fast GiST build

On Mon, Dec 7, 2020 at 2:05 AM Andrey Borodin wrote: > Here's version with tests and docs. I still have no idea how to print some > useful information about tuples keys. I suggest calling BuildIndexValueDescription() from your own custom debug instrumentation code. -- Peter Geoghegan

Re: Yet another fast GiST build

> 28 сент. 2020 г., в 13:12, Heikki Linnakangas написал(а): > > I wrote a couple of 'pageinspect' function to inspect GiST pages for this. > See attached. > <0001-Add-functions-to-pageinspect-to-inspect-GiST-indexes.patch> Here's version with tests and docs. I still have no idea how to print

Re: Yet another fast GiST build

> 5 нояб. 2020 г., в 22:20, Justin Pryzby написал(а): > > On Thu, Nov 05, 2020 at 10:11:52PM +0500, Andrey Borodin wrote: >> To test that functions are actually called for sorting build we should >> support directive sorting build like "CREATE INDEX ON A USING GIST(B) >>

Re: Yet another fast GiST build

On Thu, Nov 05, 2020 at 10:11:52PM +0500, Andrey Borodin wrote: > To test that functions are actually called for sorting build we should > support directive sorting build like "CREATE INDEX ON A USING GIST(B) > WITH(sorting=surely,and fail if not)". Maybe you could add a DEBUG1 message for

Re: Yet another fast GiST build

> 2 нояб. 2020 г., в 19:45, Heikki Linnakangas написал(а): > >> How do you think, should this patch and patch with pageinspect GiST >> functions be registered on commitfest? > > Yeah, that'd be good. I've registered both patches on January CF. pageinspect patch's code looks goot to me

Re: Yet another fast GiST build

On 30/10/2020 20:20, Andrey Borodin wrote: 27 окт. 2020 г., в 16:43, Heikki Linnakangas написал(а): static bool gbt_inet_abbrev_abort(int memtupcount, SortSupport ssup) { #if SIZEOF_DATUM == 8 return false; #else return true; #endif } Better to not set the 'abbrev_converter'

Re: Yet another fast GiST build

Thanks! > 27 окт. 2020 г., в 16:43, Heikki Linnakangas написал(а): > > gbt_ts_cmp(), gbt_time_cmp_sort() and gbt_date_cmp_sort() still have the > above issue, they still compare "upper" for no good reason. Fixed. >> +static Datum >> +gbt_bit_abbrev_convert(Datum original, SortSupport ssup) >>

Re: Yet another fast GiST build

On 21/10/2020 20:13, Andrey Borodin wrote: 7 окт. 2020 г., в 17:38, Heikki Linnakangas написал(а): On 07/10/2020 15:27, Andrey Borodin wrote: Here's draft patch with implementation of sortsupport for ints and floats. +static int +gbt_int4_cmp(Datum a, Datum b, SortSupport ssup) +{ +

Re: Yet another fast GiST build

> > > >> +return (ia->lower > ib->lower) ? 1 : -1; > >> +} > > > > We're only dealing with leaf items during index build, so the 'upper' > and 'lower' should always be equal here, right? Maybe add a comment and an > assertion on that. > > > > (It's pretty sad that the on-disk representation is

Re: Yet another fast GiST build

> 7 окт. 2020 г., в 17:38, Heikki Linnakangas написал(а): > > On 07/10/2020 15:27, Andrey Borodin wrote: >> Here's draft patch with implementation of sortsupport for ints and floats. > >> +static int >> +gbt_int4_cmp(Datum a, Datum b, SortSupport ssup) >> +{ >> +int32KEY *ia = (int32KEY

Re: Yet another fast GiST build

On 07/10/2020 15:27, Andrey Borodin wrote: Here's draft patch with implementation of sortsupport for ints and floats. +static int +gbt_int4_cmp(Datum a, Datum b, SortSupport ssup) +{ + int32KEY *ia = (int32KEY *) DatumGetPointer(a); + int32KEY *ib = (int32KEY *)

Re: Yet another fast GiST build

> 29 сент. 2020 г., в 23:04, Andrey M. Borodin > написал(а): > > > Also I'm working on btree_gist opclasses and found out that new sortsupport > function is not mentioned in gistadjustmembers(). I think this can be fixed > with gist_btree patch. Here's draft patch with implementation of

Re: Yet another fast GiST build

It became normal with -fsanitize=signed-integer-overflow,null,alignment instead of -fsanitize=undefined (which is strictly a 'default' list of needed and unnecessary things to check, can be overridden anyway but needed some reading for it) Thanks! -- Best regards, Pavel Borisov Postgres

Re: Yet another fast GiST build

Heikki Linnakangas writes: > You get the same error with: > select (float8 '1e+300')::float4; > float.c:1204:11: runtime error: 1e+300 is outside the range of > representable values of type 'float' > SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior float.c:1204:11 in > It boils down to

Re: Yet another fast GiST build

On 06/10/2020 14:05, Pavel Borisov wrote: I've been making tests with memory sanitizer Thanks! and got one another error in regression test create_index: CREATE INDEX gpointind ON point_tbl USING gist (f1); server closed the connection unexpectedly with logfile: gistproc.c:1714:28: runtime

Re: Yet another fast GiST build

I've been making tests with memory sanitizer and got one another error in regression test create_index: CREATE INDEX gpointind ON point_tbl USING gist (f1); server closed the connection unexpectedly with logfile: gistproc.c:1714:28: runtime error: 1e+300 is outside the range of representable

Re: Yet another fast GiST build

On 29/09/2020 21:04, Andrey M. Borodin wrote: 28 сент. 2020 г., в 13:12, Heikki Linnakangas написал(а): I did some testing with your patch. It seems that the rightlinks are still not always set. I didn't try to debug why. Yes, there is a bug. Now it seems to me so obvious, yet it took some

Re: Yet another fast GiST build

> 28 сент. 2020 г., в 13:12, Heikki Linnakangas написал(а): > > On 21/09/2020 17:19, Andrey M. Borodin wrote: >>> 21 сент. 2020 г., в 18:29, Andrey M. Borodin >>> написал(а): >>> >>> It was a conscious decision with incorrect motivation. I was thinking that >>> it will help to reduce

Re: Yet another fast GiST build

On 21/09/2020 17:19, Andrey M. Borodin wrote: 21 сент. 2020 г., в 18:29, Andrey M. Borodin написал(а): It was a conscious decision with incorrect motivation. I was thinking that it will help to reduce number of "false positive" inspecting right pages. But now I see that: 1. There should be no

Re: Yet another fast GiST build

On 21/09/2020 16:29, Andrey M. Borodin wrote: But thing that bothers me now: when we vacuum leaf page, we bump it's NSN. But we do not bump internal page LSN. Does this means we will follow rightlinks after vacuum? It seems superflous. Sorry, I did not understand what you said above. Vacuum

Re: Yet another fast GiST build

Heikki Linnakangas writes: > On 21/09/2020 02:06, Tom Lane wrote: >> Another interesting point is that all the other index AMs seem to WAL-log >> the new page before the smgrextend call, whereas this code is doing it >> in the other order. I strongly doubt that both patterns are equally >>

Re: Yet another fast GiST build

> 21 сент. 2020 г., в 18:29, Andrey M. Borodin > написал(а): > > It was a conscious decision with incorrect motivation. I was thinking that it > will help to reduce number of "false positive" inspecting right pages. But > now I see that: > 1. There should be no such "false positives" that

Re: Yet another fast GiST build

> 21 сент. 2020 г., в 17:15, Heikki Linnakangas написал(а): > > On 21/09/2020 12:06, Andrey M. Borodin wrote >> >> I think we don't set rightlinks during index build. > > The new GiST sorting code does not, but the regular insert-based code does. > > That's a bit questionable in the new

Re: Yet another fast GiST build

On 21/09/2020 12:06, Andrey M. Borodin wrote: 21 сент. 2020 г., в 13:45, Heikki Linnakangas написал(а): Actually, don't we have a problem with that, even before this patch? Even though we set the LSN to the magic GistBuildLSN value when we build the index, WAL replay will write the LSN of the

Re: Yet another fast GiST build

On 21/09/2020 02:06, Tom Lane wrote: Justin Pryzby writes: This also appears to break checksums. Fixed, thanks for the report! I was wondering about that, because the typical pattern for use of smgrextend for indexes seems to be RelationOpenSmgr(rel);

Re: Yet another fast GiST build

> 21 сент. 2020 г., в 13:45, Heikki Linnakangas написал(а): > > On 21/09/2020 11:08, Heikki Linnakangas wrote: >> I think they need to, so that they can stamp the page with the LSN of >> the WAL record. But GiST build is special in that regard, because it >> stamps all pages with

Re: Yet another fast GiST build

On 21/09/2020 11:08, Heikki Linnakangas wrote: I think they need to, so that they can stamp the page with the LSN of the WAL record. But GiST build is special in that regard, because it stamps all pages with GistBuildLSN. Actually, don't we have a problem with that, even before this patch?

Re: Yet another fast GiST build

On 21/09/2020 02:06, Tom Lane wrote: Justin Pryzby writes: This also appears to break checksums. Thanks, I'll go fix it. I was wondering about that, because the typical pattern for use of smgrextend for indexes seems to be RelationOpenSmgr(rel);

Re: Yet another fast GiST build

Justin Pryzby writes: > This also appears to break checksums. I was wondering about that, because the typical pattern for use of smgrextend for indexes seems to be RelationOpenSmgr(rel); PageSetChecksumInplace(page, lastblock); smgrextend(rel->rd_smgr, MAIN_FORKNUM,

Re: Yet another fast GiST build

On Sun, Sep 20, 2020 at 05:10:05PM -0400, Tom Lane wrote: > I wrote: > > It appears that hyrax (CLOBBER_CACHE_ALWAYS) is not very happy > > with this: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax=2020-09-19%2021%3A27%3A23 > > I reproduced that and traced it to a missing

Re: Yet another fast GiST build

I wrote: > It appears that hyrax (CLOBBER_CACHE_ALWAYS) is not very happy > with this: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax=2020-09-19%2021%3A27%3A23 I reproduced that and traced it to a missing RelationOpenSmgr call. Fixed now. regards, tom

Re: Yet another fast GiST build

Heikki Linnakangas writes: > Pushed. Thanks everyone! It appears that hyrax (CLOBBER_CACHE_ALWAYS) is not very happy with this: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax=2020-09-19%2021%3A27%3A23 We have a recent pass from prion, showing that -DRELCACHE_FORCE_RELEASE

Re: Yet another fast GiST build

On Thu, Sep 17, 2020 at 11:38:47AM +0300, Heikki Linnakangas wrote: > On 15/09/2020 14:36, Heikki Linnakangas wrote: > > Another patch version, fixed a few small bugs pointed out by assertion > > failures in the regression tests. > > Pushed. Thanks everyone! +/* FIXME: bump this before pushing!

Re: Yet another fast GiST build

> 17 сент. 2020 г., в 13:38, Heikki Linnakangas написал(а): > > On 15/09/2020 14:36, Heikki Linnakangas wrote: >> Another patch version, fixed a few small bugs pointed out by assertion >> failures in the regression tests. > > Pushed. Thanks everyone! That's wonderful! Thank you, Heikki!

Re: Yet another fast GiST build

On 15/09/2020 14:36, Heikki Linnakangas wrote: Another patch version, fixed a few small bugs pointed out by assertion failures in the regression tests. Pushed. Thanks everyone! - Heikki

Re: Yet another fast GiST build

At Wed, 16 Sep 2020 12:27:09 +0500, "Andrey M. Borodin" wrote in > I was thinking that machine epsilon is near 1e-300, but I was > wrong. It's actually near 1e-15. FWIW, the mantissa of double is effectively 52+1 bits, about 15.9 digits. so 1+(1e-16) is basically indistincitve from 1+(2e-16).

Re: Yet another fast GiST build

On 16/09/2020 10:27, Andrey M. Borodin wrote: Actually, I just want to understand what changes between v18 and v19 changed on-page order of items. I look into patch diff and cannot figure it out. There are only logging changes. How this affects scan? The test was failing with v18 too. -

Re: Yet another fast GiST build

> 15 сент. 2020 г., в 22:07, Heikki Linnakangas написал(а): > > regression=# SELECT f1, f1 <-> '0,1' FROM point_tbl ORDER BY f1 <-> '0,1'; >f1 | ?column? > ---+-- > (0,0) |1 > (1e-300,-1e-300) |1

Re: Yet another fast GiST build

On 15/09/2020 19:46, Andrey M. Borodin wrote: 15 сент. 2020 г., в 16:36, Heikki Linnakangas написал(а): Another patch version, fixed a few small bugs pointed out by assertion failures in the regression tests. - Heikki These changes in create_index.out do not seem correct to me

Re: Yet another fast GiST build

> 15 сент. 2020 г., в 16:36, Heikki Linnakangas написал(а): > > Another patch version, fixed a few small bugs pointed out by assertion > failures in the regression tests. > > - Heikki > These changes in create_index.out do not seem correct to me SELECT * FROM point_tbl ORDER BY f1 <->

Re: Yet another fast GiST build

On 11/09/2020 09:02, Andrey M. Borodin wrote: 10 сент. 2020 г., в 17:43, Heikki Linnakangas написал(а): One more patch version attached. I fixed some memory leaks, and fixed the abbreviation on 32-bit systems, and a bunch of small comment changes and such. The patch looks fine to me. On my

Re: Yet another fast GiST build

> 10 сент. 2020 г., в 17:43, Heikki Linnakangas написал(а): > > One more patch version attached. I fixed some memory leaks, and fixed the > abbreviation on 32-bit systems, and a bunch of small comment changes and such. > > - Heikki > The patch looks fine to me. On my machine GiST for

Re: Yet another fast GiST build

On 09/09/2020 19:50, Andrey M. Borodin wrote: 9 сент. 2020 г., в 20:39, Heikki Linnakangas написал(а): On 09/09/2020 15:20, Darafei "Komяpa" Praliaskouski wrote: On Wed, Sep 9, 2020 at 3:09 PM Heikki Linnakangas wrote: Come to think of it, the point z-order comparator could benefit a lot

Re: Yet another fast GiST build

> > Interesting to see also the size of index, it should be several times less. > > I've tested this above in CF thread and got ordered GiST index ~1.7 times smaller than non-ordered one for single column of real points. -- Best regards, Pavel Borisov Postgres Professional:

Re: Yet another fast GiST build

On Mon, Sep 7, 2020 at 7:50 PM Heikki Linnakangas wrote: > > On 07/09/2020 13:59, Pavel Borisov wrote: > I suppose there is a big jump in integer value (whether signed or > unsigned) as you cross from positive to negative floats, and then the > sort order is reversed. I have no

Re: Yet another fast GiST build

> 9 сент. 2020 г., в 20:39, Heikki Linnakangas написал(а): > > On 09/09/2020 15:20, Darafei "Komяpa" Praliaskouski wrote: >> On Wed, Sep 9, 2020 at 3:09 PM Heikki Linnakangas wrote: >>> Come to think of it, the point z-order comparator could benefit a lot >>> from key abbreviation, too. You

Re: Yet another fast GiST build

On 09/09/2020 15:20, Darafei "Komяpa" Praliaskouski wrote: On Wed, Sep 9, 2020 at 3:09 PM Heikki Linnakangas wrote: Come to think of it, the point z-order comparator could benefit a lot from key abbreviation, too. You could do the point -> zorder conversion in the abbreviation routine.

Re: Yet another fast GiST build

On 09/09/2020 15:20, Darafei "Komяpa" Praliaskouski wrote: On Wed, Sep 9, 2020 at 3:09 PM Heikki Linnakangas wrote: Come to think of it, the point z-order comparator could benefit a lot from key abbreviation, too. You could do the point -> zorder conversion in the abbreviation routine.

Re: Yet another fast GiST build

On Wed, Sep 9, 2020 at 3:09 PM Heikki Linnakangas wrote: > On 09/09/2020 13:28, Andrey M. Borodin wrote: > > Thanks Darafei! > > > >> 9 сент. 2020 г., в 12:05, Darafei Komяpa Praliaskouski > >> написал(а): > >> > >>> How does the 'sortsupport' routine interact with > >>>

Re: Yet another fast GiST build

On 09/09/2020 13:28, Andrey M. Borodin wrote: Thanks Darafei! 9 сент. 2020 г., в 12:05, Darafei Komяpa Praliaskouski написал(а): How does the 'sortsupport' routine interact with 'compress'/'decompress'? Which representation is passed to the comparator routine: the original value from the

Re: Yet another fast GiST build

Thanks Darafei! > 9 сент. 2020 г., в 12:05, Darafei Komяpa Praliaskouski > написал(а): > > > How does the 'sortsupport' routine interact with 'compress'/'decompress'? > > Which representation is passed to the comparator routine: the original > > value from the table, the compressed

Re: Yet another fast GiST build

Hi, On Wed, Sep 9, 2020 at 9:43 AM Andrey M. Borodin wrote: > > > > 9 сент. 2020 г., в 00:05, Heikki Linnakangas > написал(а): > > > > I've been reviewing the patch today. The biggest changes I've made have > been in restructuring the code in gistbuild.c for readability, but there > are a

Re: Yet another fast GiST build

> 9 сент. 2020 г., в 00:05, Heikki Linnakangas написал(а): > > I've been reviewing the patch today. The biggest changes I've made have been > in restructuring the code in gistbuild.c for readability, but there are a > bunch of smaller changes throughout. Attached is what I've got so far, >

Re: Yet another fast GiST build

On 08/09/2020 21:33, Pavel Borisov wrote: > Thanks! Did you measure the quality of the built index somehow? The > ordering shouldn't make any difference to the build speed, but it > affects the shape of the resulting index and the speed of queries > against it. Again I've

Re: Yet another fast GiST build

> > > Thanks! Did you measure the quality of the built index somehow? The > > ordering shouldn't make any difference to the build speed, but it > > affects the shape of the resulting index and the speed of queries > > against it. Again I've tried random select tests near axes and haven't noticed

Re: Yet another fast GiST build

> 7 сент. 2020 г., в 19:10, Heikki Linnakangas написал(а): > > On 07/09/2020 13:59, Pavel Borisov wrote: > I suppose there is a big jump in integer value (whether signed or > unsigned) as you cross from positive to negative floats, and then the > sort order is reversed. I have no

Re: Yet another fast GiST build

> > > >> I suppose there is a big jump in integer value (whether signed or > >> unsigned) as you cross from positive to negative floats, and then the > >> sort order is reversed. I have no idea if either of those things is a > >> problem worth fixing. That made me wonder if there might also be

Re: Yet another fast GiST build

On 24/02/2020 10:50, Andrey M. Borodin wrote: On 24 февр. 2020 г., at 01:58, Thomas Munro wrote: On Thu, Feb 20, 2020 at 10:14 AM Thomas Munro wrote: 1. We expect floats to be in IEEE format, and the sort order of IEEE floats is mostly correlated to the binary sort order of the bits

Re: Yet another fast GiST build (typo)

> 7 сент. 2020 г., в 12:14, Andrey M. Borodin написал(а): > > Maybe I'm missing something? Like forgot to log 10% of pages, or something > like that... Indeed, there was a bug. I've fixed it, and I still observe same performance gain. Best regards, Andrey Borodin.

Re: Yet another fast GiST build (typo)

> 6 сент. 2020 г., в 18:26, Heikki Linnakangas написал(а): > > On 05/09/2020 14:53, Andrey M. Borodin wrote: >> Thanks for ideas, Heikki. Please see v13 with proposed changes. > > Thanks, that was quick! > >> But I've found out that logging page-by-page slows down GiST build by >>

Re: Yet another fast GiST build (typo)

> 6 сент. 2020 г., в 18:26, Heikki Linnakangas написал(а): > > On 05/09/2020 14:53, Andrey M. Borodin wrote: >> Thanks for ideas, Heikki. Please see v13 with proposed changes. > > Thanks, that was quick! > >> But I've found out that logging page-by-page slows down GiST build by >>

Re: Yet another fast GiST build (typo)

On 05/09/2020 14:53, Andrey M. Borodin wrote: Thanks for ideas, Heikki. Please see v13 with proposed changes. Thanks, that was quick! But I've found out that logging page-by-page slows down GiST build by approximately 15% (when CPU constrained). Though In think that this is IO-wise. Hmm, any

Re: Yet another fast GiST build (typo)

> 3 сент. 2020 г., в 23:40, Heikki Linnakangas написал(а): > > On 30/08/2020 15:04, Andrey M. Borodin wrote: >>> 23 авг. 2020 г., в 14:39, Andrey M. Borodin >>> написал(а): >>> >>> Thanks for reviewing and benchmarking, Pavel! >> Pavel sent me few typos offlist. PFA v12 fixing these typos.

Re: Yet another fast GiST build (typo)

On 30/08/2020 15:04, Andrey M. Borodin wrote: 23 авг. 2020 г., в 14:39, Andrey M. Borodin написал(а): Thanks for reviewing and benchmarking, Pavel! Pavel sent me few typos offlist. PFA v12 fixing these typos. In gist_indexsortbuild(), you first build all the leaf pages. Then, you read

Re: Yet another fast GiST build (typo)

> > Pavel sent me few typos offlist. PFA v12 fixing these typos. > Thanks! > Now I consider the patch ready to be committed and mark it so on CF. Thank you! -- Best regards, Pavel Borisov Postgres Professional: http://postgrespro.com

  1   2   >