Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-17 Thread Matthias van de Meent
ssue may be caused by the new locking pattern - we > will be required to wait for all transaction started before the ending > of the phase to exit. What do you mean by "new locking pattern"? We already keep an ShareUpdateExclusiveLock on every heap table we're accessing during R/CIC, and that should already prevent any concurrent VACUUM operations, right? Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2023-12-15 Thread Matthias van de Meent
e time C/RIC is the one backend holding back cleanup of old tuples. Kind regards, Matthias van de Meent Neon (https://neon.tech) > [1]: https://postgr.es/m/20210115133858.GA18931@alvherre.pgsql > [2]: https://postgr.es/m/17485-396609c6925b982d%40postgresql.org > > >

Re: Reducing output size of nodeToString

2023-12-13 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 13:09, David Rowley wrote: > > On Thu, 7 Dec 2023 at 10:09, Matthias van de Meent > wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > tot

Re: initdb caching during tests

2023-12-07 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 15:06, Daniel Gustafsson wrote: > > > On 7 Dec 2023, at 14:50, Matthias van de Meent > > wrote: > > > Attached a patch that fixes this for both make and meson, by adding > > --no-clean to the initdb template. > > Makes sense. While

Re: initdb caching during tests

2023-12-07 Thread Matthias van de Meent
t; Cool. Pushed that way. Only change I made is to redirect the output of cp > (and/or robocopy) in pg_regress, similar to how that was done for initdb > proper. While working on some things that are prone to breaking initdb, I noticed that this template isn't generated with --no-clean, while pg

Re: Reducing output size of nodeToString

2023-12-07 Thread Matthias van de Meent
On Thu, 7 Dec 2023 at 11:26, Peter Eisentraut wrote: > > On 06.12.23 22:08, Matthias van de Meent wrote: > > PFA a patch that reduces the output size of nodeToString by 50%+ in > > most cases (measured on pg_rewrite), which on my system reduces the > > total size of pg_

Re: automating RangeTblEntry node support

2023-12-06 Thread Matthias van de Meent
of it was just updating the gen_node_support.pl code generation, and making the macros do a good job. In general I'm all for reducing special cases, so +1 on the idea. I'll have to check the specifics of the patches at a later point in time. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Reducing output size of nodeToString

2023-12-06 Thread Matthias van de Meent
, as the textual format is about to hit its entropy limits. See also [0], [1] and [2], where complaints about the verbosity of nodeToString were vocalized. Kind regards, Matthias van de Meent [0] https://www.postgresql.org/message-id/CAEze2WgGexDM63dOvndLdAWwA6uSmSsc97jmrCuNmrF1JEDK7w

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-06 Thread Matthias van de Meent
On Wed, 6 Dec 2023 at 19:55, Peter Geoghegan wrote: > > On Wed, Dec 6, 2023 at 5:27 AM Matthias van de Meent > wrote: > > 1. When scanning an index in ascending order using scankey a > 1 (so, > > one that defines a start point of the scan), we don't need to check &

Re: Bug in nbtree optimization to skip > operator comparisons (or < comparisons in backwards scans)

2023-12-06 Thread Matthias van de Meent
age is consistent. If so, then all other items on the page will also be consistent with that scankey. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: backtrace_on_internal_error

2023-12-05 Thread Matthias van de Meent
the easier API'. Forcing a change over to ereport because of stack trace spam in logs caused by elog would be quite annoying. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Extensible storage manager API - SMGR hook Redux

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 22:03, Kirill Reshke wrote: > > On Mon, 4 Dec 2023 at 22:21, Matthias van de Meent > wrote: >> >> On Mon, 4 Dec 2023 at 17:51, Kirill Reshke wrote: >> > >> > So, 0002 patch uses the `get_tablespace` function, which searches Catalo

Re: Extensible storage manager API - SMGR hook Redux

2023-12-04 Thread Matthias van de Meent
ge, but not impossible. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Proposal to use JSON for Postgres Parser format

2023-12-04 Thread Matthias van de Meent
On Mon, 31 Oct 2022 at 15:56, Michel Pelletier wrote: > On Mon, Oct 31, 2022 at 6:15 AM Matthias van de Meent > wrote: >> On Mon, 31 Oct 2022 at 13:46, Alexander Korotkov >> wrote: >>> On Fri, Oct 28, 2022 at 4:27 PM Andrew Dunstan wrote: >>>> On 2

Re: Parallel CREATE INDEX for BRIN indexes

2023-12-04 Thread Matthias van de Meent
On Sun, 3 Dec 2023 at 17:46, Tomas Vondra wrote: > On 11/30/23 18:47, Matthias van de Meent wrote: > > ... > > > > I just ran some more tests in less favorable environments, and it > > looks like I hit a bug: > > > > % SET max_parallel_workers

Re: Avoid detoast overhead when possible

2023-12-04 Thread Matthias van de Meent
On Mon, 4 Dec 2023 at 14:23, wrote: > > > Hi, > > Matthias van de Meent writes: > > > On Mon, 4 Dec 2023 at 07:56, wrote: > > > ..It would also add overhead when > > we write results to disk, such as spilling merge sorts, hash join > > spills, or

Re: Avoid detoast overhead when possible

2023-12-04 Thread Matthias van de Meent
such points surely will be cheaper than storing the full value again and again. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-30 Thread Matthias van de Meent
On Thu, 30 Nov 2023 at 01:10, Tomas Vondra wrote: > > On 11/29/23 23:59, Matthias van de Meent wrote: >> On Wed, 29 Nov 2023 at 21:56, Tomas Vondra >> wrote: >>> >>> On 11/29/23 21:30, Matthias van de Meent wrote: >>>> On Wed, 29 Nov 2023 at

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-29 Thread Matthias van de Meent
On Wed, 29 Nov 2023 at 21:56, Tomas Vondra wrote: > > On 11/29/23 21:30, Matthias van de Meent wrote: >> On Wed, 29 Nov 2023 at 18:55, Tomas Vondra >> wrote: >>> I did try to measure how much it actually saves, but none of the tests I >>> did actually found me

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-29 Thread Matthias van de Meent
mpty ranges at the end of the main loop, and if there were no tuples in the spool then the memtuple would still be at its original initialized value of 0 thus resulting in a constant false condition. I also updated some comments. Kind regards, Matthias van de Meent Neon (https://neon.tech) v6-0003-

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-29 Thread Matthias van de Meent
On Tue, 28 Nov 2023 at 18:59, Tomas Vondra wrote: > > On 11/28/23 16:39, Matthias van de Meent wrote: > > On Thu, 23 Nov 2023 at 14:35, Tomas Vondra > > wrote: > >> On 11/23/23 13:33, Matthias van de Meent wrote: > >>> The union operator may leak (lots of)

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-28 Thread Matthias van de Meent
On Thu, 23 Nov 2023 at 14:35, Tomas Vondra wrote: > On 11/23/23 13:33, Matthias van de Meent wrote: >> The union operator may leak (lots of) memory, so I think it makes >> sense to keep a context around that can be reset after we've extracted >> the merge result. >> &g

Re: POC, WIP: OR-clause support for indexes

2023-11-27 Thread Matthias van de Meent
On Mon, 27 Nov 2023, 23:16 Peter Geoghegan, wrote: > On Mon, Nov 27, 2023 at 1:04 PM Robert Haas wrote: > > The use of op_mergejoinable() seems pretty random to me. Why should we > > care about that? If somebody writes a<1 or a<2 or a<3 or a<4, you can > > transform that to a > good idea, but I

Re: Questions regarding Index AMs and natural ordering

2023-11-27 Thread Matthias van de Meent
On Fri, 24 Nov 2023, 19:58 Tom Lane, wrote: > > Peter Geoghegan writes: > > On Fri, Nov 24, 2023 at 8:44 AM Matthias van de Meent > > wrote: > >> Yes, the part where btree opclasses determine a type's ordering is > >> clear. But what I'm look

Re: Questions regarding Index AMs and natural ordering

2023-11-24 Thread Matthias van de Meent
On Thu, 23 Nov 2023 at 19:52, Peter Geoghegan wrote: > > On Thu, Nov 23, 2023 at 9:16 AM Matthias van de Meent > wrote: > > For example, btree ignores any ordering scan keys that it is given in > > btrescan, which seems fine for btree because the ordering of a btree > &

Re: Table AM Interface Enhancements

2023-11-23 Thread Matthias van de Meent
ts of usable space), and for undo-based logging you would probably need index APIs for retail index tuple deletion. Neither is supplied here, nor is described why these APIs were omitted. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Questions regarding Index AMs and natural ordering

2023-11-23 Thread Matthias van de Meent
in backend/optimizer didn't answer this question for me, either. Kind regards, Matthias van de Meent Neon (https://neon.tech) [0] https://www.postgresql.org/message-id/flat/EB2AF704-70FC-4D73-A97A-A7884A0381B5%40kleczek.org [1] https://www.postgresql.org/message-id/flat/CAH2-Wz%3DksvN_sjcnD1

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-23 Thread Matthias van de Meent
Hi, On Wed, 22 Nov 2023 at 20:16, Tomas Vondra wrote: > > On 11/20/23 20:48, Matthias van de Meent wrote: >> On Wed, 8 Nov 2023 at 12:03, Tomas Vondra >> wrote: >>> >>> Hi, >>> >>> here's an updated patch, addressing the review comme

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-20 Thread Matthias van de Meent
ere are 2 new mentions of "tuplestore" in the patch, while the structure used is tuplesort: one on form_and_spill_tuple, and one on brinbuildCallbackParallel. Please update those comments. That's it for code review. I'll do some performance comparisons and testing soon, too. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Inquiry on Generating Bitmaps from Filter Conditions in Index Scans

2023-11-20 Thread Matthias van de Meent
e that does this, but it won't be able to use much of the features available inside PostgreSQL. Kind regards, Matthias van de Meent [0] https://www.postgresql.org/message-id/flat/CANWCAZbrZ58-w1W_3pg-0tOfbx8K41_n_03_0ndGV78hJWswBA%2540mail.gmail.com

Re: RFC: Pluggable TOAST

2023-11-15 Thread Matthias van de Meent
ce savings during updates". So, while I do think there would be good reasons for typed toasting to be disabled, I don't see a good reason for only specific parts of type-specific toasting to be disabled (no reason for 'disable the append optimization for bytea, but not the splice optimization'). Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Why do indexes and sorts use the database collation?

2023-11-14 Thread Matthias van de Meent
tation that the index is based on the columns' actual collation's properties? Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Parallel CREATE INDEX for BRIN indexes

2023-11-12 Thread Matthias van de Meent
ve it's > fairly close to be committable. Are you planning on committing the patches separately, or squashed? I won't have much time this week for reviewing the patch, and it seems like these patches are incremental, so some guidance on what you want to be reviewed would be appreciated. Kind regards, M

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-11-11 Thread Matthias van de Meent
On Wed, 8 Nov 2023 at 02:53, Peter Geoghegan wrote: > > On Tue, Nov 7, 2023 at 4:20 AM Matthias van de Meent > wrote: > > On Tue, 7 Nov 2023 at 00:03, Peter Geoghegan wrote: > > > I should be able to post v6 later this week. My current plan is to > > > co

Re: Parallel aggregates in PG 16.1

2023-11-10 Thread Matthias van de Meent
ked PARALLEL SAFE to enable parallel aggregation. """ >From this, it seems like the PARALLEL = SAFE argument is missing from your aggregate definition as provided above. Kind regards, Matthias van de Meent Neon (https://neon.tech) [0] https://www.postgresql.org/docs/16/sql-createaggregate.html

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-11-09 Thread Matthias van de Meent
don't seem to support array keys for row compares, which probably is an even more niche use case: SELECT count(*) FROM tenk1 WHERE (thousand, tenthous) = ANY (ARRAY[(1, 1), (1, 2), (2, 1)]); This is no different from master, too, but it'd be nice if there was support for arrays of row operations, too, just so that composite primary keys can also be looked up with SAOPs. Kind regards, Matthias van de Meent

Re: pg_walfile_name_offset can return inconsistent values

2023-11-09 Thread Matthias van de Meent
tionpoint, xlogsegno, wal_segment_size); > +XLByteToSeg(locationpoint, xlogsegno, wal_segment_size); > XLogFileName(xlogfilename, GetWALInsertionTimeLine(), xlogsegno, > wal_segment_size); Kind regards, Matthias van de Meent

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-08 Thread Matthias van de Meent
t a truly > atomic state. Good points that I hadn't thought much about yet. I agree that atomic reset isn't worth implementing in this stats system - it's too costly and complex to do it correctly. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: ALTER TABLE uses a bistate but not for toast tables

2023-11-07 Thread Matthias van de Meent
Hi Justin, This patch has gone stale quite some time ago; CFbot does not seem to have any history of a successful apply attemps, nor do we have any succesful build history (which was introduced some time ago already). Are you planning on rebasing this patch? Kind regards, Matthias van de Meent

Re: Buffer Cache Problem

2023-11-07 Thread Matthias van de Meent
for this exclusive buffer access, and that is already limited to one process per relation, so there are no explicit concurrent super-exclusive waits in the system right now. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: 2023-11-09 release announcement draft

2023-11-07 Thread Matthias van de Meent
performance. The types' *_bloom and *_minmax opclasses were not affected and therefore do not need to be reindexed. Kind regards, Matthias van de meent.

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-11-07 Thread Matthias van de Meent
On Tue, 7 Nov 2023 at 00:03, Peter Geoghegan wrote: > > On Mon, Nov 6, 2023 at 1:28 PM Matthias van de Meent > wrote: > > I'm planning on reviewing this patch tomorrow, but in an initial scan > > through the patch I noticed there's little information about how the > &g

Re: RFC: Pluggable TOAST

2023-11-07 Thread Matthias van de Meent
nb- or bytea-typed columns, so I believe this is better solved in a different thread. Ideas like 'toast relation per column' also doesn't really solve the issue when the main table only has one bigint and one jsonb column, so I think this needs a different approach, too. I think solutions could probably best be discussed in a separate thread. Kind regards, Matthias van de Meent.

Re: Optimizing nbtree ScalarArrayOp execution, allowing multi-column ordered scans, skip scan

2023-11-06 Thread Matthias van de Meent
used in the new design? If not, I'll probably be able to discover my own understanding of the mechanism used in the patch, but if there is a framework to build that understanding on (rather than having to build it from scratch) that'd be greatly appreciated. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Add bump memory context type and use it for tuplesorts

2023-11-06 Thread Matthias van de Meent
ring merge *where* we use a + * simple slab allocator, and during a non-bounded sort where we + * use a bump allocator). Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Optimizing "boundary cases" during backward scan B-Tree index descents

2023-11-06 Thread Matthias van de Meent
ckward scans in as much detail as forward scans here. The concepts are mostly "do the same but in reverse", but the difference is noticable. Apart from these comments, no further noteworthy comments. Looks good. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Moving forward with TDE [PATCH v3]

2023-11-06 Thread Matthias van de Meent
On Sat, 4 Nov 2023 at 03:38, Andres Freund wrote: > > Hi, > > On 2023-11-02 22:09:40 +0100, Matthias van de Meent wrote: > > I'm quite surprised at the significant number of changes being made > > outside the core storage manager files. I thought that changing out >

Re: brininsert optimization opportunity

2023-11-03 Thread Matthias van de Meent
lready uses the bs_* namespace for its fields in the same file, but that's only cosmetic. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: DRAFT GIST support for ORDER BY

2023-11-03 Thread Matthias van de Meent
On Mon, 30 Oct 2023 at 14:39, Michał Kłeczek wrote: >> On 30 Oct 2023, at 13:31, Matthias van de Meent >> wrote: >> >>> The problem is though that right now handling of ORDER BY column clauses is >>> tightly coupled to BTree. >>> It would be good to

Re: Popcount optimization using AVX512

2023-11-03 Thread Matthias van de Meent
which could want to cover bytestreams of >BLCKSZ) the only popcount usages I could find were on objects that fit on a page, i.e. <8KiB in size. How does performance compare for bitstreams of such sizes, especially after any CPU clock implications are taken into account? Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Matthias van de Meent
On Thu, 2 Nov 2023 at 22:25, Tomas Vondra wrote: > > > > On 11/2/23 21:02, Matthias van de Meent wrote: > > On Thu, 2 Nov 2023 at 20:32, Tomas Vondra > > wrote: > >> On 11/2/23 20:09, stepan rutz wrote: > >>> db1=# ex

Re: Moving forward with TDE [PATCH v3]

2023-11-02 Thread Matthias van de Meent
would be the most obvious change to implement cluster-wide encryption with the least code touched, as relations don't need to know whether the files they're writing are encrypted, right? Is there a reason to not implement this at the smgr level that I overlooked in the documentation of these patches

Re: Add new option 'all' to pg_stat_reset_shared()

2023-11-02 Thread Matthias van de Meent
stats systems much myself, but skew between metrics caused by different reset points can be difficult to detect and debug, so I think an atomic call to reset all these stats could be worth implementing. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-11-02 Thread Matthias van de Meent
but gives the measured data to the user; allowing them to derive their own precise bandwidth if they're so inclined. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2023-11-01 Thread Matthias van de Meent
On Wed, 1 Nov 2023 at 07:47, Pavel Stehule wrote: > > Hi > > út 31. 10. 2023 v 22:12 odesílatel Matthias van de Meent > napsal: >> This patch was originally suggested at [0], but it was mentioned that >> they could be pulled out into it's own thread. Earlier, th

btree: downlink right separator/HIKEY optimization

2023-10-31 Thread Matthias van de Meent
is enabled). By adding this, the number of _bt_compare calls per _bt_search is often reduced by one per btree level. Kind regards, Matthias van de Meent Neon (https://neon.tech) PS. Best served with dynamic prefix truncation [1] and btree specialization [0]. [0] https://www.postgresql.org

btree: implement dynamic prefix truncation (was: Improving btree performance through specializing by key shape, take 2)

2023-10-31 Thread Matthias van de Meent
indexes on the cc-complete sample dataset, with the current worst-case index shape getting a 60%+ improved performance on INSERTs in the tests at [0]. Kind regards, Matthias van de Meent Neon (https://neon.tech) PS. Best served with the downlink right separator/HIKEY optimization (separate patch

Re: DRAFT GIST support for ORDER BY

2023-10-30 Thread Matthias van de Meent
correct ordering of int8/bigint when you use `my_column <-> UINT64_MAX` because as far as the floating point math is concerned, 0 is about as far away from INT64_MAX as (say) 20 and -21. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: RFC: Pluggable TOAST

2023-10-26 Thread Matthias van de Meent
nd would thus allow for a lower memory and storage footprint of the column's datums during query processing. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: RFC: Pluggable TOAST

2023-10-26 Thread Matthias van de Meent
ould be opt-in for a type, thus all functions that use that type's internals should be aware of the different on-disk format for toasted values and should thus be able to handle it gracefully. Kind regards, Matthias van de Meent Neon (https://neon.tech) [0] https://www.postgresql.org/message-id/flat/224711f9-83b7-a307-b17f-4457ab73aa0a%40sigaev.ru

Re: PostgreSQL domains and NOT NULL constraint

2023-10-23 Thread Matthias van de Meent
find > that possibility compelling enough to justify the performance hit > of recomputing every constraint just in case it acts like that. Makes sense. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Lowering the default wal_blocksize to 4K

2023-10-11 Thread Matthias van de Meent
On Wed, 11 Oct 2023 at 01:29, Andres Freund wrote: > > Hi, > > On 2023-10-10 21:30:44 +0200, Matthias van de Meent wrote: > > On Tue, 10 Oct 2023 at 06:14, Andres Freund wrote: > > > On 2023-10-09 23:16:30 -0400, Tom Lane wrote: > > >> Andres Freund

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Matthias van de Meent
per half, uneven pages the lower half of the ID. This should allow for enough integrity checks without further increasing the size of XLogPageHeader in most installations. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Lowering the default wal_blocksize to 4K

2023-10-10 Thread Matthias van de Meent
dditional WAL getting written and a doubling of the number of XLog pages being touched (both initialization and the smaller memcpy for records that would now cross an extra page boundary). However, for all of these issues I doubt that they actually matter much in the grand scheme of things, so I defin

Re: Comparing two double values method

2023-10-10 Thread Matthias van de Meent
fix it, as it would decrease the cases where we detect broken GUC registrations. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Pre-proposal: unicode normalized text

2023-10-06 Thread Matthias van de Meent
ng of columns would help reduce the pain: I would no longer have to re-initialize the database or cluster from 0. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: pg16: invalid page/page verification failed

2023-10-05 Thread Matthias van de Meent
uming it's a block of a different relation kind, then it's also likely it was originally located elsewhere in that other relation, indeed causing the checksum failure. You can further validate this by looking at the page header's pd_special value - if it is 8176, that'd be another indicator for it being a btree. Kind regards, Matthias van de Meent.

Re: Change of behaviour for creating same type name in multiple schemas

2023-10-05 Thread Matthias van de Meent
one point me to where the code was changed ? This was with commit 70988b7b [0] in July 2022, based on this thread [1] (moved from -bugs). Kind regards, Matthias van de Meent Neon (https://neon.tech) [0] https://github.com/postgres/postgres/commits/70988b7b0a0bd03c59a2314d0b5bcf2135692349 [1] https:/

Re: Allow deleting enumerated values from an existing enumerated data type

2023-10-03 Thread Matthias van de Meent
entries? The only real issue that I can think of is making sure that concurrent backends don't modify this data, but that shouldn't be very different from the other locks we already have to take in e.g. ALTER TYPE ... DROP ATTRIBUTE. Kind regards, Matthias van de Meent

Re: Index AmInsert Parameter Confused?

2023-09-27 Thread Matthias van de Meent
On Wed, 27 Sept 2023 at 05:03, jacktby jacktby wrote: > > > > > 2023年9月27日 00:45,Matthias van de Meent 写道: > > > > On Tue, 26 Sept 2023 at 18:38, jacktby jacktby wrote: > >> > >> typedef bool (*aminsert_function) (Relation indexRela

Re: Index AmInsert Parameter Confused?

2023-09-26 Thread Matthias van de Meent
that contains the indexed values, so that the index knows which tuple to refer to. Note that access/amapi.h describes only index AM APIs; it does not cover the table AM APIs descibed in access/tableam.h Kind regards, Matthias van de Meent

Re: XLog size reductions: Reduced XLog record header size for PG17

2023-09-25 Thread Matthias van de Meent
On Wed, 20 Sept 2023 at 07:06, Michael Paquier wrote: > > On Tue, Sep 19, 2023 at 12:07:07PM +0200, Matthias van de Meent wrote: > > V5 is a rebased version of v4, and includes the latest patch from > > "smaller XLRec block header" [0] as 0001. > > 0001 an

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-25 Thread Matthias van de Meent
On Tue, 19 Sept 2023 at 01:03, Andres Freund wrote: > > Hi, > > On 2023-05-18 19:22:26 +0300, Heikki Linnakangas wrote: > > On 18/05/2023 17:59, Matthias van de Meent wrote: > > > It changes the block IDs used to fit in 6 bits, using the upper 2 bits > > > of

Re: Improving btree performance through specializing by key shape, take 2

2023-09-25 Thread Matthias van de Meent
On Tue, 19 Sept 2023 at 22:49, Peter Geoghegan wrote: > > On Tue, Sep 19, 2023 at 6:28 AM Matthias van de Meent > wrote: > > > To be clear, page deletion does what I described here (it does an > > > in-place update of the downlink to the deleted page, so the same p

Re: GenBKI emits useless open;close for catalogs without rows

2023-09-22 Thread Matthias van de Meent
On Fri, 22 Sept 2023 at 00:25, Andres Freund wrote: > > Hi, > > On 2023-09-19 21:05:41 +0300, Heikki Linnakangas wrote: > > On 18/09/2023 17:50, Matthias van de Meent wrote: > > > (initdb takes about 73ms locally with syncing disabled) > > > > That's impress

Re: GenBKI emits useless open;close for catalogs without rows

2023-09-22 Thread Matthias van de Meent
On Tue, 19 Sept 2023 at 20:05, Heikki Linnakangas wrote: > > On 18/09/2023 17:50, Matthias van de Meent wrote: > > (initdb takes about 73ms locally with syncing disabled) > > That's impressive. It takes about 600 ms on my laptop. Of which about > 140 ms goes into pr

Re: Disabling Heap-Only Tuples

2023-09-19 Thread Matthias van de Meent
AEL is not nice for sure, but wouldn't > > ShareUpdateExclusiveLock be sufficient? We have a bunch of reloptions > > for which that is sufficient. > > Hmm, yeah, I think you're right. Updating the reloption after relation truncation implies having the same lock as relation truncation, i.e. AEL (if the vacuum docs are to be believed). So the AEL is not reqiored for updating the storage option (that would require SUEL), but for the block truncation operation operation. Kind regards, Matthias van de Meent Neon (http://neon.tech)

Re: Disabling Heap-Only Tuples

2023-09-19 Thread Matthias van de Meent
where previously we could've applied HOT. But we do have VACUUM and REINDEX CONCURRENTLY to clean that up without serious long-duration stop-the-world actions, while the other builtin cleanup methods don't. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Improving btree performance through specializing by key shape, take 2

2023-09-19 Thread Matthias van de Meent
But that's not currently the case for the nbtree code, and this is the next best thing, as it also has the benefit of working with all currently supported physical formats of btree indexes. Kind regards, Matthias van de Meent

Re: XLog size reductions: smaller XLRec block header for PG17

2023-09-18 Thread Matthias van de Meent
size` is signed. The issue has been fixed in the attached patch with 'int' types instead. Kind regards, Matthias van de Meent v2-0001-Reduce-overhead-of-small-block-data-in-xlog-recor.patch Description: Binary data

Re: Improving btree performance through specializing by key shape, take 2

2023-09-18 Thread Matthias van de Meent
On Wed, 30 Aug 2023 at 21:50, Matthias van de Meent wrote: > > Updated in the attached version 12 of the patchset (which is also > rebased on HEAD @ 9c13b681). No changes apart from rebase fixes and > these added comments. Rebased again to v13 to account for API changes in 9f060253

Re: GenBKI emits useless open;close for catalogs without rows

2023-09-18 Thread Matthias van de Meent
On Tue, 12 Sept 2023 at 17:51, Matthias van de Meent wrote: > > On Fri, 1 Sept 2023 at 19:52, Tom Lane wrote: > > > > Alvaro Herrera writes: > > > On 2023-Sep-01, Matthias van de Meent wrote: > > >> A potential addition to the patch would to stop ma

Re: GenBKI emits useless open;close for catalogs without rows

2023-09-12 Thread Matthias van de Meent
On Fri, 1 Sept 2023 at 19:52, Tom Lane wrote: > > Alvaro Herrera writes: > > On 2023-Sep-01, Matthias van de Meent wrote: > >> A potential addition to the patch would to stop manually closing > >> relations: initdb and check-world succeed without manual 'close' >

Re: Detoasting optionally to make Explain-Analyze less misleading

2023-09-12 Thread Matthias van de Meent
the differences in measurements between EXPLAIN and normal query processing of the tuples - after all, we don't create new memory contexts for every tuple in the normal DestRemote receiver either, right? Kind regards, Matthias van de Meent

Re: How to add a new pg oid?

2023-09-05 Thread Matthias van de Meent
rrent/sql-createtype.html, or look at the comments in pg_type.dat and the comments on TypInfo in bootstrap.c on how the built-in types are created and managed. Lastly, you could look at pg_class and the genbki documentation if you want to add new catalog types. Kind regards, Matthias van de Meent Ne

Re: Commitfest 2023-09 starts soon

2023-09-04 Thread Matthias van de Meent
"consensus" part of your mail, and just put down the real reason why each patch is being RfC-ed or rejected. That is, don't imply that there are hackers that OK-ed it when there are none, and inform patch authors directly about the reasons why the patch is being revoked; so without "see consensus in [0]". Kind regards, Matthias van de Meent

Re: Commitfest 2023-09 starts soon

2023-09-04 Thread Matthias van de Meent
patch to happen on the patch's thread before any action is taken (or at least a mention on that thread), but quite clearly that hasn't happened here. Are patch authors expected to follow any and all discussion on threads with "Commitfest" in the title? If so, shouldn't the relevant wiki pa

Re: GenBKI emits useless open;close for catalogs without rows

2023-09-01 Thread Matthias van de Meent
On Fri, 1 Sept 2023 at 19:43, Alvaro Herrera wrote: > > On 2023-Sep-01, Matthias van de Meent wrote: > > > A potential addition to the patch would to stop manually closing > > relations: initdb and check-world succeed without manual 'close' > > operations because the

GenBKI emits useless open;close for catalogs without rows

2023-09-01 Thread Matthias van de Meent
d relation apparently doesn't need closing - check-world succeeds without issues (incl. with TAP enabled). That is therefore implemented in attached patch 2 - it removes the 'close' syntax in its entirety. Kind regards, Matthias van de Meent Neon (https://neon.tech) 0002-Remove-the-bki-close-com

Re: Disabling Heap-Only Tuples

2023-08-30 Thread Matthias van de Meent
On Wed, 30 Aug 2023 at 15:31, Robert Haas wrote: > > On Wed, Aug 30, 2023 at 9:01 AM Matthias van de Meent > wrote: > > I've reworked the patch a bit to remove the "excessive bloat with low > > fillfactors when local space is available" issue that this parameter

Re: Disabling Heap-Only Tuples

2023-08-30 Thread Matthias van de Meent
On Mon, 28 Aug 2023 at 17:57, Robert Haas wrote: > > On Mon, Aug 28, 2023 at 11:50 AM Matthias van de Meent > wrote: > > Agreed on all points. But isn't that true for most most tools on bloat > > prevention and/or detection? E.g. fillfactor, autovacuum_*, ... > > No

Re: Disabling Heap-Only Tuples

2023-08-28 Thread Matthias van de Meent
On Mon, 28 Aug 2023 at 17:14, Robert Haas wrote: > > On Mon, Aug 28, 2023 at 10:52 AM Matthias van de Meent > wrote: > > In this new patch, I've updated a few comments to get mostly within > > line length limits; the name of the storage parameter is now > > &

Re: Disabling Heap-Only Tuples

2023-08-28 Thread Matthias van de Meent
On Wed, 19 Jul 2023 at 14:58, Laurenz Albe wrote: > > On Thu, 2023-07-06 at 22:18 +0200, Matthias van de Meent wrote: > > On Wed, 5 Jul 2023 at 19:55, Thom Brown wrote: > > > > > > On Wed, 5 Jul 2023 at 18:05, Matthias van de Meent > > > wrote: > >

Re: broken master regress tests

2023-08-25 Thread Matthias van de Meent
t looks like the error message matcher doesn't account for the localized version of "No such file or directory", might that be the issue? Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Disabling Heap-Only Tuples

2023-08-24 Thread Matthias van de Meent
unique (id, num_updates) ) would be assumed to remain bloated if I'd set a parameter named something_hot_something, as all updates would be non-hot and thus should not be influenced by the GUC/parameter. How about 'local_update_limit'? Kind regards, Matthias van de Meent

Re: Disabling Heap-Only Tuples

2023-08-24 Thread Matthias van de Meent
On Fri, 7 Jul 2023 at 12:18, Tomas Vondra wrote: > > On 7/7/23 11:55, Matthias van de Meent wrote: >> On Fri, 7 Jul 2023 at 06:53, Dilip Kumar wrote: >>> >>> >>> So IIUC, this parameter we can control that instead of putting the new >>> version

Re: [question] multil-column range partition prune

2023-08-10 Thread Matthias van de Meent
B: any value - A = 6, B < 6 As you can see, each partition contains a set of rows that may have any value for B, and thus these partitions cannot be pruned based on the predicate. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Extract numeric filed in JSONB more effectively

2023-08-03 Thread Matthias van de Meent
On Wed, 2 Aug 2023 at 03:05, Andy Fan wrote: > > Hi Matthias: > > On Wed, Aug 2, 2023 at 7:33 AM Andy Fan wrote: >> >> >> >> On Tue, Aug 1, 2023 at 7:03 PM Matthias van de Meent >> wrote: >>> >>> On Tue, 1 Aug 2023 at 06:39, Andy

Re: Adding a pg_servername() function

2023-08-03 Thread Matthias van de Meent
user's hostname can be very different from the hostname used in the url that connects to that PostgreSQL instance. Kind regards, Matthias van de Meent Neon (https://neon.tech)

Re: Potential memory leak in contrib/intarray's g_intbig_compress

2023-08-02 Thread Matthias van de Meent
On Fri, 14 Jul 2023 at 07:57, Michael Paquier wrote: > > On Thu, Jul 13, 2023 at 06:28:39PM +0200, Matthias van de Meent wrote: > > There are similar pfree calls in the _int_gist.c file's g_int_compress > > function, which made me think we do need to clean up after u

<    1   2   3   4   5   6   >