Re: "ERROR: could not open relation with OID 16391" error was encountered when reindexing

2024-01-16 Thread feichanghong
For this specific job, I have always wanted a try_index_open() thatwould attempt to open the index with a relkind check, perhaps we couldintroduce one and reuse it here?Yes, replacing index_open with try_index_open solves the problem. Theidea is similar to my initial report of "after successfully

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-16 Thread Kyotaro Horiguchi
At Wed, 17 Jan 2024 14:38:54 +0900, torikoshia wrote in > Hi, > > Thanks for applying! > > > + errmsg_plural("%zd row were skipped due to data type > > incompatibility", > > Sorry, I just noticed it, but 'were' should be 'was' here? > > >> BTW I'm thinking we should add a column to

Re: Add PQsendSyncMessage() to libpq

2024-01-16 Thread Michael Paquier
On Tue, Jan 16, 2024 at 02:55:12PM +0100, Alvaro Herrera wrote: > On 2024-Jan-16, Michael Paquier wrote: >> How about the attached to remove all that, then? > > Looks good, thank you. Thanks for double-checking. Done. -- Michael signature.asc Description: PGP signature

RE: speed up a logical replica setup

2024-01-16 Thread Hayato Kuroda (Fujitsu)
Dear Euler, hackers, I found that some bugs which have been reported by Shlok were not fixed, so made a top-up patch. 0001 was not changed, and 0002 contains below: * Add a timeout option for the recovery option, per [1]. The code was basically ported from pg_ctl.c. * Reject if the target

Re: [PATCH] Add additional extended protocol commands to psql: \parse and \bindx

2024-01-16 Thread Michael Paquier
On Tue, Jan 16, 2024 at 10:37:22AM +0100, Jelte Fennema-Nio wrote: > I do realize the same is true for plain \bind, but it seems > like a bug there too. Hmm. ignore_slash_options() is used to make the difference between active and inactive branches with \if. I was playing a bit with psql.sql

Re: Change GUC hashtable to use simplehash?

2024-01-16 Thread John Naylor
I spent some time rewriting the comments and a couple other cosmetic changes, and squashed into two patches: the second one has the optimized string hashing. They each have still just one demo use case. It looks pretty close to commitable, but I'll leave this up for a few days in case anyone wants

Re: "ERROR: could not open relation with OID 16391" error was encountered when reindexing

2024-01-16 Thread Michael Paquier
On Wed, Jan 17, 2024 at 12:54:26AM +0800, feichanghong wrote: >> This is extremely nonspecific, as line numbers in our code change >> constantly. Please quote a chunk of code surrounding that >> and indicate which line you are trying to stop at. > > Thanks for the suggestion, I've refined the

Re: Fix a possible socket leak at Windows (src/backend/port/win32/socket.c)

2024-01-16 Thread Michael Paquier
On Tue, Jan 16, 2024 at 05:25:39PM -0300, Ranier Vilela wrote: > Thanks for taking care of this. Yeah, that's a good catch. > Do you have plans or should I register for a commitfest? Daniel has stated that he would take care of it, so why not letting him a few days? I don't think that a CF

Re: heavily contended lwlocks with long wait queues scale badly

2024-01-16 Thread Michael Paquier
On Tue, Jan 16, 2024 at 11:24:49PM -0500, Jonathan S. Katz wrote: > On 1/16/24 1:11 AM, Michael Paquier wrote: >> I'd like to apply that, just let me know if you have any comments >> and/or objections. > > Looking at the code, I understand an argument for not backpatching given we > modify the

Reduce useless changes before reassembly during logical replication

2024-01-16 Thread li jie
Hi hackers, During logical replication, if there is a large write transaction, some spill files will be written to disk, depending on the setting of logical_decoding_work_mem. This behavior can effectively avoid OOM, but if the transaction generates a lot of change before commit, a large number

Re: index prefetching

2024-01-16 Thread Konstantin Knizhnik
On 16/01/2024 11:58 pm, Jim Nasby wrote: On 1/16/24 2:10 PM, Konstantin Knizhnik wrote: Amazon RDS is just vanilla Postgres with file system mounted on EBS (Amazon  distributed file system). EBS provides good throughput but larger latencies comparing with local SSDs. I am not sure if

Re: Add tuples_skipped to pg_stat_progress_copy

2024-01-16 Thread Masahiko Sawada
On Wed, Jan 17, 2024 at 2:22 PM torikoshia wrote: > > Hi, > > 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to > skip malformed data, but there is no way to watch the number of skipped > rows during COPY. > > Attached patch adds tuples_skipped to pg_stat_progress_copy, which >

Assertion failure with epoch when replaying standby records for 2PC

2024-01-16 Thread Michael Paquier
Hi all, rorqual has failed today with a very interesting failure: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=rorqual=2024-01-17%2005%3A06%3A31 This has caused an assertion failure for a 2PC transaction when replaying one of the tests from the main regression suite: 2024-01-17

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-16 Thread torikoshia
Hi, Thanks for applying! + errmsg_plural("%zd row were skipped due to data type incompatibility", Sorry, I just noticed it, but 'were' should be 'was' here? BTW I'm thinking we should add a column to pg_stat_progress_copy that counts soft errors. I'll suggest

Re: partitioning and identity column

2024-01-16 Thread Ashutosh Bapat
On Wed, Jan 17, 2024 at 12:30 AM Peter Eisentraut wrote: > > On 09.01.24 15:10, Ashutosh Bapat wrote: > > Here's complete patch-set. > > Looks good! Committed. > Thanks a lot Peter. -- Best Wishes, Ashutosh Bapat

Re: Make mesage at end-of-recovery less scary.

2024-01-16 Thread Michael Paquier
On Tue, Jan 16, 2024 at 02:46:02PM +0300, Aleksander Alekseev wrote: >> For now, let me explain the basis for this patch. The fundamental >> issue is that these warnings that always appear are, in practice, not >> a problem in almost all cases. Some of those who encounter them for >> the first

Add tuples_skipped to pg_stat_progress_copy

2024-01-16 Thread torikoshia
Hi, 132de9968840c introduced SAVE_ERROR_TO option to COPY and enabled to skip malformed data, but there is no way to watch the number of skipped rows during COPY. Attached patch adds tuples_skipped to pg_stat_progress_copy, which counts the number of skipped tuples because source data is

Re: Synchronizing slots from primary to standby

2024-01-16 Thread Nisha Moond
A review on v62-006: failover-ready validation steps doc - + Next, check that the logical replication slots identified above exist on + the standby server. This step can be skipped if + standby_slot_names has been correctly configured. + +test_standby=# SELECT bool_and(synced

Re: Synchronizing slots from primary to standby

2024-01-16 Thread shveta malik
On Wed, Jan 17, 2024 at 6:43 AM Masahiko Sawada wrote: > > On Tue, Jan 16, 2024 at 6:40 PM shveta malik wrote: > > > > On Tue, Jan 16, 2024 at 12:59 PM Masahiko Sawada > > wrote: > > > > > > On Tue, Jan 16, 2024 at 1:07 PM Amit Kapila > > > wrote: > > > > > > > > On Tue, Jan 16, 2024 at 9:03 

Re: Improve the connection failure error messages

2024-01-16 Thread Nisha Moond
Thanks for reviewing, please find my response inline. On Wed, Jan 17, 2024 at 4:56 AM Peter Smith wrote: > > On Sat, Jan 13, 2024 at 12:36 AM Aleksander Alekseev > wrote: > > > > Hi, > > > > Thanks for the patch. > > > > > Due to this behavior, it is not possible to add a test to show the > > >

Re: [PATCH] Add support function for containment operators

2024-01-16 Thread jian he
On Wed, Jan 17, 2024 at 5:46 AM Tom Lane wrote: > > But perhaps someone has an argument for a different rule? > > Anyway, pending discussion of that point, I think the code is good > to go. I don't like the test cases much though: they expend many more > cycles than necessary. You could prove

Re: Synchronizing slots from primary to standby

2024-01-16 Thread Peter Smith
On Tue, Jan 16, 2024 at 10:57 PM shveta malik wrote: > ... > v62-006: > Separated the failover-ready validation steps into this separate > doc-patch (which were earlier present in v61-002 and v61-003). Also > addressed some of the doc comments by Peter in [1]. > Thanks Hou-San for providing this

Re: Synchronizing slots from primary to standby

2024-01-16 Thread Peter Smith
Here is a review comment for the latest v62-0002 changes. == src/backend/replication/logical/slotsync.c 1. + if (namestrcmp(>data.plugin, remote_slot->plugin) == 0 && + slot->data.database == dbid && + remote_slot->restart_lsn == slot->data.restart_lsn && + remote_slot->catalog_xmin ==

Re: heavily contended lwlocks with long wait queues scale badly

2024-01-16 Thread Jonathan S. Katz
On 1/16/24 1:11 AM, Michael Paquier wrote: On Thu, Jan 11, 2024 at 09:47:33AM -0500, Jonathan S. Katz wrote: I have similar data sources to Nathan/Michael and I'm trying to avoid piling on, but one case that's interesting occurred after a major version upgrade from PG10 to PG14 on a database

Re: Fix a typo of func DecodeInsert()

2024-01-16 Thread Yongtao Huang
Thank you. I prefer to keep the comments of these three functions *DecodeInsert()*, *DecodeUpdate()*, and *DecodeDelete()* aligned. ``` /* * Parse XLOG_HEAP_INSERT (not MULTI_INSERT!) records into tuplebufs. * * Inserts can contain the new tuple. */ static void

Re: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Tom Lane
David Rowley writes: > On Wed, 17 Jan 2024 at 15:28, Maiquel Grassi wrote: >> On Wed, 17 Jan 2024 at 14:36, David Rowley wrote: >>> If you were looking for something to optimize in this rough area, then >>> perhaps adding some kind of "Backward WindowAgg" node (by overloading >>> the existing

Re: locked reads for atomics

2024-01-16 Thread Li, Yong
> On Nov 28, 2023, at 05:00, Nathan Bossart wrote: > > External Email > > Here's a v2 of the patch set in which I've attempted to address all > feedback. I've also added a pg_write_membarrier_u* pair of functions that > provide an easy way to write to an atomic variable with full barrier >

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread John Naylor
I wrote: > > Hmm, I wonder if that's a side-effect of the "create" functions doing > > their own allocations and returning a pointer. Would it be less tricky > > if the structs were declared where we need them and passed to "init" > > functions? If this is a possibility, I thought I'd first send

Re: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread David Rowley
On Wed, 17 Jan 2024 at 15:28, Maiquel Grassi wrote: > On Wed, 17 Jan 2024 at 14:36, David Rowley wrote: > > If you were looking for something to optimize in this rough area, then > > perhaps adding some kind of "Backward WindowAgg" node (by overloading > > the existing node) to allow queries

RE: Random pg_upgrade test failure on drongo

2024-01-16 Thread Hayato Kuroda (Fujitsu)
Dear hackers, > > Thanks to both of you. I have pushed the patch. > I have been tracking the BF animals these days, and this failure has not seen anymore. I think we can close the topic. Again, thanks for all efforts. Best Regards, Hayato Kuroda FUJITSU LIMITED

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread John Naylor
On Wed, Jan 17, 2024 at 8:39 AM Masahiko Sawada wrote: > > On Wed, Jan 17, 2024 at 9:20 AM John Naylor wrote: > > > > On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada > > wrote: > > > Just changing "items" to be the local tidstore struct could make the > > > code tricky a bit, since max_bytes

Re: introduce dynamic shared memory registry

2024-01-16 Thread Bharath Rupireddy
On Tue, Jan 16, 2024 at 9:37 PM Nathan Bossart wrote: > > The autoprewarm change (0003) does use this variable. I considered making > it optional (i.e., you could pass in NULL if you didn't want it), but I > didn't feel like the extra code in GetNamedDSMSegment() to allow this was > worth it so

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
As far as I see your proposal, you want to allow something that is undefined to be reversed. I don't think this is a good idea at all. As mentioned by others, you should have ORDER BY clauses and just add a DESC. * Okay, now I'm convinced of that. If you were looking for something to

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-16 Thread Jelte Fennema-Nio
On Tue, 16 Jan 2024 at 21:36, Robert Haas wrote: > Sorry for not getting back to this right away; there are quite a few > threads competing for my attention. No worries, I know it's a busy time. > But I think we would want to have those changes in all supported > branches before we could think

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
As far as I see your proposal, you want to allow something that is undefined to be reversed. I don't think this is a good idea at all. As mentioned by others, you should have ORDER BY clauses and just add a DESC. If you were looking for something to optimize in this rough area, then perhaps

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread Masahiko Sawada
On Wed, Jan 17, 2024 at 9:20 AM John Naylor wrote: > > On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada wrote: > > Just changing "items" to be the local tidstore struct could make the > > code tricky a bit, since max_bytes and num_items are on the shared > > memory while "items" is a local

Re: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread David Rowley
On Wed, 17 Jan 2024 at 08:51, Michał Kłeczek wrote: > I think that’s the main issue: what (semantically) does row_number() mean in > that case? You could equally well generate random numbers? Well, not quite random as at least row_number() would ensure the number is unique in the result set.

Re: Add pgindent test to check if codebase is correctly formatted

2024-01-16 Thread Bruce Momjian
On Tue, Jan 16, 2024 at 07:32:47PM -0600, Tristan Partin wrote: > It requires changes to at least the Meson build files. pg_bsd_indent is not > marked for installation currently. There is a TODO there. pgindent has no > install_data() for instance. pg_bsd_indent seemingly gets installed >

Re: Add pgindent test to check if codebase is correctly formatted

2024-01-16 Thread Tristan Partin
On Tue Jan 16, 2024 at 7:27 PM CST, Bruce Momjian wrote: On Tue, Jan 16, 2024 at 07:22:23PM -0600, Tristan Partin wrote: > I think a good solution would be to distribute pgindent and pg_bsd_indent. > At Neon, we are trying to format our extension code using pgindent. I am > sure there are other

Re: Add pgindent test to check if codebase is correctly formatted

2024-01-16 Thread Bruce Momjian
On Tue, Jan 16, 2024 at 07:22:23PM -0600, Tristan Partin wrote: > I think a good solution would be to distribute pgindent and pg_bsd_indent. > At Neon, we are trying to format our extension code using pgindent. I am > sure there are other extension authors out there too that format using >

Add pgindent test to check if codebase is correctly formatted

2024-01-16 Thread Tristan Partin
Had some time to watch code run through an extensive test suite, so thought I would propose this patch that is probably about 75% of the way to the stated $subject. I had to add in a hack for Meson, and I couldn't figure out a good hack for autotools. I think a good solution would be to

Re: reorganize "Shared Memory and LWLocks" section of docs

2024-01-16 Thread Bharath Rupireddy
On Tue, Jan 16, 2024 at 9:22 PM Nathan Bossart wrote: > > I fixed this in v4. LGTM. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Synchronizing slots from primary to standby

2024-01-16 Thread Masahiko Sawada
On Tue, Jan 16, 2024 at 6:40 PM shveta malik wrote: > > On Tue, Jan 16, 2024 at 12:59 PM Masahiko Sawada > wrote: > > > > On Tue, Jan 16, 2024 at 1:07 PM Amit Kapila wrote: > > > > > > On Tue, Jan 16, 2024 at 9:03 AM shveta malik > > > wrote: > > > > > > > > On Sat, Jan 13, 2024 at 12:54 PM

Re: Fix a typo of func DecodeInsert()

2024-01-16 Thread Richard Guo
On Wed, Jan 17, 2024 at 8:47 AM Yongtao Huang wrote: > Hi all, > I think the comment above the function DecodeInsert() > in src/backend/replication/logical/decode.c should be > + * *Inserts *can contain the new tuple. > , rather than > - * *Deletes *can contain the new tuple. > Nice catch. +1.

Fix a typo of func DecodeInsert()

2024-01-16 Thread Yongtao Huang
Hi all, I think the comment above the function DecodeInsert() in src/backend/replication/logical/decode.c should be + * *Inserts *can contain the new tuple. , rather than - * *Deletes *can contain the new tuple. Please correct me if I'm wrong, thanks a lot.

Re: Synchronizing slots from primary to standby

2024-01-16 Thread Peter Smith
About v62-0001: As stated in the patch comment: But note that this commit does not yet include the capability to actually sync the replication slot; the next patch will address that. ~~~ Because of this, I think it might be prudent to separate the documentation portion from this patch so that

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
But as you are the one arguing for the new feature demonstrating that the status quo is deficient is your job. --//-- I performed these three tests(take a look below) quite simple but functional, so that we can get an idea of the performance. Apparently, we have a higher cost in using

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-16 Thread John Naylor
On Tue, Jan 16, 2024 at 1:18 PM Masahiko Sawada wrote: > Just changing "items" to be the local tidstore struct could make the > code tricky a bit, since max_bytes and num_items are on the shared > memory while "items" is a local pointer to the shared tidstore. Thanks for trying it this way! I

Re: Improve the connection failure error messages

2024-01-16 Thread Peter Smith
On Sat, Jan 13, 2024 at 12:36 AM Aleksander Alekseev wrote: > > Hi, > > Thanks for the patch. > > > Due to this behavior, it is not possible to add a test to show the > > error message as it is done for CREATE SUBSCRIPTION. > > Let me know if you think there is another way to add this test. > > I

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Melanie Plageman
On Tue, Jan 16, 2024 at 2:23 PM Robert Haas wrote: > > On Tue, Jan 16, 2024 at 11:28 AM Melanie Plageman > wrote: > > All LGTM. > > Committed. Attached v8 patch set is rebased over this. In 0004, I've taken the approach you seem to favor and combined the FSM updates from the prune and no prune

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 4:28 PM Jim Nasby wrote: > On 1/12/24 12:45 PM, Robert Haas wrote: > > P.P.S. to everyone: Yikes, this logic is really confusing. > > Having studied all this code several years ago when it was even simpler > - it was *still* very hard to grok even back then. I *greatly >

Re: index prefetching

2024-01-16 Thread Jim Nasby
On 1/16/24 2:10 PM, Konstantin Knizhnik wrote: Amazon RDS is just vanilla Postgres with file system mounted on EBS (Amazon  distributed file system). EBS provides good throughput but larger latencies comparing with local SSDs. I am not sure if read-ahead works for EBS. Actually, EBS only

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
The people in this community are quite capable and willing to write a contrary opinion to mine. Not sure how to make “this new proposed function shouldn’t be added to core”, and trying to explain why not, non-oppressive. I can add “thank you for taking the time to try and improve PostgreSQL”

Re: [PATCH] Add support function for containment operators

2024-01-16 Thread Tom Lane
jian he writes: > [ v5-0001-Simplify-containment-in-range-constants-with-supp.patch ] I spent some time reviewing and cleaning up this code. The major problem I noted was that it doesn't spend any effort thinking about cases where it'd be unsafe or undesirable to apply the transformation. In

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Jim Nasby
On 1/12/24 12:45 PM, Robert Haas wrote: P.P.S. to everyone: Yikes, this logic is really confusing. Having studied all this code several years ago when it was even simpler - it was *still* very hard to grok even back then. I *greatly appreciate* the effort that y'all are putting into

Re: UUID v7

2024-01-16 Thread Jelte Fennema-Nio
On Tue, 16 Jan 2024 at 22:02, Przemysław Sztoch wrote: > But replace uuidv7_extract_time(uuid) with uuid_extract_time(uuid) - function > should be able extract timestamp from v1/v6/v7 I'm fine with this. > I would highly recommend to add: > uuidv5(namespace uuid, name text) -> uuid > using

Re: UUID v7

2024-01-16 Thread Przemysław Sztoch
Another question: how did you choose between using TimestampTz and Timestamp types? I realize that internally it's all the same. Maybe Timestamp will be slightly better since the way it is displayed doesn't depend on the session settings. Many people I talked to find this part of TimestampTz

Re: UUID v7

2024-01-16 Thread Przemysław Sztoch
Jelte Fennema-Nio wrote on 1/16/2024 9:25 PM: On Tue, 16 Jan 2024 at 19:17, Andrey M. Borodin wrote: So currently my preference for the function names would be: - uuidv4() -> alias for gen_random_uuid() - uuidv7() - uuidv7(timestamptz) - uuid_extract_ver(uuid) - uuid_extract_var(uuid) -

Re: cleanup patches for incremental backup

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 3:22 PM Matthias van de Meent wrote: > > One other thought is that the incremental backup only replaces > > relation files with incremental files, and it never does anything > > about FSM files. So the statement that it only contains data that was > > potentially changed

Re: Revise the Asserts added to bimapset manipulation functions

2024-01-16 Thread David Rowley
On Tue, 16 Jan 2024 at 21:00, Richard Guo wrote: > Thank you so much for all the work you have put into making this patch > perfect. I reviewed through the v3 patch and I do not have further > comments. I think it's in good shape now. Thanks for looking again. I pushed the patch after

New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread David G. Johnston
On Tuesday, January 16, 2024, Maiquel Grassi wrote: > However, initially, I have one more obstacle in your feedback. If I use > count(*) over() - row_number() over(), it gives me an offset of one unit. > To resolve this, I need to add 1. > > > This way, simulating a reverse row_number() becomes

Re: Add new protocol message to change GUCs for usage with future protocol-only GUCs

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 8:43 AM Jelte Fennema-Nio wrote: > I haven't removed 0008 yet, since I'd like some feedback first if that > makes sense. But I did add two new patches in the middle of the > patchset (which shift the later patch numbers by 2): > > 0007: Adds a new \parameterset

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
However, initially, I have one more obstacle in your feedback. If I use count(*) over() - row_number() over(), it gives me an offset of one unit. To resolve this, I need to add 1. This way, simulating a reverse row_number() becomes even more laborious. I don’t really understand why you think

Re: UUID v7

2024-01-16 Thread Jelte Fennema-Nio
On Tue, 16 Jan 2024 at 19:17, Andrey M. Borodin wrote: > Jelte, what is your opinion on naming the function which extracts timestamp > from UUID v7? I looked at a few more datatypes: json, jsonb & hstore. The get_ prefix is not used there at all, so I'm still opposed to that. But they seem to

Re: Fix a possible socket leak at Windows (src/backend/port/win32/socket.c)

2024-01-16 Thread Ranier Vilela
Em seg., 15 de jan. de 2024 às 09:43, Daniel Gustafsson escreveu: > > On 13 Jan 2024, at 22:38, Ranier Vilela wrote: > > > In the pgwin32_socket function (src/backend/port/win32/socket.c), there > is a possible socket leak if the socket cannot be made non-blocking. > > I don't know Windows well

Re: cleanup patches for incremental backup

2024-01-16 Thread Matthias van de Meent
On Tue, 16 Jan 2024 at 16:39, Robert Haas wrote: > > On Mon, Jan 15, 2024 at 3:31 PM Matthias van de Meent > wrote: > > Off-list I was notified that the new WAL summarizer process was not > > yet added to the glossary, so PFA a patch that does that. > > In passing, it also adds "incremental

Re: UUID v7

2024-01-16 Thread Przemysław Sztoch
Andrey Borodin wrote on 1/16/2024 1:15 PM: Sergey, Przemysław, Jelte, thanks for your feedback. Here's v9. Changes: 1. Swapped type of the argument to timestamptz in gen_uuid_v7() Please update docs part about optional timestamp argument. 2. Renamed get_uuid_v7_time() to uuid_v7_time()

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
You can do: -(ROW_NUMBER() OVER ()) AS descending (note “-“ in front) I don't have a base column to use for "order by," I think that’s the main issue: what (semantically) does row_number() mean in that case? You could equally well generate random numbers? --//-- What I want to do is

Re: UUID v7

2024-01-16 Thread Jelte Fennema-Nio
On Tue, 16 Jan 2024 at 15:44, Andrey M. Borodin wrote: > > On 16 Jan 2024, at 18:00, Aleksander Alekseev > > wrote: > > Not 100% sure what this is for. Any chance this could be part of another > > patch? > Nope, it’s necessary there. Without these changes catalog functions cannot > have

Re: index prefetching

2024-01-16 Thread Konstantin Knizhnik
On 16/01/2024 6:25 pm, Tomas Vondra wrote: On 1/16/24 09:13, Konstantin Knizhnik wrote: Hi, On 12/01/2024 6:42 pm, Tomas Vondra wrote: Hi, Here's an improved version of this patch, finishing a lot of the stuff that I alluded to earlier - moving the code from indexam.c, renaming a bunch of

New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread David G. Johnston
On Tuesday, January 16, 2024, Maiquel Grassi wrote: > > > However, initially, I have one more obstacle in your feedback. If I use > count(*) over() - row_number() over(), it gives me an offset of one unit. > To resolve this, I need to add 1. > > This way, simulating a reverse row_number() becomes

Re: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Michał Kłeczek
> On 16 Jan 2024, at 16:51, Maiquel Grassi wrote: > > > Imagine I have a dataset that is returned to my front-end, and I want to > reverse enumerate them (exactly the concept of Math enumerating integers). > The row_number does the ascending enumeration, but I need the descending >

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
I doubt it is materially different, you need that count regardless so the effort is expended no matter if you put it in an SQL expression or build it into the window function. But as you are the one arguing for the new feature demonstrating that the status quo is deficient is your job.

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 11:28 AM Melanie Plageman wrote: > All LGTM. Committed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: psql JSON output format

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 11:07 AM Laurenz Albe wrote: > "Round-trip safety" is not so important. If you want to move data from > PostgreSQL to PostgreSQL, you use the plain or the binary format. > The CSV format by default renders NULL and empty strings identical, and > I don't think anybody

Re: partitioning and identity column

2024-01-16 Thread Peter Eisentraut
On 09.01.24 15:10, Ashutosh Bapat wrote: Here's complete patch-set. Looks good! Committed.

Re: UUID v7

2024-01-16 Thread Andrey M. Borodin
> On 16 Jan 2024, at 21:49, Sergey Prokhorenko > wrote: > > It is not clear how to interpret uuid_v7_time(): > • uuid_v7 to time() (extracting the timestamp) > • time() to uuid_v7 (generation of the uuid_v7) > It is worth improving the naming, for example, adding prepositions.

New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread David G. Johnston
On Tuesday, January 16, 2024, Maiquel Grassi wrote: > Hi, > > Count() over() - row_number() over() > >But if my dataset is significantly large? Wouldn't calling two window > functions instead of one be much slower? >Is *count() over() - row_number() over()* faster than *row_number_desc()

Re: ALTER TYPE OWNER fails to recurse to multirange

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 11:46 AM Tom Lane wrote: > They're by no means independent. What would it mean to have a > multirange without the underlying range type? It would mean just that - no more, and no less. If it's possible to imagine a data type that stores pairs of values from the

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
Hi, Count() over() - row_number() over() But if my dataset is significantly large? Wouldn't calling two window functions instead of one be much slower? Is count() over() - row_number() over() faster than row_number_desc() over()? Maiquel.

Re: index prefetching

2024-01-16 Thread Robert Haas
On Tue, Jan 16, 2024 at 11:25 AM Tomas Vondra wrote: > > 3. It doesn't perform prefetch of leave pages for IOS, only referenced > > heap pages which are not marked as all-visible. It seems to me that if > > optimized has chosen IOS (and not bitmap heap scan for example), then > > there should be

Re: UUID v7

2024-01-16 Thread Sergey Prokhorenko
Andrey, It is not clear how to interpret uuid_v7_time(): - uuid_v7 to time() (extracting the timestamp) - time() to uuid_v7  (generation of the uuid_v7) It is worth improving the naming, for example, adding prepositions. Sergey prokhorenkosergeyprokhore...@yahoo.com.au On

Re: "ERROR: could not open relation with OID 16391" error was encountered when reindexing

2024-01-16 Thread feichanghong
> This is extremely nonspecific, as line numbers in our code change > constantly. Please quote a chunk of code surrounding that > and indicate which line you are trying to stop at. Thanks for the suggestion, I've refined the steps below to reproduce: 1. Initialize the data ``` DROP TABLE IF

Re: psql JSON output format

2024-01-16 Thread Andrew Dunstan
On 2024-01-16 Tu 11:07, Laurenz Albe wrote: On Tue, 2024-01-09 at 16:51 +, Dean Rasheed wrote: On Tue, 9 Jan 2024 at 14:35, Christoph Berg wrote: Getting it print numeric/boolean without quotes was actually easy, as well as json(b). Implemented as the attached v2 patch. But: not quoting

Re: ALTER TYPE OWNER fails to recurse to multirange

2024-01-16 Thread Tom Lane
Robert Haas writes: > On Mon, Jan 15, 2024 at 2:28 PM Tom Lane wrote: >> I'm reasoning by analogy to array types, which are automatically >> created and automatically updated to keep the same ownership >> etc. properties as their base type. To the extent that multirange >> types don't act

Re: "ERROR: could not open relation with OID 16391" error was encountered when reindexing

2024-01-16 Thread Tom Lane
"=?ISO-8859-1?B?ZmVpY2hhbmdob25n?=" writes: > 2. session1 reindex and gdb break at index.c:3585 This is extremely nonspecific, as line numbers in our code change constantly. Please quote a chunk of code surrounding that and indicate which line you are trying to stop at.

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Melanie Plageman
On Tue, Jan 16, 2024 at 10:24 AM Robert Haas wrote: > > On Mon, Jan 15, 2024 at 4:03 PM Melanie Plageman > wrote: > > > Perhaps it isn't important, but I find this wording confusing. You > > mention lazy_scan_prune() and then mention that "the whole test is in > > one place instead of spread

Re: index prefetching

2024-01-16 Thread Tomas Vondra
On 1/16/24 09:13, Konstantin Knizhnik wrote: > Hi, > > On 12/01/2024 6:42 pm, Tomas Vondra wrote: >> Hi, >> >> Here's an improved version of this patch, finishing a lot of the stuff >> that I alluded to earlier - moving the code from indexam.c, renaming a >> bunch of stuff, etc. I've also

Re: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread David G. Johnston
On Tuesday, January 16, 2024, Maiquel Grassi wrote: > Hello David, how are you? > > Firstly, I apologize if I wasn't clear in what I intended to propose. I > used a very specific example here, and it wasn't very clear what I really > wanted to bring up for discussion. > > I understand that it's

Re: "ERROR: could not open relation with OID 16391" error was encountered when reindexing

2024-01-16 Thread feichanghong
I have provided a python script in the attachment to minimize the reproduction of the issue. I'm sorry that I lost the attached script in my last reply, but I've added it in this reply. You can also use psql to reproduce it with the following steps: 1. Initialize the data ``` DROP TABLE IF

Re: introduce dynamic shared memory registry

2024-01-16 Thread Nathan Bossart
On Tue, Jan 16, 2024 at 10:28:29AM +0530, Bharath Rupireddy wrote: > I think it's better for GetNamedDSMSegment() to error out on empty > 'name' and size 0. This makes the user-facing function > GetNamedDSMSegment more concrete. Agreed, thanks for the suggestion. > +void * >

Re: psql JSON output format

2024-01-16 Thread Laurenz Albe
On Tue, 2024-01-09 at 16:51 +, Dean Rasheed wrote: > On Tue, 9 Jan 2024 at 14:35, Christoph Berg wrote: > > > > Getting it print numeric/boolean without quotes was actually easy, as > > well as json(b). Implemented as the attached v2 patch. > > > > But: not quoting json means that NULL and

Re: Custom explain options

2024-01-16 Thread Konstantin Knizhnik
On 16/01/2024 5:38 pm, Tomas Vondra wrote: By "broken" you mean that you prefetch items only from a single leaf page, so immediately after reading the next one nothing is prefetched. Correct? Yes, exactly. It means that reading first heap page from next leaf page will be done without

Re: reorganize "Shared Memory and LWLocks" section of docs

2024-01-16 Thread Nathan Bossart
On Tue, Jan 16, 2024 at 08:20:19AM -0600, Nathan Bossart wrote: > On Tue, Jan 16, 2024 at 10:02:15AM +0530, Bharath Rupireddy wrote: >> The v3 patch looks good to me except for a nitpick: the input >> parameter for RequestAddinShmemSpace is 'Size' not 'int' >> >> >> void

RE: New Window Function: ROW_NUMBER_DESC() OVER() ?

2024-01-16 Thread Maiquel Grassi
Hello David, how are you? Firstly, I apologize if I wasn't clear in what I intended to propose. I used a very specific example here, and it wasn't very clear what I really wanted to bring up for discussion. I understand that it's possible to order the "returned dataset" using "order by ...

Re: cleanup patches for incremental backup

2024-01-16 Thread Robert Haas
On Mon, Jan 15, 2024 at 3:31 PM Matthias van de Meent wrote: > Off-list I was notified that the new WAL summarizer process was not > yet added to the glossary, so PFA a patch that does that. > In passing, it also adds "incremental backup" to the glossary, and > updates the documented types of

Re: Custom explain options

2024-01-16 Thread Tomas Vondra
On 1/15/24 21:42, Konstantin Knizhnik wrote: > > On 15/01/2024 5:08 pm, Tomas Vondra wrote: >> >> My patch does not care about prefetching internal index pages. Yes, it's >> a limitation, but my assumption is the internal pages are maybe 0.1% of >> the index, and typically very hot / cached.

Re: Emit fewer vacuum records by reaping removable tuples during pruning

2024-01-16 Thread Robert Haas
On Mon, Jan 15, 2024 at 4:03 PM Melanie Plageman wrote: > It doesn't pass the number of LP_DEAD items back to the caller. It > passes a boolean. Oops. > Perhaps it isn't important, but I find this wording confusing. You > mention lazy_scan_prune() and then mention that "the whole test is in >

Re: POC PATCH: copy from ... exceptions to: (was Re: VLDB Features)

2024-01-16 Thread Alexander Korotkov
Hi, > Thanks for updating the patch! You're welcome! > Here is a minor comment: > > > +/* > > + * Extract a defGetCopySaveErrorToChoice value from a DefElem. > > + */ > > Should be Extract a "CopySaveErrorToChoice"? Fixed. > BTW I'm thinking we should add a column to pg_stat_progress_copy

Re: POC: GROUP BY optimization

2024-01-16 Thread Alexander Korotkov
sortref, > + *group_clauses); > + if (!sgc) > + /* The grouping clause does not cover this pathkey */ > + break; > > I think we need to check or assert 'pathkey->pk_eclass->ec_sortref' is > valid bef

  1   2   >