Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Tomas Vondra
On 1/29/24 15:15, Ronan Dunklau wrote: > Le lundi 29 janvier 2024, 13:17:07 CET Tomas Vondra a écrit : >>> Did you try running an strace on the process ? That may give you some >>> hindsights into what malloc is doing. A more sophisticated approach would >>> be using stap and plugging it into

Re: PG versus libxml2 2.12.x

2024-01-29 Thread Peter Eisentraut
On 27.01.24 20:04, Tom Lane wrote: Buildfarm member caiman has been failing build for a couple weeks now. The reason turns out to be that recent libxml2 has decided to throw a "const" into the signature required for custom error handlers. (API compatibility? What's that?) I don't mind adopting

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Ronan Dunklau
Le lundi 29 janvier 2024, 13:17:07 CET Tomas Vondra a écrit : > > Did you try running an strace on the process ? That may give you some > > hindsights into what malloc is doing. A more sophisticated approach would > > be using stap and plugging it into the malloc probes, for example > >

Re: Wrong buffer limits check

2024-01-29 Thread Daniel Gustafsson
> On 29 Jan 2024, at 14:37, Mikhail Gribkov wrote: > I have tried to analyse Postgres code with Svace static analyzer [1] and > found something I think is a real bug. > > In pgp-decrypt.c, in prefix_init function the following check: > if (len > sizeof(tmpbuf)) > > seem to be erroneous and

Re: Use of backup_label not noted in log

2024-01-29 Thread David Steele
On 1/28/24 20:09, Michael Paquier wrote: On Fri, Jan 26, 2024 at 12:08:46PM +0900, Michael Paquier wrote: Well, I'm OK with this consensus on 1d35f705e if folks think this is useful enough for all the stable branches. I have done that down to REL_15_STABLE for now as this is able to apply

Re: UUID v7

2024-01-29 Thread Junwang Zhao
On Mon, Jan 29, 2024 at 7:38 PM Jelte Fennema-Nio wrote: > > tl;dr I believe we should remove the uuidv7(timestamp) function from > this patchset. > > On Thu, 25 Jan 2024 at 18:04, Sergey Prokhorenko > wrote: > > In this case the documentation must state that the functions > >

RE: Synchronizing slots from primary to standby

2024-01-29 Thread Zhijie Hou (Fujitsu)
On Monday, January 29, 2024 9:17 PM Zhijie Hou (Fujitsu) wrote: > > On Monday, January 29, 2024 7:30 PM Amit Kapila > wrote: > > > > On Mon, Jan 29, 2024 at 3:11 PM shveta malik > > wrote: > > > > > > PFA v71 patch set with above changes. > > > > > > > Few comments on 0001 > > Thanks for the

Re: psql: add \create_function command

2024-01-29 Thread Andrew Dunstan
On 2024-01-26 Fr 15:17, Tom Lane wrote: Pavel Stehule writes: I don't know, maybe I have a problem with the described use case. I cannot imagine holding the body and head of PL routines in different places and I don't understand the necessity to join it. It seems a little weird to me too,

Wrong buffer limits check

2024-01-29 Thread Mikhail Gribkov
Hi hackers, I have tried to analyse Postgres code with Svace static analyzer [1] and found something I think is a real bug. In pgp-decrypt.c, in prefix_init function the following check: if (len > sizeof(tmpbuf)) seem to be erroneous and should really look this way: if (len > PGP_MAX_BLOCK)

RE: Finding every use of a built-in function

2024-01-29 Thread Kurlaev Jaroslav
> plpgsq_check can show dependencies > https://github.com/okbob/plpgsql_check#dependency-list That's not exactly what I was looking for, but it's a useful tool that could help me anyway. Thanks!

Re: PG versus libxml2 2.12.x

2024-01-29 Thread Andrew Dunstan
On 2024-01-27 Sa 14:04, Tom Lane wrote: Buildfarm member caiman has been failing build for a couple weeks now. The reason turns out to be that recent libxml2 has decided to throw a "const" into the signature required for custom error handlers. (API compatibility? What's that?) I don't mind

Re: Race condition in FetchTableStates() breaks synchronization of subscription tables

2024-01-29 Thread vignesh C
On Fri, 26 Jan 2024 at 11:30, Alexander Lakhin wrote: > > Hello hackers, > > After determining a possible cause for intermittent failures of the test > subscription/031_column_list [1], I was wondering what makes another > subscription test (014_binary) fail on the buildfarm: >

RE: Synchronizing slots from primary to standby

2024-01-29 Thread Zhijie Hou (Fujitsu)
On Monday, January 29, 2024 7:30 PM Amit Kapila wrote: > > On Mon, Jan 29, 2024 at 3:11 PM shveta malik > wrote: > > > > PFA v71 patch set with above changes. > > > > Few comments on 0001 Thanks for the comments. > === > 1. > parse_subscription_options() > { > ... > /* > *

separating use of SerialSLRULock

2024-01-29 Thread Alvaro Herrera
Hello In Dilip's patch to improve SLRU concurrency, there's a requirement to prevent predicate.c's SLRU control lock from being used to control access to another shared memory structure, SerialControlData. This struct is used to keep track of the areas of the SLRU that are valid. Dilip just

Re: RFC: Logging plan of the running query

2024-01-29 Thread torikoshia
Hi, Updated the patch to fix typos and move ProcessLogQueryPlanInterruptActive from errfinish() to AbortTransaction. BTW since the thread is getting long, I list the some points of the discussion so far: # Safety concern ## Catalog access inside CFI - it seems safe if the CFI call is

Re: CREATE INDEX CONCURRENTLY on partitioned index

2024-01-29 Thread Alexander Pyhalov
Hi. I've rebased patch on master and it'seems to me there's one more issue - when we call DefineIndexConcurrentInternal() in partitioned case, it waits for transactions, locking tableId, not tabrelid - heaprelid LockRelId is constructed for parent index relation, not for child index

Re: Functions to return random numbers in a given range

2024-01-29 Thread Dean Rasheed
On Fri, 26 Jan 2024 at 20:44, David Zhang wrote: > > Thank you for the patch. > > I applied this patch manually to the master branch, resolving a conflict > in `numeric.h`. It successfully passed both `make check` and `make > check-world`. > Thanks for testing. Interestingly, the cfbot didn't

Incorrect cost for MergeAppend

2024-01-29 Thread Alexander Kuzmenkov
Hello hackers, While investigating some query plans, I noticed some code that seems to be wrong: when create_merge_append_path() estimates the cost of sorting an input, it calls cost_sort() passing subpath->parent->tuples as the number of tuples. Shouldn't it use subpath->parent->rows or even

Re: Functions to return random numbers in a given range

2024-01-29 Thread Dean Rasheed
On Thu, 28 Dec 2023 at 07:34, jian he wrote: > > Your patch works. > performance is the best amount for other options in [0]. > I don't have deep knowledge about which one is more random. > Thanks for testing. > Currently we have to explicitly mention the lower and upper bound. > but can we do

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-01-29 Thread Jelte Fennema-Nio
On Mon, 29 Jan 2024 at 12:44, Alvaro Herrera wrote: > Thanks! I committed 0001 now. I also renamed the new > pq_parse_int_param to pqParseIntParam, for consistency with other > routines there. Please rebase the other patches. Awesome! Rebased, and renamed pq_release_conn_hosts to

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Tomas Vondra
On 1/29/24 09:53, Ronan Dunklau wrote: > Le dimanche 28 janvier 2024, 22:57:02 CET Tomas Vondra a écrit : > > Hi Tomas ! > > I'll comment on glibc-malloc part as I studied that part last year, and > proposed some things here: https://www.postgresql.org/message-id/ >

Re: UUID v7

2024-01-29 Thread Jelte Fennema-Nio
On Thu, 25 Jan 2024 at 13:31, Aleksander Alekseev wrote: > PFA v14. +uuidv4 () uuid + + Both functions return a version 4 (random) UUID. This is the most commonly + used type of UUID and is appropriate when random distribution of keys does + not affect performance of an application. +

Re: Retiring is_pushed_down

2024-01-29 Thread Richard Guo
On Sun, Jan 21, 2024 at 8:37 PM vignesh C wrote: > I'm seeing that there has been no activity in this thread for nearly 6 > months, I'm planning to close this in the current commitfest unless > someone is planning to take it forward. It can be opened again when > there is more interest. I'm

Re: [PoC] Improve dead tuple storage for lazy vacuum

2024-01-29 Thread John Naylor
On Mon, Jan 29, 2024 at 2:29 PM Masahiko Sawada wrote: > > > +/* > > > + * Calculate the slab blocksize so that we can allocate at least 32 > > > chunks > > > + * from the block. > > > + */ > > > +#define RT_SLAB_BLOCK_SIZE(size) \ > > > + Max((SLAB_DEFAULT_BLOCK_SIZE / (size)) * (size), (size)

Re: New Table Access Methods for Multi and Single Inserts

2024-01-29 Thread Bharath Rupireddy
On Mon, Jan 29, 2024 at 12:57 PM Bharath Rupireddy wrote: > > On Wed, Jan 17, 2024 at 10:57 PM Bharath Rupireddy > wrote: > > > > Thank you. I'm attaching v8 patch-set here which includes use of new > > insert TAMs for COPY FROM. With this, postgres not only will have the > > new TAM for

Re: [EXTERNAL] Re: Add non-blocking version of PQcancel

2024-01-29 Thread Alvaro Herrera
On 2024-Jan-28, Jelte Fennema-Nio wrote: > On Sun, 28 Jan 2024 at 10:51, Jelte Fennema-Nio wrote: > > Both of those are fixed now. > > Okay, there turned out to also be an issue on Windows with > setKeepalivesWin32 not being available in fe-cancel.c. That's fixed > now too (as well as some

Re: UUID v7

2024-01-29 Thread Jelte Fennema-Nio
tl;dr I believe we should remove the uuidv7(timestamp) function from this patchset. On Thu, 25 Jan 2024 at 18:04, Sergey Prokhorenko wrote: > In this case the documentation must state that the functions > uuid_extract_time() and uuidv7(T) are against the RFC requirements, and that > developers

Re: Synchronizing slots from primary to standby

2024-01-29 Thread Amit Kapila
On Mon, Jan 29, 2024 at 3:11 PM shveta malik wrote: > > PFA v71 patch set with above changes. > Few comments on 0001 === 1. parse_subscription_options() { ... /* * We've been explicitly asked to not connect, that requires some * additional processing. */ if (!opts->connect &&

Re: Documentation to upgrade logical replication cluster

2024-01-29 Thread Bharath Rupireddy
On Mon, Jan 29, 2024 at 10:10 AM vignesh C wrote: > > Thanks for the comments, the attached v5 version patch has the changes > for the same. Thanks for working on this. Here are some comments on the v5 patch: 1. + + Migration of logical replication clusters is possible only when all the +

Re: MERGE ... WHEN NOT MATCHED BY SOURCE

2024-01-29 Thread Dean Rasheed
On Fri, 26 Jan 2024 at 15:57, Alvaro Herrera wrote: > > Well, firstly this is clearly a feature we want to have, even though > it's non-standard, because people use it and other implementations have > it. (Eh, so maybe somebody should be talking to the SQL standard > committee about it). As for

Re: Make COPY format extendable: Extract COPY TO format implementations

2024-01-29 Thread Sutou Kouhei
Hi, In "Re: Make COPY format extendable: Extract COPY TO format implementations" on Mon, 29 Jan 2024 11:37:07 +0800, Junwang Zhao wrote: >> > > Does it make sense to pass only non-builtin options to the custom >> > > format callback after parsing and evaluating the builtin options? That

Re: Some revises in adding sorting path

2024-01-29 Thread Richard Guo
On Mon, Jul 17, 2023 at 4:55 PM Richard Guo wrote: > But I did not find a query that makes an incremental sort in this case. > After trying for a while it seems to me that we do not need to consider > incremental sort in this case, because for a partial path of a grouped > or partially grouped

Re: proposal: psql: show current user in prompt

2024-01-29 Thread Pavel Stehule
ne 28. 1. 2024 v 22:52 odesílatel Jelte Fennema-Nio napsal: > On Sun, 28 Jan 2024 at 20:01, Pavel Stehule > wrote: > > There is another reason - compatibility with other drivers. We maintain > just libpq, but there are JDBC, Npgsql, and some native Python drivers. I > didn't checked, but maybe

Re: Synchronizing slots from primary to standby

2024-01-29 Thread Bertrand Drouvot
Hi, On Mon, Jan 29, 2024 at 02:35:52PM +0530, Amit Kapila wrote: > On Mon, Jan 29, 2024 at 2:22 PM Bertrand Drouvot > wrote: > > Looking at 0001: > > > > + When altering the > > + > linkend="sql-createsubscription-params-with-slot-name">slot_name, > > + the failover property

Re: Synchronizing slots from primary to standby

2024-01-29 Thread Amit Kapila
On Mon, Jan 29, 2024 at 2:22 PM Bertrand Drouvot wrote: > > On Mon, Jan 29, 2024 at 10:24:11AM +0530, shveta malik wrote: > > On Sat, Jan 27, 2024 at 12:02 PM Zhijie Hou (Fujitsu) > > wrote: > > > > > > Attach the V70 patch set which addressed above comments and Bertrand's > > > comments in [1]

Re: scalability bottlenecks with (many) partitions (and more)

2024-01-29 Thread Ronan Dunklau
Le dimanche 28 janvier 2024, 22:57:02 CET Tomas Vondra a écrit : Hi Tomas ! I'll comment on glibc-malloc part as I studied that part last year, and proposed some things here: https://www.postgresql.org/message-id/ 3424675.QJadu78ljV%40aivenlaptop > FWIW where does the malloc overhead come

Re: Synchronizing slots from primary to standby

2024-01-29 Thread Bertrand Drouvot
Hi, On Mon, Jan 29, 2024 at 10:24:11AM +0530, shveta malik wrote: > On Sat, Jan 27, 2024 at 12:02 PM Zhijie Hou (Fujitsu) > wrote: > > > > Attach the V70 patch set which addressed above comments and Bertrand's > > comments in [1] > > > > Since v70-0001 is pushed, rebased and attached v70_2

Returning non-terminated string in ECPG Informix-compatible function

2024-01-29 Thread o . tselebrovskiy
Greetings, everyone! While analyzing output of Svace static analyzer [1] I've found a bug. In function intoasc(interval * i, char *str) from file src/interfaces/ecpg/compatlib/informix.c we return a non-terminated string since we use memcpy on tmp which is itself NULL-teminated but last zero

Re: Change COPY ... ON_ERROR ignore to ON_ERROR ignore_row

2024-01-29 Thread Yugo NAGATA
On Fri, 26 Jan 2024 08:08:29 -0700 "David G. Johnston" wrote: > Hi, > > The option choice of "ignore" in the COPY ON_ERROR clause seems overly > generic. There would seem to be two relevant ways to ignore bad column > input data - drop the entire row or just set the column value to null. I >

<    1   2