Re: logical decoding and replication of sequences, take 2

2024-02-20 Thread Robert Haas
On Wed, Feb 21, 2024 at 1:06 PM Dilip Kumar wrote: > > But I am wondering why this flag is always set to true in > > DecodeTXNNeedSkip() irrespective of the commit or abort. Because the > > aborted transactions are not supposed to be replayed? So if my > > observation is correct that for the

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-02-20 Thread Peter Eisentraut
On 21.02.24 07:40, Michael Paquier wrote: This means that all partitioned tables would have pg_class.relam set, and that relam would never be 0: - The USING clause takes priority over default_table_access_method. - If no USING clause, default_table_access_method is the AM used Any partitions

Re: Injection points: some tools to wait and wake

2024-02-20 Thread Michael Paquier
On Wed, Feb 21, 2024 at 07:08:03AM +0900, Michael Paquier wrote: > Well, both you and Andrey are asking for it now, so let's do it. The > implementation is simple: > - Store in InjectionPointSharedState an array of wait_counts and an > array of names. There is only one condition variable. > -

Re: logical decoding and replication of sequences, take 2

2024-02-20 Thread Dilip Kumar
On Tue, Feb 20, 2024 at 4:32 PM Dilip Kumar wrote: > > On Tue, Feb 20, 2024 at 3:38 PM Robert Haas wrote: > > > Let's say fast_forward is true. Then smgr_decode() is going to skip > > recording anything about the relfilenode, so we'll identify all > > sequence changes as non-transactional. But

Re: Removing unneeded self joins

2024-02-20 Thread Richard Guo
On Mon, Feb 19, 2024 at 1:24 PM Andrei Lepikhov wrote: > On 18/2/2024 23:18, Alexander Korotkov wrote: > > On Sun, Feb 18, 2024 at 5:04 PM Alexander Korotkov > wrote: > >> On Sun, Feb 18, 2024 at 3:00 PM Alexander Lakhin > wrote: > >>> Please look at the following query which fails with an

Re: What about Perl autodie?

2024-02-20 Thread Peter Eisentraut
On 14.02.24 17:52, Peter Eisentraut wrote: A gentler way might be to start using some perlcritic policies like InputOutput::RequireCheckedOpen or the more general InputOutput::RequireCheckedSyscalls and add explicit error checking at the sites it points out. Here is a start for that. I

Re: Add lookup table for replication slot invalidation causes

2024-02-20 Thread Bharath Rupireddy
On Wed, Feb 21, 2024 at 11:56 AM Michael Paquier wrote: > > Note the recent commit 74a730631065 where Alvaro has changed for the > lwlock tranche names. That's quite elegant. Yes, that's absolutely neat. FWIW, designated initializer syntax can be used in a few more places though. I'm not sure

Test to dump and restore objects left behind by regression

2024-02-20 Thread Ashutosh Bapat
Hi All, In [1] we found that having a test to dump and restore objects left behind by regression test is missing. Such a test would cover many dump restore scenarios without much effort. It will also help identity problems described in the same thread [2] during development itself. I am starting

Re: Synchronizing slots from primary to standby

2024-02-20 Thread shveta malik
On Tue, Feb 20, 2024 at 6:19 PM Masahiko Sawada wrote: > > On Tue, Feb 20, 2024 at 12:33 PM shveta malik wrote: > Thank you for the explanation. It makes sense to me to move the check. > > > 2. If the wal_level is not logical, the server will need to restart > anyway to change the wal_level and

Re: A new message seems missing a punctuation

2024-02-20 Thread Amit Kapila
On Tue, Feb 20, 2024 at 7:25 PM shveta malik wrote: > > On Tue, Feb 20, 2024 at 5:01 PM Amit Kapila wrote: > > > > > > We do expose the required information (restart_lsn, catalog_xmin, > > synced, temporary, etc) via pg_replication_slots. So, I feel the LOG > > message here is sufficient to

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-02-20 Thread Michael Paquier
On Tue, Feb 20, 2024 at 03:47:46PM +0100, Peter Eisentraut wrote: > It would be helpful if this patch could more extensively document in its > commit message what semantic changes it makes. Various options of possible > behaviors were discussed in this thread, but it's not clear which behaviors >

Re: logical decoding and replication of sequences, take 2

2024-02-20 Thread Amit Kapila
On Wed, Feb 14, 2024 at 10:21 PM Tomas Vondra wrote: > > On 2/13/24 17:37, Robert Haas wrote: > > > In other words, the fact that some sequence changes are > > non-transactional creates ordering hazards that don't exist if there > > are no non-transactional changes. So in that way, sequences are

Re: Unlinking Parallel Hash Join inner batch files sooner

2024-02-20 Thread Andrei Lepikhov
Hi, I see in [1] that the reporter mentioned a delay between the error message in parallel HashJoin and the return control back from PSQL. Your patch might reduce this delay. Also, I have the same complaint from users who processed gigabytes of data in parallel HashJoin. Presumably, they also

Re: Add lookup table for replication slot invalidation causes

2024-02-20 Thread Michael Paquier
On Wed, Feb 21, 2024 at 09:49:37AM +0530, Bharath Rupireddy wrote: > On Wed, Feb 21, 2024 at 5:04 AM Michael Paquier wrote: > Done that way. I'm fine with the designated initialization [1] that an > ISO C99 compliant compiler offers. PostgreSQL installation guide >

BRIN integer overflow

2024-02-20 Thread Oleg Tselebrovskiy
Greetings, everyone! While analyzing output of Svace static analyzer [1] I've found a bug Function bringetbitmap that is used in BRIN's IndexAmRoutine should return an int64 value, but the actual return value is int, since totalpages is int and totalpages * 10 is also int. This could lead to

Re: WIP Incremental JSON Parser

2024-02-20 Thread Andrew Dunstan
On 2024-02-20 Tu 19:53, Jacob Champion wrote: On Tue, Feb 20, 2024 at 2:10 PM Andrew Dunstan wrote: Well, that didn't help a lot, but meanwhile the CFBot seems to have decided in the last few days that it's now happy, so full steam aead! ;-) I haven't been able to track down the root cause

Re: Speeding up COPY TO for uuids and arrays

2024-02-20 Thread Michael Paquier
On Mon, Feb 19, 2024 at 03:08:45PM +0100, Laurenz Albe wrote: > On Sat, 2024-02-17 at 12:24 -0800, Andres Freund wrote: >> I wonder if we should move the core part for converting to hex to numutils.c, >> we already have code the for the inverse. There does seem to be further >> optimization

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Ajin Cherian
On Wed, Feb 21, 2024 at 4:09 PM Hayato Kuroda (Fujitsu) < kuroda.hay...@fujitsu.com> wrote: > Dear Horiguchi-san, > > > GetConnection()@streamutil.c wants to ensure conninfo has a fallback > > database name ("replication"). However, the function seems to be > > ignoring the case where neither

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-02-20 Thread Bharath Rupireddy
On Tue, Feb 20, 2024 at 12:05 PM Bharath Rupireddy wrote: > >> [...] and was able to produce something like: > > > > postgres=# select > > slot_name,slot_type,active,active_pid,wal_status,invalidation_reason from > > pg_replication_slots; > > slot_name | slot_type | active | active_pid |

RE: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Horiguchi-san, > GetConnection()@streamutil.c wants to ensure conninfo has a fallback > database name ("replication"). However, the function seems to be > ignoring the case where neither dbname nor connection string is given, > which is the first case Kuroda-san raised. The second case is

Re: logical decoding and replication of sequences, take 2

2024-02-20 Thread Amit Kapila
On Tue, Feb 20, 2024 at 5:39 PM Tomas Vondra wrote: > > On 2/20/24 06:54, Amit Kapila wrote: > > On Thu, Dec 21, 2023 at 6:47 PM Tomas Vondra > > wrote: > >> > >> On 12/19/23 13:54, Christophe Pettus wrote: > >>> Hi, > >>> > >>> I wanted to hop in here on one particular issue: > >>> > On

Re: 'Shutdown <= SmartShutdown' check while launching processes in postmaster.

2024-02-20 Thread Tom Lane
shveta malik writes: > I would like to know that why we have 'Shutdown <= SmartShutdown' > check before launching few processes (WalReceiver, WalSummarizer, > AutoVacuum worker) while rest of the processes (BGWriter, WalWriter, > Checkpointer, Archiver etc) do not have any such check. If I have

Re: Add system identifier to backup manifest

2024-02-20 Thread Michael Paquier
On Mon, Feb 19, 2024 at 12:06:19PM +0530, Amul Sul wrote: > On Mon, Feb 19, 2024 at 4:22 AM Michael Paquier wrote: >> And the new option should be documented at the top of the init() >> routine for perldoc. > > Added in the attached version. I've done some wordsmithing on 0001 and it is OK, so

Re: Add lookup table for replication slot invalidation causes

2024-02-20 Thread Bharath Rupireddy
On Wed, Feb 21, 2024 at 5:04 AM Michael Paquier wrote: > > On Tue, Feb 20, 2024 at 05:53:03PM +0100, Jelte Fennema-Nio wrote: > > Seems like a good improvement overall. But I'd prefer the definition > > of the lookup table to use this syntax: > > > > const char *const SlotInvalidationCauses[] = {

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Ajin Cherian
On Wed, Feb 21, 2024 at 2:04 PM Kyotaro Horiguchi wrote: > > Although I haven't looked the original thread, it seems that the > dbname is used only by pg_sync_replication_slots(). If it is true, > couldn't we make the SQL function require a database name to make a > connection, instead of

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Ajin Cherian
On Wed, Feb 21, 2024 at 2:04 PM Kyotaro Horiguchi wrote: > > About the proposed patch, pg_basebackup cannot verify the validity of > the dbname. It could be problematic. > > Although I haven't looked the original thread, it seems that the > dbname is used only by pg_sync_replication_slots(). If

'Shutdown <= SmartShutdown' check while launching processes in postmaster.

2024-02-20 Thread shveta malik
Hi hackers, I would like to know that why we have 'Shutdown <= SmartShutdown' check before launching few processes (WalReceiver, WalSummarizer, AutoVacuum worker) while rest of the processes (BGWriter, WalWriter, Checkpointer, Archiver etc) do not have any such check. If I have to launch a new

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Kyotaro Horiguchi
At Tue, 20 Feb 2024 19:56:10 +0530, Robert Haas wrote in > It seems like maybe somebody should look into why this is happening, > and perhaps fix it. GetConnection()@streamutil.c wants to ensure conninfo has a fallback database name ("replication"). However, the function seems to be ignoring

RE: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Robert, > > Just FYI - here is an extreme case. And note that I have applied proposed > > patch. > > > > When `pg_basebackup -D data_N2 -R` is used: > > ``` > > primary_conninfo = 'user=hayato ... dbname=hayato ... > > ``` > > > > But when `pg_basebackup -d "" -D data_N2 -R` is used: > >

Re: Patch: Add parse_type Function

2024-02-20 Thread Erik Wienhold
On 2024-02-20 15:44 +0100, David E. Wheeler wrote: > I’ve tweaked the comments and their order in v7, attached. > > This goes back to the discussion of the error raising of > to_regtype[1], so I’ve incorporated the patch from that thread into > this patch, and set up the docs for to_regtypemod()

Re: [PoC] Federated Authn/z with OAUTHBEARER

2024-02-20 Thread Jacob Champion
Hi all, v14 rebases over latest and fixes a warning when assertions are disabled. 0006 is temporary and hacks past a couple of issues I have not yet root caused -- one of which makes me wonder if 0001 needs to be considered alongside the recent pg_combinebackup and incremental JSON work...?

Re: WIP Incremental JSON Parser

2024-02-20 Thread Jacob Champion
On Tue, Feb 20, 2024 at 2:10 PM Andrew Dunstan wrote: > Well, that didn't help a lot, but meanwhile the CFBot seems to have > decided in the last few days that it's now happy, so full steam aead! ;-) I haven't been able to track down the root cause yet, but I am able to reproduce the failure

Re: partitioning and identity column

2024-02-20 Thread Alexander Korotkov
On Tue, Feb 20, 2024 at 8:00 AM Alexander Lakhin wrote: > 20.02.2024 07:57, Ashutosh Bapat wrote: > >> Could you please name functions, which you suspect, for me to recheck them? > >> Perhaps we should consider fixing all of such functions, in light of > >> b0f7dd915 and d57b7cc33... > > Looks

Re: Lessons from using mmap()

2024-02-20 Thread Alexander Korotkov
Hi! On Wed, Feb 21, 2024 at 1:21 AM Bruce Momjian wrote: > I think we have come to the same conclusion in the past, but I thought > it would be good to share someone else's research, and it might be > helpful if we ever revisit this idea. I read this blog post before. In my personal opinion it

Re: Add bump memory context type and use it for tuplesorts

2024-02-20 Thread David Rowley
On Wed, 21 Feb 2024 at 00:02, Matthias van de Meent wrote: > > On Tue, 20 Feb 2024 at 11:02, David Rowley wrote: > > On Fri, 26 Jan 2024 at 01:29, Matthias van de Meent > > wrote: > > > >> +allocSize = MAXALIGN(sizeof(BumpContext)) + Bump_BLOCKHDRSZ + > > > >> +if (minContextSize != 0)

Re: Change the bool member of the Query structure to bits

2024-02-20 Thread Quan Zongliang
On 2024/2/20 19:18, Tomas Vondra wrote: On 2/20/24 11:11, Quan Zongliang wrote: Sorry. I forgot to save a file. This is the latest. On 2024/2/20 18:07, Quan Zongliang wrote: The Query structure has an increasing number of bool attributes. This is likely to increase in the future. And

Re: Change the bool member of the Query structure to bits

2024-02-20 Thread Quan Zongliang
On 2024/2/20 23:45, Tom Lane wrote: Quan Zongliang writes: The Query structure has an increasing number of bool attributes. This is likely to increase in the future. And they have the same properties. Wouldn't it be better to store them in bits? Common statements don't use them, so they

Re: Add lookup table for replication slot invalidation causes

2024-02-20 Thread Michael Paquier
On Tue, Feb 20, 2024 at 05:53:03PM +0100, Jelte Fennema-Nio wrote: > Seems like a good improvement overall. But I'd prefer the definition > of the lookup table to use this syntax: > > const char *const SlotInvalidationCauses[] = { > [RS_INVAL_NONE] = "none", > [RS_INVAL_WAL_REMOVED] =

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

2024-02-20 Thread Michail Nikolaev
Hello! > I think the best way for this to work would be an index method that > exclusively stores TIDs, and of which we can quickly determine new > tuples, too. I was thinking about something like GIN's format, but > using (generation number, tid) instead of ([colno, colvalue], tid) as > key data

Re: Add bump memory context type and use it for tuplesorts

2024-02-20 Thread David Rowley
On Tue, 20 Feb 2024 at 23:52, Matthias van de Meent wrote: > What I meant was that > > > (char *) block + Bump_BLOCKHDRSZ > vs > > ((char *) block) + Bump_BLOCKHDRSZ > > , when combined with my little experience with pointer addition and > precedence, and a lack of compiler at the ready at that

Re: Injection points: some tools to wait and wake

2024-02-20 Thread Michael Paquier
On Tue, Feb 20, 2024 at 05:32:38PM +0300, Andrey M. Borodin wrote: > I will try to simplify test to 2-step, but it would be much easier > to implement if injection points could be awaken independently. I don't mind implementing something that wakes only a given point name, that's just more state

Lessons from using mmap()

2024-02-20 Thread Bruce Momjian
This blog, and the blogs it links to, explains the complexities of using mmap() for database data/index file I/O. https://www.symas.com/post/are-you-sure-you-want-to-use-mmap-in-your-dbms The blog starts by stating: There are, however, severe correctness and performance issues

Re: Injection points: some tools to wait and wake

2024-02-20 Thread Michael Paquier
On Tue, Feb 20, 2024 at 03:55:08PM +, Bertrand Drouvot wrote: > Okay, makes sense to keep this as it is as a "template" in case more stuff is > added. > > + /* Counter advancing when injection_points_wake() is called */ > + int wait_counts; > > In that case

Re: System username in pg_stat_activity

2024-02-20 Thread Magnus Hagander
On Fri, Feb 16, 2024 at 9:45 PM Andres Freund wrote: > > Hi, > > On 2024-02-16 15:22:16 -0500, Tom Lane wrote: > > Magnus Hagander writes: > > > I mean, we could split it into more than one view. But adding a new > > > view for every new thing we want to show is also not very good from > > >

Re: System username in pg_stat_activity

2024-02-20 Thread Magnus Hagander
On Fri, Feb 16, 2024 at 9:31 PM Magnus Hagander wrote: > > On Fri, Feb 16, 2024 at 9:20 PM Andres Freund wrote: > > > > Hi, > > > > On 2024-02-16 20:57:59 +0100, Magnus Hagander wrote: > > > On Fri, Feb 16, 2024 at 8:41 PM Andres Freund wrote: > > > > On 2024-01-10 12:46:34 +0100, Magnus

Re: Shared detoast Datum proposal

2024-02-20 Thread Andy Fan
Tomas Vondra writes: Hi Tomas, > > I took a quick look on this thread/patch today, so let me share a couple > initial thoughts. I may not have a particularly coherent/consistent > opinion on the patch or what would be a better way to do this yet, but > perhaps it'll start a discussion ...

Re: Running the fdw test from the terminal crashes into the core-dump

2024-02-20 Thread Tom Lane
Alvaro Herrera writes: > The real problem is that a MERGE ... DO NOTHING action reports that no > permissions need to be checked on the target relation, which is not a > problem when there are other actions in the MERGE command since they add > privs to check. When DO NOTHING is the only action,

Re: Running the fdw test from the terminal crashes into the core-dump

2024-02-20 Thread Alvaro Herrera
On 2024-Feb-18, Alvaro Herrera wrote: > On 2024-Feb-18, Tom Lane wrote: > > > So I'd blame this on faulty handling of the zero-partitions case in > > the RTEPermissionInfo refactoring. (I didn't bisect to prove that > > a61b1f748 is exactly where it broke, but I do see that the query > >

Re: Proposal: Adjacent B-Tree index

2024-02-20 Thread Dilshod Urazov
> only 1 index lookup is needed. Sorry, must be "only lookups of 1 index are needed". -- Dilshod Urazov вт, 20 февр. 2024 г. в 21:09, Dilshod Urazov : > > I'm not sure why are two indexes not sufficient here? > > Did I write that they are not sufficient? The whole point is that in > relational

Re: Missing Group Key in grouped aggregate

2024-02-20 Thread Tomas Vondra
On 2/20/24 16:53, Tom Lane wrote: > =?UTF-8?Q?Erik_Nordstr=C3=B6m?= writes: >> I noticed that, beginning with PG16, grouped aggregates are missing the >> "Group Key" in the EXPLAIN output. > >> It seems the Agg node has numCols (number of grouping cols) set to zero in >> queries like > >>

Re: Shared detoast Datum proposal

2024-02-20 Thread Tomas Vondra
Hi, I took a quick look on this thread/patch today, so let me share a couple initial thoughts. I may not have a particularly coherent/consistent opinion on the patch or what would be a better way to do this yet, but perhaps it'll start a discussion ... The goal of the patch (as I understand it)

Re: Proposal: Adjacent B-Tree index

2024-02-20 Thread Dilshod Urazov
> I'm not sure why are two indexes not sufficient here? Did I write that they are not sufficient? The whole point is that in relational DBMSs which are widely used to store graphs we can optimize storage in such cases. Also we can optimize traversals e.g. if we want to get all nodes that are

Re: Integer undeflow in fprintf in dsa.c

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 17:13, Ilyasov Ian wrote: > Sorry for not answering quickly. There is no need for any apology, there is no obligation to answer within any specific timeframe. > I attached a patch to the letter with changes to take into account Daniel > Gustafsson's comment. Looks good

Re: Add lookup table for replication slot invalidation causes

2024-02-20 Thread Jelte Fennema-Nio
On Tue, 20 Feb 2024 at 12:11, Bharath Rupireddy wrote: > Thoughts? Seems like a good improvement overall. But I'd prefer the definition of the lookup table to use this syntax: const char *const SlotInvalidationCauses[] = { [RS_INVAL_NONE] = "none", [RS_INVAL_WAL_REMOVED] =

Re: Streaming read-ready sequential scan code

2024-02-20 Thread Melanie Plageman
On Tue, Feb 20, 2024 at 6:13 AM Robert Haas wrote: > > On Tue, Feb 20, 2024 at 4:35 AM Melanie Plageman > wrote: > > I've written three alternative implementations of the actual streaming > > read user for sequential scan which handle the question of where to > > allocate the streaming read

RE: Integer undeflow in fprintf in dsa.c

2024-02-20 Thread Ilyasov Ian
Sorry for not answering quickly. Thank you for your comments. I attached a patch to the letter with changes to take into account Daniel Gustafsson's comment. Kind regards, Ian Ilyasov. Juniour Software Developer at Postgres Professional Subject: [PATCH] Integer underflow fix in fprintf in

Re: Fix race condition in InvalidatePossiblyObsoleteSlot()

2024-02-20 Thread Bertrand Drouvot
Hi, On Tue, Feb 20, 2024 at 02:33:44PM +0900, Michael Paquier wrote: > On Tue, Feb 20, 2024 at 08:51:17AM +0900, Michael Paquier wrote: > > Prefixing these with "initial_" is fine, IMO. That shows the > > intention that these come from the slot's data before doing the > > termination. So I'm OK

Re: pg_restore problem to load constraints with tables

2024-02-20 Thread Tom Lane
Fabrice Chapuis writes: > When a table is reloaded wit pg_restore, it is recreated without indexes or > constraints. There are automatically skipped. Is there a reason for this? [ shrug ] That's how the -t switch is defined. If you want something else, you can use the -l and -L switches to pick

Re: Injection points: some tools to wait and wake

2024-02-20 Thread Bertrand Drouvot
Hi, On Tue, Feb 20, 2024 at 08:28:28AM +0900, Michael Paquier wrote: > On Mon, Feb 19, 2024 at 02:28:04PM +, Bertrand Drouvot wrote: > > If slock_t protects "only" one counter, then what about using > > pg_atomic_uint64 > > or pg_atomic_uint32 instead? And btw do we need wait_counts at all?

Re: Missing Group Key in grouped aggregate

2024-02-20 Thread Tom Lane
=?UTF-8?Q?Erik_Nordstr=C3=B6m?= writes: > I noticed that, beginning with PG16, grouped aggregates are missing the > "Group Key" in the EXPLAIN output. > It seems the Agg node has numCols (number of grouping cols) set to zero in > queries like > SELECT foo, count(*) FROM bar WHERE foo=1 GROUP BY

Re: Change the bool member of the Query structure to bits

2024-02-20 Thread Tom Lane
Quan Zongliang writes: > The Query structure has an increasing number of bool attributes. This is > likely to increase in the future. And they have the same properties. > Wouldn't it be better to store them in bits? Common statements don't use > them, so they have little impact. This also

Re: numeric_big in make check?

2024-02-20 Thread Dean Rasheed
On Tue, 20 Feb 2024 at 15:16, Tom Lane wrote: > > Dean Rasheed writes: > > Looking at the script itself, the addition, subtraction, > > multiplication and division tests at the top are probably pointless, > > since I would expect those operations to be tested adequately (and > > probably more

Re: numeric_big in make check?

2024-02-20 Thread Tom Lane
Dean Rasheed writes: > Looking at the script itself, the addition, subtraction, > multiplication and division tests at the top are probably pointless, > since I would expect those operations to be tested adequately (and > probably more thoroughly) by the transcendental test cases. In fact, I >

Re: bug report: some issues about pg_15_stable(8fa4a1ac61189efffb8b851ee77e1bc87360c445)

2024-02-20 Thread Dean Rasheed
On Tue, 20 Feb 2024 at 14:49, Dean Rasheed wrote: > > Also, if the concurrent update were an update of a key > column that was included in the join condition, the re-scan would > follow the update to a new matching source row, which is inconsistent > with what would happen if it were a join to a

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 13:40, Peter Eisentraut wrote: > > On 20.02.24 12:39, Daniel Gustafsson wrote: >> A fifth option is to throw away our in-tree implementations and use the >> OpenSSL >> API's for everything, which is where this thread started. If the effort to >> payoff ratio is palatable

Re: bug report: some issues about pg_15_stable(8fa4a1ac61189efffb8b851ee77e1bc87360c445)

2024-02-20 Thread Dean Rasheed
On Mon, 19 Feb 2024 at 17:48, zwj wrote: > > Hello, > >I found an issue while using the latest version of PG15 > (8fa4a1ac61189efffb8b851ee77e1bc87360c445). >This question is about 'merge into'. > >When two merge into statements are executed concurrently, I obtain the > following

Re: to_regtype() Raises Error

2024-02-20 Thread David E. Wheeler
On Feb 5, 2024, at 09:01, David E. Wheeler wrote: > Ah, thank you. Updated patch attached. I’ve moved this patch into the to_regtype patch thread[1], since it exhibits the same behavior. Best, David [1]

Re: ALTER TABLE SET ACCESS METHOD on partitioned tables

2024-02-20 Thread Peter Eisentraut
On 19.07.23 20:13, Justin Pryzby wrote: On Thu, Jun 01, 2023 at 08:50:50AM -0400, Michael Paquier wrote: On Wed, May 31, 2023 at 06:35:34PM -0500, Justin Pryzby wrote: What do you think the comment ought to say ? It already says: src/backend/catalog/heap.c- * Make a dependency

Re: Patch: Add parse_type Function

2024-02-20 Thread David E. Wheeler
On Feb 20, 2024, at 01:30, jian he wrote: > the second hint `-- grammar error expected` seems to contradict with > the results? Quite right, thank you, that’s actually a trapped error. I’ve tweaked the comments and their order in v7, attached. This goes back to the discussion of the error

pg_restore problem to load constraints with tables

2024-02-20 Thread Fabrice Chapuis
Hi, When a table is reloaded wit pg_restore, it is recreated without indexes or constraints. There are automatically skipped. Is there a reason for this? g_restore -j 8 -v -d zof /shared/pgdump/aq/backup/dbtest/shtest --no-owner --role=test -t mytable 2>&1 | tee -a dbest.log pg_restore:

Re: Injection points: some tools to wait and wake

2024-02-20 Thread Andrey M. Borodin
> On 20 Feb 2024, at 02:21, Michael Paquier wrote: > > On Mon, Feb 19, 2024 at 11:54:20AM +0300, Andrey M. Borodin wrote: >> 1. injection_points_wake() will wake all of waiters. But it's not >> suitable for complex tests. I think there must be a way to wake only >> specific waiter by

Re: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Robert Haas
On Tue, Feb 20, 2024 at 5:58 PM Hayato Kuroda (Fujitsu) wrote: > > Seems weird to me. You don't use dbname=replication to ask for a > > replication connection, so why would we ever end up with that > > anywhere? And especially in only one of two such closely related > > cases? > > Just FYI - here

Re: A new message seems missing a punctuation

2024-02-20 Thread shveta malik
On Tue, Feb 20, 2024 at 5:01 PM Amit Kapila wrote: > > > We do expose the required information (restart_lsn, catalog_xmin, > synced, temporary, etc) via pg_replication_slots. So, I feel the LOG > message here is sufficient to DEBUG (or know the details) when the > slot sync doesn't succeed. >

Re: Thoughts about NUM_BUFFER_PARTITIONS

2024-02-20 Thread Tomas Vondra
Hi, On 2/20/24 03:16, wenhui qiu wrote: > Hi Heikki Linnakangas >I saw git log found this commit: > https://github.com/postgres/postgres/commit/3acc10c997f916f6a741d0b4876126b7b08e3892 > ,I don't seem to see an email discussing this commit. As the commit log > tells us, we don't know exactly

Re: numeric_big in make check?

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 14:23, Dean Rasheed wrote: > If we did that, numeric_big would be even further down the list of > expensive tests, and I'd say it should be run by default. My motivation for raising this was to get a test which is executed as part of parallel_schedule to make failures

Re: speed up a logical replica setup

2024-02-20 Thread vignesh C
On Mon, 19 Feb 2024 at 11:15, Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > > Since it may be useful, I will post top-up patch on Monday, if there are no > > updating. > > And here are top-up patches. Feel free to check and include. > > v22-0001: Same as v21-0001. > === rebased patches ===

Re: confirmed flush lsn seems to be move backward in certain error cases

2024-02-20 Thread vignesh C
On Fri, 16 Feb 2024 at 17:39, vignesh C wrote: > > Hi, > > The following assertion failure was seen while testing one scenario > for other patch: > TRAP: failed Assert("s->data.confirmed_flush >= > s->last_saved_confirmed_flush"), File: "slot.c", Line: 1760, PID: > 545314 > postgres: checkpointer

Re: Synchronizing slots from primary to standby

2024-02-20 Thread Amit Kapila
On Tue, Feb 20, 2024 at 6:19 PM Masahiko Sawada wrote: > > Thank you for the explanation. It makes sense to me to move the check. > > As for ValidateSlotSyncParams() called by SlotSyncWorkerAllowed(), I > have two comments: > > 1. The error messages are not very descriptive and seem not to match

Re: numeric_big in make check?

2024-02-20 Thread Dean Rasheed
On Mon, 19 Feb 2024 at 15:35, Tom Lane wrote: > > I thought I'd try to acquire some actual facts here, so I compared > the code coverage shown by "make check" as of HEAD, versus "make > check" after adding numeric_big to parallel_schedule. I saw the > following lines of numeric.c as being

Re: Synchronizing slots from primary to standby

2024-02-20 Thread Masahiko Sawada
On Tue, Feb 20, 2024 at 12:44 PM Amit Kapila wrote: > > On Tue, Feb 20, 2024 at 8:25 AM Masahiko Sawada wrote: > > > > Some comments not related to the patch but to the existing code: > > > > --- > > It might have already been discussed but is the > > src/backend/replication/logical the right

Re: Synchronizing slots from primary to standby

2024-02-20 Thread Masahiko Sawada
On Tue, Feb 20, 2024 at 12:33 PM shveta malik wrote: > > On Tue, Feb 20, 2024 at 8:25 AM Masahiko Sawada wrote: > > > > > > I've reviewed the v91 patch. Here are random comments: > > Thanks for the comments. > > > --- > > /* > > * Checks the remote server info. > > * > > - * We ensure that

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Peter Eisentraut
On 20.02.24 12:39, Daniel Gustafsson wrote: A fifth option is to throw away our in-tree implementations and use the OpenSSL API's for everything, which is where this thread started. If the effort to payoff ratio is palatable to anyone then patches are for sure welcome. The problem is that, as

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 13:24, Robert Haas wrote: > > On Tue, Feb 20, 2024 at 5:09 PM Daniel Gustafsson wrote: >> A fifth option is to throw away our in-tree implementations and use the >> OpenSSL >> API's for everything, which is where this thread started. If the effort to >> payoff ratio is

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Peter Eisentraut
On 20.02.24 12:27, Robert Haas wrote: I don't think the first two of these proposals help anything. AIUI, FIPS mode is supposed to be a system wide toggle that affects everything on the machine. The third one might help if you can be compliant by just choosing not to install that extension, and

RE: Have pg_basebackup write "dbname" in "primary_conninfo"?

2024-02-20 Thread Hayato Kuroda (Fujitsu)
Dear Robert, > Seems weird to me. You don't use dbname=replication to ask for a > replication connection, so why would we ever end up with that > anywhere? And especially in only one of two such closely related > cases? Just FYI - here is an extreme case. And note that I have applied proposed

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Robert Haas
On Tue, Feb 20, 2024 at 5:09 PM Daniel Gustafsson wrote: > A fifth option is to throw away our in-tree implementations and use the > OpenSSL > API's for everything, which is where this thread started. If the effort to > payoff ratio is palatable to anyone then patches are for sure welcome.

Re: Integer undeflow in fprintf in dsa.c

2024-02-20 Thread Robert Haas
On Tue, Feb 20, 2024 at 5:30 PM Daniel Gustafsson wrote: > The message "at least 0 contiguous pages free" reads a bit nonsensical though, > wouldn't it be preferrable to check for i being zero and print a custom > message > for that case? Something like the below untested sketch? > > +

Re: logical decoding and replication of sequences, take 2

2024-02-20 Thread Tomas Vondra
On 2/20/24 06:54, Amit Kapila wrote: > On Thu, Dec 21, 2023 at 6:47 PM Tomas Vondra > wrote: >> >> On 12/19/23 13:54, Christophe Pettus wrote: >>> Hi, >>> >>> I wanted to hop in here on one particular issue: >>> On Dec 12, 2023, at 02:01, Tomas Vondra wrote: - desirability of

Re: Integer undeflow in fprintf in dsa.c

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 12:28, Ильясов Ян wrote: > ​fprintf(stderr, >"segment bin %zu (at least %d contiguous pages free):\n", >i, 1 << (i - 1)); > > In case i​ equals zero user will get "at least -2147483648 contiguous pages > free". That does indeed seem like an

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 12:18, Peter Eisentraut wrote: > I think we are going about this the wrong way. It doesn't make sense to ask > OpenSSL what a piece of code that doesn't use OpenSSL should do. Given that pgcrypto cannot be built without OpenSSL, and ideally we should be using the OpenSSL

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Daniel Gustafsson
> On 20 Feb 2024, at 12:27, Robert Haas wrote: > > On Tue, Feb 20, 2024 at 4:49 PM Peter Eisentraut wrote: >> I think there are several less weird ways to address this: >> >> * Just document it. >> >> * Make a pgcrypto-level GUC setting. >> >> * Split out these functions into a separate

Re: speed up a logical replica setup

2024-02-20 Thread vignesh C
On Mon, 19 Feb 2024 at 11:15, Hayato Kuroda (Fujitsu) wrote: > > Dear hackers, > > > Since it may be useful, I will post top-up patch on Monday, if there are no > > updating. > > And here are top-up patches. Feel free to check and include. > > v22-0001: Same as v21-0001. > === rebased patches ===

Re: A new message seems missing a punctuation

2024-02-20 Thread Amit Kapila
On Tue, Feb 20, 2024 at 4:50 PM Robert Haas wrote: > > On Tue, Feb 20, 2024 at 4:42 PM Amit Kapila wrote: > > > So why do we log a message about this? > > > > This was added after the main commit of this functionality to find > > some BF failures (where we were expecting the slot to sync but due

Integer undeflow in fprintf in dsa.c

2024-02-20 Thread Ильясов Ян
Hello hackers, Using Svace* I think I've found a little bug in src/backend/utils/mmgr/dsa.c. This bug is presented in REL_12_STABLE, REL_13_STABLE, REL_14_STABLE, REL_15_STABLE, REL_16_STABLE and master. I see that it was introduced together with dynamic shared memory areas in the commit

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Robert Haas
On Tue, Feb 20, 2024 at 4:49 PM Peter Eisentraut wrote: > I think there are several less weird ways to address this: > > * Just document it. > > * Make a pgcrypto-level GUC setting. > > * Split out these functions into a separate extension. > > * Deprecate these functions. > > Or some combination

Re: A new message seems missing a punctuation

2024-02-20 Thread Robert Haas
On Tue, Feb 20, 2024 at 4:42 PM Amit Kapila wrote: > > So why do we log a message about this? > > This was added after the main commit of this functionality to find > some BF failures (where we were expecting the slot to sync but due to > one of these conditions not being met the slot was not

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-20 Thread Peter Eisentraut
On 20.02.24 11:09, Daniel Gustafsson wrote: On 20 Feb 2024, at 10:56, Koshi Shibagaki (Fujitsu) wrote: Let me confirm the discussion in threads. I think there are two topics. 1. prohibit the use of ciphers disallowed in FIPS mode at the level of block cipher (crypt-bf, etc...) in crypt()

Re: Change the bool member of the Query structure to bits

2024-02-20 Thread Tomas Vondra
On 2/20/24 11:11, Quan Zongliang wrote: > > Sorry. I forgot to save a file. This is the latest. > > On 2024/2/20 18:07, Quan Zongliang wrote: >> >> The Query structure has an increasing number of bool attributes. This >> is likely to increase in the future. And they have the same >> properties.

Re: POC, WIP: OR-clause support for indexes

2024-02-20 Thread Ranier Vilela
Em ter., 20 de fev. de 2024 às 00:18, Andrei Lepikhov < a.lepik...@postgrespro.ru> escreveu: > On 19/2/2024 19:53, Ranier Vilela wrote: > > v17-0002 > > 1) move the vars *arrayconst and *dest, to after if, to avoid makeNode > > (palloc). > > + Const *arrayconst; > > + ScalarArrayOpExpr *dest;

Re: Streaming read-ready sequential scan code

2024-02-20 Thread Robert Haas
On Tue, Feb 20, 2024 at 4:35 AM Melanie Plageman wrote: > I've written three alternative implementations of the actual streaming > read user for sequential scan which handle the question of where to > allocate the streaming read object and how to handle changing scan > direction in different

  1   2   >