Re: Clarification on warning when connecting to 'pgbouncer' database via Pgbouncer

2025-05-27 Thread Euler Taveira
uppress this warning. [1] https://github.com/pgbouncer/pgbouncer/issues/1143 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Document default values for pgoutput options + fix missing initialization for "origin"

2025-05-19 Thread Euler Taveira
it lacks an explicit default assignment. > > To ensure consistency and make the behavior clearer, I propose explicitly > initializing the "origin" parameter as well. A patch for this is also attached > (0002 patch). LGTM. It seems an oversight from the original commit 366283961ac0. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Disallow redundant indexes

2025-04-29 Thread Euler Taveira
SQL command. The WARNING seems fragile because it doesn't solve the problem. I also don't want to disallow duplicate indexes; it can break old routines. If we want to do something about it, I would say let's add a psql meta command or even a new view to show duplicate indexes. I do think

Re: New committer: Jacob Champion

2025-04-14 Thread Euler Taveira
On Fri, Apr 11, 2025, at 5:26 PM, Jonathan S. Katz wrote: > Please join us in wishing Jacob much success and few reverts! Keep up the good work, Jacob! -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Feature freeze

2025-04-10 Thread Euler Taveira
or a new page). Hence, in doubt, it is good resource to figure out if it is end of road for that release. [1] https://www.timeanddate.com/countdown/generic?iso=20260406T12&p0=3926&msg=Feature+freeze&font=cursive&csz=1 [2] https://www.postgresql.org/developer/roadmap/ --

Re: n_ins_since_vacuum stats for aborted transactions

2025-04-10 Thread Euler Taveira
number of rows inserted since this table was last vacuumed Those rows were actually inserted. They are physically in the data files. And that's what matters for the autovacuum algorithm. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-10 Thread Euler Taveira
ilcation origin names must be no longer than %d bytes.", + MAX_RONAME_LEN))); s/Repilcation/Replication/ +#define MAX_RONAME_LEN (512) It is just a cosmetic suggestion but I usually use parentheses when it is an expression but don't include it for a single nu

Re: Large expressions in indexes can't be stored (non-TOASTable)

2025-04-08 Thread Euler Taveira
e" as type if it is an identifier; it could be the case for roname if author decided to be strict. This additional TOAST table has no or rare use. +1 for removing it. It is one less file, one less table and one less index; in short, one less source of data corruption. ;) -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-04-05 Thread Euler Taveira
4 seconds. Again, that's still unacceptable for such an optional feature. Maybe you should experiment my suggestion in the previous review. You don't need a real run to prove that pg_createsubscriber is not removing unintended objects. [1] http://postgr.es/m/6f266ce2-38d4-433f-afc9-b47c48c17...@app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-04-04 Thread Euler Taveira
On Fri, Apr 4, 2025, at 1:38 PM, Antonin Houska wrote: > Euler Taveira wrote: > > > + > > + > > + > > + The FULL parameter is deprecated in favor of > > + . > > + > > + > > + > > > > The wa

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-04-03 Thread Euler Taveira
= (Node *) n; + } I'm wondering if there is an easy way to avoid these rules. PROGRESS_COMMAND_VACUUM, PROGRESS_COMMAND_ANALYZE, PROGRESS_COMMAND_CLUSTER, + PROGRESS_COMMAND_REPACK, PROGRESS_COMMAND_CREATE_INDEX, PROGRESS_COMMAND_BASEBACKUP, PROGRESS_COMMAND_COPY, It is just a matter of style but I have the habit to include new stuff at the end. +-- Yet another code path: REPACK w/o index. +REPACK clstr_tst USING INDEX clstr_tst_c; +-- Verify that inheritance link still works You forgot to remove the USING INDEX here. I'm still review the other patches (that is basically the implementation of CONCURRENTLY) and to avoid a long review, I'm sending the 0001 review. Anyway, 0001 is independent of the other patches and should be applied separately. [1] https://www.merriam-webster.com/dictionary/repack -- Euler Taveira EDB https://www.enterprisedb.com/

wal writer: obsolete sentence

2025-03-28 Thread Euler Taveira
k so. [1] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=ad4295728e04f4107f4dcbbee4950e526becb95d [2] https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=01126dc8cf2c6e06babf1bbb4dab41a4a96d704f -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Enhance 'pg_createsubscriber' to retrieve databases automatically when no database is provided.

2025-03-25 Thread Euler Taveira
On Tue, Mar 25, 2025, at 8:07 AM, Shubham Khanna wrote: > Apologies for the oversight. I have attached the patches now. Please > find them included here. I started looking at this patch. When I started reading the commit message, I didn't understand why the options that provide names to objects ar

Re: Add Postgres module info

2025-03-24 Thread Euler Taveira
RSION 2 #defineWAL2JSON_FORMAT_MIN_VERSION 1 +#if PG_VERSION_NUM >= 18 +PG_MODULE_MAGIC_EXT( + .name = "wal2json", + .version = WAL2JSON_VERSION +); +#else PG_MODULE_MAGIC; +#endif [1] https://www.postgresql.org/docs/current/extend-extensions.html -- Euler Taveira EDB https://www.enterprisedb.com/ autoversion.patch.nocfbot Description: Binary data

Re: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication

2025-03-18 Thread Euler Taveira
_SUPPORTED), errmsg("user \"%s\" cannot replicate into relation with row-level security enabled: \"%s\"", GetUserNameFromId(GetUserId(), true), RelationGetRelationName(rel; The comments already point out directions. Feel free to write a proposal for it. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-03-17 Thread Euler Taveira
e related to > max_logical_replication_workers. Good point. Looking at the documentation, the old max_replication_slots parameter was the first one in that section so I decided to use the same order for the postgresql.conf.sample. -- Euler Taveira EDB https://www.enterpr

Re: Separate GUC for replication origins

2025-03-17 Thread Euler Taveira
is appropriate to expose implementation details about table synchronization in a end user page. If we can explain it without saying too much about the implementation details, fine. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-03-13 Thread Euler Taveira
n the second > record, because max_active_replication_origins appears before > max_logical_replication_workers in the order. Fixed. > 2) I felt max_replication_slots must be max_active_replication_origins > here in logical-replication.sgml: Fixed. -- Euler Taveira EDB https://ww

Re: Add contrib/pg_logicalsnapinspect

2025-03-13 Thread Euler Taveira
ler quiet */ + return InvalidXLogRecPtr; /* keep compiler quiet */ } -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Adding a '--clean-publisher-objects' option to 'pg_createsubscriber' utility.

2025-03-13 Thread Euler Taveira
want) but can increase the list of options you need to inform if you want to clean multiple object types. A single option is not flexible and breaks backward compatibility but it does exactly what you want with a few characters. In most scenarios, if you want to have a clean subscriber, you will remove *all* possible objects, hence, a single option is your choice. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-03-12 Thread Euler Taveira
_replication_origins. I'm also attaching 0002 if we decide that backward compatibility is not a concern so it removes it and assign 10 as the default value. I'm adding an additional suffix so CF bot doesn't grab 0002. -- Euler Taveira EDB https://www.enterprisedb.com/ From 0998

Re: Proposal: Progressive explain

2025-03-12 Thread Euler Taveira
/* state related to progressive explains */ + struct PlanState *pe_curr_node; + struct Instrumentation *pe_local_instr; + dsa_area *pe_a; Could you add some comments saying what each of these variables are for? I didn't experiment but I was wondering if there is a way to avoid the duplicates that you added to avoid the overhead. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-03-07 Thread Euler Taveira
sure the GUC description is clear about it. It seems the Replication Progress Tracking chapter needs an update to specify this information too. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: log_min_messages per backend type

2025-03-06 Thread Euler Taveira
ype and if there is a difference between WARNING and default:WARNING (both is a fallback for non-specified backend type elements). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-03-06 Thread Euler Taveira
On Thu, Mar 6, 2025, at 6:55 AM, Amit Kapila wrote: > On Wed, Mar 5, 2025 at 4:38 PM Peter Eisentraut wrote: > > > > On 11.02.25 21:25, Euler Taveira wrote: > > > Here is another patch that only changes the GUC name to > > > max_replication_origin_session

Re: log_min_messages per backend type

2025-03-06 Thread Euler Taveira
On Wed, Mar 5, 2025, at 11:53 PM, Fujii Masao wrote: > On 2025/03/05 9:33, Euler Taveira wrote: > >> > +Valid BACKENDTYPE values are > >> > ARCHIVER, > >> > +AUTOVACUUM, BACKEND, > >> > +BGWOR

Re: Separate GUC for replication origins

2025-03-04 Thread Euler Taveira
d as part of this patch. The drawback here is that once we accept -1, we cannot remove it without breaking compatibility. My preference was 1 but I'm fine with 2 too. Since it is a major release, it is natural to introduce features that break things. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: log_min_messages per backend type

2025-03-04 Thread Euler Taveira
On Wed, Feb 5, 2025, at 3:51 PM, Álvaro Herrera wrote: > On 2024-Dec-17, Euler Taveira wrote: > > > Sometimes you need to inspect some debug messages from autovacuum worker but > > you cannot apply the same setting for backends (that could rapidly fill the > > log >

Re: Support POSITION with nondeterministic collations

2025-02-11 Thread Euler Taveira
| AC | Epitaciolândia 1400407 | Normandia | RR | Normandia 1504455 | Medicilândia | PA | Medicilândia 1505437 | Ourilândia do Norte | PA | Ourilândia do Norte 1507953 | Tailândia| PA | Tailândia 1700251 | Abreulândia | TO | Abreulândia 1703008 | Babaçulândia| TO | Babaçulândia (10 rows) -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-02-11 Thread Euler Taveira
On Wed, Feb 5, 2025, at 9:49 PM, Masahiko Sawada wrote: > On Wed, Feb 5, 2025 at 4:39 PM Euler Taveira wrote: > > > > On Wed, Feb 5, 2025, at 1:56 AM, Amit Kapila wrote: > > > > On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira wrote: > > > > &g

Re: Small memory fixes for pg_createsubcriber

2025-02-11 Thread Euler Taveira
time, hence, it is fine to leak memory, the initial commit cleaned up all variables but a subsequent commit 4867f8a555c apparently didn't. Although it is just a low impact improvement, it is better to be strict and shut up SASTs. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Separate GUC for replication origins

2025-02-05 Thread Euler Taveira
On Wed, Feb 5, 2025, at 1:56 AM, Amit Kapila wrote: > On Wed, Feb 5, 2025 at 8:17 AM Euler Taveira wrote: > > > > Under reflection, an accurate name is max_replication_origin_session_setup. > > A > > counter argument is that it is a long name (top-5 length). > >

Re: Separate GUC for replication origins

2025-02-04 Thread Euler Taveira
--- > +#include "utils/guc_hooks.h" > > I think #include'ing guc.h would be more appropriate. Fixed. I also updated the pg_createsubscriber documentation that refers to max_replication_slots. Since we don't have an agreement about the name, I still kept max_replication_origins. -- Eu

Re: pg_createsubscriber TAP test wrapping makes command options hard to read.

2025-01-20 Thread Euler Taveira
On Mon, Jan 20, 2025, at 7:49 PM, Tom Lane wrote: > "Euler Taveira" writes: > > It is kind of annoying to keep version 20230309 around to run perltidy. Do > > we > > have any other alternatives? > > As it says in src/tools/pgindent/README: > > 2) Inst

Re: pg_createsubscriber TAP test wrapping makes command options hard to read.

2025-01-20 Thread Euler Taveira
a information and (ii) make sure it won't break. The commit 04c8634c0c4 that added the recovery timeout option separated the doubled verbose option; it was not intentional. We can certainly keep them together. I checked your patch with perltidy 20220613-1 and it looks good to me. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2025-01-08 Thread Euler Taveira
On Thu, Aug 15, 2024, at 5:53 PM, Doruk Yilmaz wrote: > Hello again, > > On Tue, Aug 13, 2024 at 12:48 AM Euler Taveira wrote: > > I'm curious about your use case. Is it just because the internal function > > has a > > different signature or your tool is capa

Re: Separate GUC for replication origins

2025-01-08 Thread Euler Taveira
On Thu, Dec 19, 2024, at 10:31 AM, Peter Eisentraut wrote: > On 10.12.24 19:41, Euler Taveira wrote: > > I'm attaching a patch that adds max_replication_origins. It basically > > replaces > > all of the points that refers to max_replication_slots on the subscriber. It

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-03 Thread Euler Taveira
cal replication) has any use case (maybe someone inspects these extra records for analysis) but one suggestion (separate patch) is to make 'logical' synonymous with the new 'replica' behavior (logical decoding capability). This opens the door to remove 'logical' in future releases (accepted as synonym but not documented). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: log_min_messages per backend type

2024-12-19 Thread Euler Taveira
default, DEBUG5:autovacuum' (not > specifying all possible cases?) > No. Good point. I forgot to mention that ALL as backend type could be added. -- Euler Taveira EDB https://www.enterprisedb.com/

log_min_messages per backend type

2024-12-17 Thread Euler Taveira
n_messages debug2:checkpointer, debug1:autovacuum (1 row) In the above example, it sets log level DEBUG2 to checkpointer process and DEBUG1 to autovacuum launcher and autovacuum worker processes. The other processes keep WARNING (default) as log level. -- Euler Taveira EDB https://www.enterp

Re: Add Postgres module info

2024-12-11 Thread Euler Taveira
detect what version is installed, hence, some tools cannot have some logic to probe the module version. In a managed environment, it is hard to figure out the exact version for non-CREATE-EXTENSION modules, unless it is explicitly informed by the vendor. [1] https://github.com/eulerto/wal2json/issues/181 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Suggestion to standardize comment format in pg_dump

2024-12-11 Thread Euler Taveira
t adding a crucial information. You mention consistency but since it is a different class of objects I don't think this argument holds much water. -- Euler Taveira EDB https://www.enterprisedb.com/

Separate GUC for replication origins

2024-12-10 Thread Euler Taveira
ty. I mean has a special value (like -1) that means same value as max_replication_slots. Is it worth it? (If not, it should be mentioned in the commit message.) -- Euler Taveira EDB https://www.enterprisedb.com/ From 5f9e43ce48b523e3bca8e1ef8ee9a427c676381c Mon Sep 17 00:00:00 2001 From: Euler Ta

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-05 Thread Euler Taveira
used by cachectx). I think you could remove 'mem' from this variable. My suggestions are pubcxt or pubnamescxt. Although, I prefer the former, if other publication elements are added to this context in the future. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: doc: Mention clock synchronization recommendation for hot_standby_feedback

2024-12-05 Thread Euler Taveira
ld it be converted into a ...? (See synchronous_standby_names [1] for an example.) [1] https://www.postgresql.org/docs/current/runtime-config-replication.html -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-03 Thread Euler Taveira
lvaro's proposal. [1] https://codesearch.debian.net/search?q=PGOutputData&literal=0 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Truncate logs by max_log_size

2024-09-26 Thread Euler Taveira
your log files (or even use syslog). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Adding NOTICE for differences between estimated and actual rows

2024-09-05 Thread Euler Taveira
On Thu, Sep 5, 2024, at 5:32 PM, Euler Taveira wrote: > $ cat /tmp/pcc.pl > #!/usr/bin/perl -n > print "\033[1m\033[35m$1\033[36m$2\033[32m$3\033[33m$4\033[m" while > /([|+-]+)|([0-9]+)|([a-zA-Z_]+)|([^\w])/g; > > and then you can start psql as: > > $ P

Re: Adding NOTICE for differences between estimated and actual rows

2024-09-05 Thread Euler Taveira
uot;\033[1m\033[35m$1\033[36m$2\033[32m$3\033[33m$4\033[m" while /([|+-]+)|([0-9]+)|([a-zA-Z_]+)|([^\w])/g; and then you can start psql as: $ PAGER="/c/mypager.pl" psql [1] https://stackoverflow.com/questions/5947742/how-to-change-the-output-color-of-echo-in-linux/28938235#28938235 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Proposal to have INCLUDE/EXCLUDE options for altering option values

2024-08-26 Thread Euler Taveira
nvinced that this new syntax improves the user experience. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Opinion poll: Sending an automated email to a thread when it gets added to the commitfest

2024-08-14 Thread Euler Taveira
f I understand your proposal correctly, there will be another email to the thread if the previous CF was closed and someone opened a new CF entry. Sometimes some CF entries are about the same thread. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [Patch] add new parameter to pg_replication_origin_session_setup

2024-08-12 Thread Euler Taveira
covered by CI [2][3]. [1] https://commitfest.postgresql.org/49/ [2] https://wiki.postgresql.org/wiki/Cfbot [3] http://cfbot.cputube.org/index.html -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Detailed release notes

2024-08-06 Thread Euler Taveira
ually use for the mailing list. https://postgr.es/c/da4017a694d -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Detailed release notes

2024-08-05 Thread Euler Taveira
commit message points to the discussion so it is a good start point if you want to research about that specific feature. If one commit is not sufficient for that item, we can always add multiple commits as we usually do in the release notes comments. [1] https://www.pgbouncer.org/changelog.html -- Euler Taveira EDB https://www.enterprisedb.com/

Re: rare crash - FailedAssertion snapbuild.c Line: 580

2024-08-01 Thread Euler Taveira
which may be useful for analysis). [1] https://www.postgresql.org/support/versioning/ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: make pg_createsubscriber option names more consistent

2024-07-31 Thread Euler Taveira
scriber and pg_upgrade have a lot of common terminology and > a similar operating mode, so it would make sense to keep this consistent. WFM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-07-30 Thread Euler Taveira
On Tue, Jul 30, 2024, at 7:35 AM, Peter Eisentraut wrote: > On 30.07.24 02:05, Euler Taveira wrote: > > I'm attaching a patch that implements option (c). While reading the code > > I noticed that I left a comment that should be removed by commit > > b9639138262. 00

Re: speed up a logical replica setup

2024-07-29 Thread Euler Taveira
On Mon, Jul 29, 2024, at 6:11 PM, Euler Taveira wrote: > The options are: > > (a) temporary replication slot: requires an additional replication slot. > small payload. it is extremely slow in comparison with the other > options. > (b) logical message: can be consumed by logica

Re: speed up a logical replica setup

2024-07-29 Thread Euler Taveira
load. fast. I don't have a strong preference but if I need to pick one I would choose option (c) or option (d). The option (a) is out of the question. Opinions? [1] https://www.postgresql.org/message-id/b1f0f8c7-8f01-4950-af77-339df3dc4684%40app.fastmail.com [2] https://www.postgresql.o

Re: speed up a logical replica setup

2024-07-11 Thread Euler Taveira
A4eK1%2Bp%2B7Ag6nqdFRdqowK1EmJ6bG-MtZQ_54dnFBi%3D_OO5RQ%40mail.gmail.com [2] https://www.postgresql.org/message-id/OSBPR01MB25521B15BF950D2523BBE143F5D32%40OSBPR01MB2552.jpnprd01.prod.outlook.com -- Euler Taveira EDB https://www.enterprisedb.com/ From a3be65c9bea08d3c7ffd575342a879d37b28

Re: speed up a logical replica setup

2024-07-11 Thread Euler Taveira
use it will break this test. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: improve performance of pg_dump with many sequences

2024-07-10 Thread Euler Taveira
think we have to keep this workaround since older minor > releases of PostgreSQL don't have the fix. Hmm. Right. > What pg_dump command did you test here? Did you dump the sequence data, or > was this --schema-only? time pg_dump -f - -s -d postgres -- Euler Taveira EDB https://www.enterprisedb.com/

Re: improve performance of pg_dump with many sequences

2024-07-10 Thread Euler Taveira
+ Since you apply a fix for pg_sequence_last_value function, you can simplify the query in 0003. CASE is not required. I repeated the same test but not applying 0003. patched (0001 and 0002): real 0m0,290s user 0m0,038s sys 0m0,104s I'm not sure if 0003 is worth. Maybe if you h

Re: speed up a logical replica setup

2024-07-10 Thread Euler Taveira
> # expected: 'failover_slot' I'm wondering if the attached patch is sufficient to move the restart_lsn forward. I experimented several lightweight ideas but none works. BTW the steps to create the failover slot here is similar 040_standby_failover_slots_sync.pl.

Re: speed up a logical replica setup

2024-06-25 Thread Euler Taveira
wise, prepared transactions are sent to the subscriber only when committed, and are then processed immediately by the subscriber. Hence, the replication should be working for prepared transactions even if it created the slot with twophase = false. IIRC the user won't be able to change it later. As Amit said in a previous email, once the command ALTER SUBSCRIPTION ... SET (two_phase = on) is supported, users can change it after running pg_createsubscriber. The other option is to add a command-line option to enable or disable it. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-06-25 Thread Euler Taveira
E PUBLICATION > > started and managed to assign an XID only after consistent_lsn. From the > > pgoutput perspective, that's indistinguishable from the transaction starting > > at its first WAL record, after consistent_lsn. The linked "long-standing > > data > > loss bug in initial sync of logical replication" thread doesn't have (a), > > hence its bug. How close is that to accurate? > > > > Yeah, this theory sounds right to me. The key point is that no DML > (processing of WAL corresponding to DML) before CREATE PUBLICATION ... > command would have reached pgoutput level because we would have waited > for it during snapbuild.c. Can we conclude that the race condition > discussed in the other thread won't impact this patch? As Noah said the key point is the CREATE PUBLICATION *before* creating the replication slots -- that wait transactions to complete. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: State of pg_createsubscriber

2024-06-19 Thread Euler Taveira
like Kuroda-San or me to start some or all? I will open new threads soon if you don't. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-06-18 Thread Euler Taveira
On Mon, Jun 17, 2024, at 8:04 AM, Peter Eisentraut wrote: > On 07.06.24 05:49, Euler Taveira wrote: > > Here it is a patch series to fix the issues reported in recent > > discussions. The > > patches 0001 and 0003 aim to fix the buildfarm issues. The patch 0002 > &g

Re: State of pg_createsubscriber

2024-06-18 Thread Euler Taveira
On Tue, Jun 18, 2024, at 12:59 AM, Amit Kapila wrote: > On Mon, May 20, 2024 at 12:12 PM Amit Kapila wrote: > > > > On Sun, May 19, 2024 at 11:20 PM Euler Taveira wrote: > > > > > > On Sun, May 19, 2024, at 2:30 PM, Tom Lane wrote: > > > > &g

Re: speed up a logical replica setup

2024-06-06 Thread Euler Taveira
On Wed, May 22, 2024, at 12:16 PM, Euler Taveira wrote: > I'll summarize all issues as soon as I finish the review of sync slot > support. I > think we should avoid new development if we judge that the item can be > documented as a limitation for this version. Neverthel

Re: speed up a logical replica setup

2024-05-23 Thread Euler Taveira
On Thu, May 23, 2024, at 5:54 AM, Amit Kapila wrote: > On Wed, May 22, 2024 at 8:46 PM Euler Taveira wrote: > > > > Following the same line that simplifies the code, we can: (a) add a loop in > > check_subscriber() that waits until walreceiver is available on subscriber

Re: speed up a logical replica setup

2024-05-22 Thread Euler Taveira
we need a escape mechanism for the loop (timeout?). I'll summarize all issues as soon as I finish the review of sync slot support. I think we should avoid new development if we judge that the item can be documented as a limitation for this version. Nevertheless, I will share patches so

Re: State of pg_createsubscriber

2024-05-19 Thread Euler Taveira
ed since 1 April. My bad. :( I'll post patches soon to address all of the points. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: EXPLAN redundant options

2024-05-02 Thread Euler Taveira
pto; ^ postgres=# create user foo with createdb login createdb; ERROR: conflicting or redundant options LINE 1: create user foo with createdb login createdb; ^ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-04-29 Thread Euler Taveira
On Mon, Apr 29, 2024, at 6:56 AM, Amit Kapila wrote: > On Wed, Mar 27, 2024 at 1:47 AM Euler Taveira wrote: > > > > On Tue, Mar 26, 2024, at 4:12 PM, Tomas Vondra wrote: > > > > Perhaps I'm missing something, but why is NUM_CONN_ATTEMPTS even needed? > >

Re: Idea Feedback: psql \h misses -> Offers Links?

2024-04-19 Thread Euler Taveira
using SGML files. We already did it for SQL commands. Another broader idea is to build an inverted index similar to what Index [1] provides. The main problem with this approach is to create a dependency between documentation build and psql. Maybe there is a reasonable way to obtain the links for each term. [1] https://www.postgresql.org/docs/current/bookindex.html -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-29 Thread Euler Taveira
ber of digits after the decimal separator you still don't have an integer in the internal unit, hence, you have to round it to integer. We already have functions that use integer (such as pg_terminate_backend) and interval (such as pg_sleep_for) and if i searched correctly it will be the first t

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-28 Thread Euler Taveira
uals to zero no +timeout is applied. + [1] https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-RECOVERY-CONTROL -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Fix some resources leaks (src/bin/pg_basebackup/pg_createsubscriber.c)

2024-03-27 Thread Euler Taveira
xiting. > 3. > CID 1542691: (#1 of 1): Resource leak (RESOURCE_LEAK) > leaked_storage: Variable str going out of scope leaks the storage it points > to. It will exit in the next instruction. Having said that, applying this patch is just a matter of style. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-26 Thread Euler Taveira
v_ySzuFjN7UaX9qxz5Hqcwew7Fk%3D%2BAbJbu0Kd9w%40mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
anted data. See setup_subscriber. /* * Since the publication was created before the consistent LSN, it is * available on the subscriber when the physical replica is promoted. * Remove publications from the subscriber because it has no use. */ drop_publication(conn, &dbinfo[i]); -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
run before walsender is launched. > > One possible approach is to wait until the replication starts. Alternative > one is > to ease the condition. That's my suggestion too. I reused NUM_CONN_ATTEMPTS (that was renamed to NUM_ATTEMPTS in the first patch). See second patch. -- E

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
On Mon, Mar 25, 2024, at 8:55 AM, Peter Eisentraut wrote: > On 22.03.24 04:31, Euler Taveira wrote: > > On Thu, Mar 21, 2024, at 6:49 AM, Shlok Kyal wrote: > >> There is a compilation error while building postgres with the patch > >> due to a recent commit. I have attac

Re: speed up a logical replica setup

2024-03-21 Thread Euler Taveira
have a strong preference; feel free to apply it. [1] https://www.postgresql.org/message-id/CALDaNm1Dg5tDRmaabk%2BZND4WF17NrNq52WZxCE%2B90-PGz5trQQ%40mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ v33-0001-pg_createsubscriber-creates-a-new-logical-replic.patch.gz Desc

Re: speed up a logical replica setup

2024-03-21 Thread Euler Taveira
l_senders to at least 3. If you have such an error, you will fix them all and rerun using dry run mode again to verify everything is ok. I don't have a strong preference about it. It can be changed easily (unifying the check functions or providing a return for each of the check functions). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
;)", > > dbinfo->subname, > > dbinfo->dbname); > > The attached patch has the changes to handle the same. I included a different query that does the same. See v32. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
'm not sure we should add a sentence about a pilot error. I added a comment in check_subscriber that describes this situation. I think the comment is sufficient to understand the limitation and, if it is possible in the future, a check might be added for it. I didn't include v31-0004. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
as earlier as possible to avoid leaving cases that it should trigger. > 38. > > ``` > /* Subscriber PID file */ > snprintf(pidfile, MAXPGPATH, "%s/postmaster.pid", subscriber_dir); > > /* > * If the standby server is running, stop it. Some parameters (that can > * only be set at server start) are informed by command-line options. > */ > if (stat(pidfile, &statbuf) == 0) > ``` > > Hmm. pidfile is used only here, but it is declared in main(). Can it be > separated into another funtion like is_standby_started()? It is so small that I didn't bother adding a new function for it. > 39. > > Or, we may able to introcue "restart_standby_if_needed" or something. > > 40. > > ``` > * XXX this code was extracted from BootStrapXLOG(). > ``` > > So, can we extract the common part to somewhere? Since system identifier is > related > with the controldata file, I think it can be located in controldata_util.c. I added this comment here as a reference from where I extracted the code. The referred function is from backend. Feel free to propose a separate patch for it. > 41. > > You said like below in [1], but I could not find the related fix. Can you > clarify? > > > That's a good point. We should state in the documentation that GUCs > > specified in > > the command-line options are ignored during the execution. I added a sentence for it. See "How It Works". [1] https://www.postgresql.org/docs/current/logical-replication-architecture.html#LOGICAL-REPLICATION-SNAPSHOT -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
On Mon, Mar 18, 2024, at 10:52 AM, Peter Eisentraut wrote: > On 16.03.24 16:42, Euler Taveira wrote: > > I'm attaching a new version (v30) that adds: > > I have some review comments and attached a patch with some smaller > fixups (mainly message wording and avoid fi

Re: speed up a logical replica setup

2024-03-16 Thread Euler Taveira
t Thanks for checking. I proposed an alternative patch for it [1]. Can you check it? [1] https://www.postgresql.org/message-id/34637e7f-0330-420d-8f45-1d022962d2fe%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-16 Thread Euler Taveira
rted an issue [1] related to reaching the recovery stop point before the consistent state is reached. I proposed a simple patch that fixes the issue. [1] https://www.postgresql.org/message-id/CALDaNm3VMOi0GugGvhk3motghaFRKSWMCSE2t3YX1e%2BMttToxg%40mail.gmail.com -- Euler Taveira EDB https

Re: Add publisher and subscriber to glossary documentation.

2024-03-14 Thread Euler Taveira
ed to the connection string. While the physical replication does not specify a database (meaning "any database" referring to an instance), the logical replication requires a database. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-14 Thread Euler Taveira
l.org/message-id/CANhcyEW6-dH28gLbFc5XpDTJ6JPizU%2Bt5g-aKUWJBf5W_Zriqw%40mail.gmail.com [3] https://www.postgresql.org/message-id/6423dfeb-a729-45d3-b71e-7bf1b3adb0c9%40enterprisedb.com [4] https://www.postgresql.org/message-id/d898faad-f6d7-4b0d-b816-b9dcdf490685%40app.fastmail.com --

Re: speed up a logical replica setup

2024-03-12 Thread Euler Taveira
...@mail.gmail.com [2] https://www.postgresql.org/message-id/6423dfeb-a729-45d3-b71e-7bf1b3adb0c9%40enterprisedb.com [3] https://www.postgresql.org/message-id/e390e35e-508e-4eb8-92e4-e6b066407a41%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ v28-0001-pg_createsubscriber-

Re: Identify transactions causing highest wal generation

2024-03-08 Thread Euler Taveira
e| 4642.59929618 min_exec_time | 0.011094 max_exec_time | 0.872748 mean_exec_time | 0.01948543312347807 stddev_exec_time | 0.006370786385582063 rows | 238260 . . . wal_records| 496659 wal_fpi| 19417 wal_bytes | 2085

Re: speed up a logical replica setup

2024-03-06 Thread Euler Taveira
default. > 6) The message should be "waiting for the target server to reach the > consistent state": > +#define NUM_CONN_ATTEMPTS 5 > + > + pg_log_info("waiting the target server to reach the consistent > state"); > + > + conn = connect_database(conninfo, true); Fixed. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-06 Thread Euler Taveira
d transactions. The other option is to add a sentence into the documentation. > 21. others > > As I said in [7], the catch up would not be finished if long > recovery_min_apply_delay > is used. Should we overwrite during the catch up? No. If the time-delayed logical replica [2] wa

Re: speed up a logical replica setup

2024-03-05 Thread Euler Taveira
name=postgres" -d postgres -d db1 -p 9000 -r > ./pg_createsubscriber: invalid option -- 'p' > pg_createsubscriber: hint: Try "pg_createsubscriber --help" for more > information. Oops. Good catch! I will post an updated patch soon. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-01 Thread Euler Taveira
nt overall. As Greg said none of these programs connects to the database. I don't like to break backward compatibility but in this case I suspect that it is ok. I don't recall the last time I saw a script that makes use of -d option. How often do you need a pgbench debug information? -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-01 Thread Euler Taveira
option, I'm thinking about to remove it. If the logging is enabled, the information during the pg_createsubscriber will be available. The client log can be redirected to a file for future inspection. Comments? [1] https://www.postgresql.org/message-id/TYCPR01MB12077756323B79042F29DDAEDF5

  1   2   3   4   5   >