Re: EXPLAN redundant options

2024-05-02 Thread Euler Taveira
^ postgres=# create user foo with createdb login createdb; ERROR: conflicting or redundant options LINE 1: create user foo with createdb login createdb; ^ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-04-29 Thread Euler Taveira
On Mon, Apr 29, 2024, at 6:56 AM, Amit Kapila wrote: > On Wed, Mar 27, 2024 at 1:47 AM Euler Taveira wrote: > > > > On Tue, Mar 26, 2024, at 4:12 PM, Tomas Vondra wrote: > > > > Perhaps I'm missing something, but why is NUM_CONN_ATTEMPTS even needed? > >

Re: Idea Feedback: psql \h misses -> Offers Links?

2024-04-19 Thread Euler Taveira
erted index similar to what Index [1] provides. The main problem with this approach is to create a dependency between documentation build and psql. Maybe there is a reasonable way to obtain the links for each term. [1] https://www.postgresql.org/docs/current/bookindex.html -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-29 Thread Euler Taveira
ecimal separator you still don't have an integer in the internal unit, hence, you have to round it to integer. We already have functions that use integer (such as pg_terminate_backend) and interval (such as pg_sleep_for) and if i searched correctly it will be the first timeout argument as float. --

Re: [HACKERS] make async slave to wait for lsn to be replayed

2024-03-28 Thread Euler Taveira
timeout is applied. + [1] https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-RECOVERY-CONTROL -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Fix some resources leaks (src/bin/pg_basebackup/pg_createsubscriber.c)

2024-03-27 Thread Euler Taveira
gt; 3. > CID 1542691: (#1 of 1): Resource leak (RESOURCE_LEAK) > leaked_storage: Variable str going out of scope leaks the storage it points > to. It will exit in the next instruction. Having said that, applying this patch is just a matter of style. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-26 Thread Euler Taveira
%2BAbJbu0Kd9w%40mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
data. See setup_subscriber. /* * Since the publication was created before the consistent LSN, it is * available on the subscriber when the physical replica is promoted. * Remove publications from the subscriber because it has no use. */ drop_publication(conn, [i]); -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
> One possible approach is to wait until the replication starts. Alternative > one is > to ease the condition. That's my suggestion too. I reused NUM_CONN_ATTEMPTS (that was renamed to NUM_ATTEMPTS in the first patch). See second patch. -- Euler Taveira EDB https://www.enterpris

Re: speed up a logical replica setup

2024-03-25 Thread Euler Taveira
On Mon, Mar 25, 2024, at 8:55 AM, Peter Eisentraut wrote: > On 22.03.24 04:31, Euler Taveira wrote: > > On Thu, Mar 21, 2024, at 6:49 AM, Shlok Kyal wrote: > >> There is a compilation error while building postgres with the patch > >> due to a recent commit. I have attac

Re: speed up a logical replica setup

2024-03-21 Thread Euler Taveira
rong preference; feel free to apply it. [1] https://www.postgresql.org/message-id/CALDaNm1Dg5tDRmaabk%2BZND4WF17NrNq52WZxCE%2B90-PGz5trQQ%40mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ v33-0001-pg_createsubscriber-creates-a-new-logical-replic.patch.gz Description: a

Re: speed up a logical replica setup

2024-03-21 Thread Euler Taveira
to at least 3. If you have such an error, you will fix them all and rerun using dry run mode again to verify everything is ok. I don't have a strong preference about it. It can be changed easily (unifying the check functions or providing a return for each of the check functions). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
dbinfo->subname, > > dbinfo->dbname); > > The attached patch has the changes to handle the same. I included a different query that does the same. See v32. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
t sure we should add a sentence about a pilot error. I added a comment in check_subscriber that describes this situation. I think the comment is sufficient to understand the limitation and, if it is possible in the future, a check might be added for it. I didn't include v31-0004. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
fore that, the callback will be no-op. The main reason is to catch future cases added *before* the point you want to move this call that requires a cleanup. As you said it is a no-op. My preference for atexit() calls is to add it as earlier as possible to avoid leaving cases that it should trigge

Re: speed up a logical replica setup

2024-03-20 Thread Euler Taveira
On Mon, Mar 18, 2024, at 10:52 AM, Peter Eisentraut wrote: > On 16.03.24 16:42, Euler Taveira wrote: > > I'm attaching a new version (v30) that adds: > > I have some review comments and attached a patch with some smaller > fixups (mainly message wording and avoid fixed-s

Re: speed up a logical replica setup

2024-03-16 Thread Euler Taveira
t Thanks for checking. I proposed an alternative patch for it [1]. Can you check it? [1] https://www.postgresql.org/message-id/34637e7f-0330-420d-8f45-1d022962d2fe%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-16 Thread Euler Taveira
eaching the recovery stop point before the consistent state is reached. I proposed a simple patch that fixes the issue. [1] https://www.postgresql.org/message-id/CALDaNm3VMOi0GugGvhk3motghaFRKSWMCSE2t3YX1e%2BMttToxg%40mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ v30

Re: Add publisher and subscriber to glossary documentation.

2024-03-14 Thread Euler Taveira
the connection string. While the physical replication does not specify a database (meaning "any database" referring to an instance), the logical replication requires a database. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-14 Thread Euler Taveira
hcyEW6-dH28gLbFc5XpDTJ6JPizU%2Bt5g-aKUWJBf5W_Zriqw%40mail.gmail.com [3] https://www.postgresql.org/message-id/6423dfeb-a729-45d3-b71e-7bf1b3adb0c9%40enterprisedb.com [4] https://www.postgresql.org/message-id/d898faad-f6d7-4b0d-b816-b9dcdf490685%40app.fastmail.com -- Euler Taveira EDB

Re: speed up a logical replica setup

2024-03-13 Thread Euler Taveira
com [2] https://www.postgresql.org/message-id/6423dfeb-a729-45d3-b71e-7bf1b3adb0c9%40enterprisedb.com [3] https://www.postgresql.org/message-id/e390e35e-508e-4eb8-92e4-e6b066407a41%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ v28-0001-pg_createsubscriber-creates-a-new-l

Re: Identify transactions causing highest wal generation

2024-03-08 Thread Euler Taveira
e| 4642.59929618 min_exec_time | 0.011094 max_exec_time | 0.872748 mean_exec_time | 0.01948543312347807 stddev_exec_time | 0.006370786385582063 rows | 238260 . . . wal_records| 496659 wal_fpi| 19417 wal_bytes | 2085

Re: speed up a logical replica setup

2024-03-06 Thread Euler Taveira
ld be "waiting for the target server to reach the > consistent state": > +#define NUM_CONN_ATTEMPTS 5 > + > + pg_log_info("waiting the target server to reach the consistent > state"); > + > + conn = connect_database(conninfo, true); Fixed. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-06 Thread Euler Taveira
recovery_min_apply_delay > is used. Should we overwrite during the catch up? No. If the time-delayed logical replica [2] was available, I would say that we could use the apply delay for the logical replica. The user can expect that the replica will continue to have the configured apply delay but that's

Re: speed up a logical replica setup

2024-03-05 Thread Euler Taveira
res" -d postgres -d db1 -p 9000 -r > ./pg_createsubscriber: invalid option -- 'p' > pg_createsubscriber: hint: Try "pg_createsubscriber --help" for more > information. Oops. Good catch! I will post an updated patch soon. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Avoiding inadvertent debugging mode for pgbench

2024-03-01 Thread Euler Taveira
the database. I don't like to break backward compatibility but in this case I suspect that it is ok. I don't recall the last time I saw a script that makes use of -d option. How often do you need a pgbench debug information? -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-03-01 Thread Euler Taveira
. If the logging is enabled, the information during the pg_createsubscriber will be available. The client log can be redirected to a file for future inspection. Comments? [1] https://www.postgresql.org/message-id/TYCPR01MB12077756323B79042F29DDAEDF54C2%40TYCPR01MB12077.jpnprd01.prod.outlook.com -

Re: speed up a logical replica setup

2024-02-22 Thread Euler Taveira
slots, but > only %d remain", > +num_dbs, max_repslots - cur_repslots); > + pg_log_error_hint("Consider increasing max_replication_slots > to at least %d.", > + cur_repslots + num_dbs); > + return

Re: speed up a logical replica setup

2024-02-22 Thread Euler Taveira
> Thought? It was not a good idea if you want to keep the postgresql.conf outside PGDATA. I mean you need extra steps that can be error prone (different settings between files). Shlok, I didn't read your previous email carefully. :-/ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-02-19 Thread Euler Taveira
uot;pg_createsubscriber_5" does not exist > pg_createsubscriber: error: could not drop replication slot > "pg_createsubscriber_5_242843" on database "postgres": ERROR: > replication slot "pg_createsubscriber_5_242843" does not exist That's a bug and should be fixed.

Re: speed up a logical replica setup

2024-02-19 Thread Euler Taveira
in_messages = debug2 > > Is this setting necessary for the test? No. It is here as a debugging aid. Better to include it in a separate patch. There are a few messages that I don't intend to include in the final patch. All of these modifications will be included in the next patch. I'm finishing to integrate patches proposed by Hayato [1] and some additional fixes and refactors. [1] https://www.postgresql.org/message-id/TYCPR01MB12077A8421685E5515DE408EEF5512%40TYCPR01MB12077.jpnprd01.prod.outlook.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-02-15 Thread Euler Taveira
around pg_is_in_recovery() function to be used in other 2 points. Besides that, I changed some SQL commands to avoid having superfluous whitespace in it. I also added a test for cascaded replication scenario. And clean up 041 test a bit. I didn't provide an updated documentation because I want to chec

Re: About a recently-added message

2024-02-14 Thread Euler Taveira
of a comparison expression. For pg_createsubscriber, we are using publisher requires wal_level >= logical I suggest to use something like slot synchronization requires wal_level >= logical -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-02-13 Thread Euler Taveira
On Fri, Feb 9, 2024, at 3:27 AM, vignesh C wrote: > On Wed, 7 Feb 2024 at 10:24, Euler Taveira wrote: > > > > On Fri, Feb 2, 2024, at 6:41 AM, Hayato Kuroda (Fujitsu) wrote: > > > > Thanks for updating the patch! > > Thanks for the updated patch, few comme

Re: speed up a logical replica setup

2024-02-13 Thread Euler Taveira
> Oh, thanks. > Just to confirm - pglogical set shared_preload_libraries to '', should we > follow or not? The in-core logical replication does not require any library to be loaded. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-02-07 Thread Euler Taveira
on-config.html [2] https://github.com/2ndQuadrant/pglogical/blob/REL2_x_STABLE/pglogical_create_subscriber.c#L488 [3] https://github.com/2ndQuadrant/pglogical/blob/REL2_x_STABLE/pglogical_create_subscriber.c#L529 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Possibility to disable `ALTER SYSTEM`

2024-02-07 Thread Euler Taveira
ed use cases, you will never have to change it but you can have a mechanism to change it. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-02-06 Thread Euler Taveira
-0003? I started reviewing v16-0007 but didn't finish yet. The general idea is ok. However, I'm still worried about preventing some use cases if it provides only the local connection option. What if you want to keep monitoring this instance while the transformation is happening? Let's say it has a ba

Re: speed up a logical replica setup

2024-02-01 Thread Euler Taveira
ot to remove this option. Instead, it should be optional. If it is not provided, primary_conninfo is used. [1] https://www.postgresql.org/message-id/80ce3f65-7ca3-471e-b1a4-24ac529ff4ea%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ From 2666b5f660c64d699a0be440c3701c7be00ec

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
nformation for troubleshooting. > > Good point. I included a check for pg_create_subscription role and CREATE > privilege on the specified database. > > > > Not sure, but can we do the replication origin functions by these privilege? > According to the doc[1], these ones

Re: Synchronizing slots from primary to standby

2024-01-31 Thread Euler Taveira
n the same information). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
d point. I included a check for pg_create_subscription role and CREATE privilege on the specified database. > 17. main > ``` > /* > * Change system identifier. > */ > modify_sysid(pg_resetwal_path, subscriber_dir); > ``` > > Even if I executed without -v option, an output from pg_resetw

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
On Wed, Jan 31, 2024, at 11:55 AM, Fabrízio de Royes Mello wrote: > > On Wed, Jan 31, 2024 at 11:35 AM Euler Taveira wrote: > > > > On Wed, Jan 31, 2024, at 11:25 AM, Fabrízio de Royes Mello wrote: > > > > Jumping into this a bit late here... I'm trying a s

Re: speed up a logical replica setup

2024-01-31 Thread Euler Taveira
* extract publisher connection string. Assume that there are * identical entries for physical and logical replication. If there is * not, we would fail anyway. */ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-29 Thread Euler Taveira
On Sun, Jan 28, 2024, at 10:10 PM, Euler Taveira wrote: > On Fri, Jan 26, 2024, at 4:55 AM, Hayato Kuroda (Fujitsu) wrote: >> Again, thanks for updating the patch! There are my random comments for v9. > > Thanks for checking v9. I already incorporated some of the points below

Re: Should we remove -Wdeclaration-after-statement?

2024-01-29 Thread Euler Taveira
on't see huge benefits moving from one style to the other. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-28 Thread Euler Taveira
; + > + pg_subscriber > + option > + > + > ``` > > Can you reply my comment#2 [4]? I think mandatory options should be written. I included the mandatory options into the synopsis. > 10. > Just to confirm - will you implement start_standby/stop_standby functions in > next version? It is still on my list. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-25 Thread Euler Taveira
think this error means the standby has > wrong primary_slot_name, right? I refactored this code a bit but the message is the same. It detects 2 cases: (a) you set primary_slot_name but you don't have a replication slot with the same name and (b) a cannot-happen bug that provides > 1 rows. It is a broken setup so maybe a hint saying so is enough. > 18. misc > Sometimes the pid of pg_subscriber is referred. It can be stored as global > variable. I prefer to keep getpid() call. > 19. > C99-style has been allowed, so loop variables like "i" can be declared in the > for-statement, like > > ``` > for (int i = 0; i < MAX; i++) > ``` v9 does it. > 20. > Some comments, docs, and outputs must be fixed when the name is changed. Next patch. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-25 Thread Euler Taveira
On Tue, Jan 23, 2024, at 10:29 PM, Euler Taveira wrote: > I'll post a new one soon. I'm attaching another patch that fixes some of the issues pointed out by Hayato, Shlok, and Junwang. * publication doesn't exist. The analysis [1] was done by Hayato but I didn't use the proposed patch. Inst

Re: speed up a logical replica setup

2024-01-23 Thread Euler Taveira
in my local branch and removing one of them is time consuming. For this one, I did the job but let's avoid rework. > 0003 patch contains fix for bug reported in [8]. LGTM. As I said in the other email, I included it. I'll post a new one soon. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-23 Thread Euler Taveira
in why we should create the publication earlier. (I'm renaming create_all_logical_replication_slots to setup_publisher and calling create_publication from there and also adding the proposed GUC checks in it.) -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-23 Thread Euler Taveira
gt; internally create slots and then wait for wal, etc. which makes me > sound like adding 'prepare' in the name can also explain the purpose. I think "convert" and "transform" fit for this case. However, "create", "convert" and "transform" have 6, 7 and 9 characters, respectively. I suggest that we avoid long names (subscriber already has 10 characters). My preference is pg_createsubscriber. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-11 Thread Euler Taveira
e. It was already done in v5. I did in a different way. > 5. > Unnecessary Assert in drop_replication_slot() was removed. Instead, I fixed the code and keep the assert. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-11 Thread Euler Taveira
g/message-id/TY3PR01MB988978C7362A101927070D29F56A2%40TY3PR01MB9889.jpnprd01.prod.outlook.com -- Euler Taveira EDB https://www.enterprisedb.com/ From 2341ef3dc26d48b51b13ad01ac38c79d24b1e461 Mon Sep 17 00:00:00 2001 From: Euler Taveira Date: Mon, 5 Jun 2023 14:39:40 -0400 Subject: [PATCH v5] C

Re: speed up a logical replica setup

2024-01-10 Thread Euler Taveira
't a reliable and cheap way to do it. You need to start the recovery and check if it is having some progress. (I don't have a strong opinion about requiring a standby to use this tool. It would reduce the complexity about checking if the setup has all requirements to run this tool.) -

Re: speed up a logical replica setup

2024-01-04 Thread Euler Taveira
e. If point 2 isn't an > > issue, imo point 1 is no big deal. > > > > By point 2, do you mean to have a check for "max replication slots"? > It so, the one possibility is to even increase that config, if the > required max_replication_slots is low. By point 2, I m

Re: speed up a logical replica setup

2024-01-04 Thread Euler Taveira
gt; > > I think asking users to manually remove such slots won't be a good > idea. We might want to either remove them by default or provide an > option to the user. Am I correct that the majority of the use cases these replication slots will be useless? If so, let's remove them by default and add an option to control this behavior (replication slot removal). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-03 Thread Euler Taveira
t 2 isn't an issue, imo point 1 is no big deal. [1] https://www.postgresql.org/message-id/e02a2c17-22e5-4ba6-b788-de696ab74f1e%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2024-01-03 Thread Euler Taveira
impacts the time for small to large-size databases as > compared to the traditional method? This might help us with giving > users advice on when to use this tool. We can do this bit later as > well when the patch is closer to being ready for commit. I'll share it. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2023-12-20 Thread Euler Taveira
our intentional? No. I will update the patch accordingly. I suspect that a fair amount of patches broke due to MSVC change. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-19 Thread Euler Taveira
t; > at the end [1]. > > Indeed, that should be enough to answer my comment. > > > Attached v4 patch. > > LGTM, thanks. Euler, what do you think? > LGTM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Add --check option to pgindent

2023-12-18 Thread Euler Taveira
The referenced Wiki page > will need an edit or two after this goes in, but that's easy enough. ... and pgbuildfarm client [1] needs to be changed too. [1] https://github.com/PGBuildFarm/client-code/blob/main/PGBuild/Modules/CheckIndent.pm#L55-L90 -- Euler Taveira EDB https://www.enterprisedb.com/

Re: [DOC] Introducing Quick Start Guide to PL/pgSQL and PL/Python Documentation

2023-12-18 Thread Euler Taveira
ext CF [1] so we don't miss it. [1] https://commitfest.postgresql.org/46/ -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Add a perl function in Cluster.pm to generate WAL

2023-12-18 Thread Euler Taveira
24/24, tx: 0, lsn: 0/41017C70, prev 0/41017A98, desc: SWITCH The difference is euler=# select '0/40A8'::pg_lsn - '0/4028'::pg_lsn; ?column? -- 128 (1 row) euler=# select '0/41017A98'::pg_lsn - '0/4128'::pg_lsn; ?column? -- 96880 (1 row) It is cheape

Re: Improve eviction algorithm in ReorderBuffer

2023-12-15 Thread Euler Taveira
). I didn't read your patch but do you really need to free entries one by one? If not, binaryheap_free(). -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logical decoding and replication of sequences, take 2

2023-12-14 Thread Euler Taveira
On Thu, Dec 14, 2023, at 12:44 PM, Ashutosh Bapat wrote: > I haven't found the code path from where the sequence cleanup gets > called. But it's being called. Am I missing something? ReorderBufferCleanupTXN. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Move walreceiver state assignment (to WALRCV_STREAMING) in WalReceiverMain()

2023-12-12 Thread Euler Taveira
ern is due to a slow system, the walrcv_connect() took to long in WalReceiverMain() and the code above kills the walreceiver while in the process to start it. Since you cannot control the hardcoded WALRCV_STARTUP_TIMEOUT value, you might have issues during overload periods. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Add --check option to pgindent

2023-12-12 Thread Euler Taveira
ent-diff exit with status 2 if any changes would be made + --check combination of --show-diff and --silent-diff I mean --diff=show,silent,check When you add exceptions, it starts to complicate the UI. usage("Cannot have both --silent-diff and --show-diff") if

Re: I’ve come across what I think is a bug

2023-12-07 Thread Euler Taveira
not* providing truncate as an operation. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Euler Taveira
ved ones) is not a valid JSON. Hence, a JSON parser cannot be executed until you receive the whole data set. (wal2json format 1 has this disadvantage. Format 2 was born to provide a better alternative -- each row is a valid JSON.) I'm not saying that "json array" is not useful but that for large

Re: Is WAL_DEBUG related code still relevant today?

2023-12-06 Thread Euler Taveira
t should go away eventually, though, because it's an ugly and * hard-to-explain kluge. */ int trace_recovery(int trace_level) { if (trace_level < LOG && trace_level >= trace_recovery_messages) return LOG; return trace_level; } -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Is WAL_DEBUG related code still relevant today?

2023-12-06 Thread Euler Taveira
same category as WAL_DEBUG. If it does, maybe it is worth to apply the same logic there. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2023-12-05 Thread Euler Taveira
AExHW5sCAU3NvPKd7msScQKvrBN-x_AdDQD-ZYAwOxuWG%3Doz1w%40mail.gmail.com [2] https://postgr.es/m/caexhw5vhfemfvtuhe+7xwphvzjxrexz5h3dd4uqi7cwmdmj...@mail.gmail.com -- Euler Taveira EDB https://www.enterprisedb.com/ From 003255b64910ce73f15931a43def25c37be96b81 Mon Sep 17 00:00:00 2001 From: Euler T

Re: Lifetime of commit timestamps

2023-11-17 Thread Euler Taveira
ine for convenience), hence, someone can ask why the pg_xact_commit_timestamp() returns NULL for a transaction that was executed *after* you enable track_commit_timestamp. The answer is the design used a existing mechanism to clean up data in order to avoid creating a new one. -- Euler Taveira

Re: speed up a logical replica setup

2023-11-08 Thread Euler Taveira
IdentifySystem() that is used by all of these replication tools. We can move the extern variables into parameters to have a pluggable streamutil.h. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: speed up a logical replica setup

2023-11-01 Thread Euler Taveira
On Tue, Oct 31, 2023, at 11:46 PM, shihao zhong wrote: > I think this is duplicate with https://commitfest.postgresql.org/45/4637/ > > The new status of this patch is: Waiting on Author > I withdrew the other entry. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Allowing TRUNCATE of FK target when session_replication_role=replica

2023-10-31 Thread Euler Taveira
least when > running pglogical. It is not a requirement for the subscriber. However, it increases the complexity for a real scenario (in which you set up backup and sometimes additional physical replicas) because key GUCs require a restart. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Allowing TRUNCATE of FK target when session_replication_role=replica

2023-10-31 Thread Euler Taveira
ame advantages as item 2. > Unless there are any serious objections, I will send a patch to also > allow TRUNCATE in this case. > You should start checking the previous proposal [1]. [1] https://www.postgresql.org/message-id/ff835f71-3c6c-335e-4c7b-b9e1646cf3d7%402ndquadrant.it [2] ht

Re: speed up a logical replica setup

2023-10-22 Thread Euler Taveira
On Mon, Feb 21, 2022, at 9:09 AM, Euler Taveira wrote: > A new tool called pg_subscriber does this conversion and is tightly integrated > with Postgres. After a long period of inactivity, I'm back to this client tool. As suggested by Andres, I added a new helper function to change the

Re: Is it possible to change wal_level online

2023-09-14 Thread Euler Taveira
l_level default value. There might be other threads but I didn't try hard to find them. [1] https://www.postgresql.org/message-id/20200608213215.mgk3cctlzvfuaqm6%40alap3.anarazel.de -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Support prepared statement invalidation when result types change

2023-09-13 Thread Euler Taveira
/* length of orig_param_types array */ You should expand the commit message a bit. Explain this feature. Inform the behavior change. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Pgoutput not capturing the generated columns

2023-08-01 Thread Euler Taveira
it is an option that defaults to false and doesn't impact performance. The request for features should be done in this mailing list. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: CDC/ETL system on top of logical replication with pgoutput, custom client

2023-07-31 Thread Euler Taveira
"C"} 0/36A050C0 | 454540 | {"action":"B"} 0/36A050C0 | 454540 | {"action":"I","schema":"public","table":"foo","columns":[{"name":"a","value":1}]} 0/36A051A0 | 454540 | {"action":"I","schema":"public","table":"foo","columns":[{"name":"a","value":2}]} 0/36A054D8 | 454540 | {"action":"I","schema":"public","table":"foo","columns":[{"name":"a","value":3}]} 0/36A05820 | 454540 | {"action":"C"} (17 rows) Since session C committed first, it is the first transaction available to output plugin (wal2json). Transaction 454541 is the next one that is available because it committed after session C (transaction 454542) and the first transaction that started (session A) is the last one available. You can also notice that the first transaction (454540) is the last one available. Your consumer cannot rely on LSN position or xid to track the progress. Instead, Postgres provides a replication progress mechanism [1] to do it. [1] https://www.postgresql.org/docs/current/replication-origins.html -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logicalrep_message_type throws an error

2023-07-15 Thread Euler Taveira
ent is not at the beginning of the function but *before* the message. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Allow specifying a dbname in pg_basebackup connection string

2023-07-05 Thread Euler Taveira
t also benefit from it. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logicalrep_message_type throws an error

2023-07-05 Thread Euler Taveira
representations of the same data (X and 88). Since we already provide "X" I don't think we need "88". Another option, is to remove "X" from apply_dispatch() and add "??? (88)" to logicalrep_message_type(). Opinions? -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logicalrep_message_type throws an error

2023-07-03 Thread Euler Taveira
; case > > defeats that purpose. I think we should just return "???" from outside > > switch block. > > Yeah. I don't think any gcc -Wswitch options have effect if a default is used so your suggestion is good for wrong/missing message types in the future. > PFA patch. WFM. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: logicalrep_message_type throws an error

2023-07-03 Thread Euler Taveira
efault message type: ERROR: invalid logical replication message type "X" CONTEXT: processing remote data for replication origin "pg_16638" during message type "???" in transaction 796, finished at 0/16266F8 Masahiko, since abc0910e2e0 is your patch maybe you

Re: drop table in transaction

2023-05-09 Thread Euler Taveira
On Tue, May 9, 2023, at 7:42 AM, Fabrice Chapuis wrote: > Where in the code is written the mechanism used for isolation when drop table > is executed in a transaction RemoveRelations() in src/backend/commands/tablecmds.c If you are looking for a previous layer, check ExecDropStmt(). --

Re: Initial Schema Sync for Logical Replication

2023-03-24 Thread Euler Taveira
l.org/message-id/fb7894e4-b44e-4ae3-a74d-7c5650f69f1a%40app.fastmail.com [2] https://www.postgresql.org/message-id/78149fa6-4c77-4128-8518-197a631c29c3%40app.fastmail.com -- Euler Taveira EDB https://www.enterprisedb.com/

Re: add log messages when replication slots become active and inactive (was Re: Is it worth adding ReplicationSlot active_pid to ReplicationSlotPersistentData?)

2023-03-23 Thread Euler Taveira
someone is grepping by the origin function. I prefer a sentence that includes a verb. physical replication slot \"%s\" is acquired logical replication slot \"%s\" is released Isn't the PID important for this use case? If so, of course, you can rely on log_line_prefix (%p) but if the PID is crucial for an investigation then it should also be included in the message. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Initial Schema Sync for Logical Replication

2023-03-23 Thread Euler Taveira
On Thu, Mar 23, 2023, at 8:44 AM, Amit Kapila wrote: > On Thu, Mar 23, 2023 at 2:48 AM Euler Taveira wrote: > > > > On Tue, Mar 21, 2023, at 8:18 AM, Amit Kapila wrote: > > > > Now, how do we avoid these problems even if we have our own version of > > functionalit

Re: Initial Schema Sync for Logical Replication

2023-03-22 Thread Euler Taveira
ssion [1], I think if we agree that the Alvaro's DDL deparse patch is the way to go with DDL replication, it seems wise that it should be used for initial DDL synchronization as well. [1] https://www.postgresql.org/message-id/CAA4eK1%2Bw_dFytBiv3RxbOL76_noMzmX0QGTc8uS%3Dbc2WaPVoow%40mail.gmail.com

Re: Initial Schema Sync for Logical Replication

2023-03-20 Thread Euler Taveira
onal client packages; * don't need to worry about different versions. This infrastructure can also be useful for other use cases such as: * client tools that provide create commands (such as psql, pgAdmin); * other logical replication solutions; * other logical backup solutions. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Allow logical replication to copy tables in binary format

2023-03-15 Thread Euler Taveira
on should explain that v14/v15 uses text format during initial table synchronization even if binary = true. Should there be a fallback mode (text) if initial table synchronization failed because of the binary option? Maybe a different setting (auto) to support such behavior. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: zstd compression for pg_dump

2023-02-25 Thread Euler Taveira
quot; = src/bin/pg_dump/compress_zstd.h && continue test "$f" = src/bin/pg_dump/compress_none.h && continue test "$f" = src/bin/pg_dump/parallel.h && continue test "$f" = src/bin/pg_dump/pg_backup_archiver.h && continue -- Euler Taveira EDB https://www.enterprisedb.com/

Re: Time delayed LR (WAS Re: logical replication restrictions)

2023-01-23 Thread Euler Taveira
errmsg("cannot enable %s mode for subscription with %s", + "streaming = parallel", "min_apply_delay")); + Is this code correct? I also didn't like this message. "cannot enable streaming = parallel mode f

Re: is_superuser is not documented

2022-09-09 Thread Euler Taveira
tgres=# \dfS is_superuser List of functions Schema | Name | Result data type | Argument data types | Type +--+--+-+-- (0 rows) postgres=# SHOW is_superuser; is_superuser -- on (1 row) Do you mind writing a patch

Re: including pid's for `There are XX other sessions using the database`

2022-08-19 Thread Euler Taveira
executing DROP DATABASE. -- Euler Taveira EDB https://www.enterprisedb.com/

Re: fix typos

2022-08-12 Thread Euler Taveira
e could also change the messages to say "ID"? ... or say could not drop replication origin %u, in use by PID %d AFAICS there is no "with ID" but there is "with identifier". I personally prefer to omit these additional words; it seems clear wi

  1   2   3   4   >