Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-16 Thread David Ventimiglia
Whoops! Wrong SO link. Here's the correct SO link: https://stackoverflow.com/questions/77808615/how-to-use-logical-decoding-with-pg-recvlogical-to-pass-changes-through-a-non-tr On Tue, Jan 16, 2024 at 1:15 PM David Ventimiglia < davidaventimig...@hasura.io> wrote: > Thanks for

Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-16 Thread David Ventimiglia
estions/75784345/how-to-pipe-pg-recvlogical-to-psql-for-logical-replication> . Cheers, David On Tue, Jan 16, 2024 at 12:57 PM Jim Nasby wrote: > On 1/13/24 3:34 PM, David Ventimiglia wrote: > > The business problem I'm trying to solve is: > > > > "How do I

Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-13 Thread David Ventimiglia
sing psql? > > On Sat, Jan 13, 2024 at 4:34 PM David Ventimiglia < > davidaventimig...@hasura.io> wrote: > >> The business problem I'm trying to solve is: >> >> "How do I capture logical decoding events with the wal2json output >> encoder, filter them with jq, and pipe them to psql, using pg_recvlogical?" >> >>> >>>>>

Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-13 Thread David Ventimiglia
It satisfies business constraints. On Sat, Jan 13, 2024, 5:01 PM Karsten Hilbert wrote: > Am Sat, Jan 13, 2024 at 05:53:14PM -0500 schrieb Ron Johnson: > > > *No,* that's a technology problem. What is the purpose of storing them > > back in the database using psql? > > Or even the end goal to b

Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-13 Thread David Ventimiglia
blem. Tell us the "business problem" you > are trying to solve, not the problem you're having with your solution to > the "business problem". > > (If you've already mentioned it, please restate it.) > > On Sat, Jan 13, 2024 at 11:49 AM David Ventimiglia

Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-13 Thread David Ventimiglia
output from pg_recvlogical and pipe it back into the database with psql?"* Best, David On Sat, Jan 13, 2024 at 10:29 AM Adrian Klaver wrote: > On 1/12/24 21:23, David Ventimiglia wrote: > > Let me just lay my cards on the table. What I'm really trying to do is > > capture

Re: How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-12 Thread David Ventimiglia
, 2024 at 8:42 PM Juan Rodrigo Alejandro Burgos Mella < rodrigoburgosme...@gmail.com> wrote: > > try use the following syntax (yes, with a 2 before the greater sign) > > pg_recvlogical -d postgres --slot test --start -f - 2>> sample.jsonl > > Atte > JRBM > > E

How to redirect output from PostgreSQL pg_recvlogical to a file or a pipe?

2024-01-12 Thread David Ventimiglia
Hello! How do I redirect logical decoding output from the PostgreSQL CLI tool pg_recvlogical either to a file or to another command via a pipe? I ask because when I try the obvious, no output is recorded or sent: pg_recvlogical -d postgres --slot test --start -f - >> sample.jsonl Lest there be

Re: Help with a good mental model for estimating PostgreSQL throughput

2023-10-30 Thread David Ventimiglia
s per operation * number of connections = 1000 ms / 101 ms * 1000 = ~1 - something else - impossible to determine without more information Best, David On Mon, Oct 30, 2023 at 8:46 AM Laurenz Albe wrote: > On Mon, 2023-10-30 at 08:05 -0700, David Ventimiglia wrote: > > Can s

Help with a good mental model for estimating PostgreSQL throughput

2023-10-30 Thread David Ventimiglia
Hello! Can someone help me develop a good mental model for estimating PostgreSQL throughput? Here's what I mean. Suppose I have: - 1000 connections - typical query execution time of 1ms - but additional network latency of 100ms What if at all would be an estimate of the number of oper

Re: How to optimize PostgreSQL Row Security Policies that involve related tables?

2023-08-14 Thread David Ventimiglia
Sorry, I had a typo in my Row Policy. Rather, it should look like this: create policy album_rls_policy on "Track" for select to public using ( exists (select * from "Album" where "Album"."AlbumId" = "Track"."AlbumId"

How to optimize PostgreSQL Row Security Policies that involve related tables?

2023-08-14 Thread David Ventimiglia
Hello! What are some good ways to create Row Security Policies that restrict rows in one table using a filter on a related table, and get good performance? It's difficult to describe but let me try to explain. I'm using the venerable old Chinook database