Re: RFC: compression dictionaries for JSONB

2022-04-22 Thread Aleksander Alekseev
Hi hackers, Many thanks for all your great feedback! Please see the follow-up thread '[PATCH] Compression dictionaries for JSONB': https://postgr.es/m/CAJ7c6TOtAB0z1UrksvGTStNE-herK-43bj22%3D5xVBg7S4vr5rQ%40mail.gmail.com -- Best regards, Aleksander Alekseev

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-04-22 Thread Thomas Munro
On Wed, Apr 6, 2022 at 5:07 AM Robert Haas wrote: > On Mon, Apr 4, 2022 at 10:20 PM Thomas Munro wrote: > > > The checkpointer never takes heavyweight locks, so the opportunity > > > you're describing can't arise. > > > > Hmm, oh, you probably meant the buffer interlocking > > in

[PATCH] Compression dictionaries for JSONB

2022-04-22 Thread Aleksander Alekseev
Hi hackers, This is a follow-up thread to `RFC: compression dictionaries for JSONB` [1]. I would like to share my current progress in order to get early feedback. The patch is currently in a draft state but implements the basic functionality. I did my best to account for all the great feedback I

Re: BufferAlloc: don't take two simultaneous locks

2022-04-22 Thread Yura Sokolov
Btw, I've runned tests on EPYC (80 cores). 1 key per select conns | master | patch-v11 | master 1G | patch-v11 1G ++++ 1 | 29053 | 28959 | 26715 | 25631 2 | 53714 | 53002 | 55211 |

Re: Estimating HugePages Requirements?

2022-04-22 Thread Magnus Hagander
On Wed, Apr 20, 2022, 00:12 Michael Paquier wrote: > On Thu, Mar 24, 2022 at 02:07:26PM +0100, Magnus Hagander wrote: > > But neither would the suggestion of redirecting stderr to /dev/null. > > In fact, doing the redirect it will *also* throw away any FATAL that > > happens. In fact, using the

Re: [Proposal] vacuumdb --schema only

2022-04-22 Thread Gilles Darold
Le 20/04/2022 à 19:38, Nathan Bossart a écrit : Thanks for the new patch! I think this is on the right track. On Wed, Apr 20, 2022 at 05:15:02PM +0200, Gilles Darold wrote: Le 18/04/2022 à 23:56, Nathan Bossart a écrit : - if (!tables_listed) + if (!objects_listed || objfilter ==

Re: Proposal for internal Numeric to Uint64 conversion function.

2022-04-22 Thread Amul Sul
On Fri, Mar 18, 2022 at 1:17 AM Greg Stark wrote: > > On Fri, 11 Mar 2022 at 15:17, Tom Lane wrote: > > > > Amul Sul writes: > > > > Yeah, that's true, I am too not sure if we really need to refactor > > > all those; If we want, I can give it a try. > > > > The patch as-presented isn't very

Re: How to generate a WAL record spanning multiple WAL files?

2022-04-22 Thread Bharath Rupireddy
On Wed, Apr 6, 2022 at 6:56 AM Andy Fan wrote: > > On Wed, Apr 6, 2022 at 12:41 AM Robert Haas wrote: >> >> On Tue, Apr 5, 2022 at 10:10 AM Andy Fan wrote: >> >> > I wanted to have a WAL record spanning multiple WAL files of size, say >> >> > 16MB. I'm wondering if the Full Page Images (FPIs)

Re: Fix NULL pointer reference in _outPathTarget()

2022-04-22 Thread Peter Eisentraut
On 20.04.22 18:53, Tom Lane wrote: I think we could put the if (node->fldname) inside the WRITE_INDEX_ARRAY macro. Yeah, that's another way to do it. I think though that the unresolved question is whether or not we want the field name to appear in the output when the field is null. I

Re: [PATCH] Compression dictionaries for JSONB

2022-04-22 Thread Zhihong Yu
On Fri, Apr 22, 2022 at 1:30 AM Aleksander Alekseev < aleksan...@timescale.com> wrote: > Hi hackers, > > This is a follow-up thread to `RFC: compression dictionaries for JSONB` > [1]. I would like to share my current progress in order to get early > feedback. The patch is currently in a draft

Re: Dump/Restore of non-default PKs

2022-04-22 Thread Tom Lane
Peter Eisentraut writes: > On 21.04.22 13:43, Simon Riggs wrote: >> Can you explain what you find unattractive about it? > Well, if I want to create a table with a primary key, the established > way is to say "primary key", not to have to assemble it from multiple > pieces. > I think this

Re: How to simulate sync/async standbys being closer/farther (network distance) to primary in core postgres?

2022-04-22 Thread Bharath Rupireddy
On Sat, Apr 9, 2022 at 6:38 PM Julien Rouhaud wrote: > > On Sat, Apr 09, 2022 at 02:38:50PM +0530, Bharath Rupireddy wrote: > > On Fri, Apr 8, 2022 at 10:22 PM SATYANARAYANA NARLAPURAM > > wrote: > > > > > >> > wrote: > > >> > > > > >> > > Hi, > > >> > > > > >> > > I'm thinking if there's a way

Cryptohash OpenSSL error queue in FIPS enabled builds

2022-04-22 Thread Daniel Gustafsson
In trying out an OpenSSL 3.1 build with FIPS enabled I realized that our cryptohash code had a small issue. Calling a banned cipher generated two different error messages interleaved: postgres=# select md5('foo'); ERROR: could not compute MD5 hash: unsupported postgres=# select

Re: Re: fix cost subqueryscan wrong parallel cost

2022-04-22 Thread David G. Johnston
On Wed, Apr 20, 2022 at 11:38 PM bu...@sohu.com wrote: > > > for now fuction cost_subqueryscan always using *total* rows even > parallel > > > path. like this: > > > > > > Gather (rows=3) > > > Workers Planned: 2 > > > -> Subquery Scan (rows=3) -- *total* rows, should be equal >

Re: Skipping schema changes in publication

2022-04-22 Thread Bharath Rupireddy
On Tue, Mar 22, 2022 at 12:39 PM vignesh C wrote: > > Hi, > > This feature adds an option to skip changes of all tables in specified > schema while creating publication. > This feature is helpful for use cases where the user wants to > subscribe to all the changes except for the changes present

Re: pg_walcleaner - new tool to detect, archive and delete the unneeded wal files (was Re: pg_archivecleanup - add the ability to detect, archive and delete the unneeded wal files on the primary)

2022-04-22 Thread Bharath Rupireddy
On Mon, Apr 18, 2022 at 8:48 PM Stephen Frost wrote: > > Greeting, > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > On Mon, Apr 18, 2022 at 7:41 PM Stephen Frost wrote: > > > * Bharath Rupireddy (bharath.rupireddyforpostg...@gmail.com) wrote: > > > > Thanks for the

Re: pg_receivewal fail to streams when the partial file to write is not fully initialized present in the wal receiver directory

2022-04-22 Thread Bharath Rupireddy
On Tue, Apr 19, 2022 at 10:42 AM Michael Paquier wrote: > > > I would like to know if there's any problem with the proposed fix. > > There is nothing done for the case of compressed segments, meaning > that you would see the same problem when being in the middle of > writing a segment compressed

Re: Fix NULL pointer reference in _outPathTarget()

2022-04-22 Thread Tom Lane
Peter Eisentraut writes: > On 20.04.22 18:53, Tom Lane wrote: >> Yeah, that's another way to do it. I think though that the unresolved >> question is whether or not we want the field name to appear in the output >> when the field is null. I believe that I intentionally made it not appear >>

Re: Dump/Restore of non-default PKs

2022-04-22 Thread Peter Eisentraut
On 21.04.22 13:43, Simon Riggs wrote: 1. create the table without primary key 2. create the index 3. attach the index as primary key constraint That doesn't sound attractive. Can you explain what you find unattractive about it? Well, if I want to create a table with a primary key, the

Re: why pg_walfile_name() cannot be executed during recovery?

2022-04-22 Thread Bharath Rupireddy
On Sat, Apr 9, 2022 at 10:21 PM Robert Haas wrote: > > On Sat, Apr 9, 2022 at 12:25 PM Andrey Borodin wrote: > > Please excuse me if I'm not attentive enough. I've read this thread. And I > > could not find what is the problem that you are solving. What is the > > purpose of the WAL file name

Re: Handle infinite recursion in logical replication setup

2022-04-22 Thread vignesh C
On Tue, Apr 19, 2022 at 8:29 AM Peter Smith wrote: > > I checked the latest v9-0001 patch. Below are my review comments. > > Other than these few trivial comments this 0001 patch looks good to me. > > ~~~ > > 1. src/backend/replication/pgoutput/pgoutput.c - whitespace > > @@ -1696,6 +1714,10 @@

Re: Handle infinite recursion in logical replication setup

2022-04-22 Thread vignesh C
On Wed, Apr 20, 2022 at 7:26 AM Peter Smith wrote: > > Below are my review comments for the v9-0002 patch (except I did not > yet look at the TAP tests). > > ~~~ > > 1. General comment - describe.c > > I wondered why the copy_data enum value is not displayed by the psql > \drs+ command. Should it

Why is EXECUTE granted to PUBLIC for all routines?

2022-04-22 Thread Jacek Trocinski
Hi, The default behavior on Postgres is to grant EXECUTE to PUBLIC on any function or procedure that is created. I feel this this is a security concern, especially for procedures and functions defined with the "SECURITY DEFINER" clause. Normally, we don’t want everyone on the database to be

Re: Why is EXECUTE granted to PUBLIC for all routines?

2022-04-22 Thread Tom Lane
Jacek Trocinski writes: > The default behavior on Postgres is to grant EXECUTE to PUBLIC on any > function or procedure that is created. > I feel this this is a security concern, especially for procedures and > functions defined with the "SECURITY DEFINER" clause. There is zero security concern

Re: Handle infinite recursion in logical replication setup

2022-04-22 Thread vignesh C
On Wed, Apr 20, 2022 at 11:19 AM Peter Smith wrote: > > Below are my review comments for the v9-0002 patch (TAP test part only). > > (The order of my comments is a bit muddled because I jumped around in > the file a bit while reviewing it). > > == > > 1. create_subscription - missing comment.

Re: Cryptohash OpenSSL error queue in FIPS enabled builds

2022-04-22 Thread Tom Lane
Daniel Gustafsson writes: > It turns out that OpenSSL places two errors in the queue for this operation, > and we only consume one without clearing the queue in between, so we grab an > error from the previous run. Ugh. > Consuming all (both) errors and creating a concatenated string seems

Re: Postgres perl module namespace

2022-04-22 Thread Andres Freund
On 2022-04-21 09:42:44 -0400, Andrew Dunstan wrote: > On 2022-04-21 Th 00:11, Michael Paquier wrote: > > On Wed, Apr 20, 2022 at 03:56:17PM -0400, Andrew Dunstan wrote: > >> Basically I propose just to remove any mention of the Testlib items and > >> get_free_port from the export and alias lists

Re: Building Postgres with lz4 on Visual Studio

2022-04-22 Thread Andres Freund
Hi, Michael, Dilip, I think you worked most in this area? Based on 9ca40dcd4d0cad43d95a9a253fafaa9a9ba7de24 Robert, added you too, because zstd seems to have the same issue (based on the tail of the quoted email below). On 2022-04-13 17:21:41 +0300, Melih Mutlu wrote: > I tried to build

Re: pgsql: Allow db.schema.table patterns, but complain about random garbag

2022-04-22 Thread Robert Haas
On Fri, Apr 22, 2022 at 10:24 AM Andrew Dunstan wrote: > On 2022-04-22 Fr 10:04, Tom Lane wrote: > > Andrew Dunstan writes: > >> This has upset the buildfarm's msys2 animals. There appears to be some > >> wildcard expansion going on that causes the problem. I don't know why it > >> should here

Re: pgsql: Allow db.schema.table patterns, but complain about random garbag

2022-04-22 Thread Thomas Munro
On Sat, Apr 23, 2022 at 8:06 AM Robert Haas wrote: > Sure, see also > http://postgr.es/m/CA+TgmoYRGUcFBy6VgN0+Pn4f6Wv=2h0hzluphqsy6vc8ba7...@mail.gmail.com > where Andrew's opinion on how to fix this was sought. > > I have to say the fact that IPC::Run does shell-glob expansion of its >

[PATCH] Teach pg_waldump to extract FPIs from the WAL

2022-04-22 Thread David Christensen
Hi -hackers, Enclosed is a patch to allow extraction/saving of FPI from the WAL stream via pg_waldump. Description from the commit: Extracts full-page images from the WAL stream into a target directory, which must be empty or not exist. These images are subject to the same filtering rules as

Re: pgsql: Allow db.schema.table patterns, but complain about random garbag

2022-04-22 Thread Noah Misch
On Sat, Apr 23, 2022 at 09:12:20AM +1200, Thomas Munro wrote: > On Sat, Apr 23, 2022 at 8:06 AM Robert Haas wrote: > > I have to say the fact that IPC::Run does shell-glob expansion of its > > arguments on some machines and not others seems ludicrous to me. This > > patch may be overtested, but

Re: [Proposal] vacuumdb --schema only

2022-04-22 Thread Nathan Bossart
On Fri, Apr 22, 2022 at 11:57:05AM +0200, Gilles Darold wrote: > Patch v10 attached. Thanks! I've attached a v11 with some minor editorialization. I think I was able to improve the error handling for invalid combinations of command-line options a bit, but please let me know what you think. --