Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread vignesh C
Thanks Bharath for your comments. On Wed, Oct 28, 2020 at 9:48 AM Bharath Rupireddy wrote: > > On Wed, Oct 28, 2020 at 8:29 AM vignesh C wrote: > > > > Log message for GSS connection is missing once connection > > authorization is successful. We have similar log messages for SSL > > connections

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 10:14 PM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > On Wed, Oct 28, 2020 at 7:51 PM David G. Johnston > wrote: > > > I was thinking this would be useful for orchestration. However, as you > say, its a pretty fragile method. I withdraw the

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Bharath Rupireddy
On Wed, Oct 28, 2020 at 7:51 PM David G. Johnston wrote: > >> And also in case if the given pid is not a >> backend pid, we are throwing a warning and returning false but not >> error. >> >> Similarly we can return false on timeout, if required a >> warning. Thoughts? > > IMO, if there are

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-10-28 Thread Andres Freund
Hi, On 2020-10-28 21:00:30 -0700, Andres Freund wrote: > On 2020-10-28 19:09:14 -0700, Andres Freund wrote: > > On 2020-10-28 18:13:44 -0700, Andres Freund wrote: > > > Just pushed this. Let's see what the BF says... > > > > It says that apparently something is unstable about my new test. It > >

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread vignesh C
Thanks Stephen for your comments. On Wed, Oct 28, 2020 at 9:44 PM Stephen Frost wrote: > > Greetings, > > * vignesh C (vignes...@gmail.com) wrote: > > Log message for GSS connection is missing once connection > > authorization is successful. We have similar log messages for SSL > > connections

Re: psql \df choose functions by their arguments

2020-10-28 Thread Justin Pryzby
On Thu, Oct 15, 2020 at 01:21:06PM -0400, Greg Sabino Mullane wrote: > Improve psql \df to choose functions by their arguments I think this is a good idea. This isn't working for arrays: postgres=# \df aa public | aa | integer | integer, integer| func public | aa |

Re: Spurious "apparent wraparound" via SimpleLruTruncate() rounding

2020-10-28 Thread Noah Misch
On Sat, Aug 29, 2020 at 10:34:33PM -0700, Noah Misch wrote: > On Mon, May 25, 2020 at 12:00:33AM -0700, Noah Misch wrote: > > On Mon, Apr 06, 2020 at 09:18:47PM -0700, Noah Misch wrote: > > > On Mon, Apr 06, 2020 at 02:46:09PM -0400, Tom Lane wrote: > > > > Noah Misch writes: > > > > > On Wed,

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-10-28 Thread Andres Freund
Hi, On 2020-10-28 19:09:14 -0700, Andres Freund wrote: > On 2020-10-28 18:13:44 -0700, Andres Freund wrote: > > Just pushed this. Let's see what the BF says... > > It says that apparently something is unstable about my new test. It > first passed on a few animals, but then failed a lot in a row.

Re: document pg_settings view doesn't display custom options

2020-10-28 Thread Fujii Masao
On 2020/10/29 3:45, John Naylor wrote: On Wed, Oct 28, 2020 at 2:15 PM John Naylor mailto:john.nay...@enterprisedb.com>> wrote: Starting separate threads to keep from cluttering the TODO list thread. Here's a patch for the subject, as mentioned in

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Fujii Masao
On 2020/10/28 22:05, Laurenz Albe wrote: On Wed, 2020-10-28 at 09:55 +, osumi.takami...@fujitsu.com wrote: I wrote and attached the first patch to disable WAL logging. This patch passes the regression test of check-world already and is formatted by pgindent. Without reading the code, I

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-10-28 Thread Andres Freund
On 2020-10-28 18:13:44 -0700, Andres Freund wrote: > Just pushed this. Let's see what the BF says... It says that apparently something is unstable about my new test. It first passed on a few animals, but then failed a lot in a row. Looking.

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-10-28 Thread tsunakawa.ta...@fujitsu.com
The patch looks almost good except for the minor ones: (1) + for (i = 0; i < nnodes; i++) + { + RelFileNodeBackend rnode = smgr_reln[i]->smgr_rnode; + + rnodes[i] = rnode; + } You can write: + for (i = 0; i < nnodes; i++) +

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Michael Paquier
On Thu, Oct 29, 2020 at 12:02:11AM +0100, Tomas Vondra wrote: > On Wed, Oct 28, 2020 at 05:43:08PM -0300, Fabrízio de Royes Mello wrote: >> 2) REINDEX CONCURRENTLY does not keep statistics (pg_statistc) like a >> regular REINDEX for indexes using expressions and to me it's a bug. Michael >>

Re: list of extended statistics on psql

2020-10-28 Thread Tatsuro Yamada
Hi Tomas, On 2020/10/29 4:06, Tomas Vondra wrote: On Wed, Oct 28, 2020 at 03:07:56PM +0900, Tatsuro Yamada wrote: Hi Michael-san and Hackers, On 2020/09/30 15:19, Michael Paquier wrote: On Thu, Sep 17, 2020 at 02:55:31PM +0900, Michael Paquier wrote: Could you provide at least a rebased

Re: [PATCH] SET search_path += octopus

2020-10-28 Thread Andres Freund
Hi, On 2020-10-21 10:05:40 +0530, Abhijit Menon-Sen wrote: > > It seems to me that appending and incrementing using the same syntax > > is a) confusing b) will be a limitation before long. > > I understand (a), but what sort of limitation do you foresee in (b)? > > Do you think both features

Re: list of extended statistics on psql

2020-10-28 Thread Tatsuro Yamada
Hi Tomas, On 2020/10/29 4:07, Tomas Vondra wrote: On Wed, Oct 28, 2020 at 04:20:25PM +0900, Tatsuro Yamada wrote: Hi, Results of \dX and \dX+: postgres=# \dX     List of extended statistics     Schema    |   Name    |   Definition    |

Re: recovering from "found xmin ... from before relfrozenxid ..."

2020-10-28 Thread Andres Freund
Hi, On 2020-10-27 20:51:10 -0700, Andres Freund wrote: > On 2020-10-15 01:37:35 -0700, Andres Freund wrote: > > Attached is a *prototype* implemention of this concept, which clearly is > > lacking some comment work (and is intentionally lacking some > > re-indentation). > > > > I described my

Re: More aggressive vacuuming of temporary tables

2020-10-28 Thread Andres Freund
Hi, On 2020-10-14 13:31:03 -0700, Andres Freund wrote: > I was thinking about this a bit more, and I think the answer might be to > use Min(latestCompletedXid, MyProc->xid). That would, as far as I can > tell, never miss something vacuumable in a temporary table, doesn't > require to know whether

Re: [HACKERS] logical decoding of two-phase transactions

2020-10-28 Thread Peter Smith
FYI - I have cross-checked all the v12 patch code changes against the v12 code coverage resulting from running the patch tests Those v12 code coverage results were posted in this thread previously [1]. The purpose of this study was to identify if / where there are any gaps in the testing of this

Re: [HACKERS] logical decoding of two-phase transactions

2020-10-28 Thread Peter Smith
Hi Ajin. Looking at v13 patches again I found a couple more review comments: === (1) COMMENT File: src/backend/replication/logical/proto.c Function: logicalrep_write_prepare + if (rbtxn_commit_prepared(txn)) + flags = LOGICALREP_IS_COMMIT_PREPARED; + else if (rbtxn_rollback_prepared(txn)) +

Re: pg_prewarm bgworker could break fast shutdown

2020-10-28 Thread Tom Lane
Alexander Kukushkin writes: > I the fast shutdown was initiated before pg_prewarm managed to load > buffers from the dump (and start the main loop), the pg_prewarm > bgworker process never exits on SIGTERM and effectively preventing the > clean shutdown of the cluster. I might be wrong about

Re: Online checksums verification in the backend

2020-10-28 Thread Julien Rouhaud
Hello, On Thu, Oct 29, 2020 at 7:52 AM Shinoda, Noriyoshi (PN Japan A Delivery) wrote: > > Hi, > > I have tested this great feature in the latest commit environment on Red Hat > Enterprise Linux 7.8. I modified a few blocks in a relation file to raise a > checksum error. When I executed the

RE: Disable WAL logging to speed up data loading

2020-10-28 Thread osumi.takami...@fujitsu.com
Hello > > But what if someone sets wal_level=none, performs some data > > modifications, sets wal_level=archive and after dome more processing > > decides to restore from a backup that was taken before the cluster was set > to wal_level=none? > > Then they would end up with a corrupted database,

RE: Online checksums verification in the backend

2020-10-28 Thread Shinoda, Noriyoshi (PN Japan A Delivery)
Hi, I have tested this great feature in the latest commit environment on Red Hat Enterprise Linux 7.8. I modified a few blocks in a relation file to raise a checksum error. When I executed the pg_relation_check_pages function, the backend terminated abnormally. The attached file is the

Re: Track statistics for streaming of in-progress transactions

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 08:54:53AM +0530, Amit Kapila wrote: On Fri, Oct 23, 2020 at 10:24 AM Amit Kapila wrote: On Thu, Oct 22, 2020 at 2:09 PM Amit Kapila wrote: > I have fixed the above comment and rebased the patch. I have changed the docs a bit to add more explanation about the

Re: Autovacuum worker doesn't immediately exit on postmaster death

2020-10-28 Thread Tom Lane
Victor Yegorov writes: > ср, 28 окт. 2020 г. в 19:44, Alexander Kukushkin : >> I know, nobody in their mind should do that, but, if the postmaster >> process is killed with SIGKILL signal, most backend processes >> correctly notice the fact of the postmaster process absence and exit. >> There is

Re: MultiXact\SLRU buffers configuration

2020-10-28 Thread Tomas Vondra
Hi, On Wed, Oct 28, 2020 at 12:34:58PM +0500, Andrey Borodin wrote: Tomas, thanks for looking into this! 28 окт. 2020 г., в 06:36, Tomas Vondra написал(а): This thread started with a discussion about making the SLRU sizes configurable, but this patch version only adds a local cache. Does

Re: MultiXact\SLRU buffers configuration

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 10:36:39PM +0300, Alexander Korotkov wrote: Hi, Tomas! Thank you for your review. On Wed, Oct 28, 2020 at 4:36 AM Tomas Vondra wrote: I did a quick review on this patch series. A couple comments: 0001 This looks quite suspicious to me -

Re: Autovacuum worker doesn't immediately exit on postmaster death

2020-10-28 Thread Victor Yegorov
ср, 28 окт. 2020 г. в 19:44, Alexander Kukushkin : > I know, nobody in their mind should do that, but, if the postmaster > process is killed with SIGKILL signal, most backend processes > correctly notice the fact of the postmaster process absence and exit. > There is one exception though, when

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 05:43:08PM -0300, Fabrízio de Royes Mello wrote: On Wed, Oct 28, 2020 at 4:35 PM Tomas Vondra wrote: I don't think anyone proposed to do this through autovacuum. There was a reference to auto-analyze but I think that was meant as 'run analyze automatically.' Which

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Fabrízio de Royes Mello
On Wed, Oct 28, 2020 at 4:35 PM Tomas Vondra wrote: > > I don't think anyone proposed to do this through autovacuum. There was a > reference to auto-analyze but I think that was meant as 'run analyze > automatically.' Which would work in transactions just fine, I think. > Maybe I was not very

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Laurenz Albe
On Wed, 2020-10-28 at 14:05 +0100, I wrote: > But what if someone sets wal_level=none, performs some data modifications, > sets wal_level=archive and after dome more processing decides to restore from > a backup that was taken before the cluster was set to wal_level=none? > Then they would end up

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 05:16:32PM +0800, Craig Ringer wrote: > On Wed, Oct 28, 2020 at 12:02 PM Craig Ringer > wrote: > > On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: > > > > I don't know much about how to hook into that stuff so if you have an > idea, I am all

Re: Deduplicate aggregates and transition functions in planner

2020-10-28 Thread Andres Freund
Hi, On 2020-10-28 21:10:41 +0200, Heikki Linnakangas wrote: > Currently, ExecInitAgg() performs quite a lot of work, to deduplicate > identical Aggrefs, as well as Aggrefs that can share the same transition > state. That doesn't really belong in the executor, we should perform that > work in the

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 3:24 PM Tom Lane wrote: > > Nah. What I'm imagining is just that pg_type.h contains > > #ifdef EXPOSE_TO_CLIENT_CODE > > /* > * Backwards compatibility for ancient random spellings of OID macros. > * Don't use these macros in new code. > */ > #define CASHOID MONEYOID

Re: libpq compression

2020-10-28 Thread Alvaro Herrera
On 2020-Oct-26, Konstantin Knizhnik wrote: > + while (bufptr < bufend || zpq_buffered(PqStream) != 0) /* has more data > to flush or unsent data in internal compression buffer */ > { > - int r; > - > - r = secure_write(MyProcPort, bufptr,

pg_prewarm bgworker could break fast shutdown

2020-10-28 Thread Alexander Kukushkin
Hello, I the fast shutdown was initiated before pg_prewarm managed to load buffers from the dump (and start the main loop), the pg_prewarm bgworker process never exits on SIGTERM and effectively preventing the clean shutdown of the cluster. This problem bite me a few times, but yesterday I

Re: MultiXact\SLRU buffers configuration

2020-10-28 Thread Alexander Korotkov
Hi, Tomas! Thank you for your review. On Wed, Oct 28, 2020 at 4:36 AM Tomas Vondra wrote: > I did a quick review on this patch series. A couple comments: > > > 0001 > > > This looks quite suspicious to me - SimpleLruReadPage_ReadOnly is > changed to return information about what lock was

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
Andres Freund writes: > I assume you plan to error out if oid_symbol is defined for pg_type > going forward? Right, just like we just did for pg_proc. regards, tom lane

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:18:52PM -0400, Tom Lane wrote: Tomas Vondra writes: On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: Given how simple the manual workaround is not having it be manual seems like it would be safe and straight-forward to implement. Maybe, but I

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-10-28 Thread Justin Pryzby
On Tue, Sep 08, 2020 at 02:51:26PM -0500, Justin Pryzby wrote: > On Sat, Jul 18, 2020 at 03:15:32PM -0500, Justin Pryzby wrote: > > Still waiting for feedback from a committer. > > This patch has been waiting for input from a committer on the approach I've > taken with the patches since March 10,

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 3:24 PM Tom Lane wrote: > and then the negotiation here is only about whether to make this list > longer. We don't need to complicate genbki.pl with a new facility. > Agreed, and reformat_dat_files.pl must also know about these special attributes. -- John Naylor

Re: duplicate function oid symbols

2020-10-28 Thread Andres Freund
On 2020-10-28 15:24:20 -0400, Tom Lane wrote: > Nah. What I'm imagining is just that pg_type.h contains > > #ifdef EXPOSE_TO_CLIENT_CODE > > /* > * Backwards compatibility for ancient random spellings of OID macros. > * Don't use these macros in new code. > */ > #define CASHOID MONEYOID >

Re: libpq compression

2020-10-28 Thread Andres Freund
Hi, On 2020-10-26 19:20:46 +0300, Konstantin Knizhnik wrote: > diff --git a/configure b/configure > index ace4ed5..deba608 100755 > --- a/configure > +++ b/configure > @@ -700,6 +700,7 @@ LD > LDFLAGS_SL > LDFLAGS_EX > with_zlib > +with_zstd > with_system_tzdata > with_libxslt > XML2_LIBS

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
Andres Freund writes: > On 2020-10-28 14:49:06 -0400, Tom Lane wrote: >> Moreover, this clearly shows the >> effect John mentioned that people have been copying the style of adjacent >> entries rather than making use of the standard oid_symbol convention like >> they should --- some of these

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tom Lane
Tomas Vondra writes: > On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: >> Given how simple the manual workaround is not having it be manual seems >> like it would be safe and straight-forward to implement. > Maybe, but I wouldn't be surprised if it was actually a bit trickier

Re: duplicate function oid symbols

2020-10-28 Thread Andres Freund
Hi, On 2020-10-28 14:49:06 -0400, Tom Lane wrote: > The other extant oid_symbol entries are > > PGNODETREEOID > PGNDISTINCTOID > PGDEPENDENCIESOID > PGMCVLISTOID > PGDDLCOMMANDOID > EVTTRIGGEROID > The only one of these that client code would plausibly be using is LSNOID, > and even that is a

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 12:05 PM Tom Lane wrote: > This doesn't seem clearly different from any other situation where > auto-analyze doesn't react fast enough to suit you. > I would not > call it a bug, at least not without a wholesale redefinition of > how auto-analyze is supposed to work.

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:05:39PM -0400, Tom Lane wrote: Tomas Vondra writes: On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze side-effect

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 12:00:54PM -0700, David G. Johnston wrote: On Wed, Oct 28, 2020 at 11:55 AM Tomas Vondra wrote: I agree the lack of stats may be quite annoying and cause issues, but my guess is the chances of backpatching such change are about 0.01%. We have a usable 'workaround'

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 02:29:16PM -0400, Bruce Momjian wrote: > On Wed, Oct 28, 2020 at 12:02:46PM +0800, Craig Ringer wrote: > > Yes, that's possible. But in that case the passphrase will be asked for by > > openssl only when required, and we'll need to supply an openssl askpass > > hook. > >

Deduplicate aggregates and transition functions in planner

2020-10-28 Thread Heikki Linnakangas
Hi, Currently, ExecInitAgg() performs quite a lot of work, to deduplicate identical Aggrefs, as well as Aggrefs that can share the same transition state. That doesn't really belong in the executor, we should perform that work in the planner. It doesn't change from one invocation of the plan

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Mon, Oct 26, 2020 at 9:44 PM Nikolay Samokhvalov wrote: > On Mon, Oct 26, 2020 at 7:03 PM David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> On Monday, October 26, 2020, Nikolay Samokhvalov >> wrote: >>> >>> Although, this triggers a question – should ANALYZE be automated in, >>>

Re: list of extended statistics on psql

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 04:20:25PM +0900, Tatsuro Yamada wrote: Hi, Results of \dX and \dX+: postgres=# \dX     List of extended statistics     Schema    |   Name    |   Definition    | N_distinct | Dependencies |   Mcv

Re: list of extended statistics on psql

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 03:07:56PM +0900, Tatsuro Yamada wrote: Hi Michael-san and Hackers, On 2020/09/30 15:19, Michael Paquier wrote: On Thu, Sep 17, 2020 at 02:55:31PM +0900, Michael Paquier wrote: Could you provide at least a rebased version of the patch? The CF bot is complaning here.

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tom Lane
Tomas Vondra writes: > On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: >> It would seem preferable to call the lack of auto-analyzing after these >> operations a bug and back-patch a fix that injects an analyze side-effect >> just before their completion. It doesn't have to be

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 11:55 AM Tomas Vondra wrote: > I agree the lack of stats may be quite annoying and cause issues, but my > guess is the chances of backpatching such change are about 0.01%. We > have a usable 'workaround' for this - manual analyze. > My guess is that it wouldn't be

Re: [HACKERS] Custom compression methods

2020-10-28 Thread Tomas Vondra
On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote: ... I have added the next patch to support the compression options. I am storing the compression options only for the latest compression method. Basically, based on this design we would be able to support the options which are used

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Mon, Oct 26, 2020 at 03:46:10PM -0700, David G. Johnston wrote: On Mon, Oct 26, 2020 at 3:08 PM Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: Hi all, As you all already know Postgres supports functions in index expressions (marked as immutable ofc) and for this special index

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Tomas Vondra
On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: On Mon, Oct 26, 2020 at 7:46 PM David G. Johnston < david.g.johns...@gmail.com> wrote: It would seem preferable to call the lack of auto-analyzing after these operations a bug and back-patch a fix that injects an analyze

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
Andres Freund writes: > I think changing type oid macro names is somewhat problematic - in > contrast to function oid macros the type macros are much more likely to > be used by client applications, e.g. for deciding whether to use binary > or text format for a type. > A quick code search shows

Re: document pg_settings view doesn't display custom options

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 2:15 PM John Naylor wrote: > Starting separate threads to keep from cluttering the TODO list thread. > > Here's a patch for the subject, as mentioned in > https://www.postgresql.org/message-id/20201027220555.GS4951%40momjian.us > I just realized I introduced a typo, so

Autovacuum worker doesn't immediately exit on postmaster death

2020-10-28 Thread Alexander Kukushkin
Hello, I know, nobody in their mind should do that, but, if the postmaster process is killed with SIGKILL signal, most backend processes correctly notice the fact of the postmaster process absence and exit. There is one exception though, when there are autovacuum worker processes they are

Re: cleanup temporary files after crash

2020-10-28 Thread Tomas Vondra
Hi Euler, On Wed, Oct 28, 2020 at 11:16:26AM -0300, Euler Taveira wrote: Hi, Bug #16427 mentioned that temporary files are not removed after a crash. I heard similar complaints from some customers. In the bug report [1], I proposed a new GUC to control whether temporary files are removed after

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 12:02:46PM +0800, Craig Ringer wrote: > On Wed, Oct 28, 2020 at 9:43 AM Bruce Momjian wrote: >  I have used OpenSSL with Yubikey via pksc11.  You > can see the use of it on slide 57 and following: > >        

Re: duplicate function oid symbols

2020-10-28 Thread Andres Freund
Hi, Thanks for fixing the HEAP_TABLE_AM_HANDLER_OID one. On 2020-10-28 14:08:28 -0400, John Naylor wrote: > > The only existing oid_symbol in pg_type that I think has enough > > grandfather status to be tough to change is CASHOID for "money". > > But we could imagine special-casing that with a

document pg_settings view doesn't display custom options

2020-10-28 Thread John Naylor
Starting separate threads to keep from cluttering the TODO list thread. Here's a patch for the subject, as mentioned in https://www.postgresql.org/message-id/20201027220555.GS4951%40momjian.us -- John Naylor EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 12:25 PM Tom Lane wrote: > I wondered about introducing a similar prohibition for pg_type. > That might be worth doing, since some of the grandfathered macros are clustered together, which could lead to more cases creeping in as people match new types to examples nearby.

Re: [PATCH] Add `truncate` option to subscription commands

2020-10-28 Thread Rahila Syed
Hi David, The feature seems useful to me. The code will need to be refactored due to changes in commit : b05fe7b442 Please see the following comments. 1. Is there a specific reason behind having new relstate for truncate? The current state flow is

Re: cutting down the TODO list thread

2020-10-28 Thread Andres Freund
On 2020-10-28 16:20:03 +0100, Magnus Hagander wrote: > I would personally prefer a completely seprate page Same.

Re: Internal key management system

2020-10-28 Thread Stephen Frost
Greetings, * Craig Ringer (craig.rin...@enterprisedb.com) wrote: > On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: > > TL;DR: > > * Important to check that key rotation is possible on a replica, i.e. > primary and standby can have different cluster passphrase and KEK > encrypting the

Re: duplicate function oid symbols

2020-10-28 Thread Tom Lane
John Naylor writes: >> Ok, here is a patch to fix that, and also throw an error if pg_proc.dat >> has an explicitly defined symbol. > It occurred to me I neglected to explain the error with a comment, which > I've added in v2. Pushed with a bit of tweaking of the error message. I wondered

Re: libpq compression

2020-10-28 Thread Konstantin Knizhnik
On 28.10.2020 17:57, Даниил Захлыстов wrote: Hi, thanks for the patch! I’ve made a quick review and found one issue. If the backend sends a CompressionAck message followed by some already compressed message (for example, AuthenticationOk), then there is a chance that pqReadData() will read

Re: Internal key management system

2020-10-28 Thread Bruce Momjian
On Wed, Oct 28, 2020 at 09:24:35PM +0900, Masahiko Sawada wrote: > On Tue, 27 Oct 2020 at 20:34, Bruce Momjian wrote: > > You need to use separate keys for heap/index and WAL so you can > > replicate to another server that uses a different heap/index key, but > > the same WAL. You can then

Re: cannot freeze committed xmax

2020-10-28 Thread Mark Dilger
> On Oct 28, 2020, at 8:56 AM, Konstantin Knizhnik > wrote: > > > > On 28.10.2020 18:25, Mark Dilger wrote: >> >>> On Oct 28, 2020, at 6:44 AM, Konstantin Knizhnik >>> wrote: >>> >>> Looks like there is no assumption that xmax should be set to >>> InvalidTransactionId when

Re: Log message for GSS connection is missing once connection authorization is successful.

2020-10-28 Thread Stephen Frost
Greetings, * vignesh C (vignes...@gmail.com) wrote: > Log message for GSS connection is missing once connection > authorization is successful. We have similar log messages for SSL > connections once the connection authorization is successful. This > message will help the user to identify the

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-28 Thread Alvaro Herrera
On 2020-Oct-28, Tom Lane wrote: > Alvaro Herrera writes: > > I think if a struct is used as a function argument somewhere or arrays > > of the struct are formed, then it's certain that changing that struct's > > size is going to cause problems. > > I grasp the point about arrays, but not sure

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-28 Thread Tom Lane
Alvaro Herrera writes: > On 2020-Oct-27, Tom Lane wrote: >> * Changing the set of fields in ForeignKeyOptInfo is an ABI break. >> We could minimize the risk by adding the new fields at the end in >> the back branches, but it still wouldn't be zero-risk. > It'd be useful to be able to qualify

Re: cannot freeze committed xmax

2020-10-28 Thread Konstantin Knizhnik
On 28.10.2020 18:25, Mark Dilger wrote: On Oct 28, 2020, at 6:44 AM, Konstantin Knizhnik wrote: Looks like there is no assumption that xmax should be set to InvalidTransactionId when HEAP_XMAX_INVALID bit is set. And I didn't find any check preventing cutoff_xid to be greater than XID

Re: duplicate function oid symbols

2020-10-28 Thread John Naylor
I wrote: > Ok, here is a patch to fix that, and also throw an error if pg_proc.dat > has an explicitly defined symbol. > It occurred to me I neglected to explain the error with a comment, which I've added in v2. -- John Naylor EnterpriseDB: http://www.enterprisedb.com The Enterprise

Re: Patch to fix FK-related selectivity estimates with constants

2020-10-28 Thread Alvaro Herrera
On 2020-Oct-27, Tom Lane wrote: > I had two concerns about possible extension breakage from a back-patch: > > * Changing the set of fields in ForeignKeyOptInfo is an ABI break. > We could minimize the risk by adding the new fields at the end in > the back branches, but it still wouldn't be

Re: cannot freeze committed xmax

2020-10-28 Thread Mark Dilger
> On Oct 28, 2020, at 6:44 AM, Konstantin Knizhnik > wrote: > > Looks like there is no assumption that xmax should be set to > InvalidTransactionId when HEAP_XMAX_INVALID bit is set. > And I didn't find any check preventing cutoff_xid to be greater than XID of > some transaction which was

Re: libpq compression

2020-10-28 Thread Daniil Zakhlystov
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, thanks for the patch! I’ve made a quick review and found one issue.

Re: cutting down the TODO list thread

2020-10-28 Thread Magnus Hagander
On Wed, Oct 28, 2020 at 3:35 PM Julien Rouhaud wrote: > > On Wed, Oct 28, 2020 at 9:27 PM John Naylor > wrote: > > > > On Wed, Oct 28, 2020 at 6:52 AM Magnus Hagander wrote: > >> > >> On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud wrote: > >> > > >> > On Wed, 28 Oct 2020, 17:55 Oleksandr

Re: Add Information during standby recovery conflicts

2020-10-28 Thread Fujii Masao
On 2020/10/27 9:41, Masahiko Sawada wrote: On Tue, 20 Oct 2020 at 22:02, Drouvot, Bertrand wrote: Hi, On 10/15/20 9:15 AM, Masahiko Sawada wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and

Re: cutting down the TODO list thread

2020-10-28 Thread Julien Rouhaud
On Wed, Oct 28, 2020 at 9:27 PM John Naylor wrote: > > On Wed, Oct 28, 2020 at 6:52 AM Magnus Hagander wrote: >> >> On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud wrote: >> > >> > On Wed, 28 Oct 2020, 17:55 Oleksandr Shulgin > > wrote: >> >> I'm totally on board with cleaning the list up, but

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread David G. Johnston
On Wed, Oct 28, 2020 at 6:50 AM Bharath Rupireddy < bharath.rupireddyforpostg...@gmail.com> wrote: > Thanks for the comments. > > On Wed, Oct 28, 2020 at 6:41 PM Fujii Masao > wrote: > > > > I prefer that false is returned when the timeout happens, > > like pg_promote() does. > > > > Earlier it

cleanup temporary files after crash

2020-10-28 Thread Euler Taveira
Hi, Bug #16427 mentioned that temporary files are not removed after a crash. I heard similar complaints from some customers. In the bug report [1], I proposed a new GUC to control whether temporary files are removed after a crash recovery. The current behavior is only useful for debugging

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Bharath Rupireddy
Thanks for the comments. On Wed, Oct 28, 2020 at 6:41 PM Fujii Masao wrote: > > I prefer that false is returned when the timeout happens, > like pg_promote() does. > Earlier it was suggested to error out on timeout. Since users can not guess on time it takes to terminate or become idle,

cannot freeze committed xmax

2020-10-28 Thread Konstantin Knizhnik
Hi hackers, The following error was encountered by our customers: Them have  very huge catalog (size of pg_class relation is more than 30Gb)  blowned by temporary relations. When them try to vacuum it, the following error is reported: vacuum full analyze pg_catalog.pg_class; ERROR:  cannot

Re: cutting down the TODO list thread

2020-10-28 Thread John Naylor
On Wed, Oct 28, 2020 at 6:52 AM Magnus Hagander wrote: > On Wed, Oct 28, 2020 at 11:15 AM Julien Rouhaud > wrote: > > > > On Wed, 28 Oct 2020, 17:55 Oleksandr Shulgin < > oleksandr.shul...@zalando.de wrote: > >> I'm totally on board with cleaning the list up, but how about marking > as "won't

Re: cutting down the TODO list thread

2020-10-28 Thread John Naylor
On Tue, Oct 27, 2020 at 6:05 PM Bruce Momjian wrote: > On Tue, Oct 27, 2020 at 04:54:24PM -0400, John Naylor wrote: > > > > > > On Tue, Oct 27, 2020 at 3:52 PM Bruce Momjian wrote: > > > > > > Do any of these limitations need to be documented before removing > them > > from the TODO

Re: A new function to wait for the backend exit after termination

2020-10-28 Thread Fujii Masao
On 2020/10/28 20:50, Bharath Rupireddy wrote: On Wed, Oct 21, 2020 at 6:43 PM Magnus Hagander wrote: I think it would be nicer to have a pg_terminate_backend(pid, wait=false), so a function with a second parameter which defaults to the current behaviour of not waiting. And it might be a

Re: Disable WAL logging to speed up data loading

2020-10-28 Thread Laurenz Albe
On Wed, 2020-10-28 at 09:55 +, osumi.takami...@fujitsu.com wrote: > > > I wrote and attached the first patch to disable WAL logging. > > > This patch passes the regression test of check-world already and is > > > formatted by pgindent. > > > > Without reading the code, I have my doubts about

RE: [Patch] Optimize dropping of relation buffers using dlist

2020-10-28 Thread k.jami...@fujitsu.com
Hi everyone, Attached are the updated set of patches (V28). 0004 - Truncate optimization is a new patch, while the rest are similar to V27. This passes the build, regression and TAP tests. Apologies for the delay. I'll post the benchmark test results on SSD soon, considering the suggested

Re: Allow some recovery parameters to be changed with reload

2020-10-28 Thread Fujii Masao
On 2020/10/28 21:02, Sergei Kornilov wrote: Hello Sorry for late response.  > ... but what's the corresponding hazard here, exactly? It doesn't seem  > that there's any way in which the decision one process makes affects  > the decision the other process makes. There's still a race

Re: Add important info about ANALYZE after create Functional Index

2020-10-28 Thread Fabrízio de Royes Mello
On Wed, Oct 28, 2020 at 2:15 AM Michael Paquier wrote: > > On Tue, Oct 27, 2020 at 11:06:22AM -0300, Fabrízio de Royes Mello wrote: > > When we create a new table or index they will not have statistics until an > > ANALYZE happens. This is the default behaviour and I think is not a big > >

Re: parallel distinct union and aggregate support patch

2020-10-28 Thread Tomas Vondra
Hi, On Wed, Oct 28, 2020 at 05:37:40PM +0800, bu...@sohu.com wrote: Hi Here is patch for parallel distinct union aggregate and grouping sets support using batch hash agg. Please review. how to use: set enable_batch_hashagg = on how to work: like batch sort, but not sort each batch, just save

Re: Internal key management system

2020-10-28 Thread Masahiko Sawada
On Tue, 27 Oct 2020 at 20:34, Bruce Momjian wrote: > > On Tue, Oct 27, 2020 at 03:07:22PM +0800, Craig Ringer wrote: > > On Mon, Oct 26, 2020 at 11:02 PM Stephen Frost wrote: > > > > > > TL;DR: > > > > * Important to check that key rotation is possible on a replica, i.e. > > primary and standby

Re: [PATCH] Add features to pg_stat_statements

2020-10-28 Thread Fujii Masao
On 2020/10/28 17:31, seinoyu wrote: 2020-10-22 01:31 に Fujii Masao さんは書きました: On 2020/10/12 21:18, Yuki Seino wrote: The following review has been posted through the commitfest application: make installcheck-world:  tested, passed Implements feature:   tested, passed Spec compliant:  

  1   2   >