Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-24 Thread Tom Lane
Amul Sul writes: > On Wed, Mar 24, 2021 at 8:09 PM Tom Lane wrote: >> static inline struct SMgrRelationData * >> RelationGetSmgr(Relation rel) >> { >> if (unlikely(rel->rd_smgr == NULL)) >> RelationOpenSmgr(rel); >> return rel->rd_smgr; >> } > A quick question: Can't it be a

Re: Proposal: Save user's original authenticated identity for logging

2021-03-24 Thread Michael Paquier
On Wed, Mar 24, 2021 at 04:45:35PM +, Jacob Champion wrote: > With low-coverage test suites, I think it's useful to allow as little > strange behavior as possible -- in this case, printing authorization > before authentication could signal a serious bug -- but I don't feel > too strongly about

Decoding speculative insert with toast leaks memory

2021-03-24 Thread Ashutosh Bapat
Hi All, We saw OOM in a system where WAL sender consumed Gigabttes of memory which was never released. Upon investigation, we found out that there were many ReorderBufferToastHash memory contexts linked to ReorderBuffer context, together consuming gigs of memory. They were running INSERT ... ON

Re: In-placre persistance change of a relation

2021-03-24 Thread Kyotaro Horiguchi
At Thu, 25 Mar 2021 14:08:05 +0900 (JST), Kyotaro Horiguchi wrote in > (I'm not sure when the subject was broken..) > > At Thu, 14 Jan 2021 17:32:17 +0900 (JST), Kyotaro Horiguchi > wrote in > > Commit bea449c635 conflicts with this on the change of the definition > > of

Re: In-placre persistance change of a relation

2021-03-24 Thread Kyotaro Horiguchi
(I'm not sure when the subject was broken..) At Thu, 14 Jan 2021 17:32:17 +0900 (JST), Kyotaro Horiguchi wrote in > Commit bea449c635 conflicts with this on the change of the definition > of DropRelFileNodeBuffers. The change simplified this patch by a bit:p In this version, I got rid of the

RE: Global snapshots

2021-03-24 Thread tsunakawa.ta...@fujitsu.com
From: Andrey V. Lepikhov > Current state of the patch set rebased on master, 5aed6a1fc2. > > It is development version. Here some problems with visibility still detected > in > two tests: > 1. CSN Snapshot module - TAP test on time skew. > 2. Clock SI implementation - TAP test on emulation of

Re: shared-memory based stats collector

2021-03-24 Thread Kyotaro Horiguchi
Thank you for the the lot of help! At Mon, 22 Mar 2021 16:17:37 -0700, Andres Freund wrote in > Thanks! That change shouldn't be necessary on my branch - I did > something to fix this kind of problem too. I decided that there's no > point in doing hash table lookups for the database: It's not

Re: Tying an object's ownership to datdba

2021-03-24 Thread Noah Misch
On Wed, Mar 24, 2021 at 11:57:37AM -0400, John Naylor wrote: > On Mon, Dec 28, 2020 at 12:32 AM Noah Misch wrote: > > [v2] > > Hi Noah, > > In the refactoring patch, there is a lingering comment reference to > roles_has_privs_of(). Aside from that, it looks good to me. A possible thing > to

Re: wal stats questions

2021-03-24 Thread Andres Freund
Hi, On 2021-03-25 10:51:56 +0900, Masahiro Ikeda wrote: > On 2021/03/25 8:22, Andres Freund wrote: > > 1) What is the motivation to have both prevWalUsage and pgWalUsage, > >instead of just accumulating the stats since the last submission? > >There doesn't seem to be any comment

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Tomas Vondra
On 3/25/21 1:05 AM, Tomas Vondra wrote: > ... > > 0002 is an attempt to fix an issue I noticed today - we need to handle > type changes. Until now we did not have problems with that, because we > only had attnums - so we just reset the statistics (with the exception > of functional dependencies,

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-24 Thread Amul Sul
On Wed, Mar 24, 2021 at 8:09 PM Tom Lane wrote: > > Amul Sul writes: > > On Tue, Mar 23, 2021 at 8:59 PM Tom Lane wrote: > >> On closer inspection, I believe the true culprit is c6b92041d, > >> which did this: > >> - heap_sync(state->rs_new_rel); > >> +

Re: multi-install PostgresNode

2021-03-24 Thread Michael Paquier
On Wed, Mar 24, 2021 at 03:33:51PM -0300, Alvaro Herrera wrote: > On 2021-Mar-24, Andrew Dunstan wrote: >> +# The install path set in get_new_node needs to be a directory containing >> +# bin and lib subdirectories as in a standard PostgreSQL installation, so >> this >> +# can't be used with

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-24 Thread Noah Misch
On Thu, Mar 25, 2021 at 07:44:02AM +0900, Michael Paquier wrote: > On Wed, Mar 24, 2021 at 10:50:50AM -0400, Robert Haas wrote: > > On Wed, Mar 24, 2021 at 5:56 AM Christoph Berg wrote: > >> Maybe creating the tablespace directory from within the testsuite > >> would suffice? > >> > >> CREATE

Re: About to add WAL write/fsync statistics to pg_stat_wal view

2021-03-24 Thread Masahiro Ikeda
On 2021/03/23 16:10, Fujii Masao wrote: > > > On 2021/03/22 20:25, ikedamsh wrote: >> Agreed. Users can know whether the stats is for walreceiver or not. The >> pg_stat_wal view in standby server shows for the walreceiver, and in primary >> server it shows for the others. So, I updated the

Re: [HACKERS] logical decoding of two-phase transactions

2021-03-24 Thread Peter Smith
On Wed, Mar 24, 2021 at 11:31 PM Amit Kapila wrote: > > On Tue, Mar 23, 2021 at 3:31 PM Peter Smith wrote: > > > > On Tue, Mar 23, 2021 at 10:44 AM Peter Smith wrote: > > > > > > PSA patches to fix those. > > > > > > > Hi Amit. > > > > PSA a patch to allow the ALTER SUBSCRIPTION ... REFRESH

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-24 Thread Julien Rouhaud
On Wed, Mar 24, 2021 at 01:02:00PM -0300, Alvaro Herrera wrote: > On 2021-Mar-24, Julien Rouhaud wrote: > > > From e08c9d5fc86ba722844d97000798de868890aba3 Mon Sep 17 00:00:00 2001 > > From: Bruce Momjian > > Date: Mon, 22 Mar 2021 17:43:23 -0400 > > Subject: [PATCH v20 2/3] Expose queryid in

Proposal for col LIKE $1 with generic Plan

2021-03-24 Thread Andy Fan
Thanks to the get_index_clause_from_support, we can use index for WHERE a like 'abc%' case. However this currently only works on custom plan. Think about the case where the planning takes lots of time, custom plan will not give a good result. so I want to see if we should support this for a

Re: making update/delete of inheritance trees scale better

2021-03-24 Thread Amit Langote
On Thu, Mar 25, 2021 at 4:22 AM Tom Lane wrote: > I wrote: > > Yeah, it's on my to-do list for this CF, but I expect it's going to > > take some concentrated study and other things keep intruding :-(. > > I've started to look at this seriously, Thanks a lot. > and I wanted to make a note >

Re: wal stats questions

2021-03-24 Thread Masahiro Ikeda
On 2021/03/25 8:22, Andres Freund wrote: > Hi, > > I got a few questions about the wal stats while working on the shmem > stats patch: Thanks for your reviews. > 1) What is the motivation to have both prevWalUsage and pgWalUsage, >instead of just accumulating the stats since the last

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-24 Thread Fujii Masao
On 2021/03/24 22:06, James Coleman wrote: That looks good to me. Thanks for working on this. Thanks! I pushed the patch. Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: MultiXact\SLRU buffers configuration

2021-03-24 Thread Thomas Munro
On Thu, Mar 25, 2021 at 10:31 AM Thomas Munro wrote: > We already know that increasing the number of CLOG buffers above the > current number hurts as the linear search begins to dominate > (according to the commit message for 5364b357), and it doesn't seem > great to ship a new feature that melts

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-24 Thread Andres Freund
Hi, On 2021-03-24 18:36:14 +0900, Fujii Masao wrote: > Barring any objection, I'm thinking to commit this patch. To which branches? I am *strongly* opposed to backpatching this. > /* > - * Simple signal handler for exiting quickly as if due to a crash. > + * Simple signal handler for

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-24 Thread Masahiro Ikeda
On 2021/03/24 18:36, Fujii Masao wrote: > On 2021/03/23 15:50, Fujii Masao wrote: >> + * The statistics collector is started by the postmaster as soon as the >> + * startup subprocess finishes. >> >> This comment needs to be updated? Because the stats collector can >> be invoked when the startup

Re: make the stats collector shutdown without writing the statsfiles if the immediate shutdown is requested.

2021-03-24 Thread Masahiro Ikeda
On 2021/03/24 18:36, Fujii Masao wrote: > > > On 2021/03/24 3:51, Andres Freund wrote: >> Hi, >> >> On 2021-03-23 15:50:46 +0900, Fujii Masao wrote: >>> This fact makes me wonder that if we collect the statistics about WAL >>> writing >>> from walreceiver as we discussed in other thread, the

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Justin Pryzby
On Thu, Mar 25, 2021 at 01:05:37AM +0100, Tomas Vondra wrote: > here's an updated patch. 0001 should address most of the today's review > items regarding comments etc. This is still an issue: postgres=# CREATE STATISTICS xt ON a FROM t JOIN t ON true; ERROR: schema "i" does not exist --

Re: popcount

2021-03-24 Thread David Fetter
On Tue, Mar 23, 2021 at 10:51:08AM +0100, Peter Eisentraut wrote: > On 21.03.21 02:31, David Fetter wrote: > > > I have now read the entire internet on what a suitable name for this > > > function could be. I think the emerging winner is BIT_COUNT(), which > > > already exists in MySQL, and also

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Jacob Champion
On Wed, 2021-03-24 at 14:10 -0400, Stephen Frost wrote: > * Jacob Champion (pchamp...@vmware.com) wrote: > > I could see this being a problem if two client certificate nicknames > > collide across multiple in-use databases, maybe? > > Right, in such a case either cert might get returned and it's

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-24 Thread Daniel Gustafsson
> On 25 Mar 2021, at 00:26, Alvaro Herrera wrote: > > On 2021-Mar-25, Daniel Gustafsson wrote: > >> Attached is a v2 which addresses the comments raised on the main NSS thread, >> as >> well as introduces named parameters for the server cert function to make the >> test code easier to read. >

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-24 Thread Alvaro Herrera
On 2021-Mar-25, Daniel Gustafsson wrote: > Attached is a v2 which addresses the comments raised on the main NSS thread, > as > well as introduces named parameters for the server cert function to make the > test code easier to read. I don't like this patch. I think your SSL::Server::OpenSSL and

wal stats questions

2021-03-24 Thread Andres Freund
Hi, I got a few questions about the wal stats while working on the shmem stats patch: 1) What is the motivation to have both prevWalUsage and pgWalUsage, instead of just accumulating the stats since the last submission? There doesn't seem to be any comment explaining it? Computing

Re: Refactor SSL test framework to support multiple TLS libraries

2021-03-24 Thread Daniel Gustafsson
Attached is a v2 which addresses the comments raised on the main NSS thread, as well as introduces named parameters for the server cert function to make the test code easier to read. -- Daniel Gustafsson https://vmware.com/

Re: Minimal logical decoding on standbys

2021-03-24 Thread Fabrízio de Royes Mello
On Wed, Mar 24, 2021 at 3:57 AM Drouvot, Bertrand wrote: > > Thanks for pointing out, fixed in v14 attached. > Thanks... now everything is working as expected... changed the status to Ready for Commiter: https://commitfest.postgresql.org/32/2968/ Regards, -- Fabrízio de Royes Mello

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Daniel Gustafsson
> On 24 Mar 2021, at 04:54, Michael Paquier wrote: > > On Tue, Mar 23, 2021 at 12:38:50AM +0100, Daniel Gustafsson wrote: >> Thanks again for reviewing, another version which addresses the remaining >> issues will be posted soon but I wanted to get this out to give further >> reviews >>

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-24 Thread Michael Paquier
On Wed, Mar 24, 2021 at 10:50:50AM -0400, Robert Haas wrote: > On Wed, Mar 24, 2021 at 5:56 AM Christoph Berg wrote: >> Maybe creating the tablespace directory from within the testsuite >> would suffice? >> >> CREATE TABLE foo (t text); >> COPY foo FROM PROGRAM 'mkdir @testtablespace@'; >> CREATE

Re: MultiXact\SLRU buffers configuration

2021-03-24 Thread Thomas Munro
Hi Andrey, On Sat, Mar 13, 2021 at 1:44 AM Andrey Borodin wrote: > [v10] +intmultixact_offsets_slru_buffers = 8; +intmultixact_members_slru_buffers = 16; +intsubtrans_slru_buffers = 32; +intnotify_slru_buffers = 8; +int

Re: pg_amcheck contrib application

2021-03-24 Thread Robert Haas
Mark, Here's a quick and very dirty sketch of what I think perhaps this logic could look like. This is pretty much untested and it might be buggy, but at least you can see whether we're thinking at all in the same direction. -- Robert Haas EDB: http://www.enterprisedb.com

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread Peter Eisentraut
On 24.03.21 18:58, John Naylor wrote: > As a potential follow-up, should we perhaps add named arguments?  That > might make the invocations easier to read, depending on taste. I think it's quite possible some users will prefer that. All we need is to add something like proargnames =>

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Alvaro Herrera
On 2021-Mar-24, Justin Pryzby wrote: > On Wed, Mar 24, 2021 at 05:15:46PM +, Dean Rasheed wrote: > > Hmm, I think "univariate" and "multivariate" are pretty ubiquitous, > > when used to describe statistics. You could use "single-column" and > > "multi-column", but then "column" isn't really

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread Peter Eisentraut
On 24.03.21 18:25, Erik Rijkers wrote: On 2021.03.24. 16:38 Peter Eisentraut wrote: Committed. 'In cases full units' seems strange. fixed, thanks

Re: making update/delete of inheritance trees scale better

2021-03-24 Thread Tom Lane
I wrote: > Yeah, it's on my to-do list for this CF, but I expect it's going to > take some concentrated study and other things keep intruding :-(. I've started to look at this seriously, and I wanted to make a note about something that I think we should try to do, but there seems little hope of

Re: TRUNCATE on foreign table

2021-03-24 Thread Fujii Masao
On 2021/03/13 18:57, Kazutaka Onishi wrote: I have fixed the patch to pass check-world test. :D Thanks for updating the patch! Here are some review comments from me. By default all foreign tables using postgres_fdw are assumed to be updatable. This may be overridden using

Re: pl/pgsql feature request: shorthand for argument and local variable references

2021-03-24 Thread Pavel Stehule
st 24. 3. 2021 v 8:42 odesílatel Erik Rijkers napsal: > > On 2021.03.24. 08:09 Pavel Stehule wrote: > > [...] > > [plpgsql-routine-label-20210324.patch] > > > Hi, > > In that sgml > > "the functions' arguments" > > should be > &

Re: multi-install PostgresNode

2021-03-24 Thread Alvaro Herrera
On 2021-Mar-24, Andrew Dunstan wrote: > OK, like this? Yeah, looks good! > +# Private routine to return a copy of the environment with the PATH and > (DY)LD_LIBRARY_PATH > +# correctly set when there is an install path set for the node. > +# Routines that call Postgres binaries need to call

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Tom Lane
Justin Pryzby writes: > On Wed, Mar 24, 2021 at 01:30:26PM -0400, Robert Haas wrote: >> ... So --no-toast-compression is just fine for people who are >> dumping and restoring, but it's no help at all if you want to switch >> TOAST compression methods while doing a pg_upgrade. However, what does

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Stephen Frost
Greetings, * Jacob Champion (pchamp...@vmware.com) wrote: > On Wed, 2021-03-24 at 13:00 -0400, Stephen Frost wrote: > > * Jacob Champion (pchamp...@vmware.com) wrote: > > > Right, but to clarify -- I was asking if *NSS* supports loading and > > > using separate certificate databases as part of

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread John Naylor
On Wed, Mar 24, 2021 at 1:25 PM Erik Rijkers wrote: > > 'In cases full units' seems strange. > > Not a native speaker but maybe the attached changes are improvements? -In cases full units (1 minute, 1 hour, etc.), it gives the same result as +In case of full units (1 minute, 1 hour,

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread John Naylor
On Wed, Mar 24, 2021 at 11:38 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > Committed. > > I noticed that some of the documentation disappeared between v9 and v10. > So I put that back and updated it appropriately. I also added a few > more test cases to cover some things

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Justin Pryzby
On Wed, Mar 24, 2021 at 01:30:26PM -0400, Robert Haas wrote: > On Wed, Mar 24, 2021 at 1:24 PM Justin Pryzby wrote: > > I think it's not specific to pg_upgrade, but any pg_dump |pg_restore. > > > > The analogy with tablespaces is restoring from a cluster where the > > tablespace > > is named

Re: multi-install PostgresNode

2021-03-24 Thread Andrew Dunstan
On 3/24/21 11:41 AM, Alvaro Herrera wrote: > On 2021-Mar-24, Andrew Dunstan wrote: > >> On 3/24/21 9:23 AM, Andrew Dunstan wrote: >>> On 3/24/21 8:29 AM, Alvaro Herrera wrote: >>> If we're going to do that we probably shouldn't special case any >>> particular settings, but simply take any extra

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Jacob Champion
On Wed, 2021-03-24 at 13:00 -0400, Stephen Frost wrote: > * Jacob Champion (pchamp...@vmware.com) wrote: > > Right, but to clarify -- I was asking if *NSS* supports loading and > > using separate certificate databases as part of its API. It seems like > > the internals make it possible, but I

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 1:24 PM Justin Pryzby wrote: > I think it's not specific to pg_upgrade, but any pg_dump |pg_restore. > > The analogy with tablespaces is restoring from a cluster where the tablespace > is named "vast" to one where it's named "huge". I do this by running >

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 12:14 PM Dilip Kumar wrote: > Actually, we are already doing this, I mean ALTER TABLE .. ALTER > COLUMN .. SET COMPRESSION is already updating the compression method > of the index attribute. So 0003 doesn't make sense, sorry for the > noise. However, 0001 and 0002 are

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread Erik Rijkers
> On 2021.03.24. 16:38 Peter Eisentraut > wrote: > > Committed. > 'In cases full units' seems strange. Not a native speaker but maybe the attached changes are improvements? Erik Rijkers--- ./doc/src/sgml/func.sgml.orig 2021-03-24 18:16:01.269515354 +0100 +++ ./doc/src/sgml/func.sgml

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Justin Pryzby
On Wed, Mar 24, 2021 at 12:24:38PM -0400, Robert Haas wrote: > On Wed, Mar 24, 2021 at 11:42 AM Tom Lane wrote: > > On reflection, though, I wonder if we've made pg_dump do the right > > thing anyway. There is a strong case to be made for the idea that > > when dumping from a pre-14 server, it

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 12:45 PM Tom Lane wrote: > I wouldn't be proposing this if the xversion failures were the only > reason; making them go away is just a nice side-effect. The core > point is that the charter of pg_dump is to reproduce the source > database's state, and as things stand

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Justin Pryzby
On Wed, Mar 24, 2021 at 05:15:46PM +, Dean Rasheed wrote: > On Wed, 24 Mar 2021 at 16:48, Tomas Vondra > wrote: > > > > As for the changes proposed in the create_statistics, do we really want > > to use univariate / multivariate there? Yes, the terms are correct, but > > I'm not sure how many

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Dean Rasheed
On Wed, 24 Mar 2021 at 16:48, Tomas Vondra wrote: > > As for the changes proposed in the create_statistics, do we really want > to use univariate / multivariate there? Yes, the terms are correct, but > I'm not sure how many people looking at CREATE STATISTICS will > understand them. > Hmm, I

Re: Change default of checkpoint_completion_target

2021-03-24 Thread Stephen Frost
Greetings, * Bossart, Nathan (bossa...@amazon.com) wrote: > On 3/23/21, 12:19 PM, "Stephen Frost" wrote: > > * Bossart, Nathan (bossa...@amazon.com) wrote: > > > LGTM. I just have a few small wording suggestions. > > > > Agreed, those looked like good suggestions and so I've incorporated > >

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Stephen Frost
Greetings Jacob, * Jacob Champion (pchamp...@vmware.com) wrote: > On Wed, 2021-03-24 at 09:28 +0900, Michael Paquier wrote: > > On Wed, Mar 24, 2021 at 12:05:35AM +, Jacob Champion wrote: > > > I can work around it temporarily for the > > > tests, but this will be a problem if any libpq

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Tomas Vondra
On 3/24/21 5:28 PM, Dean Rasheed wrote: > On Wed, 24 Mar 2021 at 14:48, Tomas Vondra > wrote: >> >> AFAIK the primary issue here is that the two places disagree. While >> estimate_num_groups does this >> >> varnos = pull_varnos(root, (Node *) varshere); >> if (bms_membership(varnos) ==

Re: Proposal: Save user's original authenticated identity for logging

2021-03-24 Thread Jacob Champion
On Tue, 2021-03-23 at 14:21 +0900, Michael Paquier wrote: > I am not really sure that we need to bother about the ordering of the > entries here, as long as we check for all of them within the same > fragment of the log file, so I would just go down to the simplest > solution that I posted

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Tom Lane
Robert Haas writes: > On Wed, Mar 24, 2021 at 11:42 AM Tom Lane wrote: >> On reflection, though, I wonder if we've made pg_dump do the right >> thing anyway. There is a strong case to be made for the idea that >> when dumping from a pre-14 server, it should emit >> SET default_toast_compression

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Mon, Mar 22, 2021 at 4:57 PM Robert Haas wrote: > > Fixed. > > Fixed some more. Committed. -- Robert Haas EDB: http://www.enterprisedb.com

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Justin Pryzby
On Wed, Mar 24, 2021 at 01:24:46AM -0500, Justin Pryzby wrote: > It seems like you're preferring to use pluralized "statistics" in a lot of > places that sound wrong to me. For example: > > Currently the first statistics wins, which seems silly. > I can write more separately, but I think this is

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Tomas Vondra
On 3/24/21 7:24 AM, Justin Pryzby wrote: > Most importantly, it looks like this forgets to update catalog documentation > for stxexprs and stxkind='e' > Good catch. > It seems like you're preferring to use pluralized "statistics" in a lot of > places that sound wrong to me. For example: >>

Re: postgres_fdw: IMPORT FOREIGN SCHEMA ... LIMIT TO (partition)

2021-03-24 Thread Bernd Helmle
Am Mittwoch, dem 24.03.2021 um 13:23 +0100 schrieb Matthias van de Meent: > Yes, but it should be noted that the main reason that was mentioned > as > a reason to exclude partitions is to not cause table catalog bloat, > and I argue that this argument is not as solid in the case of the >

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Dean Rasheed
On Wed, 24 Mar 2021 at 14:48, Tomas Vondra wrote: > > AFAIK the primary issue here is that the two places disagree. While > estimate_num_groups does this > > varnos = pull_varnos(root, (Node *) varshere); > if (bms_membership(varnos) == BMS_SINGLETON) > { ... } > > the

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 11:42 AM Tom Lane wrote: > On reflection, though, I wonder if we've made pg_dump do the right > thing anyway. There is a strong case to be made for the idea that > when dumping from a pre-14 server, it should emit > SET default_toast_compression = 'pglz'; > rather

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 9:32 PM Robert Haas wrote: > > On Wed, Mar 24, 2021 at 11:41 AM Dilip Kumar wrote: > > Okay, that sounds like a reasonable design idea. But the problem is > > that in index_form_tuple we only have index tuple descriptor, not the > > heap tuple descriptor. Maybe we will

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-24 Thread Jan Wieck
On 3/24/21 12:04 PM, Jan Wieck wrote: In any case I changed the options so that they behave the same way, the existing -o and -O (for old/new postmaster options) work. I don't think it would be wise to have option forwarding work differently between options for postmaster and options for

Re: default result formats setting

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 12:01 PM Tom Lane wrote: > I don't think I buy the premise that there are exactly two levels > on the client side. Thanks for sharing your thoughts on this. I agree it's a complex issue, and the idea that there are possibly more than two logical levels is, for me, maybe

Re: pg_upgrade failing for 200+ million Large Objects

2021-03-24 Thread Jan Wieck
On 3/23/21 4:55 PM, Tom Lane wrote: Jan Wieck writes: Have we even reached a consensus yet on that doing it the way, my patch is proposing, is the right way to go? Like that emitting BLOB TOC entries into SECTION_DATA when in binary upgrade mode is a good thing? Or that bunching all the SQL

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 11:41 AM Dilip Kumar wrote: > Okay, that sounds like a reasonable design idea. But the problem is > that in index_form_tuple we only have index tuple descriptor, not the > heap tuple descriptor. Maybe we will have to pass the heap tuple > descriptor as a parameter to

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-24 Thread Alvaro Herrera
On 2021-Mar-24, Julien Rouhaud wrote: > From e08c9d5fc86ba722844d97000798de868890aba3 Mon Sep 17 00:00:00 2001 > From: Bruce Momjian > Date: Mon, 22 Mar 2021 17:43:23 -0400 > Subject: [PATCH v20 2/3] Expose queryid in pg_stat_activity and > src/backend/executor/execMain.c | 9

Re: default result formats setting

2021-03-24 Thread Tom Lane
Robert Haas writes: > On Wed, Mar 24, 2021 at 10:58 AM Tom Lane wrote: >> A client that is sending -1 and assuming that it will get back >> a particular format could get broken if the GUC doesn't have the >> value it thinks, true. But I'd argue that such code is unreasonably >> non-robust.

Re: Tying an object's ownership to datdba

2021-03-24 Thread John Naylor
On Mon, Dec 28, 2020 at 12:32 AM Noah Misch wrote: > [v2] Hi Noah, In the refactoring patch, there is a lingering comment reference to roles_has_privs_of(). Aside from that, it looks good to me. A possible thing to consider is an assert that is_admin is not null where we expect that. The

Re: Support for NSS as a libpq TLS backend

2021-03-24 Thread Jacob Champion
On Wed, 2021-03-24 at 09:28 +0900, Michael Paquier wrote: > On Wed, Mar 24, 2021 at 12:05:35AM +, Jacob Champion wrote: > > I can work around it temporarily for the > > tests, but this will be a problem if any libpq clients load up multiple > > independent databases for use with separate

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Tom Lane
Andrew Dunstan writes: > On 3/20/21 3:03 PM, Tom Lane wrote: >> I fixed up some issues in 0008/0009 (mostly cosmetic, except that >> you forgot a server version check in dumpToastCompression) and >> pushed that, so we can see if it makes crake happy. > It's still produced a significant amount

Re: multi-install PostgresNode

2021-03-24 Thread Alvaro Herrera
On 2021-Mar-24, Andrew Dunstan wrote: > > On 3/24/21 9:23 AM, Andrew Dunstan wrote: > > On 3/24/21 8:29 AM, Alvaro Herrera wrote: > > If we're going to do that we probably shouldn't special case any > > particular settings, but simply take any extra arguments as extra env > > settings. And if

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Dilip Kumar
On Wed, Mar 24, 2021 at 8:41 PM Robert Haas wrote: > > On Wed, Mar 24, 2021 at 7:45 AM Dilip Kumar wrote: > > I have anyway created a patch for this as well. Including all three > > patches so we don't lose track. > > > > 0001 ->shows compression method for the index attribute in index describe

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread Peter Eisentraut
On 18.01.21 21:54, John Naylor wrote: On Mon, Nov 23, 2020 at 1:44 PM John Naylor mailto:john.nay...@enterprisedb.com>> wrote: > > On Thu, Nov 12, 2020 at 9:56 AM Peter Eisentraut > wrote: > > - After reading the discussion a few times, I'm not so

Re: default result formats setting

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 10:58 AM Tom Lane wrote: > Robert Haas writes: > > But ... if it's just a GUC, it can be set by code on the server side > > that the client knows nothing about, breaking the client. That seems > > pretty bad to me. > > It's impossible for the proposed patch to break

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2021-03-24 Thread Julien Rouhaud
On Wed, Mar 24, 2021 at 08:13:40AM -0400, Bruce Momjian wrote: > On Wed, Mar 24, 2021 at 04:51:40PM +0800, Julien Rouhaud wrote: > > > but if you do that it'd be better to avoid > > > introducing a function with one name and renaming it in your next > > > commit. > > > > Oops, I apparently messed

Re: Get memory contexts of an arbitrary backend process

2021-03-24 Thread torikoshia
On 2021-03-23 17:24, Kyotaro Horiguchi wrote: Thanks for reviewing and suggestions! At Mon, 22 Mar 2021 15:09:58 +0900, torikoshia wrote in >> If MemoryContextStatsPrint(), i.e. showing 100 children at most is >> enough, this hard limit may be acceptable. > Can't this number be passed via

Re: [HACKERS] Custom compression methods

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 7:45 AM Dilip Kumar wrote: > I have anyway created a patch for this as well. Including all three > patches so we don't lose track. > > 0001 ->shows compression method for the index attribute in index describe > 0002 -> fix the reported bug (test case included) >

Re: multi-install PostgresNode

2021-03-24 Thread Andrew Dunstan
On 3/24/21 9:23 AM, Andrew Dunstan wrote: > On 3/24/21 8:29 AM, Alvaro Herrera wrote: >> On 2021-Mar-24, Dagfinn Ilmari Mannsåker wrote: >> >>> I think it would be even neater having a method that returns the desired >>> environment and then have the other methods do: >>> >>> local %ENV =

Re: default result formats setting

2021-03-24 Thread Tom Lane
Robert Haas writes: > But ... if it's just a GUC, it can be set by code on the server side > that the client knows nothing about, breaking the client. That seems > pretty bad to me. It's impossible for the proposed patch to break *existing* clients, because they all send requested format 0 or 1,

Re: pgsql: Move tablespace path re-creation from the makefiles to pg_regres

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 5:56 AM Christoph Berg wrote: > Maybe creating the tablespace directory from within the testsuite > would suffice? > > CREATE TABLE foo (t text); > COPY foo FROM PROGRAM 'mkdir @testtablespace@'; > CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@'; Would that

Re: default result formats setting

2021-03-24 Thread Robert Haas
On Thu, Nov 5, 2020 at 3:49 PM Peter Eisentraut wrote: > We could also make it a protocol message, but it would essentially > implement the same thing, just again separately. And then you'd have no > support to inspect the current setting, test out different settings > interactively, etc. That

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Tomas Vondra
On 3/24/21 2:36 PM, Dean Rasheed wrote: > On Wed, 24 Mar 2021 at 10:22, Tomas Vondra > wrote: >> >> Thanks, it seems to be some thinko in handling in PlaceHolderVars, which >> seem to break the code's assumptions about varnos. This fixes it for me, >> but I need to look at it more closely. >> >

Re: pg_amcheck contrib application

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 9:12 AM Robert Haas wrote: > On Wed, Mar 24, 2021 at 2:13 AM Mark Dilger > wrote: > > The visibility rules fix is different in v11, relying on a visibility check > > which more closely follows the implementation of > > HeapTupleSatisfiesVacuumHorizon. > > Hmm. The

Re: [CLOBBER_CACHE]Server crashed with segfault 11 while executing clusterdb

2021-03-24 Thread Tom Lane
Amul Sul writes: > On Tue, Mar 23, 2021 at 8:59 PM Tom Lane wrote: >> On closer inspection, I believe the true culprit is c6b92041d, >> which did this: >> - heap_sync(state->rs_new_rel); >> + smgrimmedsync(state->rs_new_rel->rd_smgr, MAIN_FORKNUM); >> heap_sync was

psql lacking clearerr()

2021-03-24 Thread Alvaro Herrera
psql seems to never call clearerr() on its output file. So if it gets an error while printing a result, it'll show could not print result table: Success after each and every result, even though the output file isn't in error state anymore. It seems that the simplest fix is just to do

Re: shared memory stats: high level design decisions: consistency, dropping

2021-03-24 Thread Magnus Hagander
On Sun, Mar 21, 2021 at 11:34 PM Andres Freund wrote: > > Hi, > > On 2021-03-21 12:14:35 -0400, Tom Lane wrote: > > Andres Freund writes: > > > 1) What kind of consistency do we want from the pg_stats_* views? > > > > That's a hard choice to make. But let me set the record straight: > > when we

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Dean Rasheed
On Wed, 24 Mar 2021 at 10:22, Tomas Vondra wrote: > > Thanks, it seems to be some thinko in handling in PlaceHolderVars, which > seem to break the code's assumptions about varnos. This fixes it for me, > but I need to look at it more closely. > I think that makes sense. Reviewing the docs, I

Re: shared memory stats: high level design decisions: consistency, dropping

2021-03-24 Thread Magnus Hagander
On Tue, Mar 23, 2021 at 4:21 AM Greg Stark wrote: > > On Sun, 21 Mar 2021 at 18:16, Stephen Frost wrote: > > > > Greetings, > > > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > I also believe that the snapshotting behavior has advantages in terms > > > of being able to perform multiple

Re: multi-install PostgresNode

2021-03-24 Thread Andrew Dunstan
On 3/24/21 8:29 AM, Alvaro Herrera wrote: > On 2021-Mar-24, Dagfinn Ilmari Mannsåker wrote: > >> I think it would be even neater having a method that returns the desired >> environment and then have the other methods do: >> >> local %ENV = $self->_get_install_env(); > Hmm, is it possible

Re: pg_amcheck contrib application

2021-03-24 Thread Robert Haas
On Wed, Mar 24, 2021 at 2:13 AM Mark Dilger wrote: > The visibility rules fix is different in v11, relying on a visibility check > which more closely follows the implementation of > HeapTupleSatisfiesVacuumHorizon. Hmm. The header comment you wrote says "If a tuple might not be visible to any

Re: Nicer error when connecting to standby with hot_standby=off

2021-03-24 Thread James Coleman
On Wed, Mar 24, 2021 at 5:55 AM Fujii Masao wrote: > > > > On 2021/03/24 16:59, Alvaro Herrera wrote: > > On 2021-Mar-24, Fujii Masao wrote: > > > >> On 2021/03/24 5:59, Tom Lane wrote: > >>> Alvaro Herrera writes: > FATAL: the database system is starting up > DETAIL: WAL is being

Re: default result formats setting

2021-03-24 Thread Emre Hasegeli
I think this is a good feature that would be useful to JDBC and more. I don't know the surrounding code very well, but the patch looks good to me. I agree with Tom Lane that the name of the variable is too verbose. Maybe "auto_binary_types" is enough. Do we gain much by prefixing

  1   2   >