Re: [HACKERS] Block level parallel vacuum

2019-12-30 Thread Amit Kapila
On Mon, Dec 30, 2019 at 6:37 PM Tomas Vondra wrote: > > On Mon, Dec 30, 2019 at 10:40:39AM +0530, Amit Kapila wrote: > >On Mon, Dec 30, 2019 at 2:53 AM Tomas Vondra > > wrote: > >> > > > >+1. It is already a separate patch and I think we can even discuss > >more on it in a new thread once the

Re: Reorderbuffer crash during recovery

2019-12-30 Thread vignesh C
On Mon, Dec 30, 2019 at 11:17 AM Amit Kapila wrote: > > On Fri, Dec 27, 2019 at 8:37 PM Alvaro Herrera > wrote: > > > > On 2019-Dec-27, vignesh C wrote: > > > > > I felt amit solution also solves the problem. Attached patch has the > > > fix based on the solution proposed. > > > Thoughts? > > >

Re: Cache relation sizes?

2019-12-30 Thread Thomas Munro
On Tue, Dec 31, 2019 at 4:43 PM Kyotaro HORIGUCHI wrote: > I still believe that one shared memory element for every > non-mapped relation is not only too-complex but also too-much, as > Andres (and implicitly I) wrote. I feel that just one flag for > all works fine but partitioned flags (that is,

Re: Clarifying/rationalizing Vars' varno/varattno/varnoold/varoattno

2019-12-30 Thread Tom Lane
I wrote: > Anyway, I had started to work on getting parse analysis to label > outer-join-nullable Vars properly, and soon decided that no matter how > we do it, there's not enough information available at the point where > parse analysis makes a Var. The referenced RTE is not, in itself, > enough

Re: [HACKERS] Block level parallel vacuum

2019-12-30 Thread Amit Kapila
On Mon, Dec 30, 2019 at 6:46 PM Tomas Vondra wrote: > > On Mon, Dec 30, 2019 at 08:25:28AM +0530, Amit Kapila wrote: > >On Mon, Dec 30, 2019 at 2:53 AM Tomas Vondra > > wrote: > >> I think there's another question we need to ask - why to we introduce a > >> bitmask, instead of using regular

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Alvaro Herrera
On 2019-Dec-31, Andrew Dunstan wrote: > Turns out my arithmetic was a bit off. Animal Farm is 90 kb, Madame > Bovary 678 Kb. Ten animal farms should be enough for everybody. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: archive status ".ready" files may be created too early

2019-12-30 Thread Bossart, Nathan
On 12/23/19, 6:09 PM, "Kyotaro Horiguchi" wrote: > You're right. That doesn't seem to work. Another thing I had in my > mind was that XLogWrite had an additional flag so that > AdvanceXLInsertBuffer can tell not to mark .ready. The function is > called while it *is* writing a complete record. So

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Andrew Dunstan
On Tue, Dec 31, 2019 at 11:16 AM Tomas Vondra wrote: > > On Tue, Dec 31, 2019 at 10:25:26AM +1030, Andrew Dunstan wrote: > >On Tue, Dec 31, 2019 at 10:16 AM Andrew Dunstan > > wrote: > >> > >> On Tue, Dec 31, 2019 at 9:38 AM Tom Lane wrote: > >> > >> > > >> > This doesn't seem like a reason not

[PATCH] Fix parallel query doc typos

2019-12-30 Thread Jon Jensen
Hi, hackers. Attached are 2 tiny doc typo fixes. Jon From ed4e539d5eade0a3e20d42e35239141f14a3b24b Mon Sep 17 00:00:00 2001 From: Jon Jensen Date: Mon, 30 Dec 2019 17:45:30 -0700 Subject: [PATCH] Fix doc typos --- doc/src/sgml/parallel.sgml | 4 ++-- 1 file changed, 2 insertions(+), 2

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Tomas Vondra
On Tue, Dec 31, 2019 at 10:25:26AM +1030, Andrew Dunstan wrote: On Tue, Dec 31, 2019 at 10:16 AM Andrew Dunstan wrote: On Tue, Dec 31, 2019 at 9:38 AM Tom Lane wrote: > > This doesn't seem like a reason not to allow a higher limit, like a > megabyte or so, but I'm not sure that pushing it

remove separate postgres.(sh)description files

2019-12-30 Thread John Naylor
I'm guessing the initial data for pg_(sh)description is output into separate files because it was too difficult for the traditional shell script to maintain enough state to do otherwise. With Perl, it's just as easy to assemble the data into the same format as the rest of the catalogs and then let

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-30 Thread Peter Geoghegan
On Tue, Dec 24, 2019 at 4:29 AM Anastasia Lubennikova wrote: > I updated the patchset. > The first patch now contains only infrastructure changes > and the second one sets opcisbitwise for btree opclasses in pg_opclass.dat. We should try to formally define what we're trying to represent about

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Andrew Dunstan
On Tue, Dec 31, 2019 at 10:16 AM Andrew Dunstan wrote: > > On Tue, Dec 31, 2019 at 9:38 AM Tom Lane wrote: > > > > > This doesn't seem like a reason not to allow a higher limit, like a > > megabyte or so, but I'm not sure that pushing it to the moon would be > > wise. > > > > > Just to get a

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Andrew Dunstan
On Tue, Dec 31, 2019 at 9:38 AM Tom Lane wrote: > > This doesn't seem like a reason not to allow a higher limit, like a > megabyte or so, but I'm not sure that pushing it to the moon would be > wise. > Just to get a mental handle on the size of queries we might be allowing before truncation, I

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Tom Lane
Tomas Vondra writes: > 2) What's the overhead for increasing the value for short/long queries? > My assumption is that for short queries, it's going to be negligible. > For longer queries it may be measurable, but I'd expect longer queries > to be more expensive in general, so maybe it's still

Re: Allow an alias to be attached directly to a JOIN ... USING

2019-12-30 Thread Vik Fearing
On 30/12/2019 22:25, Peter Eisentraut wrote: > On 2019-12-24 19:13, Fabien COELHO wrote: Indeed, that seems like a problem, and it's a good question.  You can see this on unpatched master with SELECT x.filler FROM (pgbench_tellers AS t JOIN b USING (bid)) AS x. >>> >>> I'm not sure

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-30 Thread Peter Geoghegan
On Mon, Dec 30, 2019 at 9:45 AM Robert Haas wrote: > > For example, float and numeric types are "never bitwise equal", while array, > > text, and other container types are "maybe bitwise equal". An array of > > integers > > or text with C collation can be treated as bitwise equal attributes, and

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Tomas Vondra
On Mon, Dec 30, 2019 at 12:46:40PM -0800, Robert Treat wrote: On Mon, Dec 23, 2019 at 9:11 PM Robert Haas wrote: On Sat, Dec 21, 2019 at 1:25 PM Tom Lane wrote: > > What is the overhead here except the memory consumption? > > The time to copy those strings out of shared storage, any time >

Re: Allow an alias to be attached directly to a JOIN ... USING

2019-12-30 Thread Peter Eisentraut
On 2019-12-24 19:13, Fabien COELHO wrote: Indeed, that seems like a problem, and it's a good question. You can see this on unpatched master with SELECT x.filler FROM (pgbench_tellers AS t JOIN b USING (bid)) AS x. I'm not sure I understand why that problem is a blocker for this patch. As

Re: [PATCH] Increase the maximum value track_activity_query_size

2019-12-30 Thread Robert Treat
On Mon, Dec 23, 2019 at 9:11 PM Robert Haas wrote: > > On Sat, Dec 21, 2019 at 1:25 PM Tom Lane wrote: > > > What is the overhead here except the memory consumption? > > > > The time to copy those strings out of shared storage, any time > > you query pg_stat_activity. > > It seems like you're

Re: proposal: schema variables

2019-12-30 Thread Pavel Stehule
r reviewers have already done that job. > If yes, my feeling is that the patch could soon be set as "Ready for > commiter". > > Best regards. Philippe. > > The new status of this patch is: Waiting on Author > Thank you very much for your comments, and notes. Updated patch attached. Regards Pavel schema-variables-20191230.patch.gz Description: application/gzip

Re: Recognizing superuser in pg_hba.conf

2019-12-30 Thread David Fetter
On Mon, Dec 30, 2019 at 11:56:17AM +0100, Vik Fearing wrote: > On 29/12/2019 23:10, Vik Fearing wrote: > > On 29/12/2019 17:31, Tom Lane wrote: > >> Robert Haas writes: > >>> On Sat, Dec 28, 2019 at 2:02 PM Vik Fearing > >>> wrote: > I'm all for this (and even suggested it during the IRC

Re: comment regarding double timestamps; and, infinite timestamps and NaN

2019-12-30 Thread Tom Lane
Justin Pryzby writes: > On Mon, Dec 30, 2019 at 09:05:24AM -0500, Tom Lane wrote: >> Uh, what? This seems completely wrong to me. We could possibly >> promote DT_NOBEGIN and DT_NOEND to +/- infinity (not NaN), but >> I don't really see the point. They'll compare to other timestamp >> values

Re: [HACKERS] pg_shmem_allocations view

2019-12-30 Thread Robert Haas
On Wed, Dec 18, 2019 at 9:53 PM Kyotaro Horiguchi wrote: > The doc is saying that "size" is "Size of the allocation" and > "allocated_size" is "size including padding". It seems somewhat > confusing to me. I'm not sure what wording is best but I think people > use net/gross wordings to describe

Re: backup manifests

2019-12-30 Thread Robert Haas
On Tue, Dec 24, 2019 at 5:42 AM Suraj Kharage wrote: > Made the change in backup manifest as well in backup validatort patch. Thanks > to Rushabh Lathia for the offline discussion and help. > > To examine the first word of each line, I am using below check: > if (strncmp(line, "File", 4) == 0) >

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-30 Thread Robert Haas
On Mon, Dec 30, 2019 at 10:57 AM Anastasia Lubennikova wrote: > In this design "maybe" category reflects the need for an extra recheck. > > For example, float and numeric types are "never bitwise equal", while array, > text, and other container types are "maybe bitwise equal". An array of >

Re: 12's AND CHAIN doesn't chain when transaction raised an error

2019-12-30 Thread Alvaro Herrera
On 2019-Aug-13, Philip Dubé wrote: > The easiest way to see this is to BEGIN READ ONLY & then attempt an > insert. Execute either of COMMIT AND CHAIN or ROLLBACK AND CHAIN & > attempt the insert a second time > > This seems incorrect. The documentation should at least point out this > behavior

Re: proposal: schema variables

2019-12-30 Thread Philippe BEAUDOIN
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, failed Hi Pavel, I have tested the latest version of your patch. Both

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-30 Thread Anastasia Lubennikova
29.12.2019 2:56, Robert Haas wrote: On Tue, Dec 24, 2019 at 7:29 AM Anastasia Lubennikova wrote: We can make this 'opcisbitwise' parameter enum (or char) instead of boolean to mark "always bitwise", "never bitwise" and "maybe bitwise". Though, I doubt if it will be helpful in any real use

Re: comment regarding double timestamps; and, infinite timestamps and NaN

2019-12-30 Thread Justin Pryzby
On Mon, Dec 30, 2019 at 09:05:24AM -0500, Tom Lane wrote: > Justin Pryzby writes: > > That seems to be only used for ineq_histogram_selectivity() interpolation of > > histogram bins. It looks to me that at least isn't working for "special > > values", and needs to use something other than

Re: TAP testing for psql's tab completion code

2019-12-30 Thread Fabien COELHO
Hello Tom, I do not think it is a good idea, because help output is quite large, there are many of them, and we should certainly not want it stored repeatedly in output files for diffs. Hm, I don't follow --- we are most certainly not going to exercise \help for every possible SQL keyword,

Re: Disallow cancellation of waiting for synchronous replication

2019-12-30 Thread Bruce Momjian
On Sat, Dec 28, 2019 at 04:55:55PM -0500, Robert Haas wrote: > On Fri, Dec 20, 2019 at 12:04 AM Andrey Borodin wrote: > > Currently, we can have split brain with the combination of following steps: > > 0. Setup cluster with synchronous replication. Isolate primary from > > standbys. > > 1. Issue

Re: TAP testing for psql's tab completion code

2019-12-30 Thread Tom Lane
Fabien COELHO writes: >> Well, I think that where possible we ought to test using the existing >> test infrastructure -- help, for example, seems like it could perfectly >> well be tested in src/test/regress/sql/psql.sql, or we could move stuff >> out to a new set of SQL test scripts under

Re: comment regarding double timestamps; and, infinite timestamps and NaN

2019-12-30 Thread Tom Lane
Justin Pryzby writes: > selfuncs.c convert_to_scalar() says: > |* The several datatypes representing absolute times are all converted > |* to Timestamp, which is actually a double, and then we just use that > |* double value. > So I propose it should say something like: > |* The several

Re: Windows v readline

2019-12-30 Thread Tom Lane
Peter Eisentraut writes: > On 2019-09-29 22:55, Andrew Dunstan wrote: >> It would certainly be nice to have readline-enabled psql on Windows if >> possible. > I tried this out. First, it doesn't build, because readline doesn't do > the dllimport/dllexport dance on global variables, so all

Re: [HACKERS] Block level parallel vacuum

2019-12-30 Thread Tomas Vondra
On Mon, Dec 30, 2019 at 08:25:28AM +0530, Amit Kapila wrote: On Mon, Dec 30, 2019 at 2:53 AM Tomas Vondra wrote: On Sun, Dec 29, 2019 at 10:06:23PM +0900, Masahiko Sawada wrote: >> v40-0001-Introduce-IndexAM-fields-for-parallel-vacuum.patch >>

Re: [HACKERS] Block level parallel vacuum

2019-12-30 Thread Tomas Vondra
On Mon, Dec 30, 2019 at 10:40:39AM +0530, Amit Kapila wrote: On Mon, Dec 30, 2019 at 2:53 AM Tomas Vondra wrote: On Sun, Dec 29, 2019 at 10:06:23PM +0900, Masahiko Sawada wrote: >> >> v40-0003-Add-FAST-option-to-vacuum-command.patch >> >> >> I

Re: [PATCH] fix a performance issue with multiple logical-decoding walsenders

2019-12-30 Thread Pierre Ducroquet
On Sunday, December 29, 2019 1:32:31 PM CET Julien Rouhaud wrote: > On Sat, Dec 28, 2019 at 1:56 PM Pierre Ducroquet wrote: > > Thank you for your comments. > > Attached to this email is a patch with better comments regarding the > > XLogSendLogical change. > > Arguably the first test to

Re: Recognizing superuser in pg_hba.conf

2019-12-30 Thread Vik Fearing
On 29/12/2019 23:10, Vik Fearing wrote: > On 29/12/2019 17:31, Tom Lane wrote: >> Robert Haas writes: >>> On Sat, Dec 28, 2019 at 2:02 PM Vik Fearing >>> wrote: I'm all for this (and even suggested it during the IRC conversation that prompted this patch). It's rife with bikeshedding,

Re: TAP testing for psql's tab completion code

2019-12-30 Thread Fabien COELHO
That is what my patch does: it tests prompts, tab completion, help, command options… and I added tests till I covered most psql source. Well, I think that where possible we ought to test using the existing test infrastructure -- help, for example, seems like it could perfectly well be

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-12-30 Thread Amit Kapila
On Thu, Dec 26, 2019 at 12:36 PM Masahiko Sawada wrote: > > On Tue, 24 Dec 2019 at 17:21, Amit Kapila wrote: > > > > > > > > Thank you for explanation. The plan makes sense. But I think in the > > > current design it's a problem that logical replication worker doesn't > > > receive changes (and

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-12-30 Thread Amit Kapila
On Sun, Dec 29, 2019 at 1:34 PM Dilip Kumar wrote: > > I have observed some more issues > > 1. Currently, In ReorderBufferCommit, it is always expected that > whenever we get REORDER_BUFFER_CHANGE_INTERNAL_SPEC_CONFIRM, we must > have already got REORDER_BUFFER_CHANGE_INTERNAL_SPEC_INSERT and in

Re: Windows v readline

2019-12-30 Thread Peter Eisentraut
On 2019-09-29 22:55, Andrew Dunstan wrote: The configure code currently has this: # readline on MinGW has problems with backslashes in psql and other bugs. # This is particularly a problem with non-US code pages. # Therefore disable its use until we understand the cause. 2004-07-20 if test