Re: Columns correlation and adaptive query optimization

2019-12-24 Thread Konstantin Knizhnik
New version of patch implicitly adding multicolumn statistic in auto_explain extension and using it in optimizer for more precise estimation of join selectivity. This patch fixes race condition while adding statistics and restricts generated statistic name to fit in 64 bytes (NameData). --

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-12-24 Thread Amit Kapila
On Tue, Dec 24, 2019 at 11:17 AM Masahiko Sawada wrote: > > On Fri, 20 Dec 2019 at 22:30, Amit Kapila wrote: > > > > > > The main aim of this feature is to reduce apply lag. Because if we > > send all the changes together it can delay there apply because of > > network delay, whereas if most of

Re: Implementing Incremental View Maintenance

2019-12-24 Thread Tatsuo Ishii
> Unfortunately, it's not clear to me which of ON STATEMENT or ON COMMIT the > user should choose. The benefit of ON STATEMENT is that the user does not > have to create and maintain the materialized view log. But I'm not sure if > and when the benefit defeats the performance overhead on DML

Re: Should we rename amapi.h and amapi.c?

2019-12-24 Thread Julien Rouhaud
On Tue, Dec 24, 2019 at 3:57 AM Michael Paquier wrote: > > On Mon, Dec 23, 2019 at 12:28:36PM -0800, Ashwin Agrawal wrote: > > I had raised the same earlier and [1] has response from Andres, which was > > "We probably should rename it, but not in 12..." > > > > [1] > >

Re: Minimal logical decoding on standbys

2019-12-24 Thread Amit Khandekar
On Thu, 19 Dec 2019 at 01:02, Rahila Syed wrote: > > Hi, > >> Hi, do you consistently get this failure on your machine ? I am not >> able to get this failure, but I am going to analyze when/how this can >> fail. Thanks >> > Yes, I am getting it each time I run make -C src/test/recovery/ check >

pgbench - use pg logging capabilities

2019-12-24 Thread Fabien COELHO
Bonjour Michaël, hello devs, As suggested in "cce64a51", this patch make pgbench use postgres logging capabilities. I tried to use fatal/error/warning/info/debug where appropriate. Some printing to stderr remain for some pgbench specific output. The patch fixes a inconsistent test case

Re: backup manifests

2019-12-24 Thread Suraj Kharage
Thank you for review comments. On Fri, Dec 20, 2019 at 9:14 PM Robert Haas wrote: > On Fri, Dec 20, 2019 at 8:24 AM Suraj Kharage > wrote: > > Thank you for review comments. > > Thanks for the new version. > > + --verify-backup > > Whitespace. > Corrected. > > +struct manifesthash_hash

Re: Implementing Incremental View Maintenance

2019-12-24 Thread legrand legrand
Yugo Nagata wrote > On Mon, 23 Dec 2019 03:41:18 -0700 (MST) > legrand legrand > legrand_legrand@ > wrote: > > [ ...] > >> I would even >> prefer a common "table" shared between all sessions like GLOBAL TEMPORARY >> TABLE (or something similar) as described here: >>

Server crash with Master-Slave configuration.

2019-12-24 Thread Prabhat Sahu
Hi, While performing below operations with Master-Slave configuration, Slave is crashed. Below are the steps to reproduce: -- create a Slave using pg_basebackup and start: ./pg_basebackup -v -R -D d2 -p 55510 mkdir /home/centos/ts1 -- Session 1(Master): ./psql postgres -p 55510 CREATE

Re: [HACKERS] Restricting maximum keep segments by repslots

2019-12-24 Thread Kyotaro Horiguchi
I'm very sorry for being late to reply. At Wed, 2 Oct 2019 17:08:07 +0200, Jehan-Guillaume de Rorthais wrote in > On Tue, 30 Jul 2019 21:30:45 +0900 (Tokyo Standard Time) > Kyotaro Horiguchi wrote: > > > In "pg_replication_slots" view, the new "wal_status" field is misleading. > > > Consider

Re: Should we rename amapi.h and amapi.c?

2019-12-24 Thread Michael Paquier
On Tue, Dec 24, 2019 at 09:32:23AM +0100, Julien Rouhaud wrote: > Looks good to me. There are still references to amapi.c in various > .po files, but those should rather be taken care of with the next > update-po cycle right? Yes, these are updated as part of the translation updates. -- Michael

Re: logical decoding : exceeded maxAllocatedDescs for .spill files

2019-12-24 Thread Amit Khandekar
On Tue, 24 Dec 2019 at 10:41, Amit Kapila wrote: > > On Fri, Dec 20, 2019 at 9:31 AM Amit Khandekar wrote: > > Attached are the patches from master back up to 94 branch. > > > > PG 9.4 and 9.5 have a common patch to be applied : > > pg94_95_use_vfd_for_logrep.patch > > From PG 9.6 onwards, each

Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2019-12-24 Thread Mahendra Singh
Hi hackers, In other thread "[HACKERS] Block level parallel vacuum"[1], Prabhat Kumar Sahu reported a random assert failure but he got only once and he was not able to reproduce it. In that thread [2], Amit Kapila suggested some points to reproduce assert. I tried to reproduce and I was able to

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-24 Thread Anastasia Lubennikova
19.12.2019 18:19, Antonin Houska wrote: Anastasia Lubennikova wrote: I attached new version with pg_opclass documentation update. One more thing I am uncertain about  is array_ops. Arrays may contain bitwise and not bitwise element types. What is the correct value of opcisbitwise the

Re: string literal continuations in C

2019-12-24 Thread Joe Conway
On 12/23/19 2:51 PM, Alvaro Herrera wrote: > Per a recent thread, these patches remove string literals split with > \-escaped newlines. The first is for the message "materialize mode > required, but it is not allowed in this context" where it's more > prevalent, and we keep perpetuating it; the

Re: unsupportable composite type partition keys

2019-12-24 Thread Amit Langote
On Mon, Dec 23, 2019 at 6:42 PM Amit Langote wrote: > On Sun, Dec 22, 2019 at 6:13 AM Tom Lane wrote: > > > I wonder whether we couldn't also lift > > > the restriction against whole-row Vars in partition expressions. > > > Doesn't seem like there is much difference between such a Var and > > >

Re: Should we rename amapi.h and amapi.c?

2019-12-24 Thread Fabien COELHO
Bonjour Michaël, the syscache mainly, so instead the attached patch does the following changes: - amapi.h -> indexam.h - amapi.c -> indexamapi.c. Here we have an equivalent in access/table/ as tableamapi.c. - amvalidate.c -> indexamvalidate.c - amvalidate.h -> indexamvalidate.h - genam.c ->

Physical replication slot advance is not persistent

2019-12-24 Thread Alexey Kondratov
Hi Hackers, I have accidentally noticed that pg_replication_slot_advance only changes in-memory state of the slot when its type is physical. Its new value does not survive restart. Reproduction steps: 1) Create new slot and remember its restart_lsn SELECT

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-24 Thread Alvaro Herrera
> @@ -106,6 +106,18 @@ CREATE OPERATOR CLASS class="parameter">name [ DEFAUL > > > > + > +NOT BITWISE > + > + > + If present, the operator class equality is not the same as equivalence. > + For example, two numerics can compare equal but have different

Re: Building infrastructure for B-Tree deduplication that recognizes when opclass equality is also equivalence

2019-12-24 Thread Tom Lane
Alvaro Herrera writes: > Am I the only one bothered by the fact that this patch (and all > downstream discussion) reduces the term "bitwise equality" to simply > "bitwise"? It reads really strange to me, both in the resulting SQL > grammar as well as in struct names, code comments etc. "This

weird libpq GSSAPI comment

2019-12-24 Thread Alvaro Herrera
I found this comment in fe-connect.c: /* * If GSSAPI is enabled and we have a credential cache, try to * set it up before sending startup messages. If it's already * operating, don't try SSL and instead just build the startup

Re: mdclose() does not cope w/ FileClose() failure

2019-12-24 Thread Noah Misch
On Mon, Dec 23, 2019 at 07:41:49PM +0900, Kyotaro Horiguchi wrote: > At Sun, 22 Dec 2019 12:21:00 -0800, Noah Misch wrote in > > On Sun, Dec 22, 2019 at 01:19:30AM -0800, Noah Misch wrote: > > > An alternative would be to call > > > _fdvec_resize() after every FileClose(), like mdtruncate()

Re: unsupportable composite type partition keys

2019-12-24 Thread Tom Lane
Amit Langote writes: > On Tue, Dec 24, 2019 at 12:00 AM Tom Lane wrote: >> BTW, I forgot to mention: while I think the patch to forbid pseudotypes >> by using CheckAttributeType() can be back-patched, I'm leaning towards >> not back-patching the other patch. The situation where we get into >>

Re: Allow an alias to be attached directly to a JOIN ... USING

2019-12-24 Thread Fabien COELHO
On Tue, 17 Sep 2019, Alvaro Herrera wrote: Indeed, that seems like a problem, and it's a good question. You can see this on unpatched master with SELECT x.filler FROM (pgbench_tellers AS t JOIN b USING (bid)) AS x. I'm not sure I understand why that problem is a blocker for this patch.

Re: Online checksums verification in the backend

2019-12-24 Thread Masahiko Sawada
On Tue, 24 Dec 2019 at 16:09, Julien Rouhaud wrote: > > On Tue, Dec 24, 2019 at 4:23 AM Masahiko Sawada wrote: > > > > On Fri, Dec 6, 2019 at 11:51 PM Julien Rouhaud wrote: > > > > > > This brings the second consideration: how to report the list corrupted > > > blocks to end users. As I said

Re: pgsql: Superuser can permit passwordless connections on postgres_fdw

2019-12-24 Thread Michael Paquier
On Fri, Dec 20, 2019 at 10:17:20PM -0500, Tom Lane wrote: > Yeah, it's sort of annoying that the buildfarm didn't notice this > aspect of things. I'm not sure I want to spend cycles on checking > it in every test run, though. > > Maybe we could have -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS >

Re: Physical replication slot advance is not persistent

2019-12-24 Thread Kyotaro Horiguchi
At Tue, 24 Dec 2019 20:12:32 +0300, Alexey Kondratov wrote in > I dig into the code and it happens because of this if statement: > >     /* Update the on disk state when lsn was updated. */ >     if (XLogRecPtrIsInvalid(endlsn)) >     { >         ReplicationSlotMarkDirty(); >        

Re: Server crash with Master-Slave configuration.

2019-12-24 Thread Michael Paquier
On Tue, Dec 24, 2019 at 05:29:25PM +0530, Prabhat Sahu wrote: > While performing below operations with Master-Slave configuration, Slave is > crashed. > Below are the steps to reproduce: > > -- create a Slave using pg_basebackup and start: > ./pg_basebackup -v -R -D d2 -p 55510 > mkdir

table partition and column default

2019-12-24 Thread Fujii Masao
Hi, As the document explains, column defaults can be specified separately for each partition. But I found that INSERT via the partitioned table ignores that default. Is this expected behavior or bug? CREATE TABLE test (i INT, j INT) PARTITION BY RANGE (i); CREATE TABLE test1 PARTITION OF test (j

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2019-12-24 Thread Kyotaro Horiguchi
At Wed, 25 Dec 2019 11:22:03 +0900, Michael Paquier wrote in > Anyway, as dropping a temporary schema leads to an inconsistent > behavior when recreating new temporary objects in a session that > dropped it, that nobody has actually complained on the matter, and > that in concept a temporary

Re: Implementing Incremental View Maintenance

2019-12-24 Thread Tatsuo Ishii
> Materialized view reminds me of the use in a data warehouse. Oracle handles > the top in its Database Data Warehousing Guide, and Microsoft has just > started to offer the materialized view feature in its Azure Synapse Analytics > (formerly SQL Data Warehouse). AWS also has previewed

Re: unsupportable composite type partition keys

2019-12-24 Thread Amit Langote
On Wed, Dec 25, 2019 at 2:42 AM Tom Lane wrote: > Amit Langote writes: > > On Tue, Dec 24, 2019 at 12:00 AM Tom Lane wrote: > >> BTW, I forgot to mention: while I think the patch to forbid pseudotypes > >> by using CheckAttributeType() can be back-patched, I'm leaning towards > >> not

Re: Should we rename amapi.h and amapi.c?

2019-12-24 Thread Michael Paquier
On Tue, Dec 24, 2019 at 02:22:22PM +0100, Fabien COELHO wrote: > The change does not attempt to keep included files in ab order. Should it do > that, or is it fixed later by some reindentation phase? Yeah, it should. Committed after fixing all that stuff. -- Michael signature.asc Description:

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2019-12-24 Thread Michael Paquier
On Tue, Dec 24, 2019 at 04:50:58PM +0530, Mahendra Singh wrote: > We can fix this problem by either one way 1) reset myTempNamespace to > invalid while drooping schema of temp table 2) should not allow to drop > temporary table schema (Please note that it is better not to cross-post on multiple

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2019-12-24 Thread Michael Paquier
On Wed, Dec 25, 2019 at 12:18:26PM +0900, Kyotaro Horiguchi wrote: > Still the owner can drop temporary namespace on another session or > pg_toast_temp_x of the current session. Arf. Yes, this had better be isAnyTempNamespace() so as we complain about all of them. -- Michael signature.asc

Re: mdclose() does not cope w/ FileClose() failure

2019-12-24 Thread Kyotaro Horiguchi
At Tue, 24 Dec 2019 11:57:39 -0800, Noah Misch wrote in > On Mon, Dec 23, 2019 at 07:41:49PM +0900, Kyotaro Horiguchi wrote: > > If I understand it correctly, it is mentioning the number of the all > > segment files in a fork, not the length of md_seg_fds arrays at a > > certain moment. But

Re: Assert failure due to "drop schema pg_temp_3 cascade" for temporary tables and \d+ is not showing any info after drooping temp table schema

2019-12-24 Thread Mahendra Singh
On Wed, 25 Dec 2019 at 07:52, Michael Paquier wrote: > > On Tue, Dec 24, 2019 at 04:50:58PM +0530, Mahendra Singh wrote: > > We can fix this problem by either one way 1) reset myTempNamespace to > > invalid while drooping schema of temp table 2) should not allow to drop > > temporary table schema

Re: relpages of btree indexes are not truncating even after deleting all the tuples from table and doing vacuum

2019-12-24 Thread Mahendra Singh
On Tue, 24 Dec 2019 at 02:41, Peter Geoghegan wrote: > > On Mon, Dec 23, 2019 at 11:05 AM Mahendra Singh wrote: > > From above example, we can see that after deleting all the tuples from > > table and firing vacuum command, size of table is reduced but size of index > > relation is same as

Re: Avoid full GIN index scan when possible

2019-12-24 Thread Alexander Korotkov
On Sat, Nov 23, 2019 at 2:39 AM Nikita Glukhov wrote: > Attached 8th version of the patches. I've read this thread. I decided to rewrite the patch in the way, which I find simple and more clear. Attached is the draft patch written from scratch except regression tests, which were copied "as

Re: table partition and column default

2019-12-24 Thread Amit Langote
Fujii-san, On Wed, Dec 25, 2019 at 12:19 PM Fujii Masao wrote: > > Hi, > > As the document explains, column defaults can be specified separately for > each partition. But I found that INSERT via the partitioned table ignores > that default. Is this expected behavior or bug? > > CREATE TABLE test

Re: unsupportable composite type partition keys

2019-12-24 Thread Tom Lane
Amit Langote writes: > On Tue, Dec 24, 2019 at 10:59 AM Amit Langote wrote: >> Btw, does the memory leakage fix in this patch address any of the >> pending concerns that were discussed on the "hyrax vs. >> RelationBuildPartitionDesc" thread earlier this year[1]? >> [1] >>

RE: Implementing Incremental View Maintenance

2019-12-24 Thread tsunakawa.ta...@fujitsu.com
From: Tatsuo Ishii > AFAIK benefit of ON STATEMENT is the transaction can see the result of > update to the base tables. With ON COMMIT, the transaction does not > see the result until the transaction commits. > > Well, I can see use cases of IVM in both DWH and OLTP. > > For example, a user

Re: Server crash with Master-Slave configuration.

2019-12-24 Thread Prabhat Sahu
On Wed, Dec 25, 2019 at 8:01 AM Michael Paquier wrote: > On Tue, Dec 24, 2019 at 05:29:25PM +0530, Prabhat Sahu wrote: > > While performing below operations with Master-Slave configuration, Slave > is > > crashed. > > Below are the steps to reproduce: > > > > -- create a Slave using