Re: insensitive collations

2019-01-14 Thread Daniel Verite
Andreas Karlsson wrote: > > Nondeterministic collations do address this by allowing canonically > > equivalent code point sequences to compare as equal. You still need a > > collation implementation that actually does compare them as equal; ICU > > does this, glibc does not AFAICT. > >

Re: MERGE SQL statement for PG12

2019-01-14 Thread Simon Riggs
On Mon, 14 Jan 2019 at 15:45, Tomas Vondra wrote: > > > On 1/11/19 12:26 AM, Peter Geoghegan wrote: > > On Thu, Jan 10, 2019 at 1:15 PM Robert Haas > wrote: > >> I feel like there has been some other thread where this was discussed, > >> but I can't find it right now. I think that the "query

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread Tom Lane
James Coleman writes: > On Sun, Jan 13, 2019 at 3:06 PM Tom Lane wrote: >> There's still a logical problem here, which is that in order to >> prove that the ScalarArrayOpExpr yields NULL when its LHS does, >> you also have to prove that the RHS is not an empty array. > I've constructed a test

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
On Mon, Jan 14, 2019 at 11:34 AM Tom Lane wrote: > > James Coleman writes: > > On Mon, Jan 14, 2019 at 11:08 AM Tom Lane wrote: > >> I think these test cases don't actually prove much about the behavior > >> of your patch. Wouldn't they be handled by the generic OR-conversion > >> logic, since

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-01-14 Thread Tomas Vondra
On 1/14/19 12:20 PM, Dean Rasheed wrote: > (Removing Adrien from the CC list, because messages to that address > keep bouncing) > > On Sun, 13 Jan 2019 at 00:31, Tomas Vondra > wrote: >> >> On 1/10/19 6:09 PM, Dean Rasheed wrote: >>> >>> In the previous discussion around

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread Tom Lane
James Coleman writes: > Are those invariants we want to keep (and recognize as regression > tests)? If so, I can confirm that they aren't duplicative of the rest > of the file, and if not I can remove them. I can't get terribly excited about them ... if we were changing their behavior, or if

Re: insensitive collations

2019-01-14 Thread Andreas Karlsson
On 1/10/19 8:44 AM, Peter Eisentraut wrote: On 09/01/2019 19:49, Andreas Karlsson wrote: Maybe this is orthogonal and best handled elsewhere but have you when working with string equality given unicode normalization forms[1] any thought? Nondeterministic collations do address this by allowing

Re: Reducing header interdependencies around heapam.h et al.

2019-01-14 Thread Alvaro Herrera
On 2019-Jan-13, Andres Freund wrote: > On 2019-01-13 23:54:58 -0300, Alvaro Herrera wrote: > > On 2019-Jan-13, Andres Freund wrote: > > > > > Alvaro, you'd introduced the split of HeapScanDesc and HeapScanDescData > > > being in different files (in a3540b0f657c6352) - what do you think about > >

Re: MERGE SQL statement for PG12

2019-01-14 Thread Tomas Vondra
On 1/11/19 12:26 AM, Peter Geoghegan wrote: > On Thu, Jan 10, 2019 at 1:15 PM Robert Haas wrote: >> I feel like there has been some other thread where this was discussed, >> but I can't find it right now. I think that the "query construction" >> logic in transformMergeStmt is fundamentally

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread Tom Lane
James Coleman writes: > On Mon, Jan 14, 2019 at 11:08 AM Tom Lane wrote: >> I think these test cases don't actually prove much about the behavior >> of your patch. Wouldn't they be handled by the generic OR-conversion >> logic, since there's fewer than MAX_SAOP_ARRAY_SIZE items? > Which ones

Re: Reducing header interdependencies around heapam.h et al.

2019-01-14 Thread Alvaro Herrera
On 2019-Jan-13, Andres Freund wrote: > One split I am wondering about however is splitting out the sysstable_ > stuff out of genam.h. It's imo a different component and shouldn't > really be in there. Would be quite a bit of rote work to add all the > necessary includes over the backend... Yeah

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
On Mon, Jan 14, 2019 at 11:08 AM Tom Lane wrote: > > James Coleman writes: > > On Sun, Jan 13, 2019 at 3:06 PM Tom Lane wrote: > >> There's still a logical problem here, which is that in order to > >> prove that the ScalarArrayOpExpr yields NULL when its LHS does, > >> you also have to prove

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-14 Thread Tom Lane
David Rowley writes: > I ran a few benchmarks on an AWS m5d.large instance based on top of > c5c7fa261f5. The biggest regression I see is from a simple SELECT 1 at > around 5-6%. A repeat of your test of SELECT 2+2 showed about half > that regression so the simple addition function call is

Re: Reducing header interdependencies around heapam.h et al.

2019-01-14 Thread Andres Freund
Hi, On 2019-01-14 15:36:14 -0300, Alvaro Herrera wrote: > 0001 -- looks reasonable. One hunk in executor.h changes LockTupleMode > to "enum LockTupleMode", but there's no need for that. Oh, that escaped from an earlier version where I briefly forgot that one cannot portably forward-declare

Re: Prepare Transaction support for ON COMMIT DROP temporary tables

2019-01-14 Thread Dimitri Fontaine
Hi, Tom Lane writes: > I'm afraid that this patch is likely to be, if not completely broken, > at least very much less useful than one could wish by the time we get > done closing the holes discussed in this other thread: > >

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-01-14 Thread Tomas Vondra
referenced will always be true. > Hmmm. I'm pretty sure I came to the conclusion it's in fact necessary, > but I might be wrong. Will check. > Turns out you were right - the attribute_referenced piece was quite unnecessary. So I've removed it. I've also extended the regression tests to verify changing type of another column does not reset the stats. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001-multivariate-MCV-lists-20190114.patch.gz Description: application/gzip 0002-multivariate-histograms-20190114.patch.gz Description: application/gzip

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-01-14 Thread Tomas Vondra
happier if some of those asserts could be reinstated, even if only in a weaker form. regards -- Tomas Vondra http://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 0001-Add-logical_work_mem-to-limit-ReorderBuffer-20190114.patc

Re: Reducing header interdependencies around heapam.h et al.

2019-01-14 Thread Alvaro Herrera
0001 -- looks reasonable. One hunk in executor.h changes LockTupleMode to "enum LockTupleMode", but there's no need for that. AFAIK the only reason to have the struct FooBarData vs. FooBar (ptr) split is so that it's possible to refer to structs without having the full struct definition. I

Re: hostorder and failover_timeout for libpq

2019-01-14 Thread Tom Lane
Michael Paquier writes: > On Wed, Sep 19, 2018 at 02:26:53PM +0200, Ildar Musin wrote: >> At this point I'd like to ask community what in your opinion would be the >> best course of action and whether this feature should be implemented within >> libpq at all? Because from my POV there are factors

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Andres Freund
On 2018-11-07 14:25:54 -0500, Tom Lane wrote: > Robert Haas writes: > > On Fri, May 11, 2018 at 11:01 AM, Simon Riggs wrote: > I have no problem if you want to replace this with an even better > design in a later release. > > >>> Meh. The author / committer should get a patch into the

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
On Mon, Jan 14, 2019 at 11:34 AM Tom Lane wrote: > >> Hm. That would be annoying, but on reflection I think maybe we don't > >> actually need to worry about emptiness of the array after all. The > >> thing that we need to prove, at least for the implication case, is > >> "truth of the

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > On 2019-01-14 18:03:24 -0500, Tom Lane wrote: > >> Do we want to revert entirely, or leave the "recheck_on_update" option > >> present but nonfunctional? > > > I think it depends a bit on whether we want to revert in

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-01-14 18:46:18 -0500, Stephen Frost wrote: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > Andres Freund writes: > > > > On 2019-01-14 18:03:24 -0500, Tom Lane wrote: > > > >> Do we want to revert entirely, or leave the

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Tom Lane
Andres Freund writes: > On 2019-01-14 18:53:02 -0500, Tom Lane wrote: >> But I suspect just doing the revert is already going to be painful >> enough :-( > I assume you're not particularly interested in doing that? No, I'm willing to do it, and will do so tomorrow if there haven't been

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Stephen Frost
Greetings, * Andres Freund (and...@anarazel.de) wrote: > On 2019-01-14 18:55:18 -0500, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > > > > Or are you suggesting that pg_dump in v12+ would throw errors if it > > > > finds that set? Or that we'll dump it, but fail to allow

Non-deterministic IndexTuple toast compression from index_form_tuple() + amcheck false positives

2019-01-14 Thread Peter Geoghegan
While testing my nbtree heap TID keyspace patch, I came across a case where amcheck reliably reports corruption. It appeared that a 4 byte varlena index entry that was expected in an index was not actually present. However, index scan queries with the "missing" value in their qual didn't actually

Re: explain plans with information about (modified) gucs

2019-01-14 Thread Alvaro Herrera
On 2019-Jan-14, Tomas Vondra wrote: > The one slightly annoying issue is that currently all the options are > formatted as text, including e.g. cpu_tuple_cost. That's because the GUC > options may have show hook, so I can't access the value directly (not > sure if there's an option around it). I

Re: Non-deterministic IndexTuple toast compression from index_form_tuple() + amcheck false positives

2019-01-14 Thread Peter Geoghegan
On Mon, Jan 14, 2019 at 1:46 PM Peter Geoghegan wrote: > I would have said that the assumption is that a fixed source tuple > will generate identical index entries. The problem with that is that > my idea of what constitutes a fixed input now seems to have been > faulty. I didn't think that the

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Andres Freund
Hi, On 2019-01-14 18:03:24 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2018-11-07 14:25:54 -0500, Tom Lane wrote: > >> In short, it seems likely to me that large parts of this patch need to > >> be pulled out, rewritten, and then put back in different places than > >> they are today.

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Tom Lane
Andres Freund writes: > On 2019-01-14 18:03:24 -0500, Tom Lane wrote: >> Do we want to revert entirely, or leave the "recheck_on_update" option >> present but nonfunctional? > I think it depends a bit on whether we want to revert in master or > master and 11. If only master, I don't see much

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Andres Freund
On 2019-01-14 18:46:18 -0500, Stephen Frost wrote: > Greetings, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Andres Freund writes: > > > On 2019-01-14 18:03:24 -0500, Tom Lane wrote: > > >> Do we want to revert entirely, or leave the "recheck_on_update" option > > >> present but nonfunctional?

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Andres Freund writes: > > On 2019-01-14 18:53:02 -0500, Tom Lane wrote: > >> But I suspect just doing the revert is already going to be painful > >> enough :-( > > > I assume you're not particularly interested in doing that? > > No, I'm

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Andres Freund
Hi, On 2019-01-14 19:04:07 -0500, Stephen Frost wrote: > As that's the case, then I guess I'm thinking we really should make > pg_upgrade complain if it finds it during the check phase. I really > don't like having a case like this where the pg_upgrade will fail from > something that we could

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-14 Thread Tom Lane
David Rowley writes: > 1. I don't think having a table named "dual" makes a whole lot of > sense for a table with a single row. Well, I borrowed Oracle terminology there ;-) > (Uppercasing these additions would also make them look less of an > afterthought.) Don't really care, can do. > I

Re: explain plans with information about (modified) gucs

2019-01-14 Thread Tomas Vondra
Hello Sergei, > This patch correlates with my proposal > "add session information column to pg_stat_statements" > https://www.postgresql.org/message-id/3aa097d7-7c47-187b-5913-db8366cd4491%40gmail.com > They both address the problem to identify the factors that make > different execution plans

Re: Non-deterministic IndexTuple toast compression from index_form_tuple() + amcheck false positives

2019-01-14 Thread Peter Geoghegan
On Mon, Jan 14, 2019 at 1:31 PM Tom Lane wrote: > Peter Geoghegan writes: > > The heapallindexed enhancement that made it into Postgres 11 assumes > > that the representation of index tuples produced by index_form_tuple() > > (or all relevant index_form_tuple() callers) is deterministic: for > >

Re: [HACKERS] logical decoding of two-phase transactions

2019-01-14 Thread Arseny Sher
Nikhil Sontakke writes: > I'd like to believe that the latest patch set tries to address some > (if not all) of your concerns. Can you please take a look and let me > know? Hi, sure. General things: - Earlier I said that there is no point of sending COMMIT PREPARED if decoding snapshot

Re: Libpq support to connect to standby server as priority

2019-01-14 Thread Tom Lane
This patch is marked as "ready for committer", but that characterization seems way over-optimistic. It looks like there are several unsettled questions: 1. The connection parameter name and values are unlike the very similar feature in pgJDBC. I think this is a fair complaint. Now I'm not in

Re: explain plans with information about (modified) gucs

2019-01-14 Thread legrand legrand
Tomas Vondra-4 wrote > Hello Sergei, > >> This patch correlates with my proposal >> "add session information column to pg_stat_statements" >> https://www.postgresql.org/message-id/3aa097d7-7c47-187b-5913-db8366cd4491%40gmail.com >> They both address the problem to identify the factors that make

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Andres Freund
Hi, On 2019-01-14 18:55:18 -0500, Stephen Frost wrote: > * Andres Freund (and...@anarazel.de) wrote: > > > Or are you suggesting that pg_dump in v12+ would throw errors if it > > > finds that set? Or that we'll dump it, but fail to allow it into a > > > v12+ database? What if v12 sees

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-14 Thread David Rowley
On Tue, 15 Jan 2019 at 09:48, Tom Lane wrote: > > David Rowley writes: > > SELECT 1; I believe is a common query for some connection poolers as a > > sort of ping to the database. In light of that, the performance drop > > of 2 microseconds per query is not going to amount to very much in > >

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> What I'm not willing to do is write hacks for pg_upgrade or pg_dump >> to mask cases where the option has been set on a v11 index. I judge >> that it's not worth the trouble. If someone else disagrees, they >> can do that work.

Re: Reducing header interdependencies around heapam.h et al.

2019-01-14 Thread Alvaro Herrera
On 2019-Jan-14, Andres Freund wrote: > I think the whole pointer hiding game that we play is a really really > bad idea. We should just about *never* have typedefs that hide the fact > that something is a pointer. But it's hard to go away from that for > legacy reasons. > > The problem with your

Re: unique, partitioned index fails to distinguish index key from INCLUDEd columns

2019-01-14 Thread Alvaro Herrera
On 2019-Jan-09, Justin Pryzby wrote: > -- Fails to error > postgres=# CREATE UNIQUE INDEX ON t(j) INCLUDE(i); > > -- Fail to enforce uniqueness across partitions due to failure to enforce > inclusion of partition key in index KEY > postgres=# INSERT INTO t VALUES(1,1); > postgres=# INSERT INTO

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Tom Lane
Andres Freund writes: > On 2018-11-07 14:25:54 -0500, Tom Lane wrote: >> In short, it seems likely to me that large parts of this patch need to >> be pulled out, rewritten, and then put back in different places than >> they are today. I'm not sure if a complete revert is the best next >> step,

Re: COPY FROM WHEN condition

2019-01-14 Thread Tomas Vondra
On 12/13/18 8:09 AM, Surafel Temesgen wrote: > > > On Wed, Dec 12, 2018 at 9:28 PM Tomas Vondra > mailto:tomas.von...@2ndquadrant.com>> wrote: > > > Can you also update the docs to mention that the functions called from > the WHERE clause does not see effects of the COPY itself? > >

Re: Non-deterministic IndexTuple toast compression from index_form_tuple() + amcheck false positives

2019-01-14 Thread Tom Lane
Peter Geoghegan writes: > The heapallindexed enhancement that made it into Postgres 11 assumes > that the representation of index tuples produced by index_form_tuple() > (or all relevant index_form_tuple() callers) is deterministic: for > every possible heap tuple input there must be a single

Re: pgsql: Remove WITH OIDS support, change oid catalog column visibility.

2019-01-14 Thread Andres Freund
Hi, On 2019-01-07 17:03:20 +0530, Amit Khandekar wrote: > On Sat, 5 Jan 2019 at 02:09, Andres Freund wrote: > > On 2019-01-03 13:40:42 -0500, Tom Lane wrote: > > > I noticed that this patch has broken restores of existing dump files: > > > > > > psql:testbed.public.backup:82: ERROR:

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Andres Freund
Hi, On 2019-01-14 18:53:02 -0500, Tom Lane wrote: > But I suspect just doing the revert is already going to be painful > enough :-( I assume you're not particularly interested in doing that? I'm more than happy to leave this to others, but if nobody signals interest I'll give it a go, because

Re: [HACKERS] Surjective functional indexes

2019-01-14 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> After a few minutes' more thought, I think that the most attractive > >> option is to leave v11 alone and do a full revert in HEAD. In this > >> way, if anyone's attached

RE: O_DIRECT for relations and SLRUs (Prototype)

2019-01-14 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > One of the reasons why I have begun this thread is that since we have heard > about the fsync issues on Linux, I think that there is room for giving our > user base more control of their fate without relying on the Linux community > decisions to

Re: explain plans with information about (modified) gucs

2019-01-14 Thread Tomas Vondra
On 1/14/19 11:13 PM, Alvaro Herrera wrote: > On 2019-Jan-14, Tomas Vondra wrote: > >> The one slightly annoying issue is that currently all the options are >> formatted as text, including e.g. cpu_tuple_cost. That's because the GUC >> options may have show hook, so I can't access the value

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Amit Langote
Fujita-san, On 2019/01/11 20:04, Etsuro Fujita wrote: > (2019/01/11 13:46), Amit Langote wrote: >> On 2019/01/10 15:07, Etsuro Fujita wrote: >>> (The name of the flag isn't >>> good?  If so, that would be my fault because I named that flag.) >> >> If it's really just to store the fact that the

Re: strange valgrind failures (again)

2019-01-14 Thread Andres Freund
Hi, On 2019-01-15 03:41:34 +0100, Tomas Vondra wrote: > On 1/15/19 3:11 AM, Andres Freund wrote: > > On 2019-01-15 03:07:10 +0100, Tomas Vondra wrote: > >> I've started observing funny valgrind failures on Fedora 28, possibly > >> after upgrading from 3.14.0-1 to 3.14.0-7 a couple of days ago.

Re: [Sender Address Forgery]Re: error message when subscription target is a partitioned table

2019-01-14 Thread Amit Langote
On 2019/01/13 16:56, Michael Paquier wrote: > On Fri, Jan 11, 2019 at 04:04:41PM +0900, Michael Paquier wrote: >> Thanks. I can commit this version if there are no objections then. > > And done. Thank you! Regards, Amit

Re: "SELECT ... FROM DUAL" is not quite as silly as it appears

2019-01-14 Thread David Rowley
On Tue, 15 Jan 2019 at 13:17, Tom Lane wrote: > > David Rowley writes: > > 1. I don't think having a table named "dual" makes a whole lot of > > sense for a table with a single row. > > Well, I borrowed Oracle terminology there ;-) yep, but ... go on... break the mould :) -- David Rowley

Re: Reducing header interdependencies around heapam.h et al.

2019-01-14 Thread Andres Freund
Hi, On 2019-01-14 17:55:44 -0300, Alvaro Herrera wrote: > On 2019-Jan-14, Andres Freund wrote: > > > I think the whole pointer hiding game that we play is a really really > > bad idea. We should just about *never* have typedefs that hide the fact > > that something is a pointer. But it's hard to

RE: Libpq support to connect to standby server as priority

2019-01-14 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > The problem here of course is that whoever invented target_session_attrs > was unconcerned with following that precedent, so what we have is > "target_session_attrs=(any | read-write)". > Are we prepared to add some aliases in service of unifying these

Re: New vacuum option to do only freezing

2019-01-14 Thread Masahiko Sawada
On Sat, Jan 12, 2019 at 3:27 AM Robert Haas wrote: > > On Fri, Jan 11, 2019 at 1:14 AM Masahiko Sawada wrote: > > Attached the updated patch. Please review it. > > I'm quite confused by this patch. It seems to me that the easiest way > to implement this patch would be to (1) make

Re: strange valgrind failures (again)

2019-01-14 Thread Tomas Vondra
On 1/15/19 3:11 AM, Andres Freund wrote: > Hi, > > On 2019-01-15 03:07:10 +0100, Tomas Vondra wrote: >> I've started observing funny valgrind failures on Fedora 28, possibly >> after upgrading from 3.14.0-1 to 3.14.0-7 a couple of days ago. This >> time it does not seem like platform-specific

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Amit Langote
Fujita-san, On 2019/01/11 21:50, Etsuro Fujita wrote: >>> (2019/01/10 10:41), Amit Langote wrote: That's a loaded meaning and abusing it to mean something else can be challenged, but we can live with that if properly documented.  Speaking of which:   /* used by

Re: Tid scan improvements

2019-01-14 Thread Tom Lane
Edmund Horner writes: > On Sat, 22 Dec 2018 at 07:10, Tom Lane wrote: >> I'm not entirely sure why you're bothering; surely nulltestsel is >> unrelated to what this patch is about? > I found that it made a difference with selectivity of range comparisons, > because clauselist_selectivity tries

Re: unique, partitioned index fails to distinguish index key from INCLUDEd columns

2019-01-14 Thread Alvaro Herrera
On 2019-Jan-14, Justin Pryzby wrote: > On Mon, Jan 14, 2019 at 07:31:07PM -0300, Alvaro Herrera wrote: > > Doh. Fix pushed. Commit 8224de4f42cc should have changed one > > appearance of ii_NumIndexAttrs to ii_NumIndexKeyAttrs, but because of > > the nature of concurrent development, nobody

Re: Speeding up text_position_next with multibyte encodings

2019-01-14 Thread John Naylor
On Sun, Dec 23, 2018 at 9:33 AM Tomas Vondra wrote: > So, what is the expected speedup in a "good/average" case? Do we have > some reasonable real-world workload mixing these cases that could be > used as a realistic benchmark? Not sure about a realistic mix, but I investigated the tradeoffs.

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Amit Langote
On 2019/01/15 10:46, Amit Langote wrote: > On 2019/01/11 20:04, Etsuro Fujita wrote: >> One thing I intended in that commit was to set the flag to false for >> partitioned tables contained in inheritance trees where the top parent is >> a UNION ALL subquery, because we don't consider PWJ for those

Re: Copy function for logical replication slots

2019-01-14 Thread Masahiko Sawada
On Tue, Nov 27, 2018 at 3:46 AM Petr Jelinek wrote: > > Hi, > > On 26/11/2018 01:29, Masahiko Sawada wrote: > > On Sun, Nov 25, 2018 at 12:27 AM Petr Jelinek > > wrote: > >> > >> The more serious thing is: > >> > >>> + if (MyReplicationSlot) > >>> + ReplicationSlotRelease(); >

Re: strange valgrind failures (again)

2019-01-14 Thread Andres Freund
Hi, On 2019-01-15 03:07:10 +0100, Tomas Vondra wrote: > I've started observing funny valgrind failures on Fedora 28, possibly > after upgrading from 3.14.0-1 to 3.14.0-7 a couple of days ago. This > time it does not seem like platform-specific issues, though - the > failures all look like this:

Re: Prepare Transaction support for ON COMMIT DROP temporary tables

2019-01-14 Thread Michael Paquier
On Mon, Jan 14, 2019 at 07:41:18PM +0100, Dimitri Fontaine wrote: > Thanks for the review here Tom, and for linking with the other > discussion (Álvaro did that too, thanks!). I've been reviewing it > too. If you can look at the patch, reviews are welcome. There are quite a couple of patterns I

Re: Libpq support to connect to standby server as priority

2019-01-14 Thread Tom Lane
Michael Paquier writes: > On Tue, Jan 15, 2019 at 02:00:57AM +, Tsunakawa, Takayuki wrote: >> The original desire should have been the ability to connect to a >> primary or a standby. So, I think we should go back to the original >> thinking (and not complicate the feature), and create a

Safely calling index_getprocinfo() while holding an nbtree exclusive buffer lock

2019-01-14 Thread Peter Geoghegan
My nbtree patch [1] needs to call index_getprocinfo() with an exclusive buffer lock held during a leaf page split. This has an undetectable self-deadlock (LWLock self-deadlock) risk: a syscache lookup against pg_proc might have a catcache miss, ending with an index scan that needs to access the

Re: Safely calling index_getprocinfo() while holding an nbtree exclusive buffer lock

2019-01-14 Thread Tom Lane
Peter Geoghegan writes: > My nbtree patch [1] needs to call index_getprocinfo() with an > exclusive buffer lock held during a leaf page split. I think you should stop right there and ask why. Surely that info can be obtained before starting the operation? Quite aside from the deadlock hazard,

Re: could recovery_target_timeline=latest be the default in standby mode?

2019-01-14 Thread Michael Paquier
On Sun, Jan 13, 2019 at 10:17:32AM +0100, Peter Eisentraut wrote: > I'm not sure what the coverage is in detail in this area. It seems we > already have tests for not-specific-recovery-target, maybe not in this > file, but most of the other tests rely on that, no? [... Checking ...] There are no

Re: Safely calling index_getprocinfo() while holding an nbtree exclusive buffer lock

2019-01-14 Thread Peter Geoghegan
On Mon, Jan 14, 2019 at 7:12 PM Tom Lane wrote: > I think you should stop right there and ask why. Surely that info > can be obtained before starting the operation? *Thinks some more* Uh, I'm already telling the same _bt_truncate() code path that it is being called from a CREATE INDEX,

Re: Query with high planning time at version 11.1 compared versions 10.5 and 11.0

2019-01-14 Thread Ashutosh Bapat
I think, there's something better possible. Two partitioned relations won't use partition-wise join, if their partition schemes do not match. Partitioned relations with same partitioning scheme share PartitionScheme pointer. PartitionScheme structure should get an extra counter, maintaining a

RE: [Proposal] Add accumulated statistics

2019-01-14 Thread Tsunakawa, Takayuki
From: Pavel Stehule [mailto:pavel.steh...@gmail.com] > the cumulated lock statistics maybe doesn't help with debugging - but it > is very good indicator of database (in production usage) health. I think it will help both. But I don't think the sampling won't be as helpful as the precise lock

strange valgrind failures (again)

2019-01-14 Thread Tomas Vondra
Hi, I've started observing funny valgrind failures on Fedora 28, possibly after upgrading from 3.14.0-1 to 3.14.0-7 a couple of days ago. This time it does not seem like platform-specific issues, though - the failures all look like this: ==20974== Conditional jump or move depends on

Re: unique, partitioned index fails to distinguish index key from INCLUDEd columns

2019-01-14 Thread Justin Pryzby
On Mon, Jan 14, 2019 at 07:31:07PM -0300, Alvaro Herrera wrote: > On 2019-Jan-09, Justin Pryzby wrote: > > > -- Fails to error > > postgres=# CREATE UNIQUE INDEX ON t(j) INCLUDE(i); > > > > -- Fail to enforce uniqueness across partitions due to failure to enforce > > inclusion of partition key

Re: Libpq support to connect to standby server as priority

2019-01-14 Thread Michael Paquier
On Tue, Jan 15, 2019 at 02:00:57AM +, Tsunakawa, Takayuki wrote: > But as some people said, I don't think this is the right way. I > suppose what's leading to the current somewhat complicated situation > is that there was no easy way for the client to know whether the > server is the master.

Re: Tid scan improvements

2019-01-14 Thread Edmund Horner
On Sat, 22 Dec 2018 at 07:10, Tom Lane wrote: > BTW, with respect to this bit in 0001: > > @@ -1795,6 +1847,15 @@ nulltestsel(PlannerInfo *root, NullTestType > nulltesttype, Node *arg, > return (Selectivity) 0; /* keep compiler quiet */ > } > } > +else if

current_logfiles not following group access and instead follows log_file_mode permissions

2019-01-14 Thread Haribabu Kommi
current_logfiles is a meta data file, that stores the current log writing file, and this file presents in the data directory. This file doesn't follow the group access mode set at the initdb time, but it follows the log_file_mode permissions. without group access permissions, backup with group

Re: speeding up planning with partitions

2019-01-14 Thread David Rowley
On Sat, 12 Jan 2019 at 21:49, David Rowley wrote: > > Can you say what you think is wrong with this way of making a copy of the > > ECs? > > If you shallow copy with memcpy(new_ec, ec, > sizeof(EquivalenceClass));, then fields such as ec_relids will just > point to the same memory as the parent

Re: Pluggable Storage - Andres's take

2019-01-14 Thread Dmitry Dolgov
> On Mon, Jan 14, 2019 at 2:07 PM Amit Khandekar wrote: > > createPQExpBuffer() should be moved after the below statement, so that > it does not leak memory Thanks for noticing, fixed. > So how about bumping up the archive version and doing these checks ? Yeah, you're right, I've added this

Re: Pluggable Storage - Andres's take

2019-01-14 Thread Haribabu Kommi
On Tue, Dec 11, 2018 at 1:13 PM Andres Freund wrote: > Hi, > > On 2018-11-26 17:55:57 -0800, Andres Freund wrote: > Further tasks I'm not yet planning to tackle, that I'd welcome help on: > - pg_upgrade testing > I did the pg_upgrade testing from older version with some tables and views exists,

Re: Pluggable Storage - Andres's take

2019-01-14 Thread Andres Freund
Hi, On 2019-01-15 18:02:38 +1100, Haribabu Kommi wrote: > On Tue, Dec 11, 2018 at 1:13 PM Andres Freund wrote: > > > Hi, > > > > On 2018-11-26 17:55:57 -0800, Andres Freund wrote: > > Further tasks I'm not yet planning to tackle, that I'd welcome help on: > > - pg_upgrade testing > > > > I did

Re: [HACKERS] generated columns

2019-01-14 Thread Pavel Stehule
út 15. 1. 2019 v 8:14 odesílatel Michael Paquier napsal: > On Sun, Jan 13, 2019 at 03:31:23PM +0100, Pavel Stehule wrote: > > ne 13. 1. 2019 v 10:43 odesílatel Peter Eisentraut < > > peter.eisentr...@2ndquadrant.com> napsal: > >> See here: > >> > >> >

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread David Rowley
On Tue, 15 Jan 2019 at 12:24, James Coleman wrote: > While I add that though I wanted to get this updated version out to > get feedback on the approach. I had a look at this and there's a couple of things I see: 1. In: + if (IsA(clause, ScalarArrayOpExpr)) + { + ScalarArrayOpExpr *saop =

Re: SPI Interface to Call Procedure with Transaction Control Statements?

2019-01-14 Thread Jack LIU
Hi Andrew, This is my code to call the procedure with SPI_connect_ext(SPI_OPT_NONATOMIC). if (run_proc) { appendStringInfo(, "CALL \"%s\".run_maintenance_proc(p_analyze := %s, p_jobmon := %s);", partman_schema, analyze, jobmon); expected_ret = SPI_OK_UTILITY; function_run

Re: using expression syntax for partition bounds

2019-01-14 Thread Amit Langote
Thanks for the review and sorry it took me a while to reply. On 2019/01/02 22:58, Peter Eisentraut wrote: > On 26/11/2018 05:58, Amit Langote wrote: >> On 2018/11/09 14:38, Amit Langote wrote: >>> Rebased due to change in addRangeTableEntryForRelation's API. >> >> Rebased again due to changes in

Re: current_logfiles not following group access and instead follows log_file_mode permissions

2019-01-14 Thread Michael Paquier
On Tue, Jan 15, 2019 at 03:08:41PM +1100, Haribabu Kommi wrote: > current_logfiles is a meta data file, that stores the current log writing > file, and this file presents in the data directory. This file > doesn't follow the group access mode set at the initdb time, but it > follows the

Re: [HACKERS] Block level parallel vacuum

2019-01-14 Thread Masahiko Sawada
On Fri, Dec 28, 2018 at 11:43 AM Masahiko Sawada wrote: > > On Thu, Dec 20, 2018 at 3:38 PM Amit Kapila wrote: > > > > On Tue, Dec 18, 2018 at 1:29 PM Masahiko Sawada > > wrote: > > > > > > Attached the updated patches. I scaled back the scope of this patch. > > > The patch now includes only

Re: [HACKERS] generated columns

2019-01-14 Thread Michael Paquier
On Sun, Jan 13, 2019 at 03:31:23PM +0100, Pavel Stehule wrote: > ne 13. 1. 2019 v 10:43 odesílatel Peter Eisentraut < > peter.eisentr...@2ndquadrant.com> napsal: >> See here: >> >> https://www.postgresql.org/message-id/b5c27634-1d44-feba-7494-ce5a31f91...@2ndquadrant.com > > I understand - it is

Re: [WIP] CREATE SUBSCRIPTION with FOR TABLES clause (table filter)

2019-01-14 Thread Andrey Borodin
Hi! > 27 дек. 2018 г., в 12:54, Evgeniy Efimkin написал(а): > > In latest patch i removed `FOR ALL TABLES` clause and `alltables` parameter, > now it's look more simple. > Add new system view pg_user_subscirption to allow non-superuser use pg_dump > and select addition column from

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-01-14 Thread Dean Rasheed
On Sun, 13 Jan 2019 at 00:04, Tomas Vondra wrote: > On 1/12/19 8:49 AM, Dean Rasheed wrote: > > A possible refinement would be to say that if there are more than > > stats_target items more common than this mincount threshold, rather than > > excluding the least common ones to get the target

Re: Remove all "INTERFACE ROUTINES" style comments

2019-01-14 Thread Heikki Linnakangas
On 12/01/2019 03:12, Andres Freund wrote: On 2019-01-10 15:58:41 -0800, Andres Freund wrote: A number of postgres files have sections like heapam's * INTERFACE ROUTINES ... They're often out-of-date, and I personally never found them to be useful. A few people, including yours truly, have

Re: [Patch] Create a new session in postmaster by calling setsid()

2019-01-14 Thread Heikki Linnakangas
On 30/12/2018 22:56, Tom Lane wrote: This comment needs copy-editing: + * If the user hits interrupts the startup (e.g. with CTRL-C), we'd Looks good otherwise. Thanks, fixed that, and pushed. - Heikki

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-01-14 Thread Dean Rasheed
(Removing Adrien from the CC list, because messages to that address keep bouncing) On Sun, 13 Jan 2019 at 00:31, Tomas Vondra wrote: > > On 1/10/19 6:09 PM, Dean Rasheed wrote: > > > > In the previous discussion around UpdateStatisticsForTypeChange(), the > > consensus appeared to be that we

Re: Pluggable Storage - Andres's take

2019-01-14 Thread Amit Khandekar
Thanks for the patch updates. A few comments so far from me : +static void _selectTableAccessMethod(ArchiveHandle *AH, const char *tablespace); tablespace => tableam +_selectTableAccessMethod(ArchiveHandle *AH, const char *tableam) +{ + PQExpBuffer cmd = createPQExpBuffer();

Re: insensitive collations

2019-01-14 Thread Daniel Verite
Peter Eisentraut wrote: > Here is an updated patch. On a table with pre-existing contents, the creation of a unique index does not seem to detect the duplicates that are equal per the collation and different binary-wise. postgres=# \d test3ci Table "public.test3ci"

Re: Proving IS NOT NULL inference for ScalarArrayOpExpr's

2019-01-14 Thread James Coleman
On Sun, Jan 13, 2019 at 3:06 PM Tom Lane wrote: > > There's still a logical problem here, which is that in order to > prove that the ScalarArrayOpExpr yields NULL when its LHS does, > you also have to prove that the RHS is not an empty array. > > Otherwise you're up against the fact that the OR