Re: [HACKERS] Multi column range partition table

2017-07-10 Thread Noah Misch
On Sun, Jul 09, 2017 at 08:42:32AM +0100, Dean Rasheed wrote: > On 6 July 2017 at 22:43, Joe Conway wrote: > > I agree we should get this right the first time and I also agree with > > Dean's proposal, so I guess I'm a +2 > > > > On 7 July 2017 at 03:21, Amit Langote wrote: > > +1 to releasing t

Re: [HACKERS] why not parallel seq scan for slow functions

2017-07-10 Thread Dilip Kumar
On Tue, Jul 11, 2017 at 9:02 AM, Jeff Janes wrote: > If I have a slow function which is evaluated in a simple seq scan, I do not > get parallel execution, even though it would be massively useful. Unless > force_parallel_mode=ON, then I get a dummy parallel plan with one worker. > > explain selec

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > wrote: > > Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables as of Type > > "partitioned table", we wouldn't need a separate flag for marking a table > > as having partitions. > > I think that is false. Whether someth

[HACKERS] why not parallel seq scan for slow functions

2017-07-10 Thread Jeff Janes
If I have a slow function which is evaluated in a simple seq scan, I do not get parallel execution, even though it would be massively useful. Unless force_parallel_mode=ON, then I get a dummy parallel plan with one worker. explain select aid,slow(abalance) from pgbench_accounts; CREATE OR REPLAC

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-10 Thread Alvaro Herrera
Amit Kapila wrote: > On Tue, Jul 11, 2017 at 6:51 AM, AP wrote: > > On Fri, Jul 07, 2017 at 05:58:25PM +0530, Amit Kapila wrote: > >> I can understand your concerns. To address first concern we need to > >> work on one or more of following work items: (a) work on vacuums that > >> can be trigger

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-10 Thread Amit Kapila
On Tue, Jul 11, 2017 at 6:51 AM, AP wrote: > On Fri, Jul 07, 2017 at 05:58:25PM +0530, Amit Kapila wrote: >> On Fri, Jul 7, 2017 at 8:22 AM, AP wrote: >> > On Thu, Jul 06, 2017 at 05:19:59PM +0530, Amit Kapila wrote: >> >> I think if you are under development, it is always advisable to create >>

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Amit Langote
On 2017/07/11 10:34, Paul A Jungwirth wrote: >> Also, there seems to be at least some preference >> for excluding partitions by default from the \d listing. > > As another user of partitions I'll chime in and say that would be very > nice! On the other hand, with pre-10 partitions you do see all t

Re: [HACKERS] strcmp() tie-breaker for identical ICU-collated strings

2017-07-10 Thread Peter Geoghegan
On Fri, Jun 9, 2017 at 11:09 AM, Robert Haas wrote: >> Isn't that what strxfrm() is? > > Yeah, just with bugs. If ICU has a non-buggy equivalent, then we can > make this work. I agree that it probably isn't worth using strxfrm() again, simply because the glibc implementation is buggy, and glibc

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Paul A Jungwirth
> Also, there seems to be at least some preference > for excluding partitions by default from the \d listing. As another user of partitions I'll chime in and say that would be very nice! On the other hand, with pre-10 partitions you do see all the child tables with `\d`, so showing declarative par

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Kyotaro HORIGUCHI
At Mon, 10 Jul 2017 18:37:34 +0530, Amit Kapila wrote in > On Mon, Jul 10, 2017 at 3:27 PM, Kyotaro HORIGUCHI > wrote: > > Hi, > > > > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila > > wrote in > > > > > >> I am also not 100% comfortable with adding flush at two new places, > >> but that

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Amit Langote
On 2017/07/11 7:33, Robert Haas wrote: > On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > wrote: >> Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables as of Type >> "partitioned table", we wouldn't need a separate flag for marking a table >> as having partitions. > > I think that is false

Re: [HACKERS] pg_stop_backup(wait_for_archive := true) on standby server

2017-07-10 Thread Masahiko Sawada
On Mon, Jul 10, 2017 at 11:56 AM, Michael Paquier wrote: > On Sat, Jul 8, 2017 at 12:50 AM, Masahiko Sawada > wrote: >> On Fri, Jul 7, 2017 at 3:48 PM, Michael Paquier >> wrote: >>> So I would suggest the following things to address this issue: >>> 1) Generate a backup history file for backups

Re: [HACKERS] [COMMITTERS] pgsql: Allow multiple hostaddrs to go with multiple hostnames.

2017-07-10 Thread Masahiko Sawada
On Mon, Jul 10, 2017 at 10:01 PM, Heikki Linnakangas wrote: > On 07/10/2017 01:27 PM, Masahiko Sawada wrote: >> >> This commit seems be cause of the documentation compilation error. I >> think is missing. >> >> ... >> >> Attached small patch fixes this. > > > Thanks, committed! > > Strangely, it

Re: [HACKERS] More race conditions in logical replication

2017-07-10 Thread Alvaro Herrera
Alvaro Herrera wrote: > I'll next update this on or before Monday 10th at 19:00 CLT. I couldn't get to this today as I wanted. Next update on Wednesday 12th, same time. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Se

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Peter Geoghegan
On Mon, Jul 10, 2017 at 4:08 PM, Greg Stark wrote: > One thing I would like to see is features like this added to the > opclasses (or opfamilies?) using standard PG functions that return > standard PG data types. So if each opclass had a function that took > the data type in question and returned

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Greg Stark
On 10 July 2017 at 23:46, David Fetter wrote: > On Mon, Jul 10, 2017 at 05:33:34PM -0500, Robert Haas wrote: >> On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote >> wrote: >> > I posted a patch upthread which makes \d hide partitions >> > (relispartition = true relations) and include them if the newl

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Greg Stark
On 10 July 2017 at 19:40, Peter Geoghegan wrote: > Key normalization means creating a representation for internal page > items that we always just memcmp(), regardless of the details of the > underlying datatypes. One thing I would like to see is features like this added to the opclasses (or opfa

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread David Fetter
On Mon, Jul 10, 2017 at 05:33:34PM -0500, Robert Haas wrote: > On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > wrote: > > I posted a patch upthread which makes \d hide partitions > > (relispartition = true relations) and include them if the newly > > proposed '!' modifier is specified. The '+' mo

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Robert Haas
On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote wrote: > I posted a patch upthread which makes \d hide partitions (relispartition = > true relations) and include them if the newly proposed '!' modifier is > specified. The '+' modifier is being used to show additional detail of > relations chosen to

Re: [HACKERS] Add support for tuple routing to foreign partitions

2017-07-10 Thread Robert Haas
On Thu, Jun 29, 2017 at 6:20 AM, Etsuro Fujita wrote: > So, I dropped the COPY part. Ouch. I think we should try to figure out how the COPY part will be handled before we commit to a design. I have to admit that I'm a little bit fuzzy about why foreign insert routing requires all of these chang

[HACKERS] Re: Oddity in error handling of constraint violation in ExecConstraints for partitioned tables

2017-07-10 Thread Robert Haas
On Sun, Jul 9, 2017 at 3:06 AM, Noah Misch wrote: > On Fri, Jul 07, 2017 at 06:47:26PM +0900, Amit Langote wrote: >> On 2017/07/06 16:06, Etsuro Fujita wrote: >> > Looks odd to me because the error message doesn't show any DETAIL info; >> > since the CTE query, which produces the message, is the s

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Peter Geoghegan
On Mon, Jul 10, 2017 at 12:53 PM, Claudio Freire wrote: > I do have a patch that attacks suffix truncation, heap tid unification > and prefix compression all at once. That's great! I'll certainly be able to review it. > It's on a hiatus ATM, but, as you say, the implementations are highly > corr

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Peter Geoghegan
On Mon, Jul 10, 2017 at 1:23 PM, Alvaro Herrera wrote: > Claudio Freire wrote: > >> A missing optimization is that having tid unification allows VACUUM to >> implement a different strategy when it needs to clean up only a tiny >> fraction of the index. It can do the lookup by key-tid instead of >>

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Alvaro Herrera
Claudio Freire wrote: > A missing optimization is that having tid unification allows VACUUM to > implement a different strategy when it needs to clean up only a tiny > fraction of the index. It can do the lookup by key-tid instead of > scanning the whole index, which can be a win if the index is l

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Claudio Freire
On Mon, Jul 10, 2017 at 3:40 PM, Peter Geoghegan wrote: > It might appear excessive to talk about several different techniques > in one place, but that seemed like the best way to me, because there > are subtle dependencies. If most of the optimizations are pursued as a > project all at once (say,

[HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Peter Geoghegan
I've created a new Wiki page that describes a scheme for normalizing internal page items within B-Tree indexes, and the many optimizations that this can enable: https://wiki.postgresql.org/wiki/Key_normalization Key normalization means creating a representation for internal page items that we alw

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread David Fetter
On Mon, Jul 10, 2017 at 04:15:28PM +0900, Amit Langote wrote: > On 2017/07/10 15:32, Craig Ringer wrote: > > On 8 July 2017 at 00:03, David Fetter wrote: > > > >> On Fri, Jul 07, 2017 at 10:29:26AM +0900, Amit Langote wrote: > >>> Hi Mark, > >>> > >>> On 2017/07/07 9:02, Mark Kirkwood wrote: > >>

Re: [HACKERS] retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Tom Lane
Noah Misch writes: > On Mon, Jul 10, 2017 at 10:46:09AM -0400, Tom Lane wrote: >> Shall we go for broke and also remove the ASLR-disabling patch in beta2? > As I mentioned in my message eight hours ago, no. Ah, sorry, I'd managed to swap out that bit of info already. However, I've now gone and

[HACKERS] Re: retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Noah Misch
On Mon, Jul 10, 2017 at 10:46:09AM -0400, Tom Lane wrote: > Magnus Hagander writes: > > On Jul 10, 2017 16:08, "Tom Lane" wrote: > >> Okay, so that leaves us with a decision to make: push it into beta2, or > >> wait till after wrap? I find it pretty scary to push a patch with > >> portability im

[HACKERS] Re: retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Noah Misch
On Mon, Jul 10, 2017 at 10:08:53AM -0400, Tom Lane wrote: > Noah Misch writes: > > I recommend pushing your patch so the August back-branch releases have it. > > One can see by inspection that your patch has negligible effect on systems > > healthy today. I have a reasonable suspicion it will hel

Re: [HACKERS] retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Tom Lane
Magnus Hagander writes: > On Jul 10, 2017 16:08, "Tom Lane" wrote: >> Okay, so that leaves us with a decision to make: push it into beta2, or >> wait till after wrap? I find it pretty scary to push a patch with >> portability implications so soon before wrap, but a quick look at the >> buildfarm

[HACKERS] Re: retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Magnus Hagander
On Jul 10, 2017 16:08, "Tom Lane" wrote: Noah Misch writes: > On Mon, Jun 05, 2017 at 09:56:33AM -0400, Tom Lane wrote: >> Yeah, being able to reproduce the problem reliably enough to say whether >> it's fixed or not is definitely the sticking point here. I have some >> ideas about that: ... >

Re: [HACKERS] retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Tom Lane
Noah Misch writes: > On Mon, Jun 05, 2017 at 09:56:33AM -0400, Tom Lane wrote: >> Yeah, being able to reproduce the problem reliably enough to say whether >> it's fixed or not is definitely the sticking point here. I have some >> ideas about that: ... > I tried this procedure without finding a s

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2017-07-10 Thread Marina Polyakova
Hello everyone! There's the second version of my patch for pgbench. Now transactions with serialization and deadlock failures are rolled back and retried until they end successfully or their number of attempts reaches maximum. In details: - You can set the maximum number of attempts by the ap

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 3:27 PM, Kyotaro HORIGUCHI wrote: > Hi, > > At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila > wrote in > >> I am also not 100% comfortable with adding flush at two new places, >> but that makes the code for fix simpler and fundamentally there is no >> problem in doing so

[HACKERS] Re: [COMMITTERS] pgsql: Allow multiple hostaddrs to go with multiple hostnames.

2017-07-10 Thread Heikki Linnakangas
On 07/10/2017 01:27 PM, Masahiko Sawada wrote: This commit seems be cause of the documentation compilation error. I think is missing. ... Attached small patch fixes this. Thanks, committed! Strangely, it worked on my system, despite that clear mistake. Looks like the 'osx' tool is more str

Re: [HACKERS] Dumping database creation options and ACLs

2017-07-10 Thread Adrien Nayrat
On 07/03/2017 05:16 PM, Rafael Martinez wrote: > We have a discussion about this some time ago and we created a wiki page > where we tried to write down some ideas/proposals and links to threads > discussing the subject: > > https://wiki.postgresql.org/wiki/Pg_dump_improvements Thanks for this li

Re: [HACKERS] List of hostaddrs not supported

2017-07-10 Thread Heikki Linnakangas
On 07/10/2017 01:47 PM, Arthur Zakirov wrote: Hello, 2017-07-10 12:30 GMT+03:00 Heikki Linnakangas : I just remembered that this was still pending. I made the documentation changes, and committed this patch now. We're uncomfortably close to wrapping the next beta, later today, but I think it

[HACKERS] Double shared memory allocation for SLRU LWLocks

2017-07-10 Thread Alexander Korotkov
Hi, all! It seems to me that we're allocating shared memory for SLRU lwlocks twice, unless I'm missing something. SimpleLruShmemSize() calculates total SLRU shared memory size including lwlocks size. SimpleLruInit() starts with line shared = (SlruShared) ShmemInitStruct(name,

Re: [HACKERS] paths in partitions of a dummy partitioned table

2017-07-10 Thread Ashutosh Bapat
On Mon, Jul 10, 2017 at 2:59 PM, Kyotaro HORIGUCHI wrote: > Hello, Thanks for the review. >> If a partitioned table is proven dummy, set_rel_pathlist() doesn't mark the >> partition relations dummy and thus doesn't set any (dummy) paths in the >> partition relations. The lack of paths in the par

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-10 Thread Fabien COELHO
Hello Alik, Your description is not very precise. What version of Postgres is used? If there is a decline, compared to which version? Is there a link to these results? Benchmark have been done in master v10. I am attaching image with results: . Ok, thanks. More precision would be helpful,

Re: [HACKERS] List of hostaddrs not supported

2017-07-10 Thread Arthur Zakirov
Hello, 2017-07-10 12:30 GMT+03:00 Heikki Linnakangas : > > > I just remembered that this was still pending. I made the documentation > changes, and committed this patch now. > > We're uncomfortably close to wrapping the next beta, later today, but I > think it's better to get this into the hands o

Re: [HACKERS] COPY vs. transition tables

2017-07-10 Thread Andrew Gierth
> "Andrew" == Andrew Gierth writes: > "Thomas" == Thomas Munro writes: Thomas> Here it is. Added to open items. Andrew> On it. Committed. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] [COMMITTERS] pgsql: Allow multiple hostaddrs to go with multiple hostnames.

2017-07-10 Thread Masahiko Sawada
On Mon, Jul 10, 2017 at 6:30 PM, Heikki Linnakangas wrote: > Allow multiple hostaddrs to go with multiple hostnames. > > Also fix two other issues, while we're at it: > > * In error message on connection failure, if multiple network addresses > were given as the host option, as in "host=127.0.0.1,

Re: [HACKERS] COPY vs. transition tables

2017-07-10 Thread Andrew Gierth
> "Thomas" == Thomas Munro writes: Thomas> Here it is. Added to open items. On it. -- Andrew (irc:RhodiumToad) -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Kyotaro HORIGUCHI
Hi, At Mon, 10 Jul 2017 14:58:13 +0530, Amit Kapila wrote in > On Mon, Jul 10, 2017 at 10:39 AM, Kyotaro HORIGUCHI > wrote: > > At Sat, 8 Jul 2017 16:41:27 +0530, Amit Kapila > > wrote in > > > >> On Sat, Jul 8, 2017 at 9:08 AM, Robert Haas wrote: > >> > On Fri, Jul 7, 2017 at 11:36 PM, A

Re: [HACKERS] POC: Sharing record typmods between backends

2017-07-10 Thread Thomas Munro
On Thu, Jun 1, 2017 at 6:29 AM, Andres Freund wrote: > On May 31, 2017 11:28:18 AM PDT, Robert Haas wrote: >>> On 2017-05-31 13:27:28 -0400, Dilip Kumar wrote: [ ... various discussion in support of using DHT ... ] Ok, good. Here's a new version that introduces a per-session DSM segment to hold

Re: [HACKERS] List of hostaddrs not supported

2017-07-10 Thread Heikki Linnakangas
On 06/09/2017 04:26 PM, Robert Haas wrote: On Fri, Jun 9, 2017 at 6:36 AM, Heikki Linnakangas wrote: Right. I think it's a usability fail as it is; it certainly fooled me. We could make the error messages and documentation more clear. But even better to allow multiple host addresses, so that it

Re: [HACKERS] paths in partitions of a dummy partitioned table

2017-07-10 Thread Kyotaro HORIGUCHI
Hello, At Thu, 6 Jul 2017 21:05:21 +0530, Ashutosh Bapat wrote in > If a partitioned table is proven dummy, set_rel_pathlist() doesn't mark the > partition relations dummy and thus doesn't set any (dummy) paths in the > partition relations. The lack of paths in the partitions means that we can

Re: [HACKERS] hash index on unlogged tables doesn't behave as expected

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 10:39 AM, Kyotaro HORIGUCHI wrote: > At Sat, 8 Jul 2017 16:41:27 +0530, Amit Kapila > wrote in >> On Sat, Jul 8, 2017 at 9:08 AM, Robert Haas wrote: >> > On Fri, Jul 7, 2017 at 11:36 PM, Amit Kapila >> > wrote: >> >> I think we should do that as a separate patch (I ca

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-10 Thread Amit Kapila
On Mon, Jul 10, 2017 at 12:19 PM, Alik Khilazhev wrote: > Hello, Fabien! > > Your description is not very precise. What version of Postgres is used? If > there is a decline, compared to which version? Is there a link to these > results? > > > Benchmark have been done in master v10. I am attaching

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Amit Langote
On 2017/07/10 15:32, Craig Ringer wrote: > On 8 July 2017 at 00:03, David Fetter wrote: > >> On Fri, Jul 07, 2017 at 10:29:26AM +0900, Amit Langote wrote: >>> Hi Mark, >>> >>> On 2017/07/07 9:02, Mark Kirkwood wrote: I've been trying out the new partitioning in version 10. Firstly, I >> must

[HACKERS] Re: retry shm attach for windows (WAS: Re: OK, so culicidae is *still* broken)

2017-07-10 Thread Michael Paquier
On Mon, Jul 10, 2017 at 3:36 PM, Noah Misch wrote: > I recommend pushing your patch so the August back-branch releases have it. > One can see by inspection that your patch has negligible effect on systems > healthy today. I have a reasonable suspicion it will help some systems. If > others remai