Re: [HACKERS] Cache Hash Index meta page.

2016-12-20 Thread Mithun Cy
On Tue, Dec 20, 2016 at 3:51 AM, Robert Haas wrote: > On Fri, Dec 16, 2016 at 5:16 AM, Mithun Cy > wrote: > >> Shouldn't _hash_doinsert() be using the cache, too >>> >> >> Yes, we have an opportunity there, added same in code. But one

Re: [HACKERS] Migration Docs WAS: Proposal for changes to recovery.conf API

2016-12-20 Thread Josh Berkus
On 12/19/2016 01:29 PM, Peter Eisentraut wrote: > On 12/16/16 8:52 PM, Robert Haas wrote: >> > If the explanation is just a few sentences long, I see no reason not >> > to include it in the release notes. > As far as I can tell from the latest posted patch, the upgrade > instructions are > > - To

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread Alvaro Herrera
Tom Lane wrote: > Ryan Murphy writes: > > I'm concerned some new users may not understand this behavior of pg_ctl, so > > I wanted to suggest that we add some additional messaging after "server > > starting" - something like: > > > $ pg_ctl -D datadir -l logfile start > >

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread Peter Eisentraut
On 12/20/16 3:49 PM, Tom Lane wrote: > Peter Eisentraut writes: >> Maybe the fix is to make --wait the default? > I was wondering about that too ... does anyone remember the rationale > for the current behavior? Probably because that didn't work reliably before

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Stephen Frost
David, * David Fetter (da...@fetter.org) wrote: > On Tue, Dec 20, 2016 at 08:34:19AM -0500, Stephen Frost wrote: > > * Heikki Linnakangas (hlinn...@iki.fi) wrote: > > > Even if you have a separate "verifier type" column, it's not fully > > > normalized, because there's still a dependency between

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread Tom Lane
Peter Eisentraut writes: > Maybe the fix is to make --wait the default? I was wondering about that too ... does anyone remember the rationale for the current behavior? But the message for the non-wait case seems like it could stand to be improved independently

Re: [HACKERS] too low cost of Bitmap index scan

2016-12-20 Thread Tom Lane
Pavel Stehule writes: > I am trying to fix slow query on PostgreSQL 9.5.4. > The data are almost in RAM If it's all in RAM, you'd likely be well-served to lower random_page_cost. It looks to me like the planner is estimating pretty accurately how many heap fetches will

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2016-12-20 Thread Tom Lane
Robert Haas writes: > On Tue, Dec 20, 2016 at 10:09 AM, Tom Lane wrote: >> I don't understand why we'd make that a system-wide behavior at all, >> rather than expecting each process to manage its own cache. > Individual backends don't have a really

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread Tom Lane
Ryan Murphy writes: > I'm concerned some new users may not understand this behavior of pg_ctl, so > I wanted to suggest that we add some additional messaging after "server > starting" - something like: > $ pg_ctl -D datadir -l logfile start > server starting > (to wait for

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread David Fetter
On Tue, Dec 20, 2016 at 03:43:11PM -0500, Peter Eisentraut wrote: > On 12/20/16 3:31 PM, Ryan Murphy wrote: > > I'm concerned some new users may not understand this behavior of pg_ctl, > > so I wanted to suggest that we add some additional messaging after > > "server starting" - something like: >

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Michael Paquier
On Wed, Dec 21, 2016 at 1:08 AM, David Fetter wrote: > Would a view that shows only what's to the left of the first semicolon > suit this purpose? Of course it would, you would just need to make the routines now checking the shape of MD5 and SCRAM identifiers available at SQL

Re: [HACKERS] Make pg_basebackup -x stream the default

2016-12-20 Thread Michael Paquier
On Tue, Dec 20, 2016 at 10:38 PM, Fujii Masao wrote: > On Mon, Dec 19, 2016 at 7:51 PM, Vladimir Rusinov wrote: >> The server must also be configured with max_wal_senders set high >> enough to leave at least one session available for the backup. > > I

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-12-20 Thread Robert Haas
On Wed, Sep 21, 2016 at 12:52 PM, Heikki Linnakangas wrote: > I find this unification business really complicated. I think it'd be simpler > to keep the BufFiles and LogicalTapeSets separate, and instead teach > tuplesort.c how to merge tapes that live on different >

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 3:10 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Dec 20, 2016 at 10:09 AM, Tom Lane wrote: >>> I don't understand why we'd make that a system-wide behavior at all, >>> rather than expecting each

Re: [HACKERS] Rethinking our fulltext phrase-search implementation

2016-12-20 Thread Tom Lane
I wrote: > I've been thinking about how to fix the problem Andreas Seltenreich > reported at > https://postgr.es/m/87eg1y2s3x@credativ.de Attached is a proposed patch that deals with the problems discussed here and in <26706.1482087...@sss.pgh.pa.us>. Is anyone interested in reviewing this,

[HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread Ryan Murphy
Hi Postgres Devs, I had a suggestion regarding the output pg_ctl gives when you use it to start the postgres server. At first I was going to write a patch, but then I decided to just ask you guys first to see what you think. I had an issue earlier where I was trying to upgrade my postgres

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread Peter Eisentraut
On 12/20/16 3:31 PM, Ryan Murphy wrote: > I'm concerned some new users may not understand this behavior of pg_ctl, > so I wanted to suggest that we add some additional messaging after > "server starting" - something like: > > $ pg_ctl -D datadir -l logfile start > server starting > (to wait for

Re: [HACKERS] Clarifying "server starting" messaging in pg_ctl start without --wait

2016-12-20 Thread David G. Johnston
On Tue, Dec 20, 2016 at 1:49 PM, Tom Lane wrote: > Peter Eisentraut writes: > > Maybe the fix is to make --wait the default? > > I was wondering about that too ... does anyone remember the rationale > for the current behavior? But the

Re: [HACKERS] Declarative partitioning - another take

2016-12-20 Thread Amit Langote
On 2016/12/21 1:45, Alvaro Herrera wrote: > Robert Haas wrote: >> On Tue, Dec 20, 2016 at 10:27 AM, Alvaro Herrera >> wrote: >>> Even if we decide to keep the message, I think it's not very good >>> wording anyhow; as a translator I disliked it on sight. Instead of >>>

Re: [HACKERS] Replication slot xmin is not reset if HS feedback is turned off while standby is shut down

2016-12-20 Thread Craig Ringer
On 21 December 2016 at 00:52, Ants Aasma wrote: > The simple fix seems to be to always send out at least one feedback > message on each connect regardless of hot_standby_feedback setting. I agree. > Patch attached. Looks like this goes back to version 9.4. It could >

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Michael Paquier
On Tue, Dec 20, 2016 at 9:23 PM, Heikki Linnakangas wrote: > On 12/16/2016 03:31 AM, Michael Paquier wrote: > Actually, it does still perform that check. There's a new function, > plain_crypt_verify, that passwordcheck uses now. plain_crypt_verify() is > intended to work with any

Re: [HACKERS] Logical decoding on standby

2016-12-20 Thread Craig Ringer
On 20 December 2016 at 15:03, Petr Jelinek wrote: >> The biggest change in this patch, and the main intrusive part, is that >> procArray->replication_slot_catalog_xmin is no longer directly used by >> vacuum. Instead, a new ShmemVariableCache->oldestCatalogXmin

Re: [HACKERS] pg_background contrib module proposal

2016-12-20 Thread David Fetter
On Mon, Dec 19, 2016 at 09:30:32PM +0500, Andrew Borodin wrote: > 2016-12-19 4:21 GMT+05:00 David Fetter : > > Couldn't it sleep in increments smaller than a second? Like maybe > > milliseconds? Also, it's probably cleaner (or at least more > > comprehensible) to write

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Stephen Frost
David, * David Fetter (da...@fetter.org) wrote: > On Tue, Dec 20, 2016 at 06:14:40PM -0500, Stephen Frost wrote: > > * David Fetter (da...@fetter.org) wrote: > > > On Tue, Dec 20, 2016 at 08:34:19AM -0500, Stephen Frost wrote: > > > > * Heikki Linnakangas (hlinn...@iki.fi) wrote: > > > > > Even

Re: [HACKERS] Logical decoding on standby

2016-12-20 Thread Craig Ringer
On 20 December 2016 at 15:03, Petr Jelinek wrote: > But in 0003 I don't understand following code: >> + if (endpos != InvalidXLogRecPtr && !do_start_slot) >> + { >> + fprintf(stderr, >> + _("%s: cannot use

Re: [HACKERS] Proposal : Parallel Merge Join

2016-12-20 Thread Dilip Kumar
On Tue, Dec 13, 2016 at 8:34 PM, Dilip Kumar wrote: > I have created two patches as per the suggestion. > > 1. mergejoin_refactoring_v2.patch --> Move functionality of > considering various merge join path into new function. > 2. parallel_mergejoin_v2.patch -> This adds the

[HACKERS] simplehash vs. pgindent

2016-12-20 Thread Robert Haas
In commit b30d3ea824c5ccba43d3e942704f20686e7dbab8, when Andres added the simplehash stuff, he also added SH_TYPE, SH_ITERATOR, and SH_STATUS to typedefs.list. When I subsequently updated typedefs.list from the buildfarm in acddbe221b084956a0efd6e4b6c6586e8fd994d7, it of course removed those

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2016-12-20 Thread Kyotaro HORIGUCHI
Thank you for the discussion. At Tue, 20 Dec 2016 15:10:21 -0500, Tom Lane wrote in <23492.1482264...@sss.pgh.pa.us> > The bigger picture here though is that we used to have limits on syscache > size, and we got rid of them (commit 8b9bc234a, see also >

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-20 Thread Kyotaro HORIGUCHI
At Tue, 20 Dec 2016 23:47:22 +0900, Fujii Masao wrote in > On Tue, Dec 20, 2016 at 2:46 PM, Michael Paquier > wrote: > > On Tue, Dec 20, 2016 at 2:31 PM, Masahiko Sawada

Re: [HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 6:18 AM, Fabien COELHO wrote: > Would this approach be acceptable, or is modifying Nodes a no-go area? > > If it is acceptable, I can probably put together a patch and submit it. > > If not, I suggest to update the documentation to tell that >

Re: [HACKERS] pg_background contrib module proposal

2016-12-20 Thread Simon Riggs
On 9 December 2016 at 13:00, Robert Haas wrote: > That might be good, because then we wouldn't have to maintain two > copies of the code. So why are there two things at all? Why is this being worked on as well as Peter's patch? What will that give us? -- Simon Riggs

Re: [HACKERS] pg_background contrib module proposal

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 7:32 PM, Simon Riggs wrote: > On 9 December 2016 at 13:00, Robert Haas wrote: >> That might be good, because then we wouldn't have to maintain two >> copies of the code. > > So why are there two things at all? Why is this

Re: [HACKERS] bigint vs txid user confusion

2016-12-20 Thread Craig Ringer
On 17 December 2016 at 00:13, Robert Haas wrote: > On Thu, Dec 15, 2016 at 3:02 AM, Craig Ringer wrote: >> I really wish we could just change the pg_stat_activity and >> pg_stat_replication xid fields to be epoch qualified in a 64-bit wide >>

Re: [HACKERS] Declarative partitioning - another take

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 9:14 PM, Amit Langote wrote: > On 2016/12/21 1:45, Alvaro Herrera wrote: >> Robert Haas wrote: >>> On Tue, Dec 20, 2016 at 10:27 AM, Alvaro Herrera >>> wrote: Even if we decide to keep the message, I think it's

Re: [HACKERS] pgstattuple documentation clarification

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 10:01 AM, Tom Lane wrote: > Andrew Dunstan writes: >> Recently a client was confused because there was a substantial >> difference between the reported table_len of a table and the sum of the >> corresponding tuple_len,

Re: [HACKERS] simplehash vs. pgindent

2016-12-20 Thread Tom Lane
Robert Haas writes: > In commit b30d3ea824c5ccba43d3e942704f20686e7dbab8, when Andres added > the simplehash stuff, he also added SH_TYPE, SH_ITERATOR, and > SH_STATUS to typedefs.list. When I subsequently updated typedefs.list > from the buildfarm in

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread David Fetter
On Tue, Dec 20, 2016 at 06:14:40PM -0500, Stephen Frost wrote: > David, > > * David Fetter (da...@fetter.org) wrote: > > On Tue, Dec 20, 2016 at 08:34:19AM -0500, Stephen Frost wrote: > > > * Heikki Linnakangas (hlinn...@iki.fi) wrote: > > > > Even if you have a separate "verifier type" column,

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-12-20 Thread Peter Geoghegan
On Tue, Dec 20, 2016 at 2:53 PM, Robert Haas wrote: >> What I have in mind is something like the attached patch. It refactors >> LogicalTapeRead(), LogicalTapeWrite() etc. functions to take a LogicalTape >> as argument, instead of LogicalTapeSet and tape number.

[HACKERS] Why does plpython delay composite type resolution?

2016-12-20 Thread Jim Nasby
Why do functions that accept composite types delay type resolution until execution? I have a naive patch that speeds up plpy.execute() by 8% by caching interred python strings for the dictionary key names (which are repeated over and over). The next step is to just pre-allocate those strings

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 8:14 PM, Peter Geoghegan wrote: > Without meaning to sound glib, unification is the process by which > parallel CREATE INDEX has the leader read temp files from workers > sufficient to complete its final on-the-fly merge. That's not glib, but you can't in

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-20 Thread Amit Kapila
On Tue, Dec 20, 2016 at 8:04 PM, Robert Haas wrote: > On Sat, Dec 17, 2016 at 5:46 AM, Amit Kapila wrote: >> Yeah, but we are planning to add a generic flag in WaitEvent structure >> which can be used for similar purpose. However, as per your >>

Re: [HACKERS] Parallel bitmap heap scan

2016-12-20 Thread Dilip Kumar
Rebased on the current head. On Tue, Dec 13, 2016 at 12:06 PM, Dilip Kumar wrote: > On Sat, Dec 10, 2016 at 5:36 PM, Amit Kapila wrote: >> Few assorted comments: > > Thanks for the review > >> >> 1. >> + else if (needWait) >> + { >> + /* Add

Re: [HACKERS] Declarative partitioning - another take

2016-12-20 Thread Amit Langote
On 2016/12/21 13:42, Robert Haas wrote: > On Tue, Dec 20, 2016 at 9:14 PM, Amit Langote > wrote: >> On 2016/12/21 1:45, Alvaro Herrera wrote: >>> Robert Haas wrote: On Tue, Dec 20, 2016 at 10:27 AM, Alvaro Herrera wrote: >

Re: [HACKERS] Replication slot xmin is not reset if HS feedback is turned off while standby is shut down

2016-12-20 Thread Michael Paquier
On Wed, Dec 21, 2016 at 11:14 AM, Craig Ringer wrote: > I didn't see this in the CF app so I created it in > https://commitfest.postgresql.org/12/ as > https://commitfest.postgresql.org/12/924/ . I couldn't find your > PostgreSQL community username, so please log in and set

[HACKERS] Faster methods for getting SPI results

2016-12-20 Thread Jim Nasby
I've been looking at the performance of SPI calls within plpython. There's a roughly 1.5x difference from equivalent python code just in pulling data out of the SPI tuplestore. Some of that is due to an inefficiency in how plpython is creating result dictionaries, but fixing that is ultimately

Re: [HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-20 Thread Kyotaro HORIGUCHI
At Tue, 20 Dec 2016 22:42:48 -0500, Robert Haas wrote in > On Tue, Dec 20, 2016 at 6:18 AM, Fabien COELHO wrote: > > Would this approach be acceptable, or is modifying Nodes a

Re: [HACKERS] Declarative partitioning - another take

2016-12-20 Thread Amit Langote
On 2016/12/21 14:03, Amit Langote wrote: > OK, updated patch attached. Oops, incomplete patch that was. Complete patch attached this time. Thanks, Amit diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 1c219b03dd..115b98313e 100644 ---

Re: [HACKERS] bigint vs txid user confusion

2016-12-20 Thread Jim Nasby
On 12/20/16 10:20 PM, Craig Ringer wrote: Tools look at pg_class.relfrozenxid and pg_databse.datfrozenxid more than probably anything else, so making changes that ignores them is pretty pointless. Except the only useful way I know of to access *frozenxid is using age(), and even that is a

Re: [HACKERS] Replication slot xmin is not reset if HS feedback is turned off while standby is shut down

2016-12-20 Thread Ants Aasma
On Wed, Dec 21, 2016 at 4:14 AM, Craig Ringer wrote: > Re the patch, I don't like > > - static bool master_has_standby_xmin = false; > + static bool master_has_standby_xmin = true; > > without any comment. It's addressed in the comment changes on the > header

Re: [HACKERS] Why does plpython delay composite type resolution?

2016-12-20 Thread Andreas Karlsson
On 12/21/2016 04:14 AM, Jim Nasby wrote: Why do functions that accept composite types delay type resolution until execution? I have a naive patch that speeds up plpy.execute() by 8% by caching interred python strings for the dictionary key names (which are repeated over and over). The next step

Re: [HACKERS] pg_background contrib module proposal

2016-12-20 Thread Andrew Borodin
2016-12-21 4:55 GMT+05:00 David Fetter : > I see. > > I find the following a little easier to follow, and the sleeps still > work even when very short. Now I know a little more SQL :) Thank you. I'm not sure every platform supports microsecond sleeps. But it will work anyway.

Re: [HACKERS] Logical decoding on standby

2016-12-20 Thread Petr Jelinek
On 21/12/16 04:31, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek > wrote: > >> But in 0003 I don't understand following code: >>> + if (endpos != InvalidXLogRecPtr && !do_start_slot) >>> + { >>> + fprintf(stderr, >>> +

Re: [HACKERS] too low cost of Bitmap index scan

2016-12-20 Thread Pavel Stehule
2016-12-21 0:01 GMT+01:00 Tom Lane : > Pavel Stehule writes: > > I am trying to fix slow query on PostgreSQL 9.5.4. > > The data are almost in RAM > > If it's all in RAM, you'd likely be well-served to lower random_page_cost. > It looks to me like the

Re: [HACKERS] Logical decoding on standby

2016-12-20 Thread Petr Jelinek
On 21/12/16 04:06, Craig Ringer wrote: > On 20 December 2016 at 15:03, Petr Jelinek > wrote: > >>> The biggest change in this patch, and the main intrusive part, is that >>> procArray->replication_slot_catalog_xmin is no longer directly used by >>> vacuum. Instead,

Re: [HACKERS] Declarative partitioning - another take

2016-12-20 Thread Alvaro Herrera
Amit Langote wrote: > diff --git a/src/backend/commands/tablecmds.c > b/src/backend/commands/tablecmds.c > index 1c219b03dd..6a179596ce 100644 > --- a/src/backend/commands/tablecmds.c > +++ b/src/backend/commands/tablecmds.c > @@ -13297,8 +13297,10 @@ ATExecAttachPartition(List **wqueue,

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Amit Kapila
On Mon, Dec 19, 2016 at 11:05 PM, Robert Haas wrote: > On Sun, Dec 18, 2016 at 8:54 AM, Amit Kapila wrote: >>> I committed remove-hash-wrtbuf and fix_dirty_marking_v1 but I've got >>> some reservations about fix_lock_chaining_v1. ISTM that the

Re: [HACKERS] pgstattuple documentation clarification

2016-12-20 Thread Andrew Dunstan
On 12/20/2016 10:01 AM, Tom Lane wrote: Andrew Dunstan writes: Recently a client was confused because there was a substantial difference between the reported table_len of a table and the sum of the corresponding tuple_len, dead_tuple_len and free_space. The docs are

Re: [HACKERS] [COMMITTERS] pgsql: Implement table partitioning.

2016-12-20 Thread Alvaro Herrera
Robert Haas wrote: > Implement table partitioning. I thought it was odd to use rd_rel->reloftype as a boolean in ATExecAttachPartition, but apparently we do it elsewhere too, so let's leave that complaint for another day. What I also found off in the same function is that we use

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Andres Freund
On 2016-12-19 15:25:42 -0500, Robert Haas wrote: > On Mon, Dec 19, 2016 at 3:13 PM, Peter Eisentraut > wrote: > > On 12/9/16 7:52 AM, Robert Haas wrote: > >> It's kind of ironic, at least IMHO, that the DirectionFunctionCall > >> macros are anything but direct.

Re: [HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-20 Thread Fabien COELHO
I was expecting the 2 combined queries either to be separated in individual queries "SELECT data FROM Stuff WHERE id = ?" or in one large queries with three ?, but not the above result... Oops, I forgot the attachement, see repeat script on 9.6.1 -- Fabien. combined_statement_issue.sql

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Pavel Stehule
2016-12-20 10:28 GMT+01:00 Andres Freund : > On 2016-12-20 01:14:10 -0800, Andres Freund wrote: > > On 2016-12-20 09:59:43 +0100, Pavel Stehule wrote: > > > In this case some benchmark can be very important (and interesting). I > am > > > not sure if faster function execution

Re: [HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-20 Thread Fabien COELHO
I was expecting the 2 combined queries either to be separated in individual queries "SELECT data FROM Stuff WHERE id = ?" or in one large queries with three ?, but not the above result... Oops, I forgot the attachement, see repeat script on 9.6.1 After some quick investigation, I concluded

Re: [HACKERS] Logical Replication WIP

2016-12-20 Thread Erik Rijkers
On 2016-12-20 10:48, Petr Jelinek wrote: Here is another small thing: $ psql -d testdb -p 6972 psql (10devel_logical_replication_20161220_1008_db80acfc9d50) Type "help" for help. testdb=# drop publication if exists xxx; ERROR: unrecognized object type: 28 testdb=# drop subscription if

Re: [HACKERS] increasing the default WAL segment size

2016-12-20 Thread Andres Freund
Hi, On 2016-12-19 15:14:50 +0530, Beena Emerson wrote: > The attached patch removes --with-wal-segsize configure option and adds a > new initdb option --wal-segsize. The module initdb passes the wal-segsize > value into an environment variable which is used to overwrite the guc value > wal_

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Andres Freund
On 2016-12-20 01:14:10 -0800, Andres Freund wrote: > On 2016-12-20 09:59:43 +0100, Pavel Stehule wrote: > > In this case some benchmark can be very important (and interesting). I am > > not sure if faster function execution has significant benefit on Vulcano > > like executor. > > It's fairly to

Re: [HACKERS] Logical Replication WIP

2016-12-20 Thread Petr Jelinek
On 20/12/16 10:41, Erik Rijkers wrote: > On 2016-12-20 09:43, Petr Jelinek wrote: > >> Thanks, this was very useful. We had wrong attribute index arithmetics >> in the place where we verify that replica identities match well enough. > > Well, I spent a lot of time on the whole thing so I am glad

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Heikki Linnakangas
On 12/16/2016 05:48 PM, Robert Haas wrote: On Thu, Dec 15, 2016 at 8:40 AM, Stephen Frost wrote: * Heikki Linnakangas (hlinn...@iki.fi) wrote: On 12/14/2016 04:57 PM, Stephen Frost wrote: * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: On 12/14/16 5:15 AM,

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Pavel Stehule
2016-12-20 9:11 GMT+01:00 Andres Freund : > On 2016-12-19 15:25:42 -0500, Robert Haas wrote: > > On Mon, Dec 19, 2016 at 3:13 PM, Peter Eisentraut > > wrote: > > > On 12/9/16 7:52 AM, Robert Haas wrote: > > >> It's kind of ironic, at least

Re: [HACKERS] increasing the default WAL segment size

2016-12-20 Thread Beena Emerson
Hello Andres, On Tue, Dec 20, 2016 at 1:58 PM, Andres Freund wrote: > Hi, > > On 2016-12-19 15:14:50 +0530, Beena Emerson wrote: > > The attached patch removes --with-wal-segsize configure option and adds a > > new initdb option --wal-segsize. The module initdb passes the

Re: [HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-20 Thread Fabien COELHO
Hello again pgdevs, More investigations conclude that the information is lost by the parser. From a ;-separated string raw_parser returns a List which are directly the nodes of the statements, with empty statements silently skipped. The Node entry of high level statements (*Stmt) does NOT

[HACKERS] BUG: pg_stat_statements query normalization issues with combined queries

2016-12-20 Thread Fabien COELHO
Hello, While investigating a performance issue, I tried to get informations from pg_stat_statements, however I ran into another issue: it seems that when using combined queries pg_stat_statements query normalization does not work properly... 2 queries that should have been mapped to only one

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Andres Freund
On 2016-12-20 09:59:43 +0100, Pavel Stehule wrote: > In this case some benchmark can be very important (and interesting). I am > not sure if faster function execution has significant benefit on Vulcano > like executor. It's fairly to see function calls as significant overhead. In fact, I moved

Re: [HACKERS] Logical Replication WIP

2016-12-20 Thread Erik Rijkers
On 2016-12-20 09:43, Petr Jelinek wrote: Thanks, this was very useful. We had wrong attribute index arithmetics in the place where we verify that replica identities match well enough. Well, I spent a lot of time on the whole thing so I am glad it's not just something stupid I did :) BTW

Re: [HACKERS] pageinspect: Hash index support

2016-12-20 Thread Ashutosh Sharma
Hi All, I have spent some time in reviewing the latest v8 patch from Jesper and could find some issues which i would like to list down, 1) Firstly, the DATA section in the Makefile is referring to "pageinspect--1.6.sql" file and currently this file is missing. +DATA = pageinspect--1.6.sql

Re: [HACKERS] Logical Replication WIP

2016-12-20 Thread Petr Jelinek
On 20/12/16 10:56, Erik Rijkers wrote: > On 2016-12-20 10:48, Petr Jelinek wrote: > > Here is another small thing: > > $ psql -d testdb -p 6972 > psql (10devel_logical_replication_20161220_1008_db80acfc9d50) > Type "help" for help. > > testdb=# drop publication if exists xxx; > ERROR:

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-20 Thread Fujii Masao
On Tue, Dec 20, 2016 at 1:44 AM, Alvaro Herrera wrote: > Fujii Masao wrote: > >> Regarding this feature, there are some loose ends. We should work on >> and complete them until the release. > > Please list these in https://wiki.postgresql.org/wiki/Open_Items so that we >

Re: [HACKERS] Hang in pldebugger after git commit : 98a64d0

2016-12-20 Thread Robert Haas
On Sat, Dec 17, 2016 at 5:46 AM, Amit Kapila wrote: > Yeah, but we are planning to add a generic flag in WaitEvent structure > which can be used for similar purpose. However, as per your > suggestion, I have changed it only for Win32 port. Also, I kept the > change in

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-12-20 Thread Fujii Masao
On Tue, Dec 20, 2016 at 2:46 PM, Michael Paquier wrote: > On Tue, Dec 20, 2016 at 2:31 PM, Masahiko Sawada > wrote: >> Do we need to consider the sorting method and the selecting k-th >> latest LSN method? > > Honestly, nah. Tests are showing

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2016-12-20 Thread Craig Ringer
On 20 December 2016 at 21:59, Robert Haas wrote: > We could implement this by having > some process, like the background writer, > SendProcSignal(PROCSIG_HOUSEKEEPING) to every process in the system > every 10 minutes or so. ... on a rolling basis. Otherwise that'll be

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-12-20 Thread Fujii Masao
On Tue, Dec 20, 2016 at 7:11 AM, Simon Riggs wrote: > On 19 December 2016 at 21:29, Peter Eisentraut > wrote: >> On 12/16/16 8:52 PM, Robert Haas wrote: >>> If the explanation is just a few sentences long, I see no reason not >>> to

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Tom Lane
Robert Haas writes: > Well, I'm hoping there is a way to have a fast-path and a slow-path > without slowing things down too much. Seems like this discussion has veered way off into the weeds. I suggest we confine it to the stated topic; if you want to discuss ways to

Re: [HACKERS] too low cost of Bitmap index scan

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 2:13 AM, Pavel Stehule wrote: > 2016-12-19 23:28 GMT+01:00 Robert Haas : >> On Sat, Dec 17, 2016 at 3:30 AM, Pavel Stehule >> wrote: >> > -> Bitmap Heap Scan on "Zasilka" (cost=5097.39..5670.64

Re: [HACKERS] too low cost of Bitmap index scan

2016-12-20 Thread Pavel Stehule
2016-12-20 13:55 GMT+01:00 Robert Haas : > On Tue, Dec 20, 2016 at 2:13 AM, Pavel Stehule > wrote: > > 2016-12-19 23:28 GMT+01:00 Robert Haas : > >> On Sat, Dec 17, 2016 at 3:30 AM, Pavel Stehule >

[HACKERS] pgstattuple documentation clarification

2016-12-20 Thread Andrew Dunstan
Recently a client was confused because there was a substantial difference between the reported table_len of a table and the sum of the corresponding tuple_len, dead_tuple_len and free_space. The docs are fairly silent on this point, and I agree that in the absence of explanation it is

Re: [HACKERS] Declarative partitioning - another take

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 4:51 AM, Alvaro Herrera wrote: > Amit Langote wrote: > >> diff --git a/src/backend/commands/tablecmds.c >> b/src/backend/commands/tablecmds.c >> index 1c219b03dd..6a179596ce 100644 >> --- a/src/backend/commands/tablecmds.c >> +++

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Amit Kapila
On Tue, Dec 20, 2016 at 7:44 PM, Robert Haas wrote: > On Tue, Dec 20, 2016 at 9:01 AM, Amit Kapila wrote: >> On Tue, Dec 20, 2016 at 7:11 PM, Robert Haas wrote: >>> On Tue, Dec 20, 2016 at 4:51 AM, Amit Kapila

Re: [HACKERS] pageinspect: Hash index support

2016-12-20 Thread Jesper Pedersen
Hi, On 12/20/2016 05:55 AM, Ashutosh Sharma wrote: Attached is the revised patch. Please have a look and let me know your feedback. I have also changed the status for this patch in the upcoming commitfest to "needs review". Thanks. I was planning to submit a new version next week for

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2016-12-20 Thread Tom Lane
Craig Ringer writes: > On 20 December 2016 at 21:59, Robert Haas wrote: >> We could implement this by having >> some process, like the background writer, >> SendProcSignal(PROCSIG_HOUSEKEEPING) to every process in the system >> every 10 minutes or

Re: [HACKERS] pgstattuple documentation clarification

2016-12-20 Thread Tom Lane
Andrew Dunstan writes: > Recently a client was confused because there was a substantial > difference between the reported table_len of a table and the sum of the > corresponding tuple_len, dead_tuple_len and free_space. The docs are > fairly silent on this point, and I

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-12-20 Thread Simon Riggs
On 20 December 2016 at 15:03, Fujii Masao wrote: > API for crash recovery will never be changed. That is, crash recovery needs > neither recovery.trigger nor standby.trigger. When the server starts a crash > recovery without any trigger file, any recovery parameter

Re: [HACKERS] Make pg_basebackup -x stream the default

2016-12-20 Thread Fujii Masao
On Mon, Dec 19, 2016 at 7:51 PM, Vladimir Rusinov wrote: > > On Sat, Dec 17, 2016 at 2:37 PM, Magnus Hagander > wrote: >> >> Attached is an updated patch that does this. As a bonus it simplifies the >> code a bit. I also fixed an error message that I

Re: [HACKERS] Protect syscache from bloating with negative cache entries

2016-12-20 Thread Robert Haas
On Mon, Dec 19, 2016 at 6:15 AM, Kyotaro HORIGUCHI wrote: > Hello, recently one of my customer stumbled over an immoderate > catcache bloat. This isn't only an issue for negative catcache entries. A long time ago, there was a limit on the size of the relcache,

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Heikki Linnakangas
On 12/16/2016 03:31 AM, Michael Paquier wrote: On Thu, Dec 15, 2016 at 9:48 PM, Heikki Linnakangas wrote: The only way to distinguish, is to know about every verifier kind there is, and check whether rolpassword looks valid as anything else than a plaintext password. And we

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 3:11 AM, Andres Freund wrote: > I think a more efficient variant would make the function signature look > something like: > > Datum /* directly returned argument */ > pgfunc( > /* extra information about function call */ >

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 4:51 AM, Amit Kapila wrote: > We have mainly four actions for squeeze operation, add tuples to the > write page, empty overflow page, unlinks overflow page, make it free > by setting the corresponding bit in overflow page. Now, if we don't > log

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 9:01 AM, Amit Kapila wrote: > On Tue, Dec 20, 2016 at 7:11 PM, Robert Haas wrote: >> On Tue, Dec 20, 2016 at 4:51 AM, Amit Kapila wrote: >>> We have mainly four actions for squeeze operation, add

Re: [HACKERS] invalid combination of options "-D - -F t -X stream" in pg_basebackup

2016-12-20 Thread Magnus Hagander
On Tue, Dec 20, 2016 at 6:56 AM, Fujii Masao wrote: > On Tue, Dec 20, 2016 at 1:43 AM, Magnus Hagander > wrote: > > > > > > On Mon, Dec 19, 2016 at 5:39 PM, Fujii Masao > wrote: > >> > >> Hi, > >> > >> Isn't it better to forbid

Re: [HACKERS] Hash Indexes

2016-12-20 Thread Amit Kapila
On Tue, Dec 20, 2016 at 7:11 PM, Robert Haas wrote: > On Tue, Dec 20, 2016 at 4:51 AM, Amit Kapila wrote: >> We have mainly four actions for squeeze operation, add tuples to the >> write page, empty overflow page, unlinks overflow page, make it

Re: [HACKERS] Time to drop old-style (V0) functions?

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 8:44 AM, Andres Freund wrote: >> Advanced Server uses 256, and we had a customer complain recently that >> 256 wasn't high enough. So apparently the use case for functions with >> ridiculous numbers of arguments isn't exactly 0. > > Well, there's a

Re: pg_authid.rolpassword format (was Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol)

2016-12-20 Thread Robert Haas
On Tue, Dec 20, 2016 at 6:37 AM, Heikki Linnakangas wrote: > It's more convenient to carry the type information with the verifier itself, > in backend code, in pg_dump, etc. Sure, you could have a separate "transfer" > text format that has the prefix, and strip it out when the

  1   2   >