Re: [HACKERS] pg_ls_dir & friends still have a hard-coded superuser check

2017-01-27 Thread Simon Riggs
st leads to people exposing themselves unknowingly when they follow the next part which seems like official advice, yet is potentially unsafe: "access can be given to other users via GRANT". 4. Later, work towards a patch. We have some weeks to get this right. I'm willin

Re: [HACKERS] Allow interrupts on waiting standby

2017-01-27 Thread Simon Riggs
On 27 January 2017 at 01:35, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Fri, Jan 27, 2017 at 4:36 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 26 January 2017 at 19:20, Andres Freund <and...@anarazel.de> wrote: >>> I'm personally fine wi

Re: [HACKERS] Allow interrupts on waiting standby

2017-01-27 Thread Simon Riggs
On 26 January 2017 at 20:36, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Simon Riggs wrote: >> On 26 January 2017 at 19:20, Andres Freund <and...@anarazel.de> wrote: > >> > I'm personally fine with going with a CHECK_FOR_INTERRUPTS >> > f

Re: [HACKERS] Speedup twophase transactions

2017-01-27 Thread Simon Riggs
On 27 January 2017 at 11:01, Nikhil Sontakke <nikh...@2ndquadrant.com> wrote: > On 27 January 2017 at 15:37, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 27 January 2017 at 09:59, Nikhil Sontakke <nikh...@2ndquadrant.com> wrote: >>>>> But, I put

Re: [HACKERS] Speedup twophase transactions

2017-01-27 Thread Simon Riggs
fsync out these 2PC XIDs at promote > time as well for their durability. Why? The data files haven't been fsynced either at that point. If there is a bug there it doesn't just affect 2PC. What sequence of actions would cause data loss? -- Simon Riggshttp://www.2ndQuadrant

Re: [HACKERS] Allow interrupts on waiting standby

2017-01-26 Thread Simon Riggs
On 26 January 2017 at 19:20, Andres Freund <and...@anarazel.de> wrote: > On 2017-01-26 12:24:44 -0500, Robert Haas wrote: >> On Thu, Jan 26, 2017 at 7:18 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> > Currently a waiting standby doesn't allow interrup

Re: [HACKERS] Superowners

2017-01-26 Thread Simon Riggs
ration of duty between people that run a service and people that use it. That should include the ability to dump all objects, yet without any security details. And it should allow someone to setup logical replication easily, including both trigger based and new logical replication. And GRANT ON ALL s

Re: [HACKERS] Replication slot xmin is not reset if HS feedback is turned off while standby is shut down

2017-01-26 Thread Simon Riggs
n the attached patch is enough or would you like > to see anything else covered? That looks good, thanks for the patch. Applying in next few hours, barring objections; then backpatching code (without tests). -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

[HACKERS] Allow interrupts on waiting standby

2017-01-26 Thread Simon Riggs
Currently a waiting standby doesn't allow interrupts. Patch implements that. Barring objection, patching today with backpatches. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services interrupt_waiting_standby.v1.p

Re: [HACKERS] [FEATURE PATCH] pg_stat_statements with plans

2017-01-25 Thread Simon Riggs
r than fixed %. Sounds good. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [HACKERS] Superowners

2017-01-24 Thread Simon Riggs
On 24 January 2017 at 13:19, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> So I was thinking about various annoying admin/security issues >> recently, so I came up with this: a new type of user called a >> “superowner”. I

[HACKERS] Superowners

2017-01-24 Thread Simon Riggs
on their database only.** Ordinary roles can only set defaults for themselves. Certain configuration variables cannot be set this way, or can only be set if a superuser issues the command. Only superusers can change a setting for all roles in all databases. Any issues, additions or changes? -- Simon Riggs

Re: [HACKERS] Online enabling of page level checksums

2017-01-23 Thread Simon Riggs
ems most sensible to add the "enable checksums for table" function into VACUUM because then it will happen automatically via autovacuum, or you could force the issue using something like vacuumdb --jobs 4 --enable-checksums That gives you vacuum_delay and a background worker for no effort

Re: [HACKERS] Password identifiers, protocol aging and SCRAM protocol

2017-01-19 Thread Simon Riggs
pg_hba.conf makes that almost impossible, forcing a big bang approach which subsequently may never happen. As a way of solving that problem, another idea would be to make the mechanism session specific depending upon what is stored for a particular user. That allows us to have a single pg_hba.conf en

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-12 Thread Simon Riggs
On 12 January 2017 at 05:49, Fujii Masao <masao.fu...@gmail.com> wrote: > On Wed, Jan 11, 2017 at 7:36 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 11 January 2017 at 09:51, Fujii Masao <masao.fu...@gmail.com> wrote: >> >>>> 5. recovery.con

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-11 Thread Simon Riggs
Having already agreed to remove the two mentioned aspects, I'm just replying to fill in some historical details. On 11 January 2017 at 17:25, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 1/11/17 5:27 AM, Simon Riggs wrote: >> * Renaming primary_* parameters - C

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-11 Thread Simon Riggs
useful, though I am willing to hear other points and/or go with majority view -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-11 Thread Simon Riggs
On 9 January 2017 at 19:50, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 1/1/17 4:14 PM, Simon Riggs wrote: >> OK, so here's the patch, plus doc cleanup patch. > > I don't think this patch is likely to succeed if we throw in more ideas > in every ro

Re: [HACKERS] Block level parallel vacuum WIP

2017-01-09 Thread Simon Riggs
le amount of memory? Does this work negate the other work to allow VACUUM to use > 1GB memory? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-06 Thread Simon Riggs
On 5 January 2017 at 12:43, Stas Kelvich <s.kelv...@postgrespro.ru> wrote: > >> On 5 Jan 2017, at 13:49, Simon Riggs <si...@2ndquadrant.com> wrote: >> >> Surely in this case the master server is acting as the Transaction >> Manager, and it knows the mappin

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-05 Thread Simon Riggs
On 5 January 2017 at 10:21, Stas Kelvich <s.kelv...@postgrespro.ru> wrote: > Thank you for looking into this. > >> On 5 Jan 2017, at 09:43, Simon Riggs <si...@2ndquadrant.com> wrote: >>> >>> GID is now variable sized. You seem to have added this to every

Re: [HACKERS] [PATCH] PostgresNode.pm enhancements, pg_lsn helper, and some more recovery tests

2017-01-04 Thread Simon Riggs
perltidy on top of that to be honest... Should we add that to the makefile? I guess start a new thread if you think so. Anything that helps me check its correct is a good thing AFAICS. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Serv

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-04 Thread Simon Riggs
On 4 January 2017 at 21:20, Simon Riggs <si...@2ndquadrant.com> wrote: > On 31 December 2016 at 08:36, Stas Kelvich <s.kelv...@postgrespro.ru> wrote: >> Here is resubmission of patch to implement logical decoding of two-phase >> transactions (instead of treating them &

Re: [HACKERS] logical decoding of two-phase transactions

2017-01-04 Thread Simon Riggs
add this stuff to > proposed in-core logical replication. > > [1] > https://www.postgresql.org/message-id/EE7452CA-3C39-4A0E-97EC-17A414972884%40postgrespro.ru We'll need some measurements about additional WAL space or mem usage from these approaches. Thanks. -- Simon Riggs

Re: [HACKERS] ALTER SYSTEM for pg_hba.conf

2017-01-04 Thread Simon Riggs
On 4 January 2017 at 20:30, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> My next thought is ALTER SYSTEM support for pg_hba.conf, especially >> since that would make it easier to do a formal test of Haribabu's >>

Re: [HACKERS] Replication/backup defaults

2017-01-04 Thread Simon Riggs
On 3 January 2017 at 12:34, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Jan 2, 2017 at 10:55 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> In the hope of making things better in 10.0, I remove my objection. If >> people want to use wal_level

[HACKERS] ALTER SYSTEM for pg_hba.conf

2017-01-04 Thread Simon Riggs
that the keyword HOST would be replaced by REMOTE and SSL by ENCRYPTION to make it clearer. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] pg_hba_file_settings view patch

2017-01-04 Thread Simon Riggs
ews on that. Perhaps we can include a test/sample pg_hba.conf for use in tests. Since we've had crashes, I suggest running the test 1 times and checks for leaks and crashes. If its safe we can move towards commit. Thanks -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQ

Re: [HACKERS] pg_recvlogical --endpos

2017-01-04 Thread Simon Riggs
be the meat of the feature. Patches 2 and 3 committed for now. Please do everything else on the logical decoding on standby posts. Thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers

Re: [HACKERS] [PATCH] PostgresNode.pm enhancements, pg_lsn helper, and some more recovery tests

2017-01-04 Thread Simon Riggs
09A=ra150fjtkmtqt5q70piqbwytbor3c...@mail.gmail.com > . > > This subset is tracked as https://commitfest.postgresql.org/12/883/ . > > When committed I will update the decoding on standby series to omit > these pre-requisite patches. Committed, thanks. -- Simon Riggshtt

Re: [HACKERS] increasing the default WAL segment size

2017-01-04 Thread Simon Riggs
On 4 January 2017 at 13:57, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Jan 4, 2017 at 3:05 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Strange response. Nothing has been assumed. I asked for tests and you >> provided measurements. > > Sure, of ze

Re: [HACKERS] [PATCH] Rename pg_switch_xlog to pg_switch_wal

2017-01-04 Thread Simon Riggs
efit analysis. +1 This thread is at best a minor cleanup item, yet major work still remains for 10.0. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] increasing the default WAL segment size

2017-01-04 Thread Simon Riggs
On 3 January 2017 at 21:33, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Jan 3, 2017 at 3:38 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 3 January 2017 at 16:24, Robert Haas <robertmh...@gmail.com> wrote: >>> On Jan 3, 2017 at 11:16 AM, Simon

Re: [HACKERS] Measuring replay lag

2017-01-03 Thread Simon Riggs
r modes. I should >> add it was originally designed to be that way by me, so must have been >> changed later. > > You can achieve that with this patch by setting > replication_lag_sample_interval to 0. I wonder why you ignore my mention of the bug in the correct mechanism? -- Simon Ri

Re: [HACKERS] increasing the default WAL segment size

2017-01-03 Thread Simon Riggs
On 3 January 2017 at 16:24, Robert Haas <robertmh...@gmail.com> wrote: > On Jan 3, 2017 at 11:16 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 3 January 2017 at 15:44, Robert Haas <robertmh...@gmail.com> wrote: >>> Yeah. I don't think th

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-03 Thread Simon Riggs
On 3 January 2017 at 16:47, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Jan 3, 2017 at 11:21 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 3 January 2017 at 15:50, Robert Haas <robertmh...@gmail.com> wrote: >>> On Sun, Jan 1, 2017 at 4:14 PM

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-03 Thread Simon Riggs
On 3 January 2017 at 15:50, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Jan 1, 2017 at 4:14 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Trying to fit recovery targets into one parameter was really not >> feasible, though I tried. > > What was th

Re: [HACKERS] increasing the default WAL segment size

2017-01-03 Thread Simon Riggs
as been made; the only discussion is what's in the new patch. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscri

Re: [HACKERS] increasing the default WAL segment size

2017-01-03 Thread Simon Riggs
On 3 January 2017 at 13:45, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Jan 3, 2017 at 6:41 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 2 January 2017 at 21:23, Jim Nasby <jim.na...@bluetreble.com> wrote: >> >>> It's not cle

Re: [HACKERS] increasing the default WAL segment size

2017-01-03 Thread Simon Riggs
WAL file. ISTM that may reveal more work is needed to be handed off to the WALWriter process (or other issues/solutions). Once we have that information we can consider whether to apply this patch, so until then, -1 to apply this, though I am hopeful that this can be applied in this r

Re: [HACKERS] Measuring replay lag

2017-01-03 Thread Simon Riggs
;> helpful in terms of networking overhead. > > For the record, these replies are only sent approximately every > replay_lag_sample_interval (with variation depending on replay speed) > and are only 42 bytes with the new field added. > > [1] > https://www.postgresql.org/mes

Re: [HACKERS] Make pg_basebackup -x stream the default

2017-01-02 Thread Simon Riggs
On 31 December 2016 at 23:47, Michael Paquier <michael.paqu...@gmail.com> wrote: > Other than that the patch looks good to me. Tests pass. +1 -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- S

Re: [HACKERS] Replication/backup defaults

2017-01-02 Thread Simon Riggs
Should we set wal_level = replica or wal_level = logical as the default for 10.0? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make change

Re: [HACKERS] Replication slot xmin is not reset if HS feedback is turned off while standby is shut down

2017-01-02 Thread Simon Riggs
On 21 December 2016 at 13:23, Simon Riggs <si...@2ndquadrant.com> wrote: > Fix it up and I'll commit. Thanks for the report. I was hoping for some more effort from Ants to correct this. I'll commit Craig's new tests for hs feedback before this, so it can go in with a Tap test, so

Re: [HACKERS] [PATCH] PostgresNode.pm enhancements, pg_lsn helper, and some more recovery tests

2017-01-02 Thread Simon Riggs
l it AFAICS. Also can't see anywhere the LSN stuff is used either. No specific problems with the code, just don't want to commit code that isn't used anywhere, yet. I want to commit the extra tests soon, as a stronger test for my recovery.conf changes. -- Simon Riggshttp://www

Re: [HACKERS] Replication/backup defaults

2017-01-02 Thread Simon Riggs
On 2 January 2017 at 09:48, Simon Riggs <si...@2ndquadrant.com> wrote: > I'm willing to assist in a project to allow changing wal_level online > in this release. Please let's follow that path. wal_level looks like one of the easier ones to change without a server restart There

Re: [HACKERS] Replication/backup defaults

2017-01-02 Thread Simon Riggs
must listen to feedback, not just try to blast through it. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Replication/backup defaults

2017-01-02 Thread Simon Riggs
changing wal_level online in this release. Please let's follow that path. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] Replication/backup defaults

2017-01-02 Thread Simon Riggs
On 2 January 2017 at 09:21, Magnus Hagander <mag...@hagander.net> wrote: > > > On Mon, Jan 2, 2017 at 10:17 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> >> On 31 December 2016 at 15:00, Magnus Hagander <mag...@hagander.net> wrote: >> >

Re: [HACKERS] Replication/backup defaults

2017-01-02 Thread Simon Riggs
-linpro.com/ > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers > -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-01-01 Thread Simon Riggs
On 20 December 2016 at 15:11, Simon Riggs <si...@2ndquadrant.com> wrote: > On 20 December 2016 at 15:03, Fujii Masao <masao.fu...@gmail.com> wrote: > >> API for crash recovery will never be changed. That is, crash recovery needs >> neither recovery.trigger nor st

Re: [HACKERS] Replication slot xmin is not reset if HS feedback is turned off while standby is shut down

2016-12-21 Thread Simon Riggs
e ? > > There was a !HotStandbyActive() check there previously, I was not sure > if it was only to avoid sending useless messages or was necessary > because something was not initialized otherwise. Looks like it is not > needed and can be removed. Revised patch that does so attache

Re: [HACKERS] pg_background contrib module proposal

2016-12-20 Thread Simon Riggs
On 9 December 2016 at 13:00, Robert Haas <robertmh...@gmail.com> wrote: > That might be good, because then we wouldn't have to maintain two > copies of the code. So why are there two things at all? Why is this being worked on as well as Peter's patch? What will that give us? --

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-12-20 Thread Simon Riggs
overy parameter settings in > postgresql.conf are ignored. Right? Yes. There are no conceptual changes, just the API. The goals are: visibility and reloading of recovery parameters, removal of special case code. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 S

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-12-19 Thread Simon Riggs
e write that first. The release notes can warn about the old API generating an ERROR, with a multiple links to discussion of how it now works. Merry Christmas everybody, new patch in time for New Year, further discussion welcome -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreS

Re: [HACKERS] Measuring replay lag

2016-12-19 Thread Simon Riggs
't care about cascading standbys giving skewed readings. One > advantage would be that persistent WAL timestamps would still be able > to provide lag estimates if a standby has been down for a while and > was catching up, and this approach can't until it's caught up due to > lack of buffer

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Simon Riggs
On 12 December 2016 at 18:05, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Dec 12, 2016 at 12:16 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 12 December 2016 at 16:52, Robert Haas <robertmh...@gmail.com> wrote: >>> On Mon, Dec 1

Re: [HACKERS] Nested Wait Events?

2016-12-12 Thread Simon Riggs
On 12 December 2016 at 16:52, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Dec 12, 2016 at 11:33 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Last week I noticed that the Wait Event/Locks system doesn't correctly >> describe waits for tuple locks because

[HACKERS] Nested Wait Events?

2016-12-12 Thread Simon Riggs
ling to St.Ives") from current event (e.g. "Waiting for LWLock") -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

2016-12-01 Thread Simon Riggs
On 15 September 2016 at 18:51, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Sep 6, 2016 at 6:04 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 1 September 2016 at 21:28, Simon Riggs <si...@2ndquadrant.com> wrote: >>> So the only way to handle

Re: [HACKERS] XactLockTableWait doesn't set wait_event correctly

2016-12-01 Thread Simon Riggs
On 2 December 2016 at 00:28, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Nov 30, 2016 at 6:50 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Obtaining a tuple lock requires two separate actions: First we do >> LockTuple() and then we do XactLockTableWait

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-12-01 Thread Simon Riggs
On 29 November 2016 at 15:13, Simon Riggs <si...@2ndquadrant.com> wrote: > On 14 November 2016 at 15:50, Robert Haas <robertmh...@gmail.com> wrote: >> On Sat, Nov 12, 2016 at 11:09 AM, Andres Freund <and...@anarazel.de> wrote: >>> I'm very tempted to

Re: [HACKERS] XactLockTableWait doesn't set wait_event correctly

2016-11-30 Thread Simon Riggs
ions welcome. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-29 Thread Simon Riggs
rrent names are > particularly bad, and I think trying to agree on what would be better > could easily sink the whole patch. OK, so we can move forward. Thanks. I'm going to be doing final review and commit this week, at the Developer meeting on Thurs and on Friday, with input in pe

[HACKERS] EvalPlanQual() doesn't follow LockTuple() pattern

2016-11-29 Thread Simon Riggs
in EvalPlanQual and it isn't explained in comments. Simply removing the wait allows the access pattern to follow the documented lock order, and allows regression tests and isolation tests to pass without problem. Perhaps I have made an error there. Thoughts? -- Simon Riggshttp://www

[HACKERS] XactLockTableWait doesn't set wait_event correctly

2016-11-29 Thread Simon Riggs
like a bug since we get different answers from log_lock_wait and wait_event, which is annoying and confusing. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hac

Re: [HACKERS] Proposal for changes to recovery.conf API

2016-11-04 Thread Simon Riggs
Since the "lots of parameters" approach is almost exactly what we have now, I think we should do that first, get this patch committed and then sit back and discuss an improved API and see what downsides it introduces. Further delay on this isn't helpful for other patches going i

Re: [HACKERS] Indirect indexes

2016-10-19 Thread Simon Riggs
rect indexes and WARM and to what extent they overlap. My current understanding is that WARM won't help you if you update parts of a JSON document and/or use GIN indexes, but is effective without needing to add a new index type and will be faster for retrieval than indirect indexes. So everyb

Re: [HACKERS] Indirect indexes

2016-10-19 Thread Simon Riggs
index tuple and thus the size and effectiveness of the index. Perhaps its best to see the restriction to 6byte PKs as both the first phase of implementation and an optimization, rather than an ugly wart. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Supp

Re: [HACKERS] Indirect indexes

2016-10-19 Thread Simon Riggs
he pros and cons of the different index types, but I'm happy we have a wide spread of knowledge now. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresq

Re: [HACKERS] Indirect indexes

2016-10-18 Thread Simon Riggs
> PK from the index tuple, and then querying the PK index to get the > tids). > > In fact, I believe that can work with all index ams supporting index-only > scans. But if you did that, an UPDATE of a b or c would cause a non-HOT update, so would defeat the purpose of indirect

Re: [HACKERS] Indirect indexes

2016-10-18 Thread Simon Riggs
non-HOT updates. Normal UPDATEs that don't change PKs won't generate any changes to VACUUM away, so only actions that remove PK values will cause anything to be collected and removed from indexes. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Supp

Re: [HACKERS] autonomous transactions

2016-10-06 Thread Simon Riggs
it via an alternate mechanism or API, so that it can continue to be used even if the above mechanism changes. We have no need to wait for the perfect solution, even assuming we would ever agree that just one exists. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Develo

Re: [HACKERS] Fast AT ADD COLUMN with DEFAULTs

2016-10-06 Thread Simon Riggs
any value stored for them. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] autonomous transactions

2016-10-06 Thread Simon Riggs
On 7 September 2016 at 20:46, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Sep 3, 2016 at 7:09 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 2 September 2016 at 09:45, Robert Haas <robertmh...@gmail.com> wrote: >>> On Wed, Aug 31, 2016 at 7:20 AM,

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Simon Riggs
ince we know we'll want multiple files, we should be thinking about how to split things up between files. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgr

Re: [HACKERS] Logical tape pause/resume

2016-10-04 Thread Simon Riggs
memory for each run? That can be used during the merge step to avoid merging runs unless the value ranges overlap. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsq

Re: [HACKERS] Choosing parallel_degree

2016-09-14 Thread Simon Riggs
On 14 September 2016 at 14:48, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Sep 1, 2016 at 9:39 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >>>> > Can I change this to a lower setting? I would have done this before >>>> > applying >>

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-14 Thread Simon Riggs
can then scan multiple indexes at once in parallel, all accessing the shmem data structure. We should also find the compression is better when we consider chunks rather than the whole data structure at once. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] kqueue

2016-09-13 Thread Simon Riggs
uot;self-documenting" patches, from multiple sources. I think we should make it a firm requirement to explain what a patch is actually about, with extra points for including with it a test that allows us to validate that. We don't have enough committer time to waste on such things. -- Simon

Re: [HACKERS] Typo in filename identification

2016-09-12 Thread Simon Riggs
On 11 September 2016 at 23:56, Daniel Gustafsson <dan...@yesql.se> wrote: > The IDENTIFICATION filename in src/backend/storage/ipc/dsm_impl.c is > incorrectly labelling the file dsm.c. Patch fixing the typo attached. > > cheers ./daniel Applied, thanks. -- Simon Riggs

Re: [HACKERS] Re: [COMMITTERS] pgsql: Use LEFT JOINs in some system views in case referenced row doesn

2016-09-12 Thread Simon Riggs
On 12 September 2016 at 08:28, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Sep 12, 2016 at 4:19 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 12 September 2016 at 03:27, Michael Paquier >> <michael.paqu...@gmail.com> wrote: >>

Re: [HACKERS] Re: [COMMITTERS] pgsql: Use LEFT JOINs in some system views in case referenced row doesn

2016-09-12 Thread Simon Riggs
On 12 September 2016 at 03:27, Michael Paquier <michael.paqu...@gmail.com> wrote: > So I'd propose the attached for 9.6 and HEAD. The $OP commit was against HEAD only, not against 9.6 Why would we need to backpatch this commit? -- Simon Riggshttp://www.2ndQua

Re: [HACKERS] Re: [COMMITTERS] pgsql: Use LEFT JOINs in some system views in case referenced row doesn

2016-09-12 Thread Simon Riggs
walreceiver * multiple references in the docs and release notes referring to walsender so usage has already been set and I vote to just leave things that way. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] Bug in two-phase transaction recovery

2016-09-09 Thread Simon Riggs
On 8 September 2016 at 11:18, Simon Riggs <si...@2ndquadrant.com> wrote: > On 8 September 2016 at 07:43, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> On Wed, Sep 7, 2016 at 10:48 PM, Stas Kelvich <s.kelv...@postgrespro.ru> >> wrote: >>&

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-09-09 Thread Simon Riggs
I wish we can change the s_s_names syntax of 9.6 to "first k(n1, n2, > n3)" style before 9.6 releasing if we got consensus. Let's see the proposed patch, so we can evaluate the proposal. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] COPY command with RLS bug

2016-09-09 Thread Simon Riggs
th a more comprehensive set of tests rather than just fix the COPY one? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Bug in two-phase transaction recovery

2016-09-08 Thread Simon Riggs
pfree(buf); > } > This one is a good catch. I have checked also the other callers of > ReadTwoPhaseFile but I am not seeing any other leak. That's a leak, > not critical though so applying it only on HEAD would be enough IMO. Far from critical, but backpatched to 9.6 bec

Re: [HACKERS] [COMMITTERS] pgsql: Fix VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

2016-09-08 Thread Simon Riggs
backpatching, just wanted some +1s before I did it. Will do that tomorrow. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] [COMMITTERS] pgsql: Fix VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

2016-09-07 Thread Simon Riggs
On 7 September 2016 at 13:47, Fujii Masao <masao.fu...@gmail.com> wrote: > On Tue, Sep 6, 2016 at 11:41 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Fix VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL >> >> lazy_truncate_heap() was waiting for >> VACUUM_TRUNCATE_

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-07 Thread Simon Riggs
on the estimate gained from pg_stats, while adding 10% for caution. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services vacuum_mem_estimate.v1.patch Description: Binary data -- Sent via pgsql-hackers mailing

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-07 Thread Simon Riggs
e the output of VACUUM VEROBOSE? Can we produce a test that verifies the result patched/unpatched? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgre

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-06 Thread Simon Riggs
On 6 September 2016 at 19:23, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Sep 6, 2016 at 2:16 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> What occurs to me is that we can exactly predict how many tuples we >> are going to get when we autovacuum, since

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-06 Thread Simon Riggs
On 6 September 2016 at 19:09, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Sep 6, 2016 at 2:06 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 6 September 2016 at 19:00, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Robert Haas <robertmh...@gmai

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2016-09-06 Thread Simon Riggs
onsequences on some systems, but it's not the sort of disaster > Robert posits above. Is there a reason we can't use repalloc here? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pg

Re: [HACKERS] Bug in VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

2016-09-06 Thread Simon Riggs
On 6 September 2016 at 11:30, Simon Riggs <si...@2ndquadrant.com> wrote: > In vacuumlazy.c, VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL is described as > being in ms on line 85, yet it is used on line 1759 in a call to > pg_usleep, so is treated as microseconds rather than milliseconds.

Re: [HACKERS] Quorum commit for multiple synchronous replication.

2016-09-06 Thread Simon Riggs
silience > --- +1 "synchronous_method" -> "synchronization_method" I'm concerned about the performance of this code. Can we work out a way of measuring it, so we can judge how successful we are at releasing waiters quickly? Thanks For 9.6 we implemen

[HACKERS] Bug in VACUUM_TRUNCATE_LOCK_WAIT_INTERVAL

2016-09-06 Thread Simon Riggs
sets it? Not a bug, but patch attached anyway. vacuum_truncate_use_lock_timeout.v1.patch -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services vacuum_lock_wait_ms.v1.patch Description: Binary

Re: [HACKERS] LOCK TABLE .. DEFERRABLE

2016-09-06 Thread Simon Riggs
On 1 September 2016 at 21:28, Simon Riggs <si...@2ndquadrant.com> wrote: > So the only way to handle multiple locks is to do this roughly the way > Rod suggests. I've just been reading the VACUUM code and it turns out that we already use Rod's mechanism internally. So on that basis i

Re: [HACKERS] Speedup twophase transactions

2016-09-06 Thread Simon Riggs
On 6 September 2016 at 09:58, Stas Kelvich <s.kelv...@postgrespro.ru> wrote: >> On 06 Sep 2016, at 04:41, Michael Paquier <michael.paqu...@gmail.com> wrote: >> >> On Sat, Sep 3, 2016 at 10:26 PM, Michael Paquier >> <michael.paqu...@gmail.com> wrote: >&

<    1   2   3   4   5   6   7   8   9   10   >