Re: [HACKERS] foreign table creation and NOT VALID check constraints

2017-08-01 Thread Simon Riggs
On 1 August 2017 at 08:37, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2017/08/01 15:22, Simon Riggs wrote: >> On 1 August 2017 at 07:16, Amit Langote <langote_amit...@lab.ntt.co.jp> >> wrote: >>> In f27a6b15e656 (9.6 & later), we decided

Re: [HACKERS] SCRAM salt length

2017-08-17 Thread Simon Riggs
ct that this length was chosen based on the example in RFC 5802 > (SCRAM-SHA-1) section 5. But the analogous example in RFC 7677 > (SCRAM-SHA-256) section 3 uses a length of 16. Should we use that instead? 16 preferred, IMHO -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreS

Re: [HACKERS] recovery_target_time = 'now' is not an error but still impractical setting

2017-08-17 Thread Simon Riggs
sign, so I'm not surprised they don't work and have never been tested. Your suggestion of "furthest" is already the default behaviour. Why are you using 'now'? Why would you want to pick a randomly selected end time? recovery_target_time = 'allballs' sounds fun for recovering corrupte

Re: [HACKERS] PostgreSQL - Weak DH group

2017-07-13 Thread Simon Riggs
ed a > GUC along those lines, as well as docs. How does this look? > > It's late in the release cycle, but it would be nice to sneak this into v10. > Using weak 1024 bit DH parameters is arguably a security issue; it was > originally reported as such. There's a work-around for o

Re: [HACKERS] New partitioning - some feedback

2017-07-07 Thread Simon Riggs
ned table seeing 1000 records all with roughly the same name isn't helpful output from \d \d would show tables but not partitions \d would show partitions exist and how many \d+ would show partition details So the information would be available, just at different levels of detail, just as

Re: [HACKERS] New partitioning - some feedback

2017-07-07 Thread Simon Riggs
e shown in \d output, then I'd be happy if they were >> identified as such rather than just 'table' (e.g 'partition table'). > > +1. +1 to remove partitions from \d display With 1000 partitions that would just be annoying -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] Request more documentation for incompatibility of parallelism and plpgsql exec_run_select

2017-07-03 Thread Simon Riggs
y. Surely NO SCROLL and WITH HOLD cursors would work fine? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-26 Thread Simon Riggs
. Passes with and without Nikhil's new test. I plan to apply both patches tomorrow morning my time to give time for further comments. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services 2PC_rework.v2.patch Descrip

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-24 Thread Simon Riggs
the wrong one. Snapshots work differently on standbys - we store all known running xids, so the test still passes correctly, even when overflowed. I'd call that just plain luck. We behave correctly, but for the wrong reasons, at least in this case. > I don't see anything prevent wrong result

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Simon Riggs
te for us to discuss. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Simon Riggs
On 25 April 2017 at 16:28, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> I can't see any reason now why overwriteOK should exist at all. I'm >> guessing that the whole "overwriteOK" idea was an incorrect response >

Re: [HACKERS] StandbyRecoverPreparedTransactions recovers subtrans links incorrectly

2017-04-25 Thread Simon Riggs
On 23 April 2017 at 17:17, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >>> Also, when I fix that, it gets further but still crashes at the same >>> Assert in SubTransSetParent. The proximate cause this time seems to be >&

Re: [HACKERS] Improve the performance of the standby server when dropping tables on the primary server

2017-08-01 Thread Simon Riggs
looking deeper: does this code also run for temp objects? Can it be optimized for that case a little better? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] foreign table creation and NOT VALID check constraints

2017-08-01 Thread Simon Riggs
In what way does it not apply? Do you have a failure case? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Proposal: global index

2017-08-20 Thread Simon Riggs
ll never use global indexes", based upon their absolute uselessness in Oracle. > Then, wherever we're using IndexTupleData in the index AM code, we would > replace it with InMemoryIndexTuple. This should satisfy both your use > case and mine. Global indexes are a subset of indirect indexes

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-23 Thread Simon Riggs
On 23 August 2017 at 08:18, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Aug 23, 2017 at 10:59 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Tue, Aug 22, 2017 at 3:23 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >>> e

Re: [HACKERS] recovery_target_time = 'now' is not an error but still impractical setting

2017-08-18 Thread Simon Riggs
On 18 August 2017 at 07:30, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Aug 17, 2017 at 6:24 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 15 August 2017 at 15:37, Piotr Stefaniak <postg...@piotr-stefaniak.me> >> wrote: >>

Re: [HACKERS] Surjective functional indexes

2017-09-14 Thread Simon Riggs
On 14 September 2017 at 10:42, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: > > > On 13.09.2017 14:00, Simon Riggs wrote: >> >> On 13 September 2017 at 11:30, Konstantin Knizhnik >> <k.knizh...@postgrespro.ru> wrote: >> >>> The on

Re: [HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-09-14 Thread Simon Riggs
formed, so it can be re-tested later or tested on a variety of platforms. Particularly in reorderbuffer, surely we will almost immediately reuse chunks again, so is it worth issuing free() and then malloc() again soon after? Does that cause additional overhead we could also avoid? Could we possibly kee

Re: [HACKERS] Surjective functional indexes

2017-09-14 Thread Simon Riggs
On 14 September 2017 at 16:37, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: > > > On 14.09.2017 13:19, Simon Riggs wrote: >> This works by looking at overall stats, and only looks at the overall >> HOT %, so its too heavyweight and coarse. >>

Re: [HACKERS] Re: [COMMITTERS] pgsql: Perform only one ReadControlFile() during startup.

2017-09-18 Thread Simon Riggs
On 16 September 2017 at 21:27, Andres Freund <and...@anarazel.de> wrote: > On 2017-09-16 15:59:01 -0400, Tom Lane wrote: >> This does not seem like a problem that justifies a system-wide change >> that's much more delicate than you thought. +1 The change/reason ratio is

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add citext_pattern_ops for citext contrib module

2017-09-19 Thread Simon Riggs
gt; simply to test that the result is not null and have done with it. Thoughts? It makes sense to have a fully detailed output in at least one parameter setting. How about use the new psql feature for \if to skip tests if the local is different to the one for which we have detailed test results? -- Simon

Re: [HACKERS] pg_control_recovery() return value when not in recovery

2017-09-18 Thread Simon Riggs
──┘ > (1 row) Yes, that would have made sense for these to be NULL > postgres[14388][1]=# SELECT pg_is_in_recovery(); > ┌───┐ > │ pg_is_in_recovery │ > ├───┤ > │ f │ > └───┘ > (1 row) But not thi

Re: [HACKERS] Process startup infrastructure is a mess

2017-09-14 Thread Simon Riggs
ctor it since we were going to get bugs anyway. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: htt

Re: [HACKERS] Fix a typo in execReplication.c

2017-10-13 Thread Simon Riggs
Thanks for the patch, looks correct to me. >> >> Committed and back-patched to v10. Well spotted both of you! Shows that reading code and correcting comments is useful activity. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Tr

Re: [HACKERS] postgres_fdw super user checks

2017-10-05 Thread Simon Riggs
uld be a useful option on a Foreign Server to allow it to run queries as either the invoker or the owner. We have that choice for functions, so we already have the concept and syntax available. We could have another default at FDW level that specifies what the default is for that type of FDW, and if t

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-09-08 Thread Simon Riggs
to invoke it. Quick prototype seems like it will deliver quite quickly. I couldn't see a reason to use "LET" rather than just "SET" which would be the POLA choice. Thoughts? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

Re: [HACKERS] PoC plpgsql - possibility to force custom or generic plan

2017-09-08 Thread Simon Riggs
to invoke it. Quick prototype seems like it will deliver quite quickly. I couldn't see a reason to use "LET" rather than just "SET" which would be the POLA choice. Thoughts? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Re

Re: [HACKERS] [Proposal] Allow users to specify multiple tables in VACUUM commands

2017-09-04 Thread Simon Riggs
b and VACUUM a; VACUUM b; If we want to keep the code simple we must surely consider whether the patch has any utility. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Fix performance of generic atomics

2017-09-06 Thread Simon Riggs
nce gain. If there is no gain on 2-socket, at least there is no loss either. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.o

Re: [HACKERS] Proposal for changes to recovery.conf API

2017-09-05 Thread Simon Riggs
On 5 September 2017 at 06:47, Daniel Gustafsson <dan...@yesql.se> wrote: >> On 27 Mar 2017, at 10:27, Simon Riggs <si...@2ndquadrant.com> wrote: >> >> On 7 March 2017 at 23:31, Josh Berkus <j...@berkus.org> wrote: >>> On 03/02/2017 07:13 AM, David Stee

Re: [HACKERS] <> join selectivity estimate question

2017-09-06 Thread Simon Riggs
ed, which >> shows a bunch of interesting cases. I'll add this to the commitfest. > > I added some "stable" tests to your patch taking inspiration from the > test SQL file. I think those will be stable across machines and runs. > Please let me know if those look good to you. Why

Re: [HACKERS] [COMMITTERS] pgsql: Add psql variables showing server version and psql version.

2017-09-06 Thread Simon Riggs
t; \else > -- version 11 or more > \endif Based upon input from Tom and Fabien, I propose this additional doc patch. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services clarify_psql_server_version.v1.patc

Re: [HACKERS] [COMMITTERS] pgsql: Add psql variables showing server version and psql version.

2017-09-05 Thread Simon Riggs
e have SERVER_VERSION_NUM but yet it is not set. How does this \if SERVER_VERSION_NUM < x behave if unset? Presumably it fails, even though the version *is* less than x Do we need some macro or suggested special handling? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Deve

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-07 Thread Simon Riggs
parallel_workers = 4; SELECT ... so we still have only one command (the last select), yet we have multiple GUC settings beforehand. Any reason to disallow that? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Se

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-07 Thread Simon Riggs
e new >>> comment in DefineSavepoint to explain why not. > >> Okay. > > Does anyone want to do further review on this patch? If so, I'll > set the CF entry back to "Needs Review". OK, I'll review Michael's patch (and confirm my patch is dead) -- Simon Rigg

Re: [HACKERS] ALTER INDEX .. SET STATISTICS ... behaviour

2017-09-06 Thread Simon Riggs
On 6 September 2017 at 10:27, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> Other than that, I'm good to commit. >> Any last minute objections? > > The docs and comments could stand a bit closer copy-editing by a >

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-07 Thread Simon Riggs
On 7 September 2017 at 11:24, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> On 5 September 2017 at 10:22, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> Does anyone want to do further review on this patch? If so, I'll >&g

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-07 Thread Simon Riggs
On 7 September 2017 at 11:31, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> I would like to relax the restriction to allow this specific use case... >> SET work_mem = X; SET max_parallel_workers = 4; SELECT ... >>

Re: [HACKERS] Race between SELECT and ALTER TABLE NO INHERIT

2017-09-12 Thread Simon Riggs
descriptor upon termination of > inheritance (both the DETACH PARTITION and also DROP TABLE child cases). Is this requirement documented or in comments anywhere? I can't see anything about that, which is a fairly major usage point. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] generated columns

2017-09-13 Thread Simon Riggs
On 13 September 2017 at 09:09, Andreas Karlsson <andr...@proxel.se> wrote: > On 09/13/2017 04:04 AM, Simon Riggs wrote: >> >> On 31 August 2017 at 05:16, Peter Eisentraut >> <peter.eisentr...@2ndquadrant.com> wrote: >>> >>> - index support (and

Re: [HACKERS] generated columns

2017-09-12 Thread Simon Riggs
column level INSERT | UPDATE | DELETE privs on it. The generation happens automatically as part of the write action if stored, or not until select for virtual. It should be possible to have column level SELECT privs. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x

Re: [HACKERS] Surjective functional indexes

2017-09-13 Thread Simon Riggs
ibed in terms of its function, e.g. recheck_on_update Yes, I'd rather not have an option at all, just some simple code with useful effect, like we have in many other places. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Service

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2017-09-06 Thread Simon Riggs
On 1 September 2017 at 22:08, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sat, Sep 2, 2017 at 1:53 AM, Robert Haas <robertmh...@gmail.com> wrote: >> On Mon, Aug 21, 2017 at 5:30 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >>> Thinking ahead, are w

Re: [HACKERS] [COMMITTERS] pgsql: Add psql variables showing server version and psql version.

2017-09-06 Thread Simon Riggs
On 6 September 2017 at 06:38, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> Based upon input from Tom and Fabien, I propose this additional doc patch. > > I do not think any of this is appropriate, particularly not the reference

Re: [HACKERS] ALTER INDEX .. SET STATISTICS ... behaviour

2017-09-06 Thread Simon Riggs
or commiter". This looks useful and good to me. I've changed this line of code to return NULL rather than return tuple if (!HeapTupleIsValid(tuple)) return tuple; Other than that, I'm good to commit. Any last minute objections? -- Simon Riggshttp://www.2ndQuadrant.com/

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2017-09-12 Thread Simon Riggs
then maintain the plugin outside of core until the time when SQL Standard is updated and we can implement directly. We already support the ability to invent new plan nodes, so this could be considered as part of the plugin. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Developme

Re: [HACKERS] MAIN, Uncompressed?

2017-09-12 Thread Simon Riggs
On 29 August 2017 at 07:58, Simon Riggs <si...@2ndquadrant.com> wrote: > On 26 August 2017 at 05:40, Mark Kirkwood <mark.kirkw...@catalyst.net.nz> > wrote: >> On 26/08/17 12:18, Simon Riggs wrote: >> >>> On 25 August 2017 at 20:53, Tom Lane <t...@sss.

Re: [HACKERS] Surjective functional indexes

2017-09-12 Thread Simon Riggs
On 1 September 2017 at 09:47, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: > > On 01.09.2017 09:25, Simon Riggs wrote: >> >> On 1 September 2017 at 05:40, Thomas Munro >> <thomas.mu...@enterprisedb.com> wrote: >>> >>> On Fri, Ju

Re: [HACKERS] PATCH : Generational memory allocator (was PATCH: two slab-like memory allocators)

2017-09-24 Thread Simon Riggs
On 24 September 2017 at 21:32, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > Attached is an updated version of the patch, tweaking the comments. That looks good, thanks. Marking Ready for Committer to give notice before commit. -- Simon Riggshttp://www.2ndQua

Re: [HACKERS] Built-in plugin for logical decoding output

2017-09-24 Thread Simon Riggs
us to evolve that > without worrying about out of core dependencies etc. We plan to submit the next evolution of the code in 2018, in time for the early cycle of PG12. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Service

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2017-08-21 Thread Simon Riggs
ssue. We definitely don't want --no-extension-comments, but --no-comments removes ALL comments just to solve a weird problem. (Meta)Data loss, surely? Thinking ahead, are we going to add a new --no-objecttype switch every time someone wants it? It would make more sense to add something more gener

Re: [HACKERS] [COMMITTERS] pgsql: Account for catalog snapshot in PGXACT->xmin updates.

2017-08-21 Thread Simon Riggs
t did -- ThereAreNoPriorRegisteredSnapshots ...but that was before we had MVCC scans on catalogs, which changed things in unforeseen ways. The right fix is surely to do a more principled approach and renaming of the function so that it reflects the current snapshot types. As Noah mentions ther

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-21 Thread Simon Riggs
r overall cost but a lower time to complete if resources are available. We have the choice of 1) adding a new optimizable quantity, or of 2) treating cost = speed, so we actually reduce the cost of a parallel plan rather than increasing it so it is more likely to be picked. -- Si

Re: [HACKERS] Explicit relation name in VACUUM VERBOSE log

2017-08-22 Thread Simon Riggs
lifiedRelationName() and then control whether we include this new behaviour with log_qualified_object_names = on | off -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@p

Re: [HACKERS] why not parallel seq scan for slow functions

2017-08-22 Thread Simon Riggs
being currently followed for costing of parallel > plans and this patch also tries to follow the same. OK, I understand this better now, thanks. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent vi

Re: [HACKERS] MAIN, Uncompressed?

2017-08-25 Thread Simon Riggs
aster control > knobs. I'm just not in love with Simon's specific proposal, especially > not if we can't think of a better name for it than "MAINU". Extended/External would be just fine if you could set the toast target, so I think a better suggestion would be to make "toa

[HACKERS] MAIN, Uncompressed?

2017-08-25 Thread Simon Riggs
' or 'e' external immediately * 2: Store attributes with attstorage 'x' or 'e' external * 3: Inline compress attributes with attstorage 'm' * 4: Store attributes with attstorage 'm' external * -- */ Not sure what to call this new option? MAINU? Objections? -- Simon Riggs

Re: [HACKERS] MAIN, Uncompressed?

2017-08-25 Thread Simon Riggs
On 25 August 2017 at 12:24, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> It looks like we need a new Column Storage option for MAIN, Uncompressed. >> We have these Column Storage options >> Plain - inline only, uncompresse

Re: [HACKERS] MAIN, Uncompressed?

2017-08-25 Thread Simon Riggs
On 25 August 2017 at 13:21, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> Main is roughly what is wanted, yet it always tries to compress. If >> you already know that won't be useful it should be possible to turn >>

Re: [HACKERS] MAIN, Uncompressed?

2017-08-25 Thread Simon Riggs
On 25 August 2017 at 14:08, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> On 25 August 2017 at 13:21, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> If you know compression isn't useful, but you don't want to fail on >>&

Re: [HACKERS] MAIN, Uncompressed?

2017-08-29 Thread Simon Riggs
On 26 August 2017 at 05:40, Mark Kirkwood <mark.kirkw...@catalyst.net.nz> wrote: > On 26/08/17 12:18, Simon Riggs wrote: > >> On 25 August 2017 at 20:53, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> >>> Greg Stark <st...@mit.edu> writes: >>&g

Re: [HACKERS] More replication race conditions

2017-08-31 Thread Simon Riggs
ers may > discover new open items at any time, and I want to plan to get them all fixed > well in advance of shipping v10. Consequently, I will appreciate your efforts > toward speedy resolution. Thanks. > > [1] > https://www.postgresql.org/message-id/20170404140717.GA2675809%40t

Re: [HACKERS] Statement-level rollback

2017-09-01 Thread Simon Riggs
rent patch. I'll add this >> patch to CommitFest 2017-3. > > This patch needs to be rebased for the upcoming commit fest. I'm willing to review this if the patch is going to be actively worked on. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-01 Thread Simon Riggs
On 1 September 2017 at 08:09, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Fri, Sep 1, 2017 at 3:05 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> I'm not sure I see the use case for anyone using SAVEPOINTs in this >> context, so simply throwing a g

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-01 Thread Simon Riggs
On 1 September 2017 at 15:19, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> I've added tests to the recent patch to show it works. > > I don't think those test cases prove anything (ie, they work fine > on an unpatched se

Re: [HACKERS] More replication race conditions

2017-09-01 Thread Simon Riggs
On 31 August 2017 at 12:54, Simon Riggs <si...@2ndquadrant.com> wrote: >> The above-described topic is currently a PostgreSQL 10 open item. Simon, >> since you committed the patch believed to have created it, you own this open >> item. If some other commit is more

[HACKERS] Release Note changes

2017-09-04 Thread Simon Riggs
o don't see an issue with mentioning it here. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www

Re: [HACKERS] Release Note changes

2017-09-04 Thread Simon Riggs
nal utilities using logical decoding, such as pglogical or > proprietary alternatives, can also provide an alternate route, > often with lower downtime." > > pg_upgrade or pg_dump is really the only option for us using LOs. Sounds like we could change that, or at least add a WAR

Re: [HACKERS] Release Note changes

2017-09-04 Thread Simon Riggs
are third party replication solutions >> that can solve the problem, along with linking to the page with the list, >> perhaps? > > Yeah, that seems fine. A link to our docs page that covers those would be fine. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL De

Re: [HACKERS] Release Note changes

2017-09-04 Thread Simon Riggs
with >> mentioning it here. > > The existing reference is alongside a bunch of other third-party > tools. Mentioning it at the very top of our release notes would give > it a pride of place with which I'm not too comfortable. -- Simon Riggshttp://www.2ndQuadr

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-09-01 Thread Simon Riggs
sing SAVEPOINTs in this context, so simply throwing a good error message is enough. Clearly nobody is using this, so lets just lock the door. I don't think fiddling with the transaction block state machine is anything anybody wants to do in back branches, at least without a better reason than t

Re: [HACKERS] Surjective functional indexes

2017-09-01 Thread Simon Riggs
; Form_pg_attribute att = TupleDescAttr(RelationGetDescr(indexDesc), i); > > Thanks! Does the patch work fully with that change? If so, I will review. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2017-09-05 Thread Simon Riggs
simply truncate without the lock. Would need a few pushups but seems less scary then trying to make pending standby actions work well enough to commit. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pg

[HACKERS] Predicate Locks for writes?

2017-10-07 Thread Simon Riggs
ignore PredicateLockTuple() for cases where we are about to update or delete the found tuple. ISTM that a Before Row Trigger would need to add an SIRead lock since that is clearly a read. Thoughts? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] Predicate Locks for writes?

2017-10-11 Thread Simon Riggs
On 11 October 2017 at 15:33, Robert Haas <robertmh...@gmail.com> wrote: > On Sat, Oct 7, 2017 at 7:26 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> PredicateLockTuple() specifically avoids adding an SIRead lock if the >> tuple already has a write lock on

Re: [HACKERS] The case for removing replacement selection sort

2017-09-27 Thread Simon Riggs
sense to continue to support replacement selection. Forgive me if I missed the explanation, but how will we handle bounded sorts? -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers maili

Re: [HACKERS] Surjective functional indexes

2017-09-27 Thread Simon Riggs
On 15 September 2017 at 16:34, Konstantin Knizhnik <k.knizh...@postgrespro.ru> wrote: > Attached please find yet another version of the patch. Thanks. I'm reviewing it. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA,

[HACKERS] Remove secondary checkpoint

2017-10-24 Thread Simon Riggs
Remove the code that maintained two checkpoint's WAL files and all associated stuff. Try to avoid breaking anything else This * reduces disk space requirements on master * removes a minor bug in fast failover * simplifies code -- Simon Riggshttp://www.2ndQuadrant.com

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-03 Thread Simon Riggs
ecause they allow some. My approach will be to reduce the errors in the best way, not to try to copy errors Oracle makes, if any. But that error avoidance can easily be a later add-on if we prefer it that way. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Develop

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-03 Thread Simon Riggs
On 3 November 2017 at 08:26, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Nov 3, 2017 at 1:05 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >>> Therefore, if MERGE eventually uses INSERT .. ON CONFLICT >>> UPDATE when a relevant unique ind

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-06 Thread Simon Riggs
On 3 November 2017 at 16:35, Peter Geoghegan <p...@bowt.ie> wrote: > Simon Riggs <si...@2ndquadrant.com> wrote: >>>> >>>> The *only* behavioural difference I have proposed would be the *lack* >>>> of an ERROR in (some) concurrent cases. >>

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-06 Thread Simon Riggs
r that an entry exists in the index for a committed row. Since we have a unique index we use it to locate the row we know exists and UPDATE that. We don't use a new MVCC snapshot, we do what EPQ does. EPQ is already violating MVCC for UPDATEs, so why does it matter if we do it for INSERTs al

Re: [HACKERS] SQL procedures

2017-11-06 Thread Simon Riggs
seemed to have one of these two > features in mind. I think that depends on what other SQL systems one > has worked with previously.) Almost all of the meat happens in later patches, so no other review comments. That seems seems strange in a patch of this size, but its true. Procedures are

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
ible to the xact's snapshot). So if I understand you correctly, in your view MERGE should just fail with an ERROR if it runs concurrently with other DML? i.e. if a race condition between the query and an INSERT runs concurrently with another INSERT We have no interest in making that work

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-02 Thread Simon Riggs
On 2 November 2017 at 19:16, Peter Geoghegan <p...@bowt.ie> wrote: > Simon Riggs <si...@2ndquadrant.com> wrote: >> >> So if I understand you correctly, in your view MERGE should just fail >> with an ERROR if it runs concurrently with other DML? > > >

Re: [HACKERS] Statement-level rollback

2017-11-02 Thread Simon Riggs
nd-based solution is required for PL procedures and functions. We could put this as an option into PL/pgSQL, but it seems like it is a function of the transaction manager rather than the driver. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, R

Re: [HACKERS] Remove secondary checkpoint

2017-11-07 Thread Simon Riggs
On 31 October 2017 at 12:01, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Oct 31, 2017 at 2:00 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 30 October 2017 at 18:58, Simon Riggs <si...@2ndquadrant.com> wrote: >>> On 30 Oc

Re: [HACKERS] Transaction control in procedures

2017-11-08 Thread Simon Riggs
really set as described in the .h It would certainly be useful to have some state that allows sanity checking on weird state transitions. What we would benefit from is a README that gives the theory of operation, so everyone can read and agree. Presumably we would need to contact authors of main P

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-01 Thread Simon Riggs
On 31 October 2017 at 18:55, Peter Geoghegan <p...@bowt.ie> wrote: > On Tue, Oct 31, 2017 at 2:25 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> If there are challenges ahead, its reasonable to ask for test cases >> for that now especially if you think you know what t

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-03 Thread Simon Riggs
On 2 November 2017 at 17:06, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Oct 31, 2017 at 5:14 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> I've proposed a SQL Standard compliant implementation that would do >> much more than be new syntax over what we alrea

Re: [HACKERS] MERGE SQL Statement for PG11

2017-11-03 Thread Simon Riggs
fragile, in this case, though I encourage more ideas like that in the future. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes t

Re: [HACKERS] Exclude pg_internal.init from base backup

2017-11-07 Thread Simon Riggs
pg_control is always in the same place. So they're not the same thing. > > So +1 for documenting the difference in how these are handled, as this is > important to know for somebody writing an external tool for it. Changes made, moving to commit the attached patch. -- Simon Riggs

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-28 Thread Simon Riggs
On 28 October 2017 at 20:39, Peter Geoghegan <p...@bowt.ie> wrote: > On Sat, Oct 28, 2017 at 3:10 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> SQL:2011 specifically states "The extent to which an >> SQL-implementation may disallow independent cha

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-29 Thread Simon Riggs
On 28 October 2017 at 22:04, Peter Geoghegan <p...@bowt.ie> wrote: > On Sat, Oct 28, 2017 at 12:49 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Nothing I am proposing blocks later work. > > Actually, many things will block future work if you go down that

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 25 October 2017 at 00:17, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Oct 23, 2017 at 11:20 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Remove the code that maintained two checkpoint's WAL files and all >> associated stuff. >> >&

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 09:44, Robert Haas <robertmh...@gmail.com> wrote: > On Sun, Oct 29, 2017 at 1:19 AM, Simon Riggs <si...@2ndquadrant.com> wrote: >> Nothing I am proposing blocks later work. > > That's not really true. Nobody's going to be happy if MERGE has one >

Re: [HACKERS] MERGE SQL Statement for PG11

2017-10-30 Thread Simon Riggs
We can then use those test cases to drive developments. If we end up with code for multiple approaches we will be able to evaluate the differences between proposals using the test cases produced. -- Simon Riggshttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support,

Re: [HACKERS] Remove secondary checkpoint

2017-10-30 Thread Simon Riggs
On 30 October 2017 at 15:31, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Mon, Oct 30, 2017 at 2:22 PM, Simon Riggs <si...@2ndquadrant.com> wrote: >> On 25 October 2017 at 00:17, Michael Paquier <michael.paqu...@gmail.com> >> wrote: >>> -

Re: [HACKERS] Remove secondary checkpoint

2017-10-24 Thread Simon Riggs
On 24 October 2017 at 09:50, Tom Lane <t...@sss.pgh.pa.us> wrote: > Simon Riggs <si...@2ndquadrant.com> writes: >> Remove the code that maintained two checkpoint's WAL files and all >> associated stuff. > >> Try to avoid breaking anything else > >&

  1   2   3   4   5   6   7   8   9   10   >