Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
makes sense to try to make it transparent in libpq. I can see it being useful to expose some tools in libpq for it, without a doubt, so clients can do these sorts of things usefully. (There's also another whole new question: how do you pick which alternative server to connect to? But that

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
On 08/08/2014 09:02 AM, Tom Lane wrote: > Craig Ringer writes: >> On 08/08/2014 03:54 AM, Tom Lane wrote: >>> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >>> at all. What happens five years from now when we switch to some other >

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
h up before returning". Doing it the way you describe would make it nearly useless for enabling client-side failover in BDR, where half the point is that it can deal with high latency or intermittently available links to downstream replicas. -- Craig Ringer http://www.2ndQ

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-07 Thread Craig Ringer
f Pg can push the info with the commit confirmation instead of the client having to request it afterwards though. (Note that the client risks waiting forever if server1 didn't send the required commits before it died, but that's where application policy decisions come in). -- Craig Rin

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-09 Thread Craig Ringer
On 08/10/2014 12:54 AM, Andres Freund wrote: > On 2014-08-07 21:02:54 -0400, Tom Lane wrote: >> Craig Ringer writes: >>> On 08/08/2014 03:54 AM, Tom Lane wrote: >>>> FWIW, I think it's a seriously bad idea to expose LSNs in the protocol >>>> at

[HACKERS] HINT: pg_hba.conf changed since last config reload

2014-08-10 Thread Craig Ringer
hat I see around pg_hba.conf from new users, I figure anything that makes it less confusing might be a good thing if there aren't other consequences. Interested in a patch? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-14 Thread Craig Ringer
carrying extra info, that clients can optionally request only if they understand it. Nobody else needs to care or notice that anything's new. The v2 to v3 protocol switch has only now reached the point where it's realistic to to drop v2 support from clients. I'm hardly keen to do

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-18 Thread Craig Ringer
use cases where having this information reported already provides the client with all it needs. It'd only be a tiny part of fully transparent client side failover, sure, but that's not something it's realistic to aim for right from the start. Reporting of a position indicator that can be

Re: [HACKERS] Reporting the commit LSN at commit time

2014-08-19 Thread Craig Ringer
;t see this as overly different. However, granting that it is a problem, the same objection to a GUC applies to this too. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgs

Re: [HACKERS] [Postgres-xc-developers] Trove with PostgreSQL-XC

2014-08-19 Thread Craig Ringer
al public cloud) every time for each customer. > its good if we have both option. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Hokey wrong versions of libpq in apt.postgresql.org

2014-08-19 Thread Craig Ringer
't. FWIW, this is inconsistent with what yum.postgresql.org does - it takes the POLA approach of packing the libpq from the major release configured in the repo. Each major has its own sub-repo. I find it pretty hard to justify installing a 9.3 libpq alongside a 9.1 server myself. -- Craig

Re: [HACKERS] pg_dump refactor patch to remove global variables

2014-08-19 Thread Craig Ringer
small step toward library-ifying pg_dump. Huge +1. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-21 Thread Craig Ringer
gt; anything that works now. Woo hoo! Funny that this started with "hey, here's a simple, non-invasive function for looking up the status of an arbitrary xid". Mature, complex systems eh? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7

Re: [HACKERS] Logical decoding on standby

2017-03-21 Thread Craig Ringer
On 21 March 2017 at 09:05, Craig Ringer wrote: > Thanks, that's a helpful point. The commit in question is 978b2f65. I > didn't notice that it introduced XLogReader use in twophase.c, though > I should've realised given the discussion about fetching recent 2pc > info

Re: [HACKERS] Logical decoding on standby

2017-03-21 Thread Craig Ringer
Hi all Updated timeline following patch attached. There's a change in read_local_xlog_page to ensure we maintain ThisTimeLineID properly, otherwise it's just comment changes. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-21 Thread Craig Ringer
On 22 March 2017 at 09:49, Craig Ringer wrote: >> Overall, though, I think that 0001 looks far better than any previous >> iteration. It's simple. It looks safe. It seems unlikely to break >> anything that works now. Woo hoo! > > Funny that this started wi

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-03-21 Thread Craig Ringer
able to _restore_ dumps made with Pg10's pg_dumpall and pg_dump. We don't have any output-compatibility in pg_dump to limit it to features from some $older_release . But ... you can always backpatch it yourself into older Pg and build your own pg_dump and pg_dumpall. -- Craig Ringer

[HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-22 Thread Craig Ringer
agnostic data, so they're retained, just muted to logs. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From e944b3839d6ffec03040ccee1309c10057b12459 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Wed, 22 Mar 2

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 22 March 2017 at 10:51, Craig Ringer wrote: > Hi all > > Updated timeline following patch attached. > > There's a change in read_local_xlog_page to ensure we maintain > ThisTimeLineID properly, otherwise it's just comment changes. OK, so we're looking OK with

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 22 March 2017 at 21:06, Andres Freund wrote: > Hi, > > On 2017-03-21 09:05:26 +0800, Craig Ringer wrote: >> > 0002 should be doable as a whole this release, I have severe doubts that >> > 0003 as a whole has a chance for 10 - the code is in quite a raw shape, >&g

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
7;s done right, but I can't do that alone, and it's hard to progress when I don't know what will be expected until it's too late to do anything about it. I guess all we can do at this point is get the foundations in place and carry on for Pg 11, where the presence of in-core l

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 07:31, Andres Freund wrote: > On 2017-03-23 06:55:53 +0800, Craig Ringer wrote: >> I was thinking that by disallowing snapshot use and output plugin >> invocation we'd avoid the need to support cancellation on recovery >> conflicts, etc, simpl

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Craig Ringer
urrent transaction" I'm fine with "current transaction". Though I think it's kind of a moot point as I don't see any reason for an application to ever be doing the equivalent of txid_status(txid_current()) in the first place. But it's harmless enough. -- Craig Ri

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Craig Ringer
ere in the midst > checking, so labeling it as in progress is fine. That seems clear and sensible. XidInMVCCSnapshot simply tests TransactionIdPrecedes(xid, snapshot->xmin), as does HeapTupleSatisfiesHistoricMVCC . It seems reasonable enough to just examine the snapshot xmin directly in txid

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
a lot simpler and they help Pg's behaviour with slots match user expectations for how the rest of Pg behaves, i.e. if it's on the master it'll be on the replica too. And as you've said, decoding on standby is a nice-to-have, wheras I think some kind of HA support is rather more

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 11:25, Craig Ringer wrote: > Amended patch attached, with added TAP test included. Managed to omit it, sigh. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services 011_crash_recovery.pl Descriptio

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >> slots on standbys in

Re: [HACKERS] Logical decoding on standby

2017-03-22 Thread Craig Ringer
On 23 March 2017 at 12:41, Andres Freund wrote: > On 2017-03-23 12:14:02 +0800, Craig Ringer wrote: >> On 23 March 2017 at 09:39, Andres Freund wrote: >> > I still think decoding-on-standby is simply not the right approach as >> > the basic/first HA approach for logi

Re: [HACKERS] Logical decoding on standby

2017-03-23 Thread Craig Ringer
On 23 March 2017 at 00:13, Simon Riggs wrote: > On 22 March 2017 at 08:53, Craig Ringer wrote: > >> I'm splitting up the rest of the decoding on standby patch set with >> the goal of getting minimal functionality for creating and managing >> slots on standbys in

Re: [HACKERS] Logical decoding on standby

2017-03-23 Thread Craig Ringer
On 23 March 2017 at 16:07, Craig Ringer wrote: > If preferred I can instead add > > proc.h: > > #define PROC_RESERVED 0x20 > > procarray.h: > > #define PROCARRAY_REPLICATION_SLOTS 0x20 > > and then test for (flags & PROCARRAY_REPLICATION_SLOTS) Attac

Re: [HACKERS] GSOC'17 project introduction: Parallel COPY execution with errors handling

2017-03-23 Thread Craig Ringer
something pretty performance-sensitive like COPY. I guess it'd be worth setting out your goals first. Do you want to handle all the kinds of problems above? Malformed rows, rows with malformed field values, and rows that fail to satisfy a constraint? or just some subset? -- Craig Ring

Re: [HACKERS] Measuring replay lag

2017-03-23 Thread Craig Ringer
in StartLogicalReplication, and the update of sentPtr in XLogSendLogical . It's a bit misleading, since it hasn't *sent* anything, it buffers it until commit. But it's useful. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Service

Re: [HACKERS] ICU integration

2017-03-23 Thread Craig Ringer
On 24 March 2017 at 04:54, Peter Eisentraut wrote: > Fixed. Congratulations on getting this done. It's great work, and it'll make a whole class of potential bugs and platform portability warts go away if widely adopted. -- Craig Ringer http://www.2n

Re: [HACKERS] Logical decoding on standby

2017-03-23 Thread Craig Ringer
On 23 March 2017 at 17:44, Craig Ringer wrote: Minor update to catalog_xmin walsender patch to fix failure to parenthesize definition of PROCARRAY_PROC_FLAGS_MASK . This one's ready to go. Working on drop slots on DB drop now. -- Craig Ringer http://www.2ndQuadran

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-23 Thread Craig Ringer
On 24 March 2017 at 02:29, Robert Haas wrote: > On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer wrote: >> Changes made per discussion. > > Committed 0001. Much appreciated. Here's the 2nd patch rebased on top of master, with the TAP test included this time. Looks re

Re: [HACKERS] BUG: pg_dump generates corrupted gzip file in Windows

2017-03-24 Thread Craig Ringer
Is there anywhere you expected to find info in the docs? Do you know of a way to detect in Windows if the output stream is not 8-bit clean from within the application program? ... other? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training &

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-24 Thread Craig Ringer
On 25 March 2017 at 07:31, Peter Eisentraut wrote: > On 3/24/17 02:27, Craig Ringer wrote: >> On 24 March 2017 at 02:29, Robert Haas wrote: >>> On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer >>> wrote: >>>> Changes made per discussion. >>

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-25 Thread Craig Ringer
esult* from the next pending query, since it relates to result processing rather than query dispatch. But that's about all the opinion I have here. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-26 Thread Craig Ringer
On 20 March 2017 at 21:47, Stas Kelvich wrote: > >> On 20 Mar 2017, at 16:39, Craig Ringer wrote: >> >> On 20 March 2017 at 20:57, Stas Kelvich wrote: >>> >>>> On 20 Mar 2017, at 15:17, Craig Ringer wrote: >>>> >>>>>

Re: [HACKERS] Logical decoding on standby

2017-03-26 Thread Craig Ringer
seeing the implications as clearly as you do. Do you mean we should restart the whole scan of the slot array if we drop any slot? That'll be O(n log m) but since we don't expect to be working on a big array or a lot of slots it's unlikely to matter. The patch coming soon will as

Re: [HACKERS] Time to drop old-style (V0) functions?

2017-03-26 Thread Craig Ringer
g > PG_FUNCTION_INFO_V1(%s).", funcname) Doesn't make sense unless we then make it singlular, IMO, otherwise it just reads weirdly. I'd rather keep the 'funcname'. If you're writing C code it shouldn't be too much of an ask. > @@ -270,14 +269,16 @@ widge

Re: [HACKERS] Time to drop old-style (V0) functions?

2017-03-26 Thread Craig Ringer
On 27 March 2017 at 10:45, Craig Ringer wrote: > Passes "make check" and recovery tests, check-world running now. A couple of fixes pending. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent

Re: [HACKERS] Time to drop old-style (V0) functions?

2017-03-26 Thread Craig Ringer
On 27 March 2017 at 10:59, Craig Ringer wrote: > On 27 March 2017 at 10:45, Craig Ringer wrote: > >> Passes "make check" and recovery tests, check-world running now. > > A couple of fixes pending. Updated. I didn't have any way to make

Re: [HACKERS] Logical decoding on standby

2017-03-26 Thread Craig Ringer
Appropriate docs changes have been made and tests added. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From c126a10e40aba0c39a43a97da591492d6240659c Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Wed, 22 Mar 2017 13

Re: [HACKERS] PATCH: Batch/pipelining support for libpq

2017-03-27 Thread Craig Ringer
On 27 March 2017 at 15:26, Michael Paquier wrote: > On Sat, Mar 25, 2017 at 9:50 PM, Craig Ringer wrote: >> I'm fairly confident that I overlooked single row mode entirely in the >> original patch, though it's long enough ago that it's hard for me to >> re

Re: [HACKERS] Logical decoding on standby

2017-03-27 Thread Craig Ringer
On 27 March 2017 at 14:08, Craig Ringer wrote: > So this patch makes ReplicationSlotAcquire check that the slot > database matches the current database and refuse to acquire the slot > if it does not. New patch attached that drops above requirement, so slots can still be dropped fr

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-27 Thread Craig Ringer
On 27 March 2017 at 09:31, Craig Ringer wrote: > We're in the last week of the CF. If you have a patch that's nearly > ready or getting there, now would be a good time to post it for help > and input from others. > > I would really like to get this in, but we're ru

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-27 Thread Craig Ringer
I think that's how we'll need to go. Plugins can either defer processing on all 2pc xacts with catalog changes, or lock the xact. It's not perfect, but it's far from unreasonable when you consider that plugins would only be locking 2pc xacts where they expect the result of logical decoding to influence the commit/abort decision, so we won't be doing a commit/abort until we finish decoding the prepare anyway. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical decoding on standby

2017-03-27 Thread Craig Ringer
On 27 March 2017 at 16:20, Simon Riggs wrote: > On 27 March 2017 at 09:03, Craig Ringer wrote: > >> I think this one's ready to go. > > Looks like something I could commit. Full review by me while offline > today, aiming to commit tomorrow barring issues raised. Grea

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-27 Thread Craig Ringer
check this sort of thing, mixed DDL/DML, overlapping xacts, interleaved prepared xacts, etc. If they highlight problems they'll be useful for the next iteration of this patch anyway. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & S

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-27 Thread Craig Ringer
ck > behaviour > when we loose info whether prepare was sent to subscriber or not. Great, thanks. I'll try to have some TAP tests ready. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsq

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-27 Thread Craig Ringer
xacts with catalog changes to test_decoding, I cannot reproduce the expected locking issues so far. See tests in attached updated version, in contrib/test_decoding/sql/prepare.sql . Haven't done any TAP tests yet, since the pg_regress tests are so far sufficient to turn up issues. -- Craig Ri

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-27 Thread Craig Ringer
On 28 March 2017 at 10:53, Craig Ringer wrote: > However, even once I add an option to force decoding of 2pc xacts with > catalog changes to test_decoding, I cannot reproduce the expected > locking issues so far. See tests in attached updated version, in > contrib/test_decoding/sql

Re: [HACKERS] Logical decoding on standby

2017-03-28 Thread Craig Ringer
move the docs changes regarding dropping slots becoming db-specific, so I'll post a follow-up for that in a sec. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Logical decoding on standby

2017-03-28 Thread Craig Ringer
On 29 March 2017 at 08:01, Craig Ringer wrote: > I just notice that I failed to remove the docs changes regarding > dropping slots becoming db-specific, so I'll post a follow-up for that > in a sec. Attached. -- Craig Ringer http://www.2ndQuadrant.co

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-28 Thread Craig Ringer
n we have to. Also, the patch has unsolved issues with how it keeps track of whether an xact was output at prepare time or not and suppresses output at commit time. I'm inclined to shelve the patch for Pg 10. We've only got a couple of days left, the tests are still pretty minimal. We have open issues around locking, less than totally satisfactory abort handling, and potential to skip replay of transactions for both prepare and commit prepared. It's not ready to go. However, it's definitely to the point where with a little more work it'll be practical to patch into variants of Pg until we can mainstream it in Pg 11, which is nice. -- Craig Ringer

Re: [HACKERS] On How To Shorten the Steep Learning Curve Towards PG Hacking...

2017-03-28 Thread Craig Ringer
place, e.g. that you don't need to and shouldn't LWLockRelease() before elog(ERROR). That SPI doesn't manage snapshots or xacts for you (but will often silently work anyway!). etc. I've long intended to start a blog series on postgresql innards concepts, partly

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-28 Thread Craig Ringer
On 29 March 2017 at 08:57, Peter Eisentraut wrote: > On 3/22/17 03:18, Craig Ringer wrote: >> Trivial patch to change 'diag' to 'note' in TAP tests in >> src/test/recovery attached. >> >> It'll reduce the test output a little. > > Commi

[HACKERS] Fix obsolete comment in GetSnapshotData

2017-03-28 Thread Craig Ringer
aXmin: the global xmin for non-catalog tables * >= RecentGlobalXmin * -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To mak

Re: [HACKERS] Logical decoding on standby

2017-03-29 Thread Craig Ringer
On 29 March 2017 at 08:11, Craig Ringer wrote: > On 29 March 2017 at 08:01, Craig Ringer wrote: > >> I just notice that I failed to remove the docs changes regarding >> dropping slots becoming db-specific, so I'll post a follow-up for that >> in a sec. > > At

Re: [HACKERS] Logical decoding on standby

2017-03-29 Thread Craig Ringer
On 29 March 2017 at 16:44, Craig Ringer wrote: > * Split oldestCatalogXmin tracking into separate patch Regarding this, Simon raised concerns about xlog volume here. It's pretty negligible. We only write a new record when a vacuum runs after catalog_xmin advances on the slot

Re: [HACKERS] Logical decoding on standby

2017-03-29 Thread Craig Ringer
On 29 March 2017 at 23:13, Simon Riggs wrote: > On 29 March 2017 at 10:17, Craig Ringer wrote: >> On 29 March 2017 at 16:44, Craig Ringer wrote: >> >>> * Split oldestCatalogXmin tracking into separate patch >> >> Regarding this, Simon raised concerns about

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Craig Ringer
On 30 March 2017 at 11:34, Craig Ringer wrote: > On 29 March 2017 at 23:13, Simon Riggs wrote: >> On 29 March 2017 at 10:17, Craig Ringer wrote: >>> On 29 March 2017 at 16:44, Craig Ringer wrote: >>> >>>> * Split oldestCatalogXmin tracking into separ

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-30 Thread Craig Ringer
> $(PG_PROVE_FLAGS),$(PG_PROVE_FLAGS),--verbose) > And then it is possible to enforce the flag for noisy tests. That's what I'd like to go for. Even default to it, so long as I can override on command line. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Craig Ringer
On 31 March 2017 at 01:16, Andres Freund wrote: > On 2017-03-29 08:01:34 +0800, Craig Ringer wrote: >> On 28 March 2017 at 23:22, Andres Freund wrote: >> >> >> --- a/doc/src/sgml/protocol.sgml >> >> +++ b/doc/src/sgml/protocol.sgml >> >> @@ -2

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Craig Ringer
rmulations to be long and hard to read. Expressing it as "if validity has changed or value has increased" made more sense. Agree order should change. >> diff --git a/src/include/access/transam.h b/src/include/access/transam.h >> index d25a2dd..a4ecfb7 100644 >> --- a/src/i

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-30 Thread Craig Ringer
he detailed test output in tmp_check/logs. to src/test/perl/README too? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscript

Re: [HACKERS] Logical decoding on standby

2017-03-30 Thread Craig Ringer
On 31 March 2017 at 12:49, Craig Ringer wrote: > On 31 March 2017 at 01:16, Andres Freund wrote: >> The comment and code don't quite square to me - it's far from obvious >> that LogStandbySnapshot does something like that. I'd even say it's a >> bad i

Re: [HACKERS] [PATCH] Transaction traceability - txid_status(bigint)

2017-03-31 Thread Craig Ringer
On 31 Mar. 2017 22:31, "David Steele" wrote: On 3/25/17 12:12 AM, Peter Eisentraut wrote: > I'm wondering if this is a perl version/platform issue around >> >> $tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/; >> >> where we're not recognising the required output from psql when we get it. >>

Re: [HACKERS] Fix obsolete comment in GetSnapshotData

2017-04-02 Thread Craig Ringer
On 31 March 2017 at 21:59, Robert Haas wrote: > On Wed, Mar 29, 2017 at 12:00 AM, Craig Ringer wrote: >> There's an outdated reference to GetOldestXmin(true, true) in >> GetSnapshotData. It hasn't had that call signature for a long while >> now. Update the

Re: [HACKERS] delta relations in AFTER triggers

2017-04-02 Thread Craig Ringer
ested and give > people interested in other PLs better visibility. > > That, and I suspect that people will start using this infrastructure > for some very cool projects. Yeah, I was excited to see this committed. It practically begs to be used for plpgsql TABLE valued variables backe

Re: [HACKERS] Logical decoding on standby

2017-04-02 Thread Craig Ringer
On 31 March 2017 at 12:49, Craig Ringer wrote: > On 31 March 2017 at 01:16, Andres Freund wrote: >>> @@ -9633,6 +9643,12 @@ xlog_redo(XLogReaderState *record) >>> SetTransactionIdLimit(checkPoint.oldestXid, >&

Re: [HACKERS] Logical decoding on standby

2017-04-03 Thread Craig Ringer
On 3 April 2017 at 13:46, Craig Ringer wrote: > OK, updated catalog_xmin logging patch attached. Ahem, that should be v5. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 7f742f582e1f6f8f23c4e9d78cd0298180e53

Re: [HACKERS] Logical decoding on standby

2017-04-03 Thread Craig Ringer
On 3 April 2017 at 15:27, Craig Ringer wrote: > On 3 April 2017 at 13:46, Craig Ringer wrote: > >> OK, updated catalog_xmin logging patch attached. > > Ahem, that should be v5. ... and here's v6, which returns to the separate xl_xact_catalog_xmin_advance approach. pgin

Re: [HACKERS] Rewriting the test of pg_upgrade as a TAP test

2017-04-03 Thread Craig Ringer
> Patch has been tested on macos and Windows. > > I am parking that in the next commit fest. Great. Count me in as reviewer, and feel free to poke me if I get caught up in other things. I'd like to see us adopting TAP for cross-version stuff in pg_dump etc too, down the track. -- Cr

Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Craig Ringer
On 4 Apr. 2017 14:22, "Andres Freund" wrote: On 2017-01-05 03:12:09 +, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Magnus Hagander > > For the pg_ctl changes, we're going from removing all privilieges from

Re: [HACKERS] Logical decoding on standby

2017-04-04 Thread Craig Ringer
for 03 and tidying them up, but 03 should allow 01 and 02 to be reviewed in their proper context now. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.or

Re: [HACKERS] Logical decoding on standby

2017-04-04 Thread Craig Ringer
On 4 April 2017 at 22:32, Craig Ringer wrote: > Hi all > > Here's the final set of three patches on top of what's already committed. > > The first is catalog_xmin logging, which is unchanged from the prior post. > > The 2nd is support for conflict with recovery, wit

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-04 Thread Craig Ringer
d improvement >> is only 27% at this point (with 999 tuples). Presumably that's >> because it's constructing a brand new dictionary from scratch for each >> tuple. Taking a look at this now. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreS

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-04 Thread Craig Ringer
On 5 April 2017 at 08:00, Craig Ringer wrote: > Taking a look at this now. Rebased to current master with conflicts and whitespace errors fixed. Review pending. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Servic

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-04 Thread Craig Ringer
On 5 April 2017 at 08:23, Craig Ringer wrote: > On 5 April 2017 at 08:00, Craig Ringer wrote: > >> Taking a look at this now. > > Rebased to current master with conflicts and whitespace errors fixed. > Review pending. This patch fails to update the documen

Re: [HACKERS] Supporting huge pages on Windows

2017-04-04 Thread Craig Ringer
nd its call site. But pg_ctl -w start fails > emitting the following message: That won't work. You'd have to pass 0 to the flags of CreateRestrictedToken and instead supply a PrivilegesToDelete array. You'd probably GetTokenInformation and AND with a mask of ones you wanted to

Re: [HACKERS] Logical decoding on standby

2017-04-05 Thread Craig Ringer
On 5 April 2017 at 04:19, Andres Freund wrote: > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: >> I'm much happier with this. I'm still fixing some issues in the tests >> for 03 and tidying them up, but 03 should allow 01 and 02 to be >> reviewed in their pro

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-05 Thread Craig Ringer
On 6 April 2017 at 09:40, Jim Nasby wrote: > On 4/4/17 7:44 PM, Craig Ringer wrote: >> >> The patch crashes in initdb with --enable-cassert builds: > > > Thanks for the review! I'll get to the rest of it in a bit, but I'm unable > to reproduce the initdb failu

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-05 Thread Craig Ringer
On 6 April 2017 at 11:46, Craig Ringer wrote: > On 6 April 2017 at 09:40, Jim Nasby wrote: >> On 4/4/17 7:44 PM, Craig Ringer wrote: >>> >>> The patch crashes in initdb with --enable-cassert builds: >> >> >> Thanks for the review! I'll ge

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Craig Ringer
at > what plpythonu is doing. Seems fine to me. Notes on the docs aside, I am pretty happy with this and think it's reasonable to proceed with it for Pg 10. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services --

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Craig Ringer
On 6 April 2017 at 15:38, Craig Ringer wrote: > Notes on the docs aside, I am pretty happy with this and think it's > reasonable to proceed with it for Pg 10. Actually, I'm a bit hesitant about returning a static struct that you expect callers to copy and modify. But it seem

Re: [HACKERS] Logical decoding on standby

2017-04-06 Thread Craig Ringer
On 5 April 2017 at 23:25, Robert Haas wrote: > On Wed, Apr 5, 2017 at 10:32 AM, Andres Freund wrote: >> On 2017-04-05 17:18:24 +0800, Craig Ringer wrote: >>> On 5 April 2017 at 04:19, Andres Freund wrote: >>> > On 2017-04-04 22:32:40 +0800, Craig Ringer wrote: >

Re: [HACKERS] Faster methods for getting SPI results (460% improvement)

2017-04-06 Thread Craig Ringer
se the mydest > field altogether). We could document that a bit better, but it's really > not complicated. I strongly agree that this is the way DestReceiver creation should be refactored. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 S

Re: [HACKERS] logical replication apply to run with sync commit off by default

2017-04-06 Thread Craig Ringer
g sync appy back on for logical replicas will cause them to send more timely standby status updates, so commits on the upstream can return faster. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hacke

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-07 Thread Craig Ringer
client can easily enough probe for all known methods, and we might as well just tell it up front. This is hardly new. Most servers with neotiable auth, like SMTP, IMAP, etc, have the server supply a list of auth mechs. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Craig Ringer
of them. I'm aware there are some compat concerns there, but existing clients will already have no idea what the scram method is, so now's our chance to lock it in as containing a *list* of permitted methods. Even though to start with it it's hard coded. -- Craig Ringer

Re: [HACKERS] SCRAM authentication, take three

2017-04-09 Thread Craig Ringer
s of confusion for new users. Simple is good here IMO. Let users specify 'scram' and negotiate. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] SCRAM authentication, take three

2017-04-09 Thread Craig Ringer
On 10 April 2017 at 13:57, Craig Ringer wrote: > On 10 April 2017 at 12:34, Michael Paquier wrote: > >> Attached is a patch to hopefully make the discussion progress. I >> simply propose to use sasl as a keyword for pg_hba.conf, on the basis >> that SASL is the proto

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Craig Ringer
On 12 Apr. 2017 03:14, "Andrew Dunstan" wrote: On 04/11/2017 12:08 PM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Apr 11, 2017 at 11:32 AM, Andrew Dunstan >> wrote: >>> This buildfarm run as you can see takes 33m32s, and the Tap tests take a >>> combined 19m52s of that time. >> I don't

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Craig Ringer
On 12 April 2017 at 08:22, Michael Paquier wrote: > On Wed, Apr 12, 2017 at 9:12 AM, Craig Ringer > wrote: >> >> We should also have a debug --no-fsync option for initdb, or maybe allow it >> to take -o options to pass to child postgres so we can pass fsync=off . &g

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Craig Ringer
On 12 April 2017 at 08:22, Michael Paquier wrote: > On Wed, Apr 12, 2017 at 9:12 AM, Craig Ringer > wrote: >> Well, you can get a lot of information on timings in crake's latest >> builds for example, or nightjar's. >> >> Here's an interesting fact

Re: [HACKERS] TAP tests take a long time

2017-04-11 Thread Craig Ringer
On 12 April 2017 at 13:27, Craig Ringer wrote: > On 12 April 2017 at 08:22, Michael Paquier wrote: >> On Wed, Apr 12, 2017 at 9:12 AM, Craig Ringer >> wrote: >>> Well, you can get a lot of information on timings in crake's latest >>> builds for exa

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-04-11 Thread Craig Ringer
may not be the correct field to show the information. It's better than (ab)using 'query' IMO. I'd rather an abbreviated entry to address Tom's concerns about format. 'lrlaunch' or whatever. -- Craig Ringer http://www.2ndQuadrant.com/ Postgre

[HACKERS] TAP: fix undefined var warnings in PostgresNode with timeout

2017-04-12 Thread Craig Ringer
applied to 9.6 and pg 10. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services From 2745f52455c4a0c187ec879f009aac8f7d0e2ce1 Mon Sep 17 00:00:00 2001 From: Craig Ringer Date: Wed, 12 Apr 2017 20:04:34 +0800 Subject: [PATCH]

<    1   2   3   4   5   6   7   8   9   10   >