On 24 March 2017 at 02:29, Robert Haas <robertmh...@gmail.com> wrote:
> On Tue, Mar 21, 2017 at 11:35 PM, Craig Ringer <cr...@2ndquadrant.com> wrote:
>> Changes made per discussion.
>
> Committed 0001.
Much appreciated.
Here's the 2nd patch rebased on top of master, with the TAP test
included this time. Looks ready to go.
I really appreciate the time you've taken to look at this. Point me at
anything from your team you want some outside review on.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
From eca78655c966613b1b98baf9049b5c4d519d375a Mon Sep 17 00:00:00 2001
From: Craig Ringer <cr...@2ndquadrant.com>
Date: Mon, 23 Jan 2017 13:34:02 +0800
Subject: [PATCH] Introduce txid_status(bigint) to get status of an xact
If an application loses its connection while a COMMIT request is in
flight, the backend crashes mid-commit, etc, then an application may
not be sure whether or not a commit completed successfully or was
rolled back. While two-phase commit solves this it does so at a
considerable overhead, so introduce a lighter alternative.
txid_status(bigint) lets an application determine the status of a a
commit based on an xid-with-epoch as returned by txid_current() or
similar. Status may be committed, aborted, in-progress (including
prepared xacts) or null if the xact is too old for its commit status
to still be retained because it has passed the wrap-around epoch
boundary.
Applications must call txid_current() in their transactions to make
much use of this since PostgreSQL does not automatically report an xid
to the client when one is assigned.
Introduces TransactionIdInRecentPast(...) for the use of other
functions that need similar logic in future.
Authors: Craig Ringer, Robert Haas
---
doc/src/sgml/func.sgml | 27 ++++++
src/backend/utils/adt/txid.c | 132 ++++++++++++++++++++++++++++++
src/include/catalog/pg_proc.h | 2 +
src/test/recovery/t/011_crash_recovery.pl | 46 +++++++++++
src/test/regress/expected/txid.out | 68 +++++++++++++++
src/test/regress/sql/txid.sql | 38 +++++++++
6 files changed, 313 insertions(+)
create mode 100644 src/test/recovery/t/011_crash_recovery.pl
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 64f86ce..356fd33 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -17523,6 +17523,10 @@ SELECT collation for ('foo' COLLATE "de_DE");
<primary>txid_visible_in_snapshot</primary>
</indexterm>
+ <indexterm>
+ <primary>txid_status</primary>
+ </indexterm>
+
<para>
The functions shown in <xref linkend="functions-txid-snapshot">
provide server transaction information in an exportable form. The main
@@ -17573,6 +17577,11 @@ SELECT collation for ('foo' COLLATE "de_DE");
<entry><type>boolean</type></entry>
<entry>is transaction ID visible in snapshot? (do not use with subtransaction ids)</entry>
</row>
+ <row>
+ <entry><literal><function>txid_status(<parameter>bigint</parameter>)</function></literal></entry>
+ <entry><type>txid_status</type></entry>
+ <entry>report the status of the given xact - <literal>committed</literal>, <literal>aborted</literal>, <literal>in progress</literal>, or NULL if the xid is too old</entry>
+ </row>
</tbody>
</tgroup>
</table>
@@ -17643,6 +17652,24 @@ SELECT collation for ('foo' COLLATE "de_DE");
</para>
<para>
+ <function>txid_status(bigint)</> reports the commit status of a recent
+ transaction. Applications may use it to determine whether a transaction
+ committed or aborted when the application and database server become
+ disconnected while a <literal>COMMIT</literal> is in progress.
+ The status of a transaction will be reported as either
+ <literal>in progress</>,
+ <literal>committed</>, or <literal>aborted</>, provided that the
+ transaction is recent enough that the system retains the commit status
+ of that transaction. If is old enough that no references to that
+ transaction survive in the system and the commit status information has
+ been discarded, this function will return NULL. Note that prepared
+ transactions are reported as <literal>in progress</>; applications must
+ check <link
+ linkend="view-pg-prepared-xacts"><literal>pg_prepared_xacts</></> if they
+ need to determine whether the xid is a prepared transaction.
+ </para>
+
+ <para>
The functions shown in <xref linkend="functions-commit-timestamp">
provide information about transactions that have been already committed.
These functions mainly provide information about when the transactions
diff --git a/src/backend/utils/adt/txid.c b/src/backend/utils/adt/txid.c
index 772d7c7..5c64e32 100644
--- a/src/backend/utils/adt/txid.c
+++ b/src/backend/utils/adt/txid.c
@@ -21,6 +21,7 @@
#include "postgres.h"
+#include "access/clog.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -28,6 +29,7 @@
#include "miscadmin.h"
#include "libpq/pqformat.h"
#include "postmaster/postmaster.h"
+#include "storage/lwlock.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
#include "utils/snapmgr.h"
@@ -93,6 +95,70 @@ load_xid_epoch(TxidEpoch *state)
}
/*
+ * Helper to get a TransactionId from a 64-bit xid with wraparound detection.
+ *
+ * It is an ERROR if the xid is in the future. Otherwise, returns true if
+ * the transaction is still new enough that we can determine whether it
+ * committed and false otherwise. If *extracted_xid is not NULL, it is set
+ * to the low 32 bits of the transaction ID (i.e. the actual XID, without the
+ * epoch).
+ *
+ * The caller must hold CLogTruncationLock since it's dealing with arbitrary
+ * XIDs, and must continue to hold it until it's done with any clog lookups
+ * relating to those XIDs.
+ */
+static bool
+TransactionIdInRecentPast(uint64 xid_with_epoch, TransactionId *extracted_xid)
+{
+ uint32 xid_epoch = (uint32) (xid_with_epoch >> 32);
+ TransactionId xid = (TransactionId) xid_with_epoch;
+ uint32 now_epoch;
+ TransactionId now_epoch_last_xid;
+
+ GetNextXidAndEpoch(&now_epoch_last_xid, &now_epoch);
+
+ if (extracted_xid != NULL)
+ *extracted_xid = xid;
+
+ if (!TransactionIdIsValid(xid))
+ return false;
+
+ /* For non-normal transaction IDs, we can ignore the epoch. */
+ if (!TransactionIdIsNormal(xid))
+ return true;
+
+ /* If the transaction ID is in the future, throw an error. */
+ if (xid_epoch > now_epoch
+ || (xid_epoch == now_epoch && xid > now_epoch_last_xid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("transaction ID " UINT64_FORMAT " is in the future",
+ xid_with_epoch)));
+
+ /*
+ * ShmemVariableCache->oldestClogXid is protected by CLogTruncationLock,
+ * but we don't acquire that lock here. Instead, we require the caller to
+ * acquire it, because the caller is presumably going to look up the
+ * returned XID. If we took and released the lock within this function, a
+ * CLOG truncation could occur before the caller finished with the XID.
+ */
+ Assert(LWLockHeldByMe(CLogTruncationLock));
+
+ /*
+ * If the transaction ID has wrapped around, it's definitely too old to
+ * determine the commit status. Otherwise, we can compare it to
+ * ShmemVariableCache->oldestClogXid to determine whether the relevant CLOG
+ * entry is guaranteed to still exist.
+ */
+ if (xid_epoch + 1 < now_epoch
+ || (xid_epoch + 1 == now_epoch && xid < now_epoch_last_xid)
+ || TransactionIdPrecedes(xid, ShmemVariableCache->oldestClogXid))
+ return false;
+
+ return true;
+}
+
+/*
* do a TransactionId -> txid conversion for an XID near the given epoch
*/
static txid
@@ -354,6 +420,9 @@ bad_format:
*
* Return the current toplevel transaction ID as TXID
* If the current transaction does not have one, one is assigned.
+ *
+ * This value has the epoch as the high 32 bits and the 32-bit xid
+ * as the low 32 bits.
*/
Datum
txid_current(PG_FUNCTION_ARGS)
@@ -658,3 +727,66 @@ txid_snapshot_xip(PG_FUNCTION_ARGS)
SRF_RETURN_DONE(fctx);
}
}
+
+/*
+ * Report the status of a recent transaction ID, or null for wrapped,
+ * truncated away or otherwise too old XIDs.
+ *
+ * The passed epoch-qualified xid is treated as a normal xid, not a
+ * multixact id.
+ *
+ * If it points to a committed subxact the result is the subxact status even
+ * though the parent xact may still be in progress or may have aborted.
+ */
+Datum
+txid_status(PG_FUNCTION_ARGS)
+{
+ const char *status;
+ uint64 xid_with_epoch = PG_GETARG_INT64(0);
+ TransactionId xid;
+
+ /*
+ * We must protect against concurrent truncation of clog entries to avoid
+ * an I/O error on SLRU lookup.
+ */
+ LWLockAcquire(CLogTruncationLock, LW_SHARED);
+ if (TransactionIdInRecentPast(xid_with_epoch, &xid))
+ {
+ Assert(TransactionIdIsValid(xid));
+
+ if (TransactionIdIsCurrentTransactionId(xid))
+ status = gettext_noop("in progress");
+ else if (TransactionIdDidCommit(xid))
+ status = gettext_noop("committed");
+ else if (TransactionIdDidAbort(xid))
+ status = gettext_noop("aborted");
+ else
+ {
+ /*
+ * The xact is not marked as either committed or aborted in clog.
+ *
+ * It could be a transaction that ended without updating clog or
+ * writing an abort record due to a crash. We can safely assume
+ * it's aborted if it isn't committed and is older than our
+ * snapshot xmin.
+ *
+ * Otherwise it must be in-progress (or have been at the time
+ * we checked commit/abort status).
+ */
+ if (TransactionIdPrecedes(xid, GetActiveSnapshot()->xmin))
+ status = gettext_noop("aborted");
+ else
+ status = gettext_noop("in progress");
+ }
+ }
+ else
+ {
+ status = NULL;
+ }
+ LWLockRelease(CLogTruncationLock);
+
+ if (status == NULL)
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(status));
+}
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0d18ab8..a66d045 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -4978,6 +4978,8 @@ DATA(insert OID = 2947 ( txid_snapshot_xip PGNSP PGUID 12 1 50 0 0 f f f f t
DESCR("get set of in-progress txids in snapshot");
DATA(insert OID = 2948 ( txid_visible_in_snapshot PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "20 2970" _null_ _null_ _null_ _null_ _null_ txid_visible_in_snapshot _null_ _null_ _null_ ));
DESCR("is txid visible in snapshot?");
+DATA(insert OID = 3360 ( txid_status PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 25 "20" _null_ _null_ _null_ _null_ _null_ txid_status _null_ _null_ _null_ ));
+DESCR("commit status of transaction");
/* record comparison using normal comparison rules */
DATA(insert OID = 2981 ( record_eq PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "2249 2249" _null_ _null_ _null_ _null_ _null_ record_eq _null_ _null_ _null_ ));
diff --git a/src/test/recovery/t/011_crash_recovery.pl b/src/test/recovery/t/011_crash_recovery.pl
new file mode 100644
index 0000000..e5d1a0f
--- /dev/null
+++ b/src/test/recovery/t/011_crash_recovery.pl
@@ -0,0 +1,46 @@
+#
+# Tests relating to PostgreSQL crash recovery and redo
+#
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 3;
+
+my $node = get_new_node('master');
+$node->init(allows_streaming => 1);
+$node->start;
+
+my ($stdin, $stdout, $stderr) = ('', '', '');
+
+# Ensure that txid_status reports 'aborted' for xacts
+# that were in-progress during crash. To do that, we need
+# an xact to be in-progress when we crash and we need to know
+# its xid.
+my $tx = IPC::Run::start(
+ ['psql', '-qAt', '-v', 'ON_ERROR_STOP=1', '-f', '-', '-d', $node->connstr('postgres')],
+ '<', \$stdin, '>', \$stdout, '2>', \$stderr);
+$stdin .= q[
+BEGIN;
+CREATE TABLE mine(x integer);
+SELECT txid_current();
+];
+$tx->pump until $stdout =~ /[[:digit:]]+[\r\n]$/;
+
+# Status should be in-progress
+my $xid = $stdout;
+chomp($xid);
+
+is($node->safe_psql('postgres', qq[SELECT txid_status('$xid');]), 'in progress', 'own xid is in-progres');
+
+# Crash and restart the postmaster
+$node->stop('immediate');
+$node->start;
+
+# Make sure we really got a new xid
+cmp_ok($node->safe_psql('postgres', 'SELECT txid_current()'), '>', $xid,
+ 'new xid after restart is greater');
+# and make sure we show the in-progress xact as aborted
+is($node->safe_psql('postgres', qq[SELECT txid_status('$xid');]), 'aborted', 'xid is aborted after crash');
+
+$tx->kill_kill;
diff --git a/src/test/regress/expected/txid.out b/src/test/regress/expected/txid.out
index 802ccb9..015dae3 100644
--- a/src/test/regress/expected/txid.out
+++ b/src/test/regress/expected/txid.out
@@ -254,3 +254,71 @@ SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
(1 row)
COMMIT;
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+SELECT txid_status(:committed) AS committed;
+ committed
+-----------
+ committed
+(1 row)
+
+SELECT txid_status(:rolledback) AS rolledback;
+ rolledback
+------------
+ aborted
+(1 row)
+
+SELECT txid_status(:inprogress) AS inprogress;
+ inprogress
+-------------
+ in progress
+(1 row)
+
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+ txid_status
+-------------
+ committed
+(1 row)
+
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+ txid_status
+-------------
+
+(1 row)
+
+COMMIT;
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+NOTICE: Got expected error for xid in the future
+ test_future_xid_status
+------------------------
+
+(1 row)
+
+ROLLBACK;
diff --git a/src/test/regress/sql/txid.sql b/src/test/regress/sql/txid.sql
index 4aefd9e..bd6decf 100644
--- a/src/test/regress/sql/txid.sql
+++ b/src/test/regress/sql/txid.sql
@@ -59,3 +59,41 @@ SELECT txid_current_if_assigned() IS NULL;
SELECT txid_current() \gset
SELECT txid_current_if_assigned() IS NOT DISTINCT FROM BIGINT :'txid_current';
COMMIT;
+
+-- test xid status functions
+BEGIN;
+SELECT txid_current() AS committed \gset
+COMMIT;
+
+BEGIN;
+SELECT txid_current() AS rolledback \gset
+ROLLBACK;
+
+BEGIN;
+SELECT txid_current() AS inprogress \gset
+
+SELECT txid_status(:committed) AS committed;
+SELECT txid_status(:rolledback) AS rolledback;
+SELECT txid_status(:inprogress) AS inprogress;
+SELECT txid_status(1); -- BootstrapTransactionId is always committed
+SELECT txid_status(2); -- FrozenTransactionId is always committed
+SELECT txid_status(3); -- in regress testing FirstNormalTransactionId will always be behind oldestXmin
+
+COMMIT;
+
+BEGIN;
+CREATE FUNCTION test_future_xid_status(bigint)
+RETURNS void
+LANGUAGE plpgsql
+AS
+$$
+BEGIN
+ PERFORM txid_status($1);
+ RAISE EXCEPTION 'didn''t ERROR at xid in the future as expected';
+EXCEPTION
+ WHEN invalid_parameter_value THEN
+ RAISE NOTICE 'Got expected error for xid in the future';
+END;
+$$;
+SELECT test_future_xid_status(:inprogress + 10000);
+ROLLBACK;
--
2.5.5
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
