One thing I find annoying when adding new isolation tests is finding
an appropriate place for the new tests, and figuring out whether any
currently defined steps can be reused. If new steps are needed, the
result often seems to be a confusing mess of cryptically named steps,
resulting in something that's not very readable, and only seems to get
worse as more tests are added.
The attached patch allows permutation steps to be written inline,
using the syntax "<session name> { <SQL> }". Thus, for example,
instead of writing this:
permutation "delete1" "insert1" "merge2" "c1" "select2" "c2"
where all the steps are defined elsewhere, the test can be written in
a fully self-contained form like this:
permutation
s1 { DELETE FROM target WHERE key = 1; }
s1 { INSERT INTO target VALUES (1, 'insert1'); }
s2 {
MERGE INTO target t
USING (SELECT 1 as key, 'merge2' as val) s ON s.key = t.key
WHEN NOT MATCHED THEN INSERT VALUES (s.key, s.val)
WHEN MATCHED THEN UPDATE set val = t.val || ' updated by merge2';
}
s1 { COMMIT; }
s2 { SELECT * FROM target; }
s2 { COMMIT; }
That won't be appropriate for all isolation tests (particularly when
there are a large number of permutations sharing the same steps), but
I do think it would be useful a lot of the time when adding new tests.
I'm not proposing to rewrite any existing tests. 0002 (not intended
for commit) does that for a couple of spec files, just to check that
the feature works.
IMO, the inline form is much easier to read, and it makes it easier to
add new tests to existing spec files. It also makes it easier to share
proposed new tests in email threads.
Thoughts?
Regards,
Dean
From 160d9f92140a76e41684bfc2d3522c7986b17e0e Mon Sep 17 00:00:00 2001
From: Dean Rasheed <[email protected]>
Date: Sat, 21 Nov 2026 12:55:12 +0000
Subject: [PATCH v1 1/2] POC: Allow isolation test steps to be declared inline.
---
src/test/isolation/README | 34 +++++++++-
src/test/isolation/isolationtester.c | 94 +++++++++++++++++++++++-----
src/test/isolation/isolationtester.h | 5 +-
src/test/isolation/specparse.y | 42 ++++++++++++-
4 files changed, 155 insertions(+), 20 deletions(-)
diff --git a/src/test/isolation/README b/src/test/isolation/README
index 5818ca50038..be14754d5bf 100644
--- a/src/test/isolation/README
+++ b/src/test/isolation/README
@@ -84,10 +84,11 @@ session <name>
There are normally several "session" parts in a spec file. Each
session is executed in its own connection. A session part consists
- of three parts: setup, teardown and one or more "steps". The per-session
+ of three parts: setup, teardown and zero or more "steps". The per-session
setup and teardown parts have the same syntax as the per-test setup and
teardown described above, but they are executed in each session. The setup
part might, for example, contain a "BEGIN" command to begin a transaction.
+ Session names must be unique within a spec file.
Each step has the syntax
@@ -109,6 +110,31 @@ permutation <step name> ...
or leave others out. Also, each step name can be annotated with some
parenthesized markers, which are described below.
+permutation <session name> { <SQL> } ...
+
+ A permutation may also be written in a self-contained form, where each
+ step is given inline as the name of the session in which it runs, followed
+ by its SQL. For readability, this form is best split over multiple lines,
+ for example:
+
+ permutation
+ s1 { DELETE FROM target WHERE key = 1; }
+ s2 { INSERT INTO target VALUES (1, 'insert1'); }
+ s1 { COMMIT; }
+ s2 { COMMIT; }
+
+ Here the steps are executed in the order written: first the DELETE in
+ session s1, then the INSERT in session s2, and so on. Inline steps are
+ assigned generated names of the form "<session name>[<n>]", where n is a
+ counter that starts at one for each session used in the permutation, so
+ that the names are unique within a permutation and can be identified in
+ the test output. Also, each session name can be annotated with some
+ parenthesized markers, which are described below.
+
+ Note also that inline steps may be mixed with references to named steps
+ within the same permutation (making the permutation no longer
+ self-contained, but potentially reducing duplication of test code).
+
Session and step names are SQL identifiers, either plain or double-quoted.
A difference from standard SQL is that no case-folding occurs, so that
FOO and "FOO" are the same name while FOO and Foo are different,
@@ -160,10 +186,14 @@ discouraged since the extra files are a nuisance for maintenance.
Instead, it's usually possible to stabilize the test output by applying
special markers to some of the step names listed in a permutation line.
-The general form of a permutation entry is
+The general form of a permutation entry is either
<step name> [ ( <marker> [ , <marker> ... ] ) ]
+or
+
+ <session name> [ ( <marker> [ , <marker> ... ] ) ] { <SQL> }
+
where each marker defines a "blocking condition". The step will not be
reported as completed before all the blocking conditions are satisfied.
The possible markers are:
diff --git a/src/test/isolation/isolationtester.c b/src/test/isolation/isolationtester.c
index e64dc020b18..ac71736396a 100644
--- a/src/test/isolation/isolationtester.c
+++ b/src/test/isolation/isolationtester.c
@@ -285,6 +285,21 @@ check_testspec(TestSpec *testspec)
}
}
+ /* Verify that all session names are unique. */
+ for (i = 0; i < testspec->nsessions; i++)
+ {
+ for (j = i + 1; j < testspec->nsessions; j++)
+ {
+ if (strcmp(testspec->sessions[i]->name,
+ testspec->sessions[j]->name) == 0)
+ {
+ fprintf(stderr, "duplicate session name: %s\n",
+ testspec->sessions[i]->name);
+ exit(1);
+ }
+ }
+ }
+
/* Set the session index fields in steps. */
for (i = 0; i < testspec->nsessions; i++)
{
@@ -295,8 +310,9 @@ check_testspec(TestSpec *testspec)
}
/*
- * If we have manually-specified permutations, link PermutationSteps to
- * Steps, and fill in blocker links.
+ * If we have manually-specified permutations, link named PermutationSteps
+ * to Steps, assign inline Steps sessions and names, and fill in blocker
+ * links.
*/
for (i = 0; i < testspec->npermutations; i++)
{
@@ -305,22 +321,72 @@ check_testspec(TestSpec *testspec)
for (j = 0; j < p->nsteps; j++)
{
PermutationStep *pstep = p->steps[j];
- Step **this = (Step **) bsearch(pstep->name,
- allsteps,
- nallsteps,
- sizeof(Step *),
- step_bsearch_cmp);
- if (this == NULL)
+ /* If it's a named PermutationStep, link it to a Step */
+ if (pstep->name)
{
- fprintf(stderr, "undefined step \"%s\" specified in permutation\n",
- pstep->name);
- exit(1);
+ Step **this = (Step **) bsearch(pstep->name,
+ allsteps,
+ nallsteps,
+ sizeof(Step *),
+ step_bsearch_cmp);
+
+ if (this == NULL)
+ {
+ fprintf(stderr, "undefined step \"%s\" specified in permutation\n",
+ pstep->name);
+ exit(1);
+ }
+ pstep->step = *this;
+
+ /* Mark the step used, for check below */
+ pstep->step->used = true;
}
- pstep->step = *this;
- /* Mark the step used, for check below */
- pstep->step->used = true;
+ /* Otherwise, it's an inline Step; assign it to a session */
+ else
+ {
+ for (k = 0; k < testspec->nsessions; k++)
+ {
+ if (strcmp(testspec->sessions[k]->name,
+ pstep->step->sessionname) == 0)
+ {
+ pstep->step->session = k;
+ break;
+ }
+ }
+ if (pstep->step->session < 0)
+ {
+ fprintf(stderr, "undefined session \"%s\" specified by step in permutation\n",
+ pstep->step->sessionname);
+ exit(1);
+ }
+
+ /* Generate a name for the Step (unique within permutation) */
+ k = 1;
+ while (true)
+ {
+ char *name = psprintf("%s[%d]",
+ pstep->step->sessionname, k);
+ bool unique = true;
+
+ for (int s = 0; s < p->nsteps; s++)
+ {
+ if (p->steps[s]->name &&
+ strcmp(p->steps[s]->name, name) == 0)
+ {
+ unique = false;
+ break;
+ }
+ }
+ if (unique)
+ {
+ pstep->name = pstep->step->name = name;
+ break;
+ }
+ k++;
+ }
+ }
}
/*
diff --git a/src/test/isolation/isolationtester.h b/src/test/isolation/isolationtester.h
index a9328a94516..a3717f0499b 100644
--- a/src/test/isolation/isolationtester.h
+++ b/src/test/isolation/isolationtester.h
@@ -32,8 +32,9 @@ typedef struct
struct Step
{
- char *name;
+ char *name; /* initially NULL for an inline step */
char *sql;
+ char *sessionname; /* only used for inline steps */
/* These fields are filled by check_testspec(): */
int session; /* identifies owning session */
bool used; /* has step been used in a permutation? */
@@ -48,7 +49,7 @@ typedef enum
typedef struct
{
- char *stepname;
+ char *stepname; /* initially NULL for an inline step */
PermutationStepBlockerType blocktype;
int num_notices; /* only used for PSB_NUM_NOTICES */
/* These fields are filled by check_testspec(): */
diff --git a/src/test/isolation/specparse.y b/src/test/isolation/specparse.y
index f6b9058e557..f25cc6f2890 100644
--- a/src/test/isolation/specparse.y
+++ b/src/test/isolation/specparse.y
@@ -46,7 +46,7 @@ TestSpec parseresult; /* result of parsing is left here */
%type <str> opt_setup opt_teardown
%type <str> setup
%type <ptr_list> step_list session_list permutation_list opt_permutation_list
-%type <ptr_list> permutation_step_list blocker_list
+%type <ptr_list> permutation_step_list blocker_list opt_steps
%type <session> session
%type <step> step
%type <permutation> permutation
@@ -121,7 +121,7 @@ session_list:
;
session:
- SESSION identifier opt_setup step_list opt_teardown
+ SESSION identifier opt_setup opt_steps opt_teardown
{
$$ = pg_malloc_object(Session);
$$->name = $2;
@@ -132,6 +132,15 @@ session:
}
;
+opt_steps:
+ /* EMPTY */
+ {
+ $$.elements = NULL;
+ $$.nelements = 0;
+ }
+ | step_list
+ ;
+
step_list:
step_list step
{
@@ -155,6 +164,7 @@ step:
$$ = pg_malloc_object(Step);
$$->name = $2;
$$->sql = $3;
+ $$->sessionname = NULL; /* not an inline step */
$$->session = -1; /* until filled */
$$->used = false;
}
@@ -231,6 +241,34 @@ permutation_step:
$$->nblockers = $3.nelements;
$$->step = NULL;
}
+ | identifier sqlblock
+ {
+ $$ = pg_malloc_object(PermutationStep);
+ $$->name = NULL; /* until filled */
+ $$->blockers = NULL;
+ $$->nblockers = 0;
+ $$->step = pg_malloc_object(Step);
+
+ $$->step->name = NULL; /* until filled */
+ $$->step->sql = $2;
+ $$->step->sessionname = $1;
+ $$->step->session = -1; /* until filled */
+ $$->step->used = true;
+ }
+ | identifier '(' blocker_list ')' sqlblock
+ {
+ $$ = pg_malloc_object(PermutationStep);
+ $$->name = NULL; /* until filled */
+ $$->blockers = (PermutationStepBlocker **) $3.elements;
+ $$->nblockers = $3.nelements;
+ $$->step = pg_malloc_object(Step);
+
+ $$->step->name = NULL; /* until filled */
+ $$->step->sql = $5;
+ $$->step->sessionname = $1;
+ $$->step->session = -1; /* until filled */
+ $$->step->used = true;
+ }
;
blocker_list:
--
2.51.0
From 61eb670518c395b49dedfb5892dc5a16255671ef Mon Sep 17 00:00:00 2001
From: Dean Rasheed <[email protected]>
Date: Sat, 21 Nov 2026 12:56:49 +0000
Subject: [PATCH v1 2/2] Test inline isolation test steps.
Not intended to be pushed.
---
.../expected/inline-alter-domain-validate.out | 23 ++++++
.../expected/inline-deadlock-hard.out | 36 +++++++++
src/test/isolation/isolation_schedule | 2 +
.../specs/inline-alter-domain-validate.spec | 28 +++++++
.../isolation/specs/inline-deadlock-hard.spec | 76 +++++++++++++++++++
5 files changed, 165 insertions(+)
create mode 100644 src/test/isolation/expected/inline-alter-domain-validate.out
create mode 100644 src/test/isolation/expected/inline-deadlock-hard.out
create mode 100644 src/test/isolation/specs/inline-alter-domain-validate.spec
create mode 100644 src/test/isolation/specs/inline-deadlock-hard.spec
diff --git a/src/test/isolation/expected/inline-alter-domain-validate.out b/src/test/isolation/expected/inline-alter-domain-validate.out
new file mode 100644
index 00000000000..7356dee4ff0
--- /dev/null
+++ b/src/test/isolation/expected/inline-alter-domain-validate.out
@@ -0,0 +1,23 @@
+Parsed test spec with 3 sessions
+
+starting permutation: s1[1] s2[1] s3[1] s3[2] s1[2] s3[3] s3[4]
+step s1[1]: DO $$ BEGIN PERFORM pg_advisory_lock(8888); END $$;
+step s2[1]: WITH wait AS MATERIALIZED (SELECT pg_advisory_lock(8888)) INSERT INTO alter_domain_validate_t SELECT (-1)::alter_domain_validate_d FROM wait; <waiting ...>
+step s3[1]: ALTER DOMAIN alter_domain_validate_d ADD CONSTRAINT alter_domain_validate_d_pos CHECK (VALUE > 0) NOT VALID;
+step s3[2]: ALTER DOMAIN alter_domain_validate_d VALIDATE CONSTRAINT alter_domain_validate_d_pos; <waiting ...>
+step s1[2]: DO $$ BEGIN PERFORM pg_advisory_unlock(8888); END $$;
+step s2[1]: <... completed>
+step s3[2]: <... completed>
+ERROR: column "a" of table "alter_domain_validate_t" contains values that violate the new constraint
+step s3[3]: SELECT convalidated FROM pg_constraint WHERE conname = 'alter_domain_validate_d_pos';
+convalidated
+------------
+f
+(1 row)
+
+step s3[4]: SELECT count(*) FROM alter_domain_validate_t;
+count
+-----
+ 1
+(1 row)
+
diff --git a/src/test/isolation/expected/inline-deadlock-hard.out b/src/test/isolation/expected/inline-deadlock-hard.out
new file mode 100644
index 00000000000..19d6bfe5a37
--- /dev/null
+++ b/src/test/isolation/expected/inline-deadlock-hard.out
@@ -0,0 +1,36 @@
+Parsed test spec with 8 sessions
+
+starting permutation: s1[1] s2[1] s3[1] s4[1] s5[1] s6[1] s7[1] s8[1] s1[2] s2[2] s3[2] s4[2] s5[2] s6[2] s7[2] s8[2] s8[3] s7[3] s6[3] s5[3] s4[3] s3[3] s2[3] s1[3]
+step s1[1]: LOCK TABLE a1;
+step s2[1]: LOCK TABLE a2;
+step s3[1]: LOCK TABLE a3;
+step s4[1]: LOCK TABLE a4;
+step s5[1]: LOCK TABLE a5;
+step s6[1]: LOCK TABLE a6;
+step s7[1]: LOCK TABLE a7;
+step s8[1]: LOCK TABLE a8;
+step s1[2]: LOCK TABLE a2; <waiting ...>
+step s2[2]: LOCK TABLE a3; <waiting ...>
+step s3[2]: LOCK TABLE a4; <waiting ...>
+step s4[2]: LOCK TABLE a5; <waiting ...>
+step s5[2]: LOCK TABLE a6; <waiting ...>
+step s6[2]: LOCK TABLE a7; <waiting ...>
+step s7[2]: LOCK TABLE a8; <waiting ...>
+step s8[2]: LOCK TABLE a1; <waiting ...>
+step s8[2]: <... completed>
+ERROR: deadlock detected
+step s7[2]: <... completed>
+step s8[3]: COMMIT;
+step s7[3]: COMMIT;
+step s6[2]: <... completed>
+step s6[3]: COMMIT;
+step s5[2]: <... completed>
+step s5[3]: COMMIT;
+step s4[2]: <... completed>
+step s4[3]: COMMIT;
+step s3[2]: <... completed>
+step s3[3]: COMMIT;
+step s2[2]: <... completed>
+step s2[3]: COMMIT;
+step s1[2]: <... completed>
+step s1[3]: COMMIT;
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 3831004b74d..6acf7697fe4 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -1,3 +1,5 @@
+test: inline-alter-domain-validate
+test: inline-deadlock-hard
test: read-only-anomaly
test: read-only-anomaly-2
test: read-only-anomaly-3
diff --git a/src/test/isolation/specs/inline-alter-domain-validate.spec b/src/test/isolation/specs/inline-alter-domain-validate.spec
new file mode 100644
index 00000000000..ed71c5d5417
--- /dev/null
+++ b/src/test/isolation/specs/inline-alter-domain-validate.spec
@@ -0,0 +1,28 @@
+# Test inline permutation steps from alter-domain-validate.spec
+
+setup
+{
+ CREATE DOMAIN alter_domain_validate_d AS int;
+ CREATE TABLE alter_domain_validate_t (a alter_domain_validate_d);
+}
+
+teardown
+{
+ DROP TABLE alter_domain_validate_t;
+ DROP DOMAIN alter_domain_validate_d;
+}
+
+session s1
+session s2
+session s3
+
+permutation
+ s1 { DO $$ BEGIN PERFORM pg_advisory_lock(8888); END $$; }
+ # CoerceToDomain initializes the domain constraint list during executor
+ # startup, before this CTE waits on the advisory lock.
+ s2 { WITH wait AS MATERIALIZED (SELECT pg_advisory_lock(8888)) INSERT INTO alter_domain_validate_t SELECT (-1)::alter_domain_validate_d FROM wait; }
+ s3 { ALTER DOMAIN alter_domain_validate_d ADD CONSTRAINT alter_domain_validate_d_pos CHECK (VALUE > 0) NOT VALID; }
+ s3 { ALTER DOMAIN alter_domain_validate_d VALIDATE CONSTRAINT alter_domain_validate_d_pos; }
+ s1 { DO $$ BEGIN PERFORM pg_advisory_unlock(8888); END $$; }
+ s3 { SELECT convalidated FROM pg_constraint WHERE conname = 'alter_domain_validate_d_pos'; }
+ s3 { SELECT count(*) FROM alter_domain_validate_t; }
diff --git a/src/test/isolation/specs/inline-deadlock-hard.spec b/src/test/isolation/specs/inline-deadlock-hard.spec
new file mode 100644
index 00000000000..f3c27788155
--- /dev/null
+++ b/src/test/isolation/specs/inline-deadlock-hard.spec
@@ -0,0 +1,76 @@
+# Test inline permutation steps from deadlock-hard.spec
+
+setup
+{
+ CREATE TABLE a1 ();
+ CREATE TABLE a2 ();
+ CREATE TABLE a3 ();
+ CREATE TABLE a4 ();
+ CREATE TABLE a5 ();
+ CREATE TABLE a6 ();
+ CREATE TABLE a7 ();
+ CREATE TABLE a8 ();
+}
+
+teardown
+{
+ DROP TABLE a1, a2, a3, a4, a5, a6, a7, a8;
+}
+
+session s1
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s2
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s3
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s4
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s5
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s6
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s7
+setup { BEGIN; SET deadlock_timeout = '100s'; }
+
+session s8
+setup { BEGIN; SET deadlock_timeout = '10ms'; }
+
+# Note: when s8[2] detects the deadlock and fails, s7[2] is released, making
+# it timing-dependent which query completion is received first by the tester.
+# To ensure output stability, add a blocking mark to force s8[2]'s completion
+# to be reported first. There is a second timing dependency, too: the tester
+# might or might not observe s8[2] during its short lock wait state. Apply a
+# dummy blocking mark to s8[2] to ensure it will be reported as "waiting"
+# regardless of that.
+
+permutation
+ s1 { LOCK TABLE a1; }
+ s2 { LOCK TABLE a2; }
+ s3 { LOCK TABLE a3; }
+ s4 { LOCK TABLE a4; }
+ s5 { LOCK TABLE a5; }
+ s6 { LOCK TABLE a6; }
+ s7 { LOCK TABLE a7; }
+ s8 { LOCK TABLE a8; }
+ s1 { LOCK TABLE a2; }
+ s2 { LOCK TABLE a3; }
+ s3 { LOCK TABLE a4; }
+ s4 { LOCK TABLE a5; }
+ s5 { LOCK TABLE a6; }
+ s6 { LOCK TABLE a7; }
+ s7("s8[2]") { LOCK TABLE a8; }
+ s8(*) { LOCK TABLE a1; }
+ s8 { COMMIT; }
+ s7 { COMMIT; }
+ s6 { COMMIT; }
+ s5 { COMMIT; }
+ s4 { COMMIT; }
+ s3 { COMMIT; }
+ s2 { COMMIT; }
+ s1 { COMMIT; }
--
2.51.0