Tom Lane wrote: > Alvaro Herrera <alvhe...@alvh.no-ip.org> writes:
> > I think we could solve this by putting in the same parallel group only > > slow tests that mostly sleeps, ie. nothing that would monopolize CPU for > > long enough to cause a problem. Concretely: > > test: timeouts tuplelock-update deadlock-hard deadlock-soft-2 > > OK, but there'd better be a comment there explaining the concern > very precisely, or somebody will break it. Here's a concrete proposal. Runtime is 45.7 seconds on my laptop. It can be further reduced, but not by more than a second or two unless you get in the business of modifying other tests. (I only modified deadlock-soft-2 because it saves 5 seconds). Admittedly the new isolation_schedule file is a bit ugly. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>From f80a2fd9e3320d033017e17a648ac1ca13396ef0 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera <alvhe...@alvh.no-ip.org> Date: Thu, 25 Jan 2018 17:11:03 -0300 Subject: [PATCH] Parallelize isolation tests a bit It is possible to run isolation tests in parallel, which saves quite a bit of runtime (from 76 to 45 seconds in my machine). Test 'timeout' has strict timing requirements, which may fail in animals that are slow (either because of running under valgrind, doing cache clobbering or expensive memory checks, or hardware is just slow), so only put it in the same group with other tests that mostly sleep, to avoid disruption. Discussion: https://postgr.es/m/20180124231006.z7spaz5gkzbdvob5@alvherre.pgsql --- src/test/isolation/expected/deadlock-soft-2.out | 12 ++-- src/test/isolation/isolation_schedule | 84 +++++-------------------- src/test/isolation/specs/deadlock-soft-2.spec | 22 +++---- 3 files changed, 33 insertions(+), 85 deletions(-) diff --git a/src/test/isolation/expected/deadlock-soft-2.out b/src/test/isolation/expected/deadlock-soft-2.out index 14b0343ba4..5c33a5cdaa 100644 --- a/src/test/isolation/expected/deadlock-soft-2.out +++ b/src/test/isolation/expected/deadlock-soft-2.out @@ -1,12 +1,12 @@ Parsed test spec with 4 sessions starting permutation: s1a s2a s2b s3a s4a s1b s1c s2c s3c s4c -step s1a: LOCK TABLE a1 IN SHARE UPDATE EXCLUSIVE MODE; -step s2a: LOCK TABLE a2 IN ACCESS SHARE MODE; -step s2b: LOCK TABLE a1 IN SHARE UPDATE EXCLUSIVE MODE; <waiting ...> -step s3a: LOCK TABLE a2 IN ACCESS EXCLUSIVE MODE; <waiting ...> -step s4a: LOCK TABLE a2 IN ACCESS EXCLUSIVE MODE; <waiting ...> -step s1b: LOCK TABLE a2 IN SHARE UPDATE EXCLUSIVE MODE; <waiting ...> +step s1a: LOCK TABLE aa1 IN SHARE UPDATE EXCLUSIVE MODE; +step s2a: LOCK TABLE aa2 IN ACCESS SHARE MODE; +step s2b: LOCK TABLE aa1 IN SHARE UPDATE EXCLUSIVE MODE; <waiting ...> +step s3a: LOCK TABLE aa2 IN ACCESS EXCLUSIVE MODE; <waiting ...> +step s4a: LOCK TABLE aa2 IN ACCESS EXCLUSIVE MODE; <waiting ...> +step s1b: LOCK TABLE aa2 IN SHARE UPDATE EXCLUSIVE MODE; <waiting ...> step s1b: <... completed> step s1c: COMMIT; step s2b: <... completed> diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule index 74d7d59546..0df00c6a53 100644 --- a/src/test/isolation/isolation_schedule +++ b/src/test/isolation/isolation_schedule @@ -1,68 +1,16 @@ -test: read-only-anomaly -test: read-only-anomaly-2 -test: read-only-anomaly-3 -test: read-write-unique -test: read-write-unique-2 -test: read-write-unique-3 -test: read-write-unique-4 -test: simple-write-skew -test: receipt-report -test: temporal-range-integrity -test: project-manager -test: classroom-scheduling -test: total-cash -test: referential-integrity -test: ri-trigger -test: partial-index -test: two-ids -test: multiple-row-versions -test: index-only-scan -test: deadlock-simple -test: deadlock-hard -test: deadlock-soft -test: deadlock-soft-2 -test: fk-contention -test: fk-deadlock -test: fk-deadlock2 -test: eval-plan-qual -test: lock-update-delete -test: lock-update-traversal -test: insert-conflict-do-nothing -test: insert-conflict-do-nothing-2 -test: insert-conflict-do-update -test: insert-conflict-do-update-2 -test: insert-conflict-do-update-3 -test: insert-conflict-toast -test: delete-abort-savept -test: delete-abort-savept-2 -test: aborted-keyrevoke -test: multixact-no-deadlock -test: multixact-no-forget -test: lock-committed-update -test: lock-committed-keyupdate -test: update-locked-tuple -test: propagate-lock-delete -test: tuplelock-conflict -test: tuplelock-update -test: freeze-the-dead -test: nowait -test: nowait-2 -test: nowait-3 -test: nowait-4 -test: nowait-5 -test: skip-locked -test: skip-locked-2 -test: skip-locked-3 -test: skip-locked-4 -test: drop-index-concurrently-1 -test: multiple-cic -test: alter-table-1 -test: alter-table-2 -test: alter-table-3 -test: alter-table-4 -test: create-trigger -test: sequence-ddl -test: async-notify -test: vacuum-reltuples -test: timeouts -test: vacuum-concurrent-drop +# In the first group of parallel tests, test "timeout" might be susceptible to +# additional load placed on the machine, so avoid anything in this group that +# does much more other than sleeping. +test: timeouts tuplelock-update deadlock-hard deadlock-soft-2 + +test: read-only-anomaly read-write-unique temporal-range-integrity classroom-scheduling nowait-3 skip-locked-3 alter-table-3 +test: read-only-anomaly-2 read-write-unique-2 project-manager total-cash nowait-4 alter-table-4 +test: read-only-anomaly-3 read-write-unique-4 referential-integrity ri-trigger partial-index sequence-ddl aborted-keyrevoke +test: read-write-unique-3 two-ids multiple-row-versions index-only-scan deadlock-simple async-notify fk-contention +test: deadlock-soft simple-write-skew fk-deadlock alter-table-1 multiple-cic vacuum-reltuples skip-locked-4 receipt-report +test: fk-deadlock2 eval-plan-qual lock-update-delete insert-conflict-do-nothing insert-conflict-toast +test: insert-conflict-do-nothing-2 delete-abort-savept multixact-no-deadlock nowait-5 create-trigger +test: insert-conflict-do-update lock-committed-update multixact-no-forget lock-committed-keyupdate lock-update-traversal alter-table-2 +test: insert-conflict-do-update-2 delete-abort-savept-2 propagate-lock-delete tuplelock-conflict freeze-the-dead +test: insert-conflict-do-update-3 update-locked-tuple nowait skip-locked drop-index-concurrently-1 +test: skip-locked-2 vacuum-concurrent-drop nowait-2 diff --git a/src/test/isolation/specs/deadlock-soft-2.spec b/src/test/isolation/specs/deadlock-soft-2.spec index 46b73bc082..9c643bc026 100644 --- a/src/test/isolation/specs/deadlock-soft-2.spec +++ b/src/test/isolation/specs/deadlock-soft-2.spec @@ -1,6 +1,6 @@ # Soft deadlock requiring reversal of multiple wait-edges. s1 must -# jump over both s3 and s4 and acquire the lock on a2 immediately, -# since s3 and s4 are hard-blocked on a1. +# jump over both s3 and s4 and acquire the lock on aa2 immediately, +# since s3 and s4 are hard-blocked on aa1. # The expected output for this test assumes that isolationtester will # detect step s1b as waiting before the deadlock detector runs and @@ -10,35 +10,35 @@ setup { - CREATE TABLE a1 (); - CREATE TABLE a2 (); + CREATE TABLE aa1 (); + CREATE TABLE aa2 (); } teardown { - DROP TABLE a1, a2; + DROP TABLE aa1, aa2; } session "s1" setup { BEGIN; SET deadlock_timeout = '5s'; } -step "s1a" { LOCK TABLE a1 IN SHARE UPDATE EXCLUSIVE MODE; } -step "s1b" { LOCK TABLE a2 IN SHARE UPDATE EXCLUSIVE MODE; } +step "s1a" { LOCK TABLE aa1 IN SHARE UPDATE EXCLUSIVE MODE; } +step "s1b" { LOCK TABLE aa2 IN SHARE UPDATE EXCLUSIVE MODE; } step "s1c" { COMMIT; } session "s2" setup { BEGIN; SET deadlock_timeout = '100s'; } -step "s2a" { LOCK TABLE a2 IN ACCESS SHARE MODE; } -step "s2b" { LOCK TABLE a1 IN SHARE UPDATE EXCLUSIVE MODE; } +step "s2a" { LOCK TABLE aa2 IN ACCESS SHARE MODE; } +step "s2b" { LOCK TABLE aa1 IN SHARE UPDATE EXCLUSIVE MODE; } step "s2c" { COMMIT; } session "s3" setup { BEGIN; SET deadlock_timeout = '100s'; } -step "s3a" { LOCK TABLE a2 IN ACCESS EXCLUSIVE MODE; } +step "s3a" { LOCK TABLE aa2 IN ACCESS EXCLUSIVE MODE; } step "s3c" { COMMIT; } session "s4" setup { BEGIN; SET deadlock_timeout = '100s'; } -step "s4a" { LOCK TABLE a2 IN ACCESS EXCLUSIVE MODE; } +step "s4a" { LOCK TABLE aa2 IN ACCESS EXCLUSIVE MODE; } step "s4c" { COMMIT; } permutation "s1a" "s2a" "s2b" "s3a" "s4a" "s1b" "s1c" "s2c" "s3c" "s4c" -- 2.11.0
