Re: [COMMITTERS] pgsql: Fix inadequacies in recently added wait events

[Thomas Munro]

On Wed, Aug 9, 2017 at 7:39 AM, Alvaro Herrera  wrote:
> Fix inadequacies in recently added wait events
>
> In commit 9915de6c1cb2, we introduced a new wait point for replication
> slots and incorrectly labelled it as wait event PG_WAIT_LOCK.  That's
> wrong, so invent an appropriate new wait event instead, and document it
> properly.
>
> While at it, fix numerous other problems in the vicinity:
> - two different walreceiver wait events were being mixed up in a single
>   wait event (which wasn't documented either); split it out so that they
>   can be distinguished, and document the new events properly.
>
> - ParallelBitmapPopulate was documented but didn't exist.
>
> - ParallelBitmapScan was not documented (I think this should be called
>   "ParallelBitmapScanInit" instead.)
>
> - Logical replication wait events weren't documented
>
> - various symbols had been added in dartboard order in various places.
>   Put them in alphabetical order instead, as was originally intended.

All of the above seem like good candidates for a checker script in
src/tools/check_XXX.pl, a bit like the others I've talked about
recently [1][2].

[1] 
https://www.postgresql.org/message-id/CAEepm=3da6ajyxiuk03pakt1t7egwann9cajc773ct1rdpo...@mail.gmail.com
[2] 
https://www.postgresql.org/message-id/CAEepm=0b7ym9mzsviq1d-hnt4koarvejvsfayvfyknv-pvd...@mail.gmail.com

-- 
Thomas Munro
http://www.enterprisedb.com


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Tag refs/tags/REL9_6_4 was created

[pgsql]

Tag refs/tags/REL9_6_4 was created.

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Tag refs/tags/REL9_3_18 was created

[pgsql]

Tag refs/tags/REL9_3_18 was created.

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Tag refs/tags/REL9_4_13 was created

[pgsql]

Tag refs/tags/REL9_4_13 was created.

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Tag refs/tags/REL9_2_22 was created

[pgsql]

Tag refs/tags/REL9_2_22 was created.

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Tag refs/tags/REL_10_BETA3 was created

[pgsql]

Tag refs/tags/REL_10_BETA3 was created.

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Tag refs/tags/REL9_5_8 was created

[pgsql]

Tag refs/tags/REL9_5_8 was created.

-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix typo in comment

[Alvaro Herrera]

Fix typo in comment

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/f5d54ef97abdd1df3d6bfe0320a565ecce742abe

Modified Files
--
src/backend/utils/sort/tuplesort.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix datumSerialize infrastructure to not crash on non-varlena da

[Tom Lane]

Fix datumSerialize infrastructure to not crash on non-varlena data.

Commit 1efc7e538 did a poor job of emulating existing logic for touching
Datums that might be expanded-object pointers.  It didn't check for typlen
being -1 first, which meant it could crash on fixed-length pass-by-ref
values, and probably on cstring values as well.  It also didn't use
DatumGetPointer before VARATT_IS_EXTERNAL_EXPANDED, which while currently
harmless is not according to documentation nor prevailing style.

I also think the lack of any explanation as to why datumSerialize makes
these particular nonobvious choices is pretty awful, so fix that.

Per report from Jarred Ward.  Back-patch to 9.6 where this code came in.

Discussion: https://postgr.es/m/6f61e6d2-2f5e-4794-9479-a429be1ce...@simple.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/9bf4068cc321a4d44ac54089ab651a49d89bb567

Modified Files
--
src/backend/utils/adt/datum.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Reword some unclear comments

[Alvaro Herrera]

Reword some unclear comments

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/77d2c00af78ee12ae0d1cea34605f1e7af3f6d93

Modified Files
--
src/backend/access/heap/heapam.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: doc: Add missing pieces to logical replication protocol doc

[Peter Eisentraut]

doc: Add missing pieces to logical replication protocol doc

Reported-by: Kyle Conroy 

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/13f03a001e4d841b6a27c0c9c3fe14e1fb2aad80

Modified Files
--
doc/src/sgml/protocol.sgml | 22 +-
1 file changed, 21 insertions(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix yet another race condition in recovery/t/001_stream_rep.pl.

[Tom Lane]

Fix yet another race condition in recovery/t/001_stream_rep.pl.

In commit 5c77690f6, we added polling in front of most of the
get_slot_xmins calls in 001_stream_rep.pl, but today's results from
buildfarm member nightjar show that at least one more poll loop
is needed.

Proactively add a poll loop before the next-to-last get_slot_xmins call
as well.  It may be that there is no race condition there because the
standby_2 server is shut down at that point, but I'm quite tired of
fighting with this test script.  The empirical evidence that it's safe,
from the buildfarm, is no stronger than the evidence for the other
call that nightjar just proved unsafe.

The only remaining get_slot_xmins calls without wait_slot_xmins
protection are the first two, which should be OK since nothing has
happened at that point.  It's tempting to ignore that special case
and merge get_slot_xmins and wait_slot_xmins into a single function.
I didn't go that far though.

Discussion: https://postgr.es/m/18436.1502228...@sss.pgh.pa.us

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/4576a69354fa2efc1bafa50df1c104c1a80c64e5

Modified Files
--
src/test/recovery/t/001_stream_rep.pl | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Reword some unclear comments

[Alvaro Herrera]

Reword some unclear comments

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/962b26b674d29fa2834afbd760a9229bc42a5876

Modified Files
--
src/backend/access/heap/heapam.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Reword some unclear comments

[Alvaro Herrera]

Reword some unclear comments

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/9cc510fe89a3f013091bf7234ca893ba74a15094

Modified Files
--
src/backend/access/heap/heapam.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Reword some unclear comments

[Alvaro Herrera]

Reword some unclear comments

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1924ef41668773ad62d845eebc11ff35f17446ad

Modified Files
--
src/backend/access/heap/heapam.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Reword some unclear comments

[Alvaro Herrera]

Reword some unclear comments

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e829db0b19bda06996a1c7104ca1251c8eacaa2c

Modified Files
--
src/backend/access/heap/heapam.c | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix datumSerialize infrastructure to not crash on non-varlena da

[Tom Lane]

Fix datumSerialize infrastructure to not crash on non-varlena data.

Commit 1efc7e538 did a poor job of emulating existing logic for touching
Datums that might be expanded-object pointers.  It didn't check for typlen
being -1 first, which meant it could crash on fixed-length pass-by-ref
values, and probably on cstring values as well.  It also didn't use
DatumGetPointer before VARATT_IS_EXTERNAL_EXPANDED, which while currently
harmless is not according to documentation nor prevailing style.

I also think the lack of any explanation as to why datumSerialize makes
these particular nonobvious choices is pretty awful, so fix that.

Per report from Jarred Ward.  Back-patch to 9.6 where this code came in.

Discussion: https://postgr.es/m/6f61e6d2-2f5e-4794-9479-a429be1ce...@simple.com

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/fe578cbd4b4262496d44cb5cd110eae3c4aff48c

Modified Files
--
src/backend/utils/adt/datum.c | 18 +-
1 file changed, 13 insertions(+), 5 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix replication origin-related race conditions

[Alvaro Herrera]

Fix replication origin-related race conditions

Similar to what was fixed in commit 9915de6c1cb2 for replication slots,
but this time it's related to replication origins: DROP SUBSCRIPTION
attempts to drop the replication origin, but that fails if the
replication worker process hasn't yet marked it unused.  This causes
failures in the buildfarm:
ERROR:  could not drop replication origin with OID 1, in use by PID 34069

Like the aforementioned commit, fix by having the process running DROP
SUBSCRIPTION sleep until the worker marks the the replication origin
struct as free.  This uses a condition variable on each replication
origin shmem state struct, so that the session trying to drop can sleep
and expect to be awakened by the process keeping the origin open.

Also fix a SGML markup in the previous commit.

Discussion: https://postgr.es/m/20170808001433.rozlseaf4m2wkw3n@alvherre.pgsql

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/b2c95a3798ff39fc24d71b6655ddfe0e4cb3f378

Modified Files
--
doc/src/sgml/monitoring.sgml |  8 +++--
src/backend/commands/subscriptioncmds.c  |  2 +-
src/backend/postmaster/pgstat.c  |  3 ++
src/backend/replication/logical/origin.c | 57 ++--
src/include/pgstat.h |  1 +
src/include/replication/origin.h |  2 +-
6 files changed, 58 insertions(+), 15 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Fix inadequacies in recently added wait events

[Alvaro Herrera]

Fix inadequacies in recently added wait events

In commit 9915de6c1cb2, we introduced a new wait point for replication
slots and incorrectly labelled it as wait event PG_WAIT_LOCK.  That's
wrong, so invent an appropriate new wait event instead, and document it
properly.

While at it, fix numerous other problems in the vicinity:
- two different walreceiver wait events were being mixed up in a single
  wait event (which wasn't documented either); split it out so that they
  can be distinguished, and document the new events properly.

- ParallelBitmapPopulate was documented but didn't exist.

- ParallelBitmapScan was not documented (I think this should be called
  "ParallelBitmapScanInit" instead.)

- Logical replication wait events weren't documented

- various symbols had been added in dartboard order in various places.
  Put them in alphabetical order instead, as was originally intended.

Discussion: https://postgr.es/m/20170808181131.mu4fjepuh5m75cyq@alvherre.pgsql

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/030273b7ea468ed4b3073dfd1f2ad88e3129df6a

Modified Files
--
doc/src/sgml/monitoring.sgml   | 32 +--
src/backend/postmaster/pgstat.c| 36 +-
.../libpqwalreceiver/libpqwalreceiver.c|  4 +--
src/backend/replication/slot.c |  3 +-
src/include/pgstat.h   | 16 +-
5 files changed, 64 insertions(+), 27 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers