Hi, Since 67c20979ce7, promotion waits until every process absorbs a ProcSignalBarrier, even with wal_level=replica and no slots. A backend blocked sending to a client never absorbs it. If a client on the standby pauses while fetching a result, promotion has no upper bound. A paused client that stayed connected still held it after 300 seconds. If the client goes away, promotion waits until TCP gives up, about 16 minutes. 18.6 promotes in 0.1 seconds. This is item 8 in [1].
Manu found why in [2]. ProcessClientWriteInterrupt() only acts on ProcDiePending, so the barrier stays pending while secure_write() waits. Once the send buffer is full it hangs every time. The same wait blocks crash failover and standby replay when the primary creates its first logical slot. With synchronous_commit=remote_apply, commits on the primary then wait too. DROP DATABASE has had the same problem since 15. The patch absorbs barriers after a blocked wait in secure_read() or secure_write() wakes up, if interrupts are not held. The normal path does not change. We may be in the middle of a message, and AbsorbDataChecksumsBarrier() can throw. So any error is made FATAL and kept from the client, as Andres suggested in [3]. With a temporary injection point there, the client sees the connection close. Without that, the error lands inside a DataRow and the client hangs. On the REL_19_STABLE head 84572589029 on Linux, all 38 runs with a stuck client hung on 19, and all finished within 210 ms with the patch. The attached file has each case and the controls. DROP DATABASE waits for a barrier too. A stuck client has blocked it since 15, and 14 is fine. 0001 applies to 15 and later as is, so I think it should go back to 15. Not covered is a backend blocked while sending an error message, where interrupts are held. Andres described it in [3]. [1] https://postgr.es/m/cagrkxqsfdjqzp_kwrz7dv6lhx-csrkdnynnxgntrscenqtk...@mail.gmail.com [2] https://postgr.es/m/[email protected] [3] https://postgr.es/m/CAO6_XqoZb2czT=i=vbqjwbwwo3d21cnmm-epfyfx0m-o+ab...@mail.gmail.com
v2-0001-Absorb-ProcSignalBarriers-while-waiting-for-the-c.patch
Description: Binary data
v2-0002-Test-barrier-absorption-with-a-stuck-client.patch
Description: Binary data
