Hello hackers,
12.12.2025 02:54, Thomas Munro wrote:
Thanks! Could also be environmental. Maybe the snow arrived and you
adjusted your thermostat? :-) But even then we'd still need to figure
out what's different on those branches... One difference is autoconf
vs meson. Both say -g -O2, but only Meson says -fPIC. Can you
reproduce it on later branches with configure/make?
I'd like to revive this thread, while we have no snow in EU yet. We had
some off-list discussion of this topic at the end of the past year, but
didn't root-cause the issue then. But this week I have finally dedicated
some time to dive into this with my Orange Pi device, which is able to
reproduce those crashes more or less as often as greenfly.
I came to the following modification:
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -4250,3 +4250,10 @@ LockBuffer(Buffer buffer, int mode)
else if (mode == BUFFER_LOCK_EXCLUSIVE)
+{
+for (int i = 0; i < 100; i++)
+{
+LWLockAcquire(BufferDescriptorGetContentLock(buf), LW_EXCLUSIVE);
+LWLockRelease(BufferDescriptorGetContentLock(buf));
+}
LWLockAcquire(BufferDescriptorGetContentLock(buf),
LW_EXCLUSIVE);
+}
else
which makes a build produced on REL_15_STABLE with:
CC=clang-20 CFLAGS="-O1" ./configure -q --enable-debug --enable-cassert
--enable-tap-tests && make -s -j8
fail for me reliably (0 of 10 runs succeeded) as below:
PROVE_TESTS="t/001*" make -s check -C src/bin/pgbench
...
Files=1, Tests=3, 9 wallclock secs ( 0.11 usr 0.03 sys + 4.83 cusr 1.02
csys = 5.99 CPU)
Result: FAIL
TRAP: FailedAssertion("!(oldstate & LW_VAL_EXCLUSIVE)", File: "lwlock.c", Line:
1843, PID: 64342)
postgres: main: orangepi postgres [local] CREATE
TYPE(ExceptionalCondition+0x72)[0x2ab2066546]
postgres: main: orangepi postgres [local] CREATE
TYPE(LWLockRelease+0x58e)[0x2ab2385160]
postgres: main: orangepi postgres [local] CREATE
TYPE(_bt_relbuf+0x14)[0x2ab20d5060]
postgres: main: orangepi postgres [local] CREATE TYPE(+0x13bc2a)[0x2ab20d1c2a]
...
(other observed result is a hang, once per ~10 runs)
Please find attached the assembler code for the LWLockWakeup() function
that is known to produce the assertion failure and note two instructions
I commented there:
3eee4a: lw a3,4(s6) # 1014: old_state =
pg_atomic_read_u32(&lock->state)
3eee4e: lw a5,8(s6) # 1026:
proclist_is_empty(&lock->waiters)
The following minimal modification:
--- a/src/backend/storage/lmgr/lwlock.c
+++ b/src/backend/storage/lmgr/lwlock.c
@@ -1016,2 +1016,3 @@ LWLockWakeup(LWLock *lock)
{
+__asm__ __volatile__("nop" ::: "memory");
desired_state = old_state;
which turns that pair into:
3eee4a: lw a3,4(s6)
3eee4e: nop
3eee50: lw a5,8(s6)
makes the test pass reliably for me (10+ iterations).
So it's not surprising now that this crash depends on postgres version
(v14/v15 only), compiler (gcc doesn't produce the code needed), its'
options (not reproduced with -O0), etc.
I've generated a minimal standalone reproducer for this anomaly (please
find it attached too), which replicates an essential part of
LWLockWakeup():
sb pending_wakeup, (waiting)
lw old_state, 4(lock)
lw head, 8(lock)
and desired_state, old_state, mask
1: lr.w cur, (state)
bne cur, old_state, 2f
sc.w tmp, desired_state, (state)
bnez tmp, 1b
2:
and demonstrates that LR/SC mechanism doesn't work as expected at least on
the given RISC-V CPU (8 SpacemiT X60 cores in 2 clusters):
Machine model: ky x1 orangepi-rv2 board
...
processor : 7
hart : 7
model name : Ky(R) X1
isa :
rv64imafdcv_zicbom_zicboz_zicntr_zicond_zicsr_zifencei_zihintpause_zihpm_zfh_zfhmin_zca_zcd_zba_zbb_zbc_zbs_zkt_zve32f_zve32x_zve64d_zve64f_zve64x_zvfh_zvfhmin_zvkt_sscofpmf_sstc_svinval_svnapot_svpbmt
mmu : sv39
uarch : ky,x60
mvendorid : 0x710
marchid : 0x8000000058000001
mimpid : 0x1000000049772200
Let me share results of a few experiments:
clang-20 -O1 ~/lrsc-anomaly.c -o ~/lrsc-anomaly
~/lrsc-anomaly
LOST UPDATE in round 28: value=0x70001d01 expected=0x00001d01
LOST UPDATE in round 29: value=0x70001e01 expected=0x00001e01
LOST UPDATE in round 30: value=0x70001f01 expected=0x00001f01
LOST UPDATE in round 31: value=0x70002001 expected=0x00002001
LOST UPDATE in round 32: value=0x70002101 expected=0x00002101
LOST UPDATE in round 33: value=0x70002201 expected=0x00002201
LOST UPDATE in round 34: value=0x70002301 expected=0x00002301
LOST UPDATE in round 35: value=0x70002401 expected=0x00002401
LOST UPDATE in round 37: value=0x70002601 expected=0x00002601
LOST UPDATE in round 38: value=0x70002701 expected=0x00002701
page_offset=0 rounds=1000000 retried=1824 lost=15523: FAILED
It reproduces the issue: process A's SC succeeds despite B's increment
happened in between.
for n in `seq 0 15`; do ~/lrsc-anomaly -n $n | tail -1; done
page_offset=0 rounds=1000000 retried=5374 lost=16160: FAILED
page_offset=1 rounds=1000000 retried=29563 lost=16438: FAILED
page_offset=2 rounds=1000000 retried=1251 lost=17209: FAILED
page_offset=3 rounds=1000000 retried=14960 lost=22: FAILED
page_offset=4 rounds=1000000 retried=12090 lost=19: FAILED
page_offset=5 rounds=1000000 retried=13561 lost=15: FAILED
page_offset=6 rounds=1000000 retried=15037 lost=32: FAILED
page_offset=7 rounds=1000000 retried=3124 lost=16774: FAILED
page_offset=8 rounds=1000000 retried=19813 lost=0: OK
page_offset=9 rounds=1000000 retried=20210 lost=0: OK
page_offset=10 rounds=1000000 retried=17992 lost=0: OK
page_offset=11 rounds=1000000 retried=18207 lost=0: OK
page_offset=12 rounds=1000000 retried=17903 lost=0: OK
page_offset=13 rounds=1000000 retried=18779 lost=0: OK
page_offset=14 rounds=1000000 retried=18384 lost=0: OK
page_offset=15 rounds=1000000 retried=18328 lost=0: OK
That is, the effect depends not only on that lw pair, but also on offset
of "waiting" in "sb pending_wakeup, (waiting)" preceding it.
The anomaly's probability also depends on whether these two cores/harts
are in the same cluster or not. For example, with cores 0 and 1 that belong
to cluster 0:
for n in `seq 0 4 15`; do taskset -c 0,1 ~/lrsc-anomaly -n $n | tail -1; done
page_offset=0 rounds=1000000 retried=1194 lost=16909: FAILED
page_offset=4 rounds=1000000 retried=1147 lost=16464: FAILED
page_offset=8 rounds=1000000 retried=18658 lost=0: OK
page_offset=12 rounds=1000000 retried=18154 lost=0: OK
And now with two cores that belong to different clusters:
for n in `seq 0 4 15`; do taskset -c 0,4 ~/lrsc-anomaly -n $n | tail -1; done
page_offset=0 rounds=1000000 retried=13350 lost=45: FAILED
page_offset=4 rounds=1000000 retried=15566 lost=37: FAILED
page_offset=8 rounds=1000000 retried=18223 lost=0: OK
page_offset=12 rounds=1000000 retried=17466 lost=0: OK
Also, nop inserted as above:
/* old_state = pg_atomic_read_u32(&lock->state) */
" lw %[old_state], 4(%[lock])\n"
+"nop\n"
/* proclist_is_empty(&lock->waiters) */
" lw %[head], 8(%[lock])\n"
changes the picture:
for n in `seq 0 15`; do taskset -c 0,1 ~/lrsc-anomaly -n $n | tail -1; done
page_offset=0 rounds=1000000 retried=15306 lost=17557: FAILED
page_offset=1 rounds=1000000 retried=2624 lost=16711: FAILED
page_offset=2 rounds=1000000 retried=1468 lost=16940: FAILED
page_offset=3 rounds=1000000 retried=815 lost=17098: FAILED
page_offset=4 rounds=1000000 retried=31171 lost=0: OK
page_offset=5 rounds=1000000 retried=17372 lost=0: OK
page_offset=6 rounds=1000000 retried=31803 lost=0: OK
page_offset=7 rounds=1000000 retried=19194 lost=0: OK
page_offset=8 rounds=1000000 retried=5777 lost=0: OK
page_offset=9 rounds=1000000 retried=5375 lost=0: OK
page_offset=10 rounds=1000000 retried=11920 lost=0: OK
page_offset=11 rounds=1000000 retried=5334 lost=0: OK
page_offset=12 rounds=1000000 retried=5087 lost=0: OK
page_offset=13 rounds=1000000 retried=12040 lost=0: OK
page_offset=14 rounds=1000000 retried=8479 lost=0: OK
page_offset=15 rounds=1000000 retried=4965 lost=0: OK
It kind of confirms the behavior observed with Postgres, when nop shifted
the code out of the problem area.
It also depends on number of distinct lines (buffers) in rotation; with
#define NBUFFERS 2
the probability drops almost to zero:
taskset -c 0,1 ~/lrsc-anomaly
LOST UPDATE in round 300830: value=0x7000ba01 expected=0x0000ba01
LOST UPDATE in round 300831: value=0x7000bb01 expected=0x0000bb01
page_offset=0 rounds=1000000 retried=2 lost=2: FAILED
Sorry for the long scroll, but it looks like this anomaly should be added
to our collection of the outer world's defects discovered during Postgres
testing.
Best regards,
Alexander
src/backend/postgres: file format elf64-littleriscv
Disassembly of section .plt:
Disassembly of section .text:
00000000003eebd2 <LWLockRelease>:
3eebd2: addi sp,sp,-144
3eebd4: sd ra,136(sp)
3eebd6: sd s0,128(sp)
3eebd8: sd s1,120(sp)
3eebda: sd s2,112(sp)
3eebdc: sd s3,104(sp)
3eebde: sd s4,96(sp)
3eebe0: sd s5,88(sp)
3eebe2: sd s6,80(sp)
3eebe4: sd s7,72(sp)
3eebe6: sd s8,64(sp)
3eebe8: sd s9,56(sp)
3eebea: sd s10,48(sp)
3eebec: mv s6,a0
3eebee: auipc a0,0x402
3eebf2: addi s0,a0,1994 # 7f13b8 <NamedLWLockTrancheRequests>
3eebf6: lw a0,12(s0)
3eebf8: auipc a2,0x402
3eebfc: addiw a1,a0,-1
3eec00: slti a3,a1,-1
3eec04: addiw a3,a3,-1
3eec06: or a3,a3,a1
3eec08: addi a2,a2,-1216 # 7f0738 <held_lwlocks>
3eec0c: addiw a0,a0,-1
3eec0e: bltz a0,3eec66 <LWLockRelease+0x94>
3eec12: slli a4,a0,0x4
3eec16: add a4,a4,a2
3eec18: ld a4,0(a4)
3eec1a: bne s6,a4,3eec0c <LWLockRelease+0x3a>
3eec1e: bltz a0,3eec6c <LWLockRelease+0x9a>
3eec22: slli a3,a0,0x4
3eec26: add a3,a3,a2
3eec28: lw a3,8(a3)
3eec2a: sw a1,12(s0)
3eec2c: bge a0,a1,3eec50 <LWLockRelease+0x7e>
3eec30: slli a0,a0,0x20
3eec32: slli a1,a1,0x4
3eec34: srli a0,a0,0x1c
3eec36: add a1,a1,a2
3eec38: add a0,a0,a2
3eec3a: addi a0,a0,16
3eec3c: addi a1,a1,16
3eec3e: ld a2,0(a0)
3eec40: ld a4,8(a0)
3eec42: sd a2,-16(a0)
3eec46: sd a4,-8(a0)
3eec4a: addi a0,a0,16
3eec4c: bne a0,a1,3eec3e <LWLockRelease+0x6c>
3eec50: addi s1,s6,4
3eec54: addi a0,s6,7
3eec58: andi a0,a0,-4
3eec5a: beqz a3,3eec94 <LWLockRelease+0xc2>
3eec5c: bne a0,s1,3ef184 <LWLockRelease+0x5b2>
3eec60: li a1,-1
3eec62: li a2,1
3eec64: j 3eeca0 <LWLockRelease+0xce>
3eec66: mv a0,a3
3eec68: bgez a0,3eec22 <LWLockRelease+0x50>
3eec6c: li a0,21
3eec6e: li a1,0
3eec70: auipc ra,0xffce2
3eec74: jalr -1810(ra) # d055e <errstart_cold>
3eec78: beqz a0,3ef00a <LWLockRelease+0x438>
3eec7c: lhu a0,0(s6)
3eec80: li a1,47
3eec84: bltu a1,a0,3eefb8 <LWLockRelease+0x3e6>
3eec88: auipc a1,0x3e6
3eec8c: ld a1,-1600(a1) # 7d4648 <_GLOBAL_OFFSET_TABLE_+0xb68>
3eec90: slli a0,a0,0x3
3eec92: j 3eefda <LWLockRelease+0x408>
3eec94: bne a0,s1,3ef1cc <LWLockRelease+0x5fa>
3eec98: lui a1,0xff000
3eec9c: lui a2,0x1000
3eeca0: neg a2,a2
3eeca4: amoadd.w.aqrl a2,a2,(s1)
3eeca8: addw a1,a1,a2
3eecaa: slli a2,a1,0x27
3eecae: bltz a2,3ef13c <LWLockRelease+0x56a>
3eecb2: lui a2,0x61000
3eecb6: addiw a2,a2,-1 # 60ffffff <__BSS_END__+0x607e47df>
3eecb8: and a1,a1,a2
3eecba: lui a2,0x60000
3eecbe: bne a1,a2,3eeeaa <LWLockRelease+0x2d8>
3eecc2: lui a1,0x80000
3eecc6: addi a1,a1,-1 # ffffffff7fffffff
<__BSS_END__+0xffffffff7f7e47df>
3eecc8: sw a1,0(sp)
3eecca: sw a1,4(sp)
3eeccc: bne a0,s1,3ef1a8 <LWLockRelease+0x5d6>
3eecd0: lui s2,0x10000
3eecd4: amoor.w.aqrl a0,s2,(s1)
3eecd8: slli a0,a0,0x23
3eecda: bgez a0,3eed1c <LWLockRelease+0x14a>
3eecde: auipc a0,0x1ed
3eece2: li s3,907
3eece6: auipc a1,0x1ed
3eecea: addi s4,a0,-788 # 5db9ca <hextbl+0x272cf>
3eecee: addi s0,a1,264 # 5dbdee <hextbl+0x276f3>
3eecf2: sd zero,8(sp)
3eecf4: sw zero,16(sp)
3eecf6: sd s4,24(sp)
3eecf8: sw s3,32(sp)
3eecfa: sd s0,40(sp)
3eecfc: addi a0,sp,8
3eecfe: jal 3f6dcc <perform_spin_delay>
3eed02: lwu a0,0(s1)
3eed06: slli a0,a0,0x23
3eed08: bltz a0,3eecfc <LWLockRelease+0x12a>
3eed0c: addi a0,sp,8
3eed0e: jal 3f6e62 <finish_spin_delay>
3eed12: amoor.w.aqrl a0,s2,(s1)
3eed16: slli a0,a0,0x23
3eed18: bltz a0,3eecf2 <LWLockRelease+0x120>
3eed1c: lw a1,8(s6)
3eed20: lui a3,0x80000
3eed24: addiw a3,a3,-1 # 7fffffff <__BSS_END__+0x7f7e47df>
3eed26: mv a2,a3
3eed28: beq a1,a3,3eed44 <LWLockRelease+0x172>
3eed2c: auipc a0,0x3e5
3eed30: ld a0,196(a0) # 7d3df0 <_GLOBAL_OFFSET_TABLE_+0x310>
3eed34: ld a0,0(a0)
3eed36: ld a0,0(a0)
3eed38: li a2,880
3eed3c: mul a2,a1,a2
3eed40: add a0,a0,a2
3eed42: lw a2,96(a0)
3eed44: lui a7,0x20000
3eed48: beq a1,a3,3eee34 <LWLockRelease+0x262>
3eed4c: li s2,0
3eed4e: li s3,1
3eed50: auipc a0,0x3e5
3eed54: li s4,880
3eed58: ld s5,160(a0) # 7d3df0 <_GLOBAL_OFFSET_TABLE_+0x310>
3eed5c: lui s9,0x80000
3eed60: addiw s9,s9,-1 # 7fffffff <__BSS_END__+0x7f7e47df>
3eed62: li s8,2
3eed64: li s7,1
3eed66: j 3eed6e <LWLockRelease+0x19c>
3eed68: mv a1,s10
3eed6a: beq s10,s9,3eee2c <LWLockRelease+0x25a>
3eed6e: ld a0,0(s5)
3eed72: ld a0,0(a0)
3eed74: mv s10,a2
3eed76: mul s0,a1,s4
3eed7a: andi a2,s2,1
3eed7e: add s0,s0,a0
3eed80: beqz a2,3eed88 <LWLockRelease+0x1b6>
3eed82: lbu a2,95(s0)
3eed86: beqz a2,3eee26 <LWLockRelease+0x254>
3eed88: lw a2,96(s0)
3eed8a: bnez a2,3eed92 <LWLockRelease+0x1c0>
3eed8c: lw a3,100(s0)
3eed8e: beqz a3,3ef118 <LWLockRelease+0x546>
3eed92: lw a3,100(s0)
3eed94: bne a3,s9,3eedbc <LWLockRelease+0x1ea>
3eed98: lw a0,8(s6)
3eed9c: bne a0,a1,3ef0d0 <LWLockRelease+0x4fe>
3eeda0: sw a2,8(s6)
3eeda4: lw a0,96(s0)
3eeda6: beq a0,s9,3eedca <LWLockRelease+0x1f8>
3eedaa: ld a2,0(s5)
3eedae: ld a2,0(a2)
3eedb0: lw a3,100(s0)
3eedb2: mul a0,a0,s4
3eedb6: add a0,a0,a2
3eedb8: sw a3,100(a0)
3eedba: j 3eedd8 <LWLockRelease+0x206>
3eedbc: mul a3,a3,s4
3eedc0: add a0,a0,a3
3eedc2: sw a2,96(a0)
3eedc4: lw a0,96(s0)
3eedc6: bne a0,s9,3eedaa <LWLockRelease+0x1d8>
3eedca: lw a0,12(s6)
3eedce: bne a0,a1,3ef0f4 <LWLockRelease+0x522>
3eedd2: lw a0,100(s0)
3eedd4: sw a0,12(s6)
3eedd8: sw zero,96(s0)
3eeddc: sw zero,100(s0)
3eede0: mv a0,sp
3eede2: jal 3eeaaa <proclist_push_tail_offset>
3eede6: lbu a0,94(s0)
3eedea: bne a0,s3,3ef088 <LWLockRelease+0x4b6>
3eedee: lbu a0,95(s0)
3eedf2: sb s8,94(s0)
3eedf6: addi a1,a0,-2
3eedfa: snez a2,a1
3eedfe: seqz a1,a1
3eee02: and s7,a1,s7
3eee06: or s2,s2,a2
3eee0a: seqz a0,a0
3eee0e: bnez a0,3eee2c <LWLockRelease+0x25a>
3eee10: mv a2,s9
3eee12: beq s10,s9,3eed68 <LWLockRelease+0x196>
3eee16: ld a0,0(s5)
3eee1a: ld a0,0(a0)
3eee1c: mul a1,s10,s4
3eee20: add a0,a0,a1
3eee22: lw a2,96(a0)
3eee24: j 3eed68 <LWLockRelease+0x196>
3eee26: li a0,0
3eee28: li s2,1
3eee2a: beqz a0,3eee10 <LWLockRelease+0x23e>
3eee2c: andi a0,s7,1
3eee30: slli a7,a0,0x1d
3eee34: lw a1,0(sp)
3eee36: lui a2,0x80000
3eee3a: addiw a2,a2,-1 # 7fffffff <__BSS_END__+0x7f7e47df>
3eee3c: beq a1,a2,3eee4a <LWLockRelease+0x278>
3eee40: lwu a1,0(s1)
3eee44: slli a1,a1,0x21
3eee46: bgez a1,3ef1f0 <LWLockRelease+0x61e>
3eee4a: lw a3,4(s6) # 1014:
old_state = pg_atomic_read_u32(&lock->state)
3eee4e: lw a5,8(s6) # 1026:
proclist_is_empty(&lock->waiters)
3eee52: lui a1,0xd0000
3eee56: addiw a1,a1,-1 # ffffffffcfffffff
<__BSS_END__+0xffffffffcf7e47df>
3eee58: and a4,a3,a1
3eee5c: or a4,a4,a7
3eee60: beq a5,a2,3eee6a <LWLockRelease+0x298>
3eee64: lui a2,0xf0000
3eee68: j 3eee6e <LWLockRelease+0x29c>
3eee6a: lui a2,0xb0000
3eee6e: addiw a2,a2,-1 # ffffffffafffffff
<__BSS_END__+0xffffffffaf7e47df>
3eee70: and a4,a4,a2
3eee72: lr.w.aqrl a2,(s1)
3eee76: bne a2,a3,3eeeda <LWLockRelease+0x308>
3eee7a: sc.w.rl a5,a4,(s1)
3eee7e: bnez a5,3eee72 <LWLockRelease+0x2a0>
3eee80: lw a0,0(sp)
3eee82: lui a1,0x80000
3eee86: addiw a1,a1,-1 # 7fffffff <__BSS_END__+0x7f7e47df>
3eee88: mv a2,a1
3eee8a: beq a0,a1,3eeea6 <LWLockRelease+0x2d4>
3eee8e: auipc a2,0x3e5
3eee92: ld a2,-158(a2) # 7d3df0 <_GLOBAL_OFFSET_TABLE_+0x310>
3eee96: ld a2,0(a2)
3eee98: ld a2,0(a2)
3eee9a: li a3,880
3eee9e: mul a3,a0,a3
3eeea2: add a2,a2,a3
3eeea4: lw a2,96(a2)
3eeea6: bne a0,a1,3eef1c <LWLockRelease+0x34a>
3eeeaa: auipc a0,0x3e6
3eeeae: ld a0,294(a0) # 7d4fd0 <_GLOBAL_OFFSET_TABLE_+0x14f0>
3eeeb2: lw a1,0(a0)
3eeeb4: beqz a1,3ef160 <LWLockRelease+0x58e>
3eeeb8: lw a1,0(a0)
3eeeba: addi a1,a1,-1
3eeebc: sw a1,0(a0)
3eeebe: ld ra,136(sp)
3eeec0: ld s0,128(sp)
3eeec2: ld s1,120(sp)
3eeec4: ld s2,112(sp)
3eeec6: ld s3,104(sp)
3eeec8: ld s4,96(sp)
3eeeca: ld s5,88(sp)
3eeecc: ld s6,80(sp)
3eeece: ld s7,72(sp)
3eeed0: ld s8,64(sp)
3eeed2: ld s9,56(sp)
3eeed4: ld s10,48(sp)
3eeed6: addi sp,sp,144
3eeed8: ret
3eeeda: lui a3,0xf0000
3eeede: lui s0,0xb0000
3eeee2: lui a5,0x80000
3eeee6: addiw a6,a3,-1 # ffffffffefffffff
<__BSS_END__+0xffffffffef7e47df>
3eeeea: addiw s0,s0,-1 # ffffffffafffffff
<__BSS_END__+0xffffffffaf7e47df>
3eeeec: addiw a5,a5,-1 # 7fffffff <__BSS_END__+0x7f7e47df>
3eeeee: j 3eef0e <LWLockRelease+0x33c>
3eeef0: sext.w a3,a2
3eeef4: and a2,a2,a1
3eeef6: or a2,a2,a7
3eeefa: and a4,a4,a2
3eeefc: lr.w.aqrl a2,(s1)
3eef00: bne a2,a3,3eef0a <LWLockRelease+0x338>
3eef04: sc.w.rl a0,a4,(s1)
3eef08: bnez a0,3eeefc <LWLockRelease+0x32a>
3eef0a: beq a2,a3,3eee80 <LWLockRelease+0x2ae>
3eef0e: lw a3,8(s6)
3eef12: mv a4,s0
3eef14: beq a3,a5,3eeef0 <LWLockRelease+0x31e>
3eef18: mv a4,a6
3eef1a: j 3eeef0 <LWLockRelease+0x31e>
3eef1c: auipc a1,0x3e5
3eef20: ld s3,-300(a1) # 7d3df0 <_GLOBAL_OFFSET_TABLE_+0x310>
3eef24: li s2,880
3eef28: lui s1,0x80000
3eef2c: addiw s1,s1,-1 # 7fffffff <__BSS_END__+0x7f7e47df>
3eef2e: j 3eef36 <LWLockRelease+0x364>
3eef30: mv a0,s0
3eef32: beq s0,s1,3eeeaa <LWLockRelease+0x2d8>
3eef36: ld a1,0(s3)
3eef3a: ld a4,0(a1)
3eef3c: mul a1,a0,s2
3eef40: add a1,a1,a4
3eef42: lw a3,96(a1)
3eef44: mv s0,a2
3eef46: bnez a3,3eef4e <LWLockRelease+0x37c>
3eef48: lw a2,100(a1)
3eef4a: beqz a2,3ef0ac <LWLockRelease+0x4da>
3eef4e: lw a2,100(a1)
3eef50: bne a2,s1,3eef74 <LWLockRelease+0x3a2>
3eef54: lw a2,0(sp)
3eef56: bne a2,a0,3ef040 <LWLockRelease+0x46e>
3eef5a: sw a3,0(sp)
3eef5c: lw a2,96(a1)
3eef5e: beq a2,s1,3eef82 <LWLockRelease+0x3b0>
3eef62: ld a0,0(s3)
3eef66: ld a0,0(a0)
3eef68: lw a3,100(a1)
3eef6a: mul a2,a2,s2
3eef6e: add a0,a0,a2
3eef70: sw a3,100(a0)
3eef72: j 3eef8c <LWLockRelease+0x3ba>
3eef74: mul a2,a2,s2
3eef78: add a2,a2,a4
3eef7a: sw a3,96(a2)
3eef7c: lw a2,96(a1)
3eef7e: bne a2,s1,3eef62 <LWLockRelease+0x390>
3eef82: lw a2,4(sp)
3eef84: bne a2,a0,3ef064 <LWLockRelease+0x492>
3eef88: lw a0,100(a1)
3eef8a: sw a0,4(sp)
3eef8c: sw zero,96(a1)
3eef90: sw zero,100(a1)
3eef94: fence rw,w
3eef98: ld a0,24(a1)
3eef9a: sb zero,94(a1)
3eef9e: jal 36b150 <PGSemaphoreUnlock>
3eefa2: mv a2,s1
3eefa4: beq s0,s1,3eef30 <LWLockRelease+0x35e>
3eefa8: ld a0,0(s3)
3eefac: ld a0,0(a0)
3eefae: mul a1,s0,s2
3eefb2: add a0,a0,a1
3eefb4: lw a2,96(a0)
3eefb6: j 3eef30 <LWLockRelease+0x35e>
3eefb8: li a1,74
3eefbc: bltu a1,a0,3ef012 <LWLockRelease+0x440>
3eefc0: li a1,1
3eefc2: slli a1,a1,0x20
3eefc4: addi a1,a1,-48
3eefc8: add a0,a0,a1
3eefca: addi a1,a1,47
3eefce: and a0,a0,a1
3eefd0: auipc a1,0x3c8
3eefd4: slli a0,a0,0x3
3eefd6: addi a1,a1,32 # 7b6ff0 <BuiltinTrancheNames>
3eefda: add a0,a0,a1
3eefdc: ld a1,0(a0)
3eefde: auipc a0,0x1ed
3eefe2: addi a0,a0,-1105 # 5dbb8d <hextbl+0x27492>
3eefe6: auipc ra,0x118
3eefea: jalr -906(ra) # 506c5c <errmsg_internal>
3eefee: auipc a0,0x1ed
3eeff2: auipc a1,0x1ed
3eeff6: addi a0,a0,-1572 # 5db9ca <hextbl+0x272cf>
3eeffa: addi a2,a1,-1105 # 5dbba1 <hextbl+0x274a6>
3eeffe: li a1,1823
3ef002: auipc ra,0x118
3ef006: jalr -642(ra) # 506d80 <errfinish>
3ef00a: auipc ra,0xffcde
3ef00e: jalr -362(ra) # ccea0 <abort@plt>
3ef012: lw a1,4(s0)
3ef014: addi a0,a0,-75
3ef018: slli a0,a0,0x30
3ef01a: srli a2,a0,0x30
3ef01e: bge a2,a1,3ef036 <LWLockRelease+0x464>
3ef022: ld a1,40(s0)
3ef024: srli a0,a0,0x2d
3ef026: add a0,a0,a1
3ef028: ld a1,0(a0)
3ef02a: bnez a1,3eefde <LWLockRelease+0x40c>
3ef02c: auipc a0,0x1c4
3ef030: addi a1,a0,-40 # 5b3004 <year_lengths+0x5b66c>
3ef034: j 3eefde <LWLockRelease+0x40c>
3ef036: auipc a0,0x1c4
3ef03a: addi a1,a0,-50 # 5b3004 <year_lengths+0x5b66c>
3ef03e: j 3eefde <LWLockRelease+0x40c>
3ef040: auipc a0,0x1eb
3ef044: auipc a1,0x169
3ef048: auipc a2,0x1eb
3ef04c: addi a0,a0,1494 # 5da616 <hextbl+0x25f1b>
3ef050: addi a1,a1,-1379 # 557ae1 <year_lengths+0x149>
3ef054: addi a2,a2,1305 # 5da561 <hextbl+0x25e66>
3ef058: li a3,123
3ef05c: auipc ra,0xffce1
3ef060: jalr 1144(ra) # d04d4 <ExceptionalCondition>
3ef064: auipc a0,0x1eb
3ef068: auipc a1,0x169
3ef06c: auipc a2,0x1eb
3ef070: addi a0,a0,1513 # 5da64d <hextbl+0x25f52>
3ef074: addi a1,a1,-1415 # 557ae1 <year_lengths+0x149>
3ef078: addi a2,a2,1269 # 5da561 <hextbl+0x25e66>
3ef07c: li a3,131
3ef080: auipc ra,0xffce1
3ef084: jalr 1108(ra) # d04d4 <ExceptionalCondition>
3ef088: auipc a0,0x1ed
3ef08c: auipc a1,0x169
3ef090: auipc a2,0x1ed
3ef094: addi a0,a0,-1310 # 5dbb6a <hextbl+0x2746f>
3ef098: addi a1,a1,-1451 # 557ae1 <year_lengths+0x149>
3ef09c: addi a2,a2,-1734 # 5db9ca <hextbl+0x272cf>
3ef0a0: li a3,996
3ef0a4: auipc ra,0xffce1
3ef0a8: jalr 1072(ra) # d04d4 <ExceptionalCondition>
3ef0ac: auipc a0,0x1eb
3ef0b0: auipc a1,0x169
3ef0b4: auipc a2,0x1eb
3ef0b8: addi a0,a0,1462 # 5da662 <hextbl+0x25f67>
3ef0bc: addi a1,a1,-1487 # 557ae1 <year_lengths+0x149>
3ef0c0: addi a2,a2,1197 # 5da561 <hextbl+0x25e66>
3ef0c4: li a3,119
3ef0c8: auipc ra,0xffce1
3ef0cc: jalr 1036(ra) # d04d4 <ExceptionalCondition>
3ef0d0: auipc a0,0x1eb
3ef0d4: auipc a1,0x169
3ef0d8: auipc a2,0x1eb
3ef0dc: addi a0,a0,1350 # 5da616 <hextbl+0x25f1b>
3ef0e0: addi a1,a1,-1523 # 557ae1 <year_lengths+0x149>
3ef0e4: addi a2,a2,1161 # 5da561 <hextbl+0x25e66>
3ef0e8: li a3,123
3ef0ec: auipc ra,0xffce1
3ef0f0: jalr 1000(ra) # d04d4 <ExceptionalCondition>
3ef0f4: auipc a0,0x1eb
3ef0f8: auipc a1,0x169
3ef0fc: auipc a2,0x1eb
3ef100: addi a0,a0,1369 # 5da64d <hextbl+0x25f52>
3ef104: addi a1,a1,-1559 # 557ae1 <year_lengths+0x149>
3ef108: addi a2,a2,1125 # 5da561 <hextbl+0x25e66>
3ef10c: li a3,131
3ef110: auipc ra,0xffce1
3ef114: jalr 964(ra) # d04d4 <ExceptionalCondition>
3ef118: auipc a0,0x1eb
3ef11c: auipc a1,0x169
3ef120: auipc a2,0x1eb
3ef124: addi a0,a0,1354 # 5da662 <hextbl+0x25f67>
3ef128: addi a1,a1,-1595 # 557ae1 <year_lengths+0x149>
3ef12c: addi a2,a2,1089 # 5da561 <hextbl+0x25e66>
3ef130: li a3,119
3ef134: auipc ra,0xffce1
3ef138: jalr 928(ra) # d04d4 <ExceptionalCondition>
3ef13c: auipc a0,0x1ed
3ef140: auipc a1,0x169
3ef144: auipc a2,0x1ed
3ef148: addi a0,a0,-1421 # 5dbbaf <hextbl+0x274b4>
3ef14c: addi a1,a1,-1631 # 557ae1 <year_lengths+0x149>
3ef150: addi a2,a2,-1914 # 5db9ca <hextbl+0x272cf>
3ef154: li a3,1843
3ef158: auipc ra,0xffce1
3ef15c: jalr 892(ra) # d04d4 <ExceptionalCondition>
3ef160: auipc a0,0x181
3ef164: auipc a1,0x169
3ef168: auipc a2,0x1ed
3ef16c: addi a0,a0,-1304 # 56fc48 <year_lengths+0x182b0>
3ef170: addi a1,a1,-1667 # 557ae1 <year_lengths+0x149>
3ef174: addi a2,a2,-1950 # 5db9ca <hextbl+0x272cf>
3ef178: li a3,1873
3ef17c: auipc ra,0xffce1
3ef180: jalr 856(ra) # d04d4 <ExceptionalCondition>
3ef184: auipc a0,0x17f
3ef188: auipc a1,0x17f
3ef18c: auipc a2,0x17f
3ef190: addi a0,a0,1487 # 56e753 <year_lengths+0x16dbb>
3ef194: addi a1,a1,-1227 # 56dcbd <year_lengths+0x16325>
3ef198: addi a2,a2,-1214 # 56dcce <year_lengths+0x16336>
3ef19c: li a3,403
3ef1a0: auipc ra,0xffce1
3ef1a4: jalr 820(ra) # d04d4 <ExceptionalCondition>
3ef1a8: auipc a0,0x17f
3ef1ac: auipc a1,0x17f
3ef1b0: auipc a2,0x17f
3ef1b4: addi a0,a0,1451 # 56e753 <year_lengths+0x16dbb>
3ef1b8: addi a1,a1,-1263 # 56dcbd <year_lengths+0x16325>
3ef1bc: addi a2,a2,-1250 # 56dcce <year_lengths+0x16336>
3ef1c0: li a3,374
3ef1c4: auipc ra,0xffce1
3ef1c8: jalr 784(ra) # d04d4 <ExceptionalCondition>
3ef1cc: auipc a0,0x17f
3ef1d0: auipc a1,0x17f
3ef1d4: auipc a2,0x17f
3ef1d8: addi a0,a0,1415 # 56e753 <year_lengths+0x16dbb>
3ef1dc: addi a1,a1,-1299 # 56dcbd <year_lengths+0x16325>
3ef1e0: addi a2,a2,-1286 # 56dcce <year_lengths+0x16336>
3ef1e4: li a3,403
3ef1e8: auipc ra,0xffce1
3ef1ec: jalr 748(ra) # d04d4 <ExceptionalCondition>
3ef1f0: auipc a0,0x1ed
3ef1f4: auipc a1,0x169
3ef1f8: auipc a2,0x1ec
3ef1fc: addi a0,a0,-980 # 5dbe1c <hextbl+0x27721>
3ef200: addi a1,a1,-1811 # 557ae1 <year_lengths+0x149>
3ef204: addi a2,a2,2002 # 5db9ca <hextbl+0x272cf>
3ef208: li a3,1007
3ef20c: auipc ra,0xffce1
3ef210: jalr 712(ra) # d04d4 <ExceptionalCondition>
/*
* lrsc_tiny.c
* Two-core reproducer for lost LR/SC updates on RISC-V.
*
* Core A releases an LWLock as LWLockRelease() does (state = &lock->state = lock + 4; waiting: a byte on another
* page, at the page offset of state + n):
*
* sb pending_wakeup, (waiting)
* lw old_state, 4(lock)
* lw head, 8(lock)
* and desired_state, old_state, mask
* 1: lr.w cur, (state)
* bne cur, old_state, 2f
* sc.w tmp, desired_state, (state)
* bnez tmp, 1b
* 2:
*
* while core B adds 1 to state. Depending on n, A's SC might succeed although B's add came in between; the exit
* status is then 1.
*/
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>
#define FLAGS (7u << 28) /* HAS_WAITERS, RELEASE_OK, LOCKED */
/* set by A's release in round r, kept by the CAS */
#define TAG(r) (((r) % 0xff + 1) << 8)
#define TAG_MASK (0xff << 8)
#define PAGE_SIZE 4096
#define NBUFFERS 512 /* 32 KiB of 64-byte BufferDescs */
struct shared
{
uint32_t done __attribute__((aligned(128)));
uint8_t pgproc[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); /* waiter->lwWaiting */
uint8_t buffers[NBUFFERS][64];
};
/* round r's word: at offset 40 as content_lock.state in a BufferDesc */
static uint32_t *
word(struct shared *sh, int r)
{
return (uint32_t *) &sh->buffers[r % NBUFFERS][40];
}
/* LWLockWakeup()'s last store and final CAS: mark the waiter, clear FLAGS; returns 1 if B came in first */
static int
wakeup_unlock(uint32_t *state, uint8_t *waiting)
{
char *lock = (char *) state - 4; /* the LWLock: tranche, state, waiters */
long old_state, head, desired_state, cur, tmp;
__asm__ __volatile__(
/* waiter->lwWaiting = LW_WS_PENDING_WAKEUP */
" sb %[pending_wakeup], 0(%[waiting])\n"
/* old_state = pg_atomic_read_u32(&lock->state) */
" lw %[old_state], 4(%[lock])\n"
/* proclist_is_empty(&lock->waiters) */
" lw %[head], 8(%[lock])\n"
/* desired_state = old_state & ~(HAS_WAITERS | RELEASE_OK | LOCKED) */
" and %[desired_state], %[old_state], %[mask]\n"
/* pg_atomic_compare_exchange_u32(&lock->state, &old_state, ...) */
"1: lr.w %[cur], (%[state])\n"
" bne %[cur], %[old_state], 2f\n"
" sc.w %[tmp], %[desired_state], (%[state])\n"
" bnez %[tmp], 1b\n"
"2:\n"
: [old_state] "=&r"(old_state), [head] "=&r"(head), [cur] "=&r"(cur), [tmp] "=&r"(tmp),
[desired_state] "=&r"(desired_state)
: [lock] "r"(lock), [state] "r"(state), [waiting] "r"(waiting), [pending_wakeup] "r"(2L),
[mask] "r"((long) ~FLAGS)
: "memory");
if (cur == old_state)
return 0;
/* B came in: retry from what the LR saw, as the while (true) loop does */
uint32_t o = (uint32_t) cur;
while (!__atomic_compare_exchange_n(state, &o, o & ~FLAGS, 0, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST))
;
return 1;
}
int
main(int argc, char **argv)
{
int n = 0, c, r;
int retried = 0, lost = 0;
struct shared *sh = mmap(NULL, sizeof(*sh), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
if (sh == MAP_FAILED)
return 2;
while ((c = getopt(argc, argv, "n:")) != -1)
if (c == 'n')
n = atoi(optarg);
else
return 2;
if (fork() == 0)
{
/*
* B: once A has released state, add 1; as many rounds as A. Only A's release sets this round's TAG (its CAS
* keeps it), and a reused buffer still has an older one, so B's add comes after that store.
*/
for (r = 1; r <= 1000000; r++)
{
uint32_t *state = word(sh, r);
while ((__atomic_load_n(state, __ATOMIC_RELAXED) & TAG_MASK) != TAG(r))
;
__atomic_fetch_add(state, 1, __ATOMIC_SEQ_CST);
__atomic_store_n(&sh->done, r, __ATOMIC_RELEASE);
}
_exit(0);
}
/* A: release state, clear FLAGS, check B's +1 */
for (r = 1; r <= 1000000; r++)
{
uint32_t *state = word(sh, r), v;
/* vary A's timing relative to B */
for (volatile uint32_t j = 0; j < (r >> 12) % 257; j++)
;
__atomic_store_n(state, FLAGS | TAG(r), __ATOMIC_RELAXED);
retried += wakeup_unlock(state, sh->pgproc + (((uintptr_t) state + n) & (PAGE_SIZE - 1)));
while (__atomic_load_n(&sh->done, __ATOMIC_ACQUIRE) != r)
;
v = __atomic_load_n(state, __ATOMIC_SEQ_CST);
if (v != TAG(r) + 1 && lost++ < 10)
printf("LOST UPDATE in round %d: value=0x%08x expected=0x%08x\n", r, v, TAG(r) + 1);
}
printf("page_offset=%d rounds=%d retried=%d lost=%d: %s\n", n, r - 1, retried, lost, lost ? "FAILED" : "OK");
return lost != 0;
}
