Hi,
Thank you for your feeeback.
Also OpenBSD 6.9beta snapshot is crashing when I setup RAID5 with three
"Samsung PRO 860 1TB" SSDs.
OpenBSD obsd69b.it-infra.org 6.9 GENERIC.MP#368 amd64
obsd69b# dmesg | grep -i bios
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xdc312018 (61 entries)
bios0: vendor American Megatrends Inc. version "2201" date 03/23/2015
bios0: ASUSTeK COMPUTER INC. CROSSHAIR V FORMULA-Z
acpi0 at bios0: ACPI 5.0
# ---
sha512sum install69_amd64_snapshot.img
9bcf7feae186966cecf0de041d283ef27678ef350fd5c17e953e3d27d719aa8e3b034634b65ac04794d8568a7bb4807f7673fcdb6a86a66808ba0a55f4087368
install69_amd64_snapshot.img
md5sum -b install69_amd64_snapshot.img
0408ba32001dccb6465f76faefc6eb8a *install69_amd64_snapshot.img
OpenBSD 6.8 or 6.9beta do not crash writing big files to RAID5 when the
SSDs are "Samsung PRO 850 1TB" or "Intel DC S3520 150GB"
Today I have used only 6G SATA cables for the RAID configuration.
What I really worry about is the fact, that OpenBSD 6.8 or 6.9beta is
crashing with RAID5 based on Samsung PRO 860 512MB or 1TB SSD drives.
Kind regards
Mark
On 01.03.21 15:35, Atanas Vladimirov wrote:
On 2021-02-27 03:35, Mark Schneider wrote:
Hi,
I face system crash on OpenBSD 6.8 when trying to use softraid RAID5
drive trying to write big files (like 10GBytes) to it.
I can reproduce the error (tested on two different systems with
OpenBSD 6.8 installed on an SSD drive or an USB stick). The RAID5
drive itself consist of six Samsung PRO 860 512GB SSDs.
In short:
bioctl -c 5 -l sd0a,sd1a,sd2a,sd3a,sd4a,sd5a softraid0
obsdssdarc# disklabel sd7
# /dev/rsd7c:
type: SCSI
disk: SCSI disk
label: SR RAID 5
duid: a50fb9a25bf07243
flags:
bytes/sector: 512
sectors/track: 255
tracks/cylinder: 511
sectors/cylinder: 130305
cylinders: 38379
total sectors: 5001073280
boundstart: 0
boundend: 5001073280
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 5001073280 0 4.2BSD 8192 65536 52270
c: 5001073280 0 unused
#
--------------------------------------------------------------------------------
obsdssdarc# time dd if=/dev/urandom of=/arc-ssd/1GB-urandom.bin bs=1M
count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 8.120 secs (132218264 bytes/sec)
0m08.13s real 0m00.00s user 0m08.14s system
# Working as expected
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
obsdssdarc# time dd if=/dev/urandom of=/arc-ssd/10GB-urandom.bin
bs=10M count=1024
# Error messages
uvm_fault(0xffffffff821f5490, 0x40, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at sr_validate_io+0x44: cmpl $0,0x40(%r9)
ddb{2}>
# Crashing OpenBSD 6.8
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# After reboot:
obsdssdarc# mount /dev/sd7a /arc-ssd/
mount_ffs: /dev/sd7a on /arc-ssd: Device not configure
obsdssdarc# grep sd7 /var/run/dmesg.boot
softraid0: trying to bring up sd7 degraded
softraid0: sd7 was not shutdown properly
softraid0: sd7 is offline, will not be brought online
More details in attached files. Thanks a lot in advance for short
feedback.
Kind regards
Mark
Hi,
I can't reproduce this on a resent snapshot:
```
temp# bioctl sd5
Volume Status Size Device
softraid0 1 Online 2356352188416 sd5 RAID5
0 Online 785450747904 1:0.0 noencl <sd0d>
1 Online 785450747904 1:1.0 noencl <sd1d>
2 Online 785450747904 1:2.0 noencl <sd2d>
3 Online 785450747904 1:3.0 noencl <sd3d>
temp# time dd if=/dev/urandom of=20GB-urandom.bin bs=10M count=2048
2048+0 records in
2048+0 records out
21474836480 bytes transferred in 1845.313 secs (11637500 bytes/sec)
30m45.95s real 0m00.01s user 2m21.61s system
temp# ls -lh
total 41948288
-rw-r--r-- 1 root wheel 20.0G Mar 1 13:05 20GB-urandom.bin
```
```
OpenBSD 6.9-beta (GENERIC.MP) #366: Sun Feb 28 07:15:39 MST 2021
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17134055424 (16340MB)
avail mem = 16599408640 (15830MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xeb5a0 (56 entries)
bios0: vendor American Megatrends Inc. version "2.3a" date 01/06/2021
bios0: Supermicro X9SCL/X9SCM
```
# OpenBSD 6.9beta RAID5 configuration with three 1TB "Samsung SSD PRO 860"
drives
sysctl hw.disknames
disklabel sd1
disklabel -E sd1
disklabel -E sd2
odisklabel -E sd3
bioctl -c 5 -l sd1a,sd2a,sd3a softraid0
disklabel -E sd4
newfs sd4a
obsdarc# mkdir /arc-3xssd
obsdarc# mount /dev/sd4a /arc-3xssd/
obsdarc# df -h | grep 3xssd
/dev/sd4a 1.8T 8.0K 1.8T 0% /arc-3x1TB-ssd860
# ------------------------------------------------------------------------------
obsd69b# dd if=/dev/urandom of=/arc-3x1TB-ssd860/1GB-urandom.bin bs=1M
count=1024
# Error messages
uvm_fault(0xffffffff821c3608, 0x40, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at sr_validate_io+0x44: cmpl $0,0x40(%r9)
ddb{4}>
obsd69b# bioctl sd4
Volume Status Size Device
softraid0 0 Online 2048418512896 sd4 RAID5
0 Online 1024209272832 0:0.0 noencl <sd1a>
1 Online 1024209272832 0:1.0 noencl <sd2a>
2 Online 1024209272832 0:2.0 noencl <sd3a>
obsd69b# disklabel sd4
# /dev/rsd4c:
type: SCSI
disk: SCSI disk
label: SR RAID 5
duid: 2f9692cd2e3a048f
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 249039
total sectors: 4000817408
boundstart: 0
boundend: 4000817408
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 4000817408 0 4.2BSD 8192 65536 52270
c: 4000817408 0 unused
obsd69b# bioctl sd1
sd1: <ATA, Samsung SSD 860, RVM0>, serial S42NNF0N110543P
obsd69b# bioctl sd2
sd2: <ATA, Samsung SSD 860, RVM0>, serial S42NNF0MA01951H
obsd69b# bioctl sd3
sd3: <ATA, Samsung SSD 860, RVM0>, serial S42NNF0M603477P
obsd69b# disklabel sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label: Samsung SSD 860
duid: cb0d589d6d25894e
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 124519
total sectors: 2000409264
boundstart: 0
boundend: 2000409264
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 2000409264 0 RAID
c: 2000409264 0 unused
obsd69b# # -------------------------------------------------------
obsd69b# disklabel sd2
# /dev/rsd2c:
type: SCSI
disk: SCSI disk
label: Samsung SSD 860
duid: 2f129644e0687c3e
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 124519
total sectors: 2000409264
boundstart: 0
boundend: 2000409264
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 2000409264 0 RAID
c: 2000409264 0 unused
obsd69b# # -------------------------------------------------------
obsd69b# disklabel sd3
# /dev/rsd3c:
type: SCSI
disk: SCSI disk
label: Samsung SSD 860
duid: 044f97d5c339c078
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 124519
total sectors: 2000409264
boundstart: 0
boundend: 2000409264
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 2000409264 0 RAID
c: 2000409264 0 unused
# ------------------------------------------------------------------------------
obsd69b# dd if=/dev/urandom of=/arc-3x1TB-ssd860/10MB-urandom.bin bs=1M
count=10
10+0 records in
10+0 records out
10485760 bytes transferred in 0.059 secs (176067598 bytes/sec)
obsd69b# dd if=/dev/urandom of=/arc-3x1TB-ssd860/100MB-urandom.bin bs=1M
count=102
102+0 records in
102+0 records out
106954752 bytes transferred in 0.607 secs (175942846 bytes/sec)
obsd69b# dd if=/dev/urandom of=/arc-3x1TB-ssd860/1GB-urandom.bin bs=1M
count=1024
# Error messages
uvm_fault(0xffffffff82242ae8, 0x40, 0, 1) -> e
kernel: page fault trap, code=0
Stopped at sr_validate_io+0x44: cmpl $0,0x40(%r9)
ddb{6}>
# OpenBSD 6.9beta RAID5 with three Intel SSD DC S3520 Series 150GB
1 sysctl hw.disknames
2 dd if=/dev/zero of=/dev/rsd1c bs=1m count=1024
3 dd if=/dev/zero of=/dev/rsd2c bs=1m count=1024
4 dd if=/dev/zero of=/dev/rsd3c bs=1m count=1024
5 disklabel -E sd1
6 disklabel -E sd2
7 disklabel -E sd3
8 bioctl -c 5 -l sd1a,sd2a,sd3a softraid0
9 disklabel -E sd4
10 newfs sd4a
11 ls -l /
12 mount /dev/sd4a /arc-3x150MB-ssd-DC_S3520
16 bioctl sd4
17 bioctl sd1
18 bioctl sd2
19 bioctl sd3
obsd69b# dd if=/dev/urandom of=/arc-3x150MB-ssd-DC_S3520/1GB-urandom.bin bs=1M
count=1024
1024+0 records in
1024+0 records out
1073741824 bytes transferred in 16.396 secs (65485091 bytes/sec)
obsd69b# dd if=/dev/urandom of=/arc-3x150MB-ssd-DC_S3520/10GB-urandom.bin
bs=10M count=1024
1024+0 records in
1024+0 records out
10737418240 bytes transferred in 146.004 secs (73541829 bytes/sec)
obsd69b# bioctl sd4
Volume Status Size Device
softraid0 0 Online 300079251456 sd4 RAID5
0 Online 150039674880 0:0.0 noencl <sd1a>
1 Online 150039674880 0:1.0 noencl <sd2a>
2 Online 150039674880 0:2.0 noencl <sd3a>
obsd69b# bioctl sd1
sd1: <ATA, VK000150GWCNN, 4IDV>, serial BTDV744506JG150MGN
obsd69b# bioctl sd2
sd2: <ATA, VK000150GWCNN, 4IDV>, serial BTDV744506FH150MGN
obsd69b# bioctl sd3
sd3: <ATA, VK000150GWCNN, 4IDV>, serial BTDV744505Q5150MGN
obsd69b# disklabel sd4
# /dev/rsd4c:
type: SCSI
disk: SCSI disk
label: SR RAID 5
duid: 6d47413e0a4d04ae
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 36482
total sectors: 586092288
boundstart: 0
boundend: 586092288
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 586092288 0 4.2BSD 4096 32768 26062
c: 586092288 0 unused
obsd69b# # --------------------------------------------------------
obsd69b# disklabel sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label: VK000150GWCNN
duid: 2b944d630814b838
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 18241
total sectors: 293046768
boundstart: 0
boundend: 293046768
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 293046768 0 RAID
c: 293046768 0 unused
obsd69b# # --------------------------------------------------------
obsd69b# disklabel sd2
# /dev/rsd2c:
type: SCSI
disk: SCSI disk
label: VK000150GWCNN
duid: 33bcea27bcd27ae5
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 18241
total sectors: 293046768
boundstart: 0
boundend: 293046768
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 293046768 0 RAID
c: 293046768 0 unused
obsd69b# # --------------------------------------------------------
obsd69b# disklabel sd3
# /dev/rsd3c:
type: SCSI
disk: SCSI disk
label: VK000150GWCNN
duid: dd214d21bc000e8b
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 18241
total sectors: 293046768
boundstart: 0
boundend: 293046768
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 293046768 0 RAID
c: 293046768 0 unused
obsdarc# history -0
1 sysctl hw.disknames
2 dd if=/dev/zero of=/dev/rsd1c bs=1m count=1024
3 dd if=/dev/zero of=/dev/rsd2c bs=1m count=1024
4 dd if=/dev/zero of=/dev/rsd3c bs=1m count=1024
5 dd if=/dev/zero of=/dev/rsd4c bs=1m count=1024
6 dd if=/dev/zero of=/dev/rsd5c bs=1m count=1024
7 dd if=/dev/zero of=/dev/rsd6c bs=1m count=1024
8 disklabel -E sd1
9 disklabel -E sd2
10 disklabel -E sd2
11 disklabel -E sd3
12 disklabel -E sd4
13 disklabel -E sd5
14 disklabel -E sd6
15 bioctl -c 5 -l sd1a,sd2a,sd3a,sd4a,sd5a,sd6a softraid0
16 disklabel -E sd7
17 newfs sd7a
18 mkdir /arc-ssd-850PRO-raid5
19 mount /dev/sd7a /arc-ssd-850PRO-raid5/
20 df -h
/dev/sd7a 4.6T 8.0K 4.4T 0% /arc-ssd-850PRO-raid5
obsdarc# disklabel sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label: Samsung SSD 850
duid: 60c8aae4fbe0a302
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 124519
total sectors: 2000409264
boundstart: 0
boundend: 2000409264
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 2000409264 0 RAID
c: 2000409264 0 unused
...
obsdarc# disklabel sd7
# /dev/rsd7c:
type: SCSI
disk: SCSI disk
label: SR RAID 5
duid: 53d64b664874270c
flags:
bytes/sector: 512
sectors/track: 255
tracks/cylinder: 511
sectors/cylinder: 130305
cylinders: 76758
total sectors: 10002043520
boundstart: 0
boundend: 10002043520
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 10002043520 0 4.2BSD 8192 65536 52270
c: 10002043520 0 unused
obsdarc# dd if=/dev/urandom of=/arc-ssd-850PRO-raid5/10GB-urandom.bin bs=10M
count=1024
1024+0 records in
1024+0 records out
10737418240 bytes transferred in 119.049 secs (90192901 bytes/sec)
