Re: Installer disk info improvement (was - Re: querying scsi id/wwn for scsi disk)

[Jiri B]

On Sun, Apr 02, 2017 at 06:14:50PM -0400, Ted Unangst wrote:
> Robert Peichaer wrote:
> > Parsing dmesg output always tends to be fragile, but what about this?
> > Use whatever is enclosed in <> in the dmesg output for a disk and get
> > the size from disklabel.
> 
> This looks insane. If somebody can tell us what output they want, we can
> provide it in a more useful interface (sysctl, etc.) Then it might be useful
> in other scenarios too.

Hi,

yes IMO parsing /var/run/dmesg.boot is silly. It would be better way to have
better interface to list disks, although I'm not able to provide diffs.

My use case was running OpenBSD under qemu-kvm with direct-lun iscsi disks,
all having same lun size, passed via qemu natively or via local block device
on a Linux host. And my concern was how to distinguish these equaly big luns
inside installer.

j.

Re: Installer disk info improvement (was - Re: querying scsi id/wwn for scsi disk)

[Ted Unangst]

Robert Peichaer wrote:
> Parsing dmesg output always tends to be fragile, but what about this?
> Use whatever is enclosed in <> in the dmesg output for a disk and get
> the size from disklabel.

This looks insane. If somebody can tell us what output they want, we can
provide it in a more useful interface (sysctl, etc.) Then it might be useful
in other scenarios too.

Re: Installer disk info improvement (was - Re: querying scsi id/wwn for scsi disk)

[Robert Peichaer]

On Thu, Mar 30, 2017 at 09:10:01AM -0400, Jiri B wrote:
> > > > diff -u -p -r1.988 install.sub
> > > > --- distrib/miniroot/install.sub13 Mar 2017 17:08:31 -  
> > > > 1.988
> > > > +++ distrib/miniroot/install.sub30 Mar 2017 10:44:01 -
> > > > @@ -264,13 +264,7 @@ diskinfo() {
> > > > local _d
> > > >  
> > > > for _d; do
> > > > -   make_dev $_d
> > > > -   echo -n "$_d: "
> > > > -   disklabel -dpg $_d 2>/dev/null |
> > > > -   sed -e '/^label: /{s,,,;s/ *$//;s/^$/ > > > label>/;h;d;}' \
> > > > -   -e '/.*# total bytes: \(.*\)/{s//(\1)/;H;}' \
> > > > -   -e '$!d;x;s/\n/ /'
> > > > -   rm -f /dev/{r,}$_d?
> > > > +   sed -n "/^$_d/p" /var/run/dmesg.boot
> > > > done
> > > >  }
> > > > 
> > > 
> > > Your proposition is good for the installer? I doubt it.
> > > 
> > > j.
> > 
> > AFAICT the function diskinfo() is only called once in the installer: if
> > you press ? a the prompt for the root disk. So my diff just changes the
> > output in this case, no other functionality is affected.
> > 
> > What causes your doubt?
> 
> Robert,
> 
> could we use something like this? From dmesg we can get current
> vendor, model, size plus serial if it does exist, 'sd0' could be grepped
> before sed or we could put variable inside sed itself:
> 
> sed -e '/^sd0 at.*: <[A-Z]*, \([^,]*\).*fixed *\(.*\)/{s//\1 <\2>/;s/< 
> *>$//;h;d;}' -e '/sd0: \([^,]*\).*/{s//(\1)/;H;}' -e '$!d;x;s/\n/ 
> /' /var/run/dmesg.boot
> SAMSUNG MZ7TE256  (244198MB)
> 
> If there's no serial it maybe could print this?
> 
> cat /var/run/dmesg.boot | sed 's/fixed.*/fixed/;' | sed -e '/^sd0 at.*: 
> <[A-Z]*, \([^,]*\).*fixed *\(.*\)/{s//\1 <\2>/;s/< *>$//;h;d;}' -e 
> '/sd0: \([^,]*\).*/{s//(\1)/;H;}' -e '$!d;x;s/\n/ /'
> SAMSUNG MZ7TE256  (244198MB)
> 
> What do you think?
> 
> PS: sed is really hardcore :)
> 
> j.

Parsing dmesg output always tends to be fragile, but what about this?
Use whatever is enclosed in <> in the dmesg output for a disk and get
the size from disklabel.

VMware ESXi:
sd0 at scsibus1 targ 0 lun 0:  SCSI3 0/direct 
fixed naa.5000c2994057dedf

QEMU using virtio:
sd0 at scsibus1 targ 0 lun 0:  SCSI3 0/direct fixed

My notebook:
sd0 at scsibus1 targ 0 lun 0:  SCSI3 0/direct fixed 
naa.500a07511210a1b8

>From an ppc imac:
wd0 at pciide0 channel 0 drive 0: 


sd0: VirtIO, Block Device, (10.0G)
sd0: ATA, VMware Virtual S,  naa.5000c2994057dedf (51.2G)
sd0: ATA, MTFDDAK512MBF, LN01 naa.500a07511210a1b8 (512.0G)
wd0: Samsung SSD 840 EVO 120GB (120.0G)


Index: install.sub
===
RCS file: /cvs/src/distrib/miniroot/install.sub,v
retrieving revision 1.989
diff -u -p -p -u -r1.989 install.sub
--- install.sub 31 Mar 2017 18:36:49 -  1.989
+++ install.sub 2 Apr 2017 20:13:04 -
@@ -266,10 +266,11 @@ diskinfo() {
for _d; do
make_dev $_d
echo -n "$_d: "
-   disklabel -dpg $_d 2>/dev/null |
-   sed -e '/^label: /{s,,,;s/ *$//;s/^$//;h;d;}' \
-   -e '/.*# total bytes: \(.*\)/{s//(\1)/;H;}' \
-   -e '$!d;x;s/\n/ /'
+   set --  $(sed -n "/^$_d at /{s/^.* .* \(naa\..*\)$/ 
\1/;s/> .*$//;p;}" \
+   /var/run/dmesg.boot 2>/dev/null) \
+   $(disklabel -dpg $_d 2>/dev/null |
+   sed -n '/.*# total bytes: \(.*\)/s//(\1)/p')
+   echo "$*"
rm -f /dev/{r,}$_d?
done
 }
===
Stats: --- 4 lines 166 chars
Stats: +++ 5 lines 220 chars
Stats: 1 lines
Stats: 54 chars

-- 
-=[rpe]=-

Installer disk info improvement (was - Re: querying scsi id/wwn for scsi disk)

[Jiri B]

> > > diff -u -p -r1.988 install.sub
> > > --- distrib/miniroot/install.sub  13 Mar 2017 17:08:31 -  1.988
> > > +++ distrib/miniroot/install.sub  30 Mar 2017 10:44:01 -
> > > @@ -264,13 +264,7 @@ diskinfo() {
> > >   local _d
> > >  
> > >   for _d; do
> > > - make_dev $_d
> > > - echo -n "$_d: "
> > > - disklabel -dpg $_d 2>/dev/null |
> > > - sed -e '/^label: /{s,,,;s/ *$//;s/^$//;h;d;}' \
> > > - -e '/.*# total bytes: \(.*\)/{s//(\1)/;H;}' \
> > > - -e '$!d;x;s/\n/ /'
> > > - rm -f /dev/{r,}$_d?
> > > + sed -n "/^$_d/p" /var/run/dmesg.boot
> > >   done
> > >  }
> > > 
> > 
> > Your proposition is good for the installer? I doubt it.
> > 
> > j.
> 
> AFAICT the function diskinfo() is only called once in the installer: if
> you press ? a the prompt for the root disk. So my diff just changes the
> output in this case, no other functionality is affected.
> 
> What causes your doubt?

Robert,

could we use something like this? From dmesg we can get current
vendor, model, size plus serial if it does exist, 'sd0' could be grepped
before sed or we could put variable inside sed itself:

sed -e '/^sd0 at.*: <[A-Z]*, \([^,]*\).*fixed *\(.*\)/{s//\1 <\2>/;s/< *>$//;h;d;}' -e '/sd0: \([^,]*\).*/{s//(\1)/;H;}' -e '$!d;x;s/\n/ /' 
/var/run/dmesg.boot
SAMSUNG MZ7TE256  (244198MB)

If there's no serial it maybe could print this?

cat /var/run/dmesg.boot | sed 's/fixed.*/fixed/;' | sed -e '/^sd0 at.*: 
<[A-Z]*, \([^,]*\).*fixed *\(.*\)/{s//\1 <\2>/;s/< *>$//;h;d;}' -e 
'/sd0: \([^,]*\).*/{s//(\1)/;H;}' -e '$!d;x;s/\n/ /'
SAMSUNG MZ7TE256  (244198MB)

What do you think?

PS: sed is really hardcore :)

j.

Re: querying scsi id/wwn for scsi disk

[Bruno Flueckiger]

On Thu, Mar 30, 2017 at 07:21:22AM -0400, Jiri B wrote:
> On Thu, Mar 30, 2017 at 12:59:00PM +0200, Bruno Flueckiger wrote:
> > I see your point with the installer. Default labels make the disks
> > indistinguishable. The following diff prints the raw infos from dmesg
> > rather than the current list of disks:
> > 
> > Index: distrib/miniroot/install.sub
> > ===
> > RCS file: /cvs/src/distrib/miniroot/install.sub,v
> > retrieving revision 1.988
> > diff -u -p -r1.988 install.sub
> > --- distrib/miniroot/install.sub13 Mar 2017 17:08:31 -  1.988
> > +++ distrib/miniroot/install.sub30 Mar 2017 10:44:01 -
> > @@ -264,13 +264,7 @@ diskinfo() {
> > local _d
> >  
> > for _d; do
> > -   make_dev $_d
> > -   echo -n "$_d: "
> > -   disklabel -dpg $_d 2>/dev/null |
> > -   sed -e '/^label: /{s,,,;s/ *$//;s/^$//;h;d;}' \
> > -   -e '/.*# total bytes: \(.*\)/{s//(\1)/;H;}' \
> > -   -e '$!d;x;s/\n/ /'
> > -   rm -f /dev/{r,}$_d?
> > +   sed -n "/^$_d/p" /var/run/dmesg.boot
> > done
> >  }
> > 
> 
> Your proposition is good for the installer? I doubt it.
> 
> j.

AFAICT the function diskinfo() is only called once in the installer: if
you press ? a the prompt for the root disk. So my diff just changes the
output in this case, no other functionality is affected.

What causes your doubt?

Re: querying scsi id/wwn for scsi disk

[Jiri B]

On Thu, Mar 30, 2017 at 12:59:00PM +0200, Bruno Flueckiger wrote:
> I see your point with the installer. Default labels make the disks
> indistinguishable. The following diff prints the raw infos from dmesg
> rather than the current list of disks:
> 
> Index: distrib/miniroot/install.sub
> ===
> RCS file: /cvs/src/distrib/miniroot/install.sub,v
> retrieving revision 1.988
> diff -u -p -r1.988 install.sub
> --- distrib/miniroot/install.sub  13 Mar 2017 17:08:31 -  1.988
> +++ distrib/miniroot/install.sub  30 Mar 2017 10:44:01 -
> @@ -264,13 +264,7 @@ diskinfo() {
>   local _d
>  
>   for _d; do
> - make_dev $_d
> - echo -n "$_d: "
> - disklabel -dpg $_d 2>/dev/null |
> - sed -e '/^label: /{s,,,;s/ *$//;s/^$//;h;d;}' \
> - -e '/.*# total bytes: \(.*\)/{s//(\1)/;H;}' \
> - -e '$!d;x;s/\n/ /'
> - rm -f /dev/{r,}$_d?
> + sed -n "/^$_d/p" /var/run/dmesg.boot
>   done
>  }
> 

Your proposition is good for the installer? I doubt it.

j.

Re: querying scsi id/wwn for scsi disk

[Bruno Flueckiger]

On Thu, Mar 30, 2017 at 04:53:51AM -0400, Jiri B wrote:
> On Thu, Mar 30, 2017 at 10:25:18AM +0200, Bruno Flueckiger wrote:
> > > how to query scsi id or wwn for a scsi disk in OpenBSD? I'd like to get 
> > > this
> > > info and extend installer to provide more info about disks (because 
> > > currently
> > > it's imposible to distinguish between scsi disks if they are same size and
> > > originate from same iscsi target and passed to OpenBSD via qemu-kvm).
> > > 
> > > So what's OpenBSD equivalent scsi query for Linux commands?
> > > 
> > > # lsscsi -iws | tail -n1
> > > [6:0:0:10]   disk0x6006048c8f0ff1a5c7ef85c8d1c95  /dev/sdd   
> > > 36006048c8f0ff1a5c7ef85c8d1c95481  16.1GB
> > > 
> > > # /usr/lib/udev/scsi_id -xg /dev/sdd
> > > ID_SCSI=1
> > > ID_VENDOR=EMC
> > > ID_VENDOR_ENC=EMC\x20\x20\x20\x20\x20
> > > ID_MODEL=Celerra
> > > ID_MODEL_ENC=Celerra\x20\x20\x20\x20\x20\x20\x20\x20\x20
> > > ID_REVISION=0002
> > > ID_TYPE=disk
> > > ID_SERIAL=36006048c8f0ff1a5c7ef85c8d1c95481
> > > ID_SERIAL_SHORT=6006048c8f0ff1a5c7ef85c8d1c95481
> > > ID_WWN=0x6006048c8f0ff1a5
> > > ID_WWN_VENDOR_EXTENSION=0xc7ef85c8d1c95481
> > > ID_WWN_WITH_EXTENSION=0x6006048c8f0ff1a5c7ef85c8d1c95481
> > > ID_SCSI_SERIAL=EMC-Celerra-iSCSI-VLU-fs179_T5_LUN10_CKM00120100230
> > 
> > $ dmesg | grep scsi
> > 
> > sd1 at scsibus2 targ 0 lun 0:  SCSI3
> > 0/direct fixed naa.6001405635870b3d9e95d40c9d9221d1
> > sd2 at scsibus2 targ 0 lun 1:  SCSI3
> > 0/direct fixed naa.6001405dcc70b1dd909ed44f8db0d6d6
> 
>  disklabel sd1 | grep label
> 
> For sd1 and sd2 please. This is what is printed in installer. See diskinfo()
> in src/distrib/miniroot/install.sub
> 
> If it does print just 'iSCSI Storage 3.1' then this is not very
> useful info in the installer (although one can break and investigate dmesg)
> 
> So maybe disklabel should have better 'label' or we could print more info
> directly in the installer.
> 
> I'm also not sure sysctl hw.disknames output is best one, it does show duid
> which is OpenBSD specific (compare disklabel with and without '-d').
> 
> It seems there's no good solution fitting all cases (softraid, usb disks,
> both are scsi-like devices).
> 
> j.

# disklabel sd1 | grep label
label: iSCSI Storage

# disklabel sd2 | grep label
label: iSCSI Storage

I see your point with the installer. Default labels make the disks
indistinguishable. The following diff prints the raw infos from dmesg
rather than the current list of disks:

Index: distrib/miniroot/install.sub
===
RCS file: /cvs/src/distrib/miniroot/install.sub,v
retrieving revision 1.988
diff -u -p -r1.988 install.sub
--- distrib/miniroot/install.sub13 Mar 2017 17:08:31 -  1.988
+++ distrib/miniroot/install.sub30 Mar 2017 10:44:01 -
@@ -264,13 +264,7 @@ diskinfo() {
local _d
 
for _d; do
-   make_dev $_d
-   echo -n "$_d: "
-   disklabel -dpg $_d 2>/dev/null |
-   sed -e '/^label: /{s,,,;s/ *$//;s/^$//;h;d;}' \
-   -e '/.*# total bytes: \(.*\)/{s//(\1)/;H;}' \
-   -e '$!d;x;s/\n/ /'
-   rm -f /dev/{r,}$_d?
+   sed -n "/^$_d/p" /var/run/dmesg.boot
done
 }

Re: querying scsi id/wwn for scsi disk

[Jiri B]

On Thu, Mar 30, 2017 at 10:25:18AM +0200, Bruno Flueckiger wrote:
> > how to query scsi id or wwn for a scsi disk in OpenBSD? I'd like to get this
> > info and extend installer to provide more info about disks (because 
> > currently
> > it's imposible to distinguish between scsi disks if they are same size and
> > originate from same iscsi target and passed to OpenBSD via qemu-kvm).
> > 
> > So what's OpenBSD equivalent scsi query for Linux commands?
> > 
> > # lsscsi -iws | tail -n1
> > [6:0:0:10]   disk0x6006048c8f0ff1a5c7ef85c8d1c95  /dev/sdd   
> > 36006048c8f0ff1a5c7ef85c8d1c95481  16.1GB
> > 
> > # /usr/lib/udev/scsi_id -xg /dev/sdd
> > ID_SCSI=1
> > ID_VENDOR=EMC
> > ID_VENDOR_ENC=EMC\x20\x20\x20\x20\x20
> > ID_MODEL=Celerra
> > ID_MODEL_ENC=Celerra\x20\x20\x20\x20\x20\x20\x20\x20\x20
> > ID_REVISION=0002
> > ID_TYPE=disk
> > ID_SERIAL=36006048c8f0ff1a5c7ef85c8d1c95481
> > ID_SERIAL_SHORT=6006048c8f0ff1a5c7ef85c8d1c95481
> > ID_WWN=0x6006048c8f0ff1a5
> > ID_WWN_VENDOR_EXTENSION=0xc7ef85c8d1c95481
> > ID_WWN_WITH_EXTENSION=0x6006048c8f0ff1a5c7ef85c8d1c95481
> > ID_SCSI_SERIAL=EMC-Celerra-iSCSI-VLU-fs179_T5_LUN10_CKM00120100230
> 
> $ dmesg | grep scsi
> 
> sd1 at scsibus2 targ 0 lun 0:  SCSI3
> 0/direct fixed naa.6001405635870b3d9e95d40c9d9221d1
> sd2 at scsibus2 targ 0 lun 1:  SCSI3
> 0/direct fixed naa.6001405dcc70b1dd909ed44f8db0d6d6

 disklabel sd1 | grep label

For sd1 and sd2 please. This is what is printed in installer. See diskinfo()
in src/distrib/miniroot/install.sub

If it does print just 'iSCSI Storage 3.1' then this is not very
useful info in the installer (although one can break and investigate dmesg)

So maybe disklabel should have better 'label' or we could print more info
directly in the installer.

I'm also not sure sysctl hw.disknames output is best one, it does show duid
which is OpenBSD specific (compare disklabel with and without '-d').

It seems there's no good solution fitting all cases (softraid, usb disks,
both are scsi-like devices).

j.

Re: querying scsi id/wwn for scsi disk

[Bruno Flueckiger]

On Thu, Mar 30, 2017 at 04:04:51AM -0400, Jiri B wrote:
> Hi,
> 
> how to query scsi id or wwn for a scsi disk in OpenBSD? I'd like to get this
> info and extend installer to provide more info about disks (because currently
> it's imposible to distinguish between scsi disks if they are same size and
> originate from same iscsi target and passed to OpenBSD via qemu-kvm).
> 
> Currently OpenBSD does show for such SCSI disk something like (taken from 
> disklabel):
> 
> ...
> Which disk is the root disk? ('?' for details) [sd0] ?
> sd0: Celerra (20.0G)
>  ^^^  ^
> 
> # scsi -f /dev/rsd0c -c "12 0 0 0 64 0" -i 0x64 "s8 z8 z16 z4"
> EMC Celerra 0002
> 
> So what's OpenBSD equivalent scsi query for Linux commands?
> 
> # lsscsi -iws | tail -n1
> [6:0:0:10]   disk0x6006048c8f0ff1a5c7ef85c8d1c95  /dev/sdd   
> 36006048c8f0ff1a5c7ef85c8d1c95481  16.1GB
> 
> # /usr/lib/udev/scsi_id -xg /dev/sdd
> ID_SCSI=1
> ID_VENDOR=EMC
> ID_VENDOR_ENC=EMC\x20\x20\x20\x20\x20
> ID_MODEL=Celerra
> ID_MODEL_ENC=Celerra\x20\x20\x20\x20\x20\x20\x20\x20\x20
> ID_REVISION=0002
> ID_TYPE=disk
> ID_SERIAL=36006048c8f0ff1a5c7ef85c8d1c95481
> ID_SERIAL_SHORT=6006048c8f0ff1a5c7ef85c8d1c95481
> ID_WWN=0x6006048c8f0ff1a5
> ID_WWN_VENDOR_EXTENSION=0xc7ef85c8d1c95481
> ID_WWN_WITH_EXTENSION=0x6006048c8f0ff1a5c7ef85c8d1c95481
> ID_SCSI_SERIAL=EMC-Celerra-iSCSI-VLU-fs179_T5_LUN10_CKM00120100230
> 
> Thank you for help, it would help me playing with iscsi luns on OpenBSD.
> 
> j.

Infos about disks are printed to the console as soon as a disk is
attached. So you can use dmesg and grep for scsi to get the info you
want:

$ dmesg | grep scsi

scsibus0 at mpath0: 256 targets
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 0 lun 0:  SCSI3 0/direct fixed
naa.
vscsi0 at root
scsibus2 at vscsi0: 256 targets
scsibus3 at softraid0: 256 targets
sd1 at scsibus2 targ 0 lun 0:  SCSI3
0/direct fixed naa.6001405635870b3d9e95d40c9d9221d1
sd2 at scsibus2 targ 0 lun 1:  SCSI3
0/direct fixed naa.6001405dcc70b1dd909ed44f8db0d6d6

Cheers,
Bruno