On Tue, Mar 12, 2024 at 1:28 PM Dennis Clarke <dcla...@blastwave.org> wrote: > > > This is a somewhat odd problem and may have nothing to do with iSCSI > config at all. Suffice it to say that I have the following in the server > /etc/rc.conf : > > # > # the iSCSI initiator > iscsid_enable="YES" > iscsictl_enable="YES" > iscsictl_flags="-Aa" > # > > During boot I see this on the console : > > > cannot import 'proteus': no suchpid 55 (zpool) is attempting to use > unsafe AIO requests - not logging anymore > pool or dataset > Destroy and re-create the pool from > a backup source. > cachefile import failed, retrying > no pools available to import > > > Sure enough the machine brings up a 10Gbit link with jumboframes *after* > the above messages : > > > ix0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> > metric 0 mtu 9000 > > options=4e53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG> > ether 8c:dc:d4:ae:18:b8 > inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 > media: Ethernet autoselect (10Gbase-Twinax > <full-duplex,rxpause,txpause>) > status: active > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > > > Then a little later I see iscsi doing its goodness : > > > da0 at iscsi1 bus 0 scbus8 target 0 lun 0 > da0: <FREEBSD CTLDISK 0001> Fixed Direct Access SPC-5 SCSI device > da0: Serial Number MYSERIAL0000 > da0: 150.000MB/s transfers > da0: Command Queueing enabled > da0: 2097152MB (4294967296 512 byte sectors) > add net ::0.0.0.0: gateway ::1 > Starting iscsid. > Starting iscsictl. > > The storage exists just fine and iSCSI seems to be doing its thing : > > root@titan:~ # > root@titan:~ # camcontrol devlist > <ST8000NM002A-2KE102 NN03> at scbus0 target 0 lun 0 (pass0,ada0) > <Samsung SSD 870 QVO 1TB SVQ01B6Q> at scbus1 target 0 lun 0 (pass1,ada1) > <AHCI SGPIO Enclosure 2.00 0001> at scbus2 target 0 lun 0 (ses0,pass2) > <AHCI SGPIO Enclosure 2.00 0001> at scbus6 target 0 lun 0 (ses1,pass3) > <SAMSUNG MZVKW512HMJP-000L7 6L6QCXA7> at scbus7 target 0 lun 1 (pass4,nda0) > <FREEBSD CTLDISK 0001> at scbus8 target 0 lun 0 (da0,pass5) > root@titan:~ # > root@titan:~ # gpart show da0 > => 40 4294967216 da0 GPT (2.0T) > 40 8 - free - (4.0K) > 48 4294967200 1 freebsd-zfs (2.0T) > 4294967248 8 - free - (4.0K) > > root@titan:~ # > > However the zpool therein is not seen : > > root@titan:~ # > root@titan:~ # zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP > HEALTH ALTROOT > iota 7.27T 597G 6.68T - - 0% 8% 1.00x > ONLINE - > t0 444G 40.8G 403G - - 4% 9% 1.00x > ONLINE - > root@titan:~ # > > > Of course I can manually import it : > > > root@titan:~ # zpool import > pool: proteus > id: 15277728307274839698 > state: ONLINE > status: Some supported features are not enabled on the pool. > (Note that they may be intentionally disabled if the > 'compatibility' property is set.) > action: The pool can be imported using its name or numeric identifier, > though > some features will not be available without an explicit 'zpool > upgrade'. > config: > > proteus ONLINE > da0p1 ONLINE > root@titan:~ # > > It seems as if there is something out of sequence and the iSCSI > processes should be happening earlier in the boot process? I really do > not know and am wondering why that zpool proteus on the iSCSI storage > needs to be manually import'ed after a reboot. > > Any insights would be wonderful. > > -- > Dennis Clarke > RISC-V/SPARC/PPC/ARM/CISC > UNIX and Linux spoken
Yes, this looks exactly like an ordering problem. zpools get imported early in the boot process, under the assumption that most of them are local. Networking comes up later, under the assumption that networking might require files that are mounted on ZFS. For you, I suggest setting proteus's cachefile to a non-default location and importing it from /etc/rc.local, like this: zpool set cachefile=/var/cache/iscsi-zpools.cache proteus Then in /etc/rc.local: zpool import -a -c /var/cache/iscsi-zpools.cache -o cachefile=/var/cache/iscsi-zpools.cache -Alan
