Hi there Thanks --- Gilberto Nunes Ferreira
(47) 3025-5907 (47) 99676-7530 - Whatsapp / Telegram Skype: gilberto.nunes36 Em qui., 12 de dez. de 2019 às 00:19, Kriengsak Panphotong <is...@buu.ac.th> escreveu: > Hi list, > > I found solution on this link. > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932307 > > you need add : "find_multipaths yes" to /etc/multipath.conf > > # multipath.conf example > defaults { > polling_interval 2 > path_selector "round-robin 0" > path_grouping_policy multibus > uid_attribute ID_SERIAL > rr_min_io 100 > failback immediate > no_path_retry queue > user_friendly_names yes > find_multipaths yes > } > > after add , restart multipath-tools (/etc/init.d/multipath-tools restart) > its work great for me. > > ------------------------------ > *จาก:* pve-user <pve-user-boun...@pve.proxmox.com> ในนามของ Gilberto > Nunes <gilberto.nune...@gmail.com> > *ส่ง:* 12 ธันวาคม 2562 02:38 > *ถึง:* pve-user@pve.proxmox.com <pve-user@pve.proxmox.com> > *ชื่อเรื่อง:* Re: [PVE-User] Need help with iscsi Dell MD3200i and > MULTIPATH > > But still I am in doubt... This is some Debian Buster bug???? > > > --- > Gilberto Nunes Ferreira > > (47) 3025-5907 > (47) 99676-7530 - Whatsapp / Telegram > > Skype: gilberto.nunes36 > > > > > > Em qua., 11 de dez. de 2019 às 16:25, Gilberto Nunes < > gilberto.nune...@gmail.com> escreveu: > > > After spending hours trying to fix this, I finally figure it out: > > After fill the file /etc/multipath/wwids with the wwids of the iscsi > disk, > > like this: > > cat /etc/multipath/wwids > > # Multipath wwids, Version : 1.0 > > # NOTE: This file is automatically maintained by multipath and > multipathd. > > # You should not need to edit this file in normal circumstances. > > # > > # Valid WWIDs: > > /361418770003f0d2800000a6c5df0a959/ > > > > Now everything is work properly > > multipath -ll > > mylun (361418770003f0d2800000a6c5df0a959) dm-2 DELL,MD32xxi > > size=1.0G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 > > rdac' wp=rw > > `-+- policy='round-robin 0' prio=9 status=active > > |- 15:0:0:0 sdb 8:16 active ready running > > `- 16:0:0:0 sdc 8:32 active ready running > > ls /dev/mapper/ > > control mylun pve-root pve-swap > > > > I don't know what's happen here, but in my case I needed to do this > > manually! > > This file in /etc/mulitpath/wwids should not be full fill automatically > by > > multipath??? So weird! > > --- > > Gilberto Nunes Ferreira > > > > (47) 3025-5907 > > (47) 99676-7530 - Whatsapp / Telegram > > > > Skype: gilberto.nunes36 > > > > > > > > > > > > Em qua., 11 de dez. de 2019 às 15:55, Gilberto Nunes < > > gilberto.nune...@gmail.com> escreveu: > > > >> Hi there > >> > >> I am using PVE 6.1-3 and trying to use iscsi over Dell Storage MD3200i. > >> I have successful deploy the storage to give access to block device via > 2 > >> different IPs > >> How ever, I can not deploy multipath properly. > >> When I try multipath -ll give me nothing or multipath -v3 tells me that > >> the wwid is not in the file > >> BTW, I have used this site as reference: > >> https://pve.proxmox.com/wiki/ISCSI_Multipath > >> <https://pve.proxmox.com/wiki/ISCSI_Multipath#Dell> > >> > >> > >> > https://icicimov.github.io/blog/virtualization/Adding-iSCSI-shared-volume-to-Proxmox-to-support-Live-Migration/ > >> > >> > >> Here's the /etc/multipath.conf: > >> defaults { > >> polling_interval 3 > >> path_selector "round-robin 0" > >> max_fds "max" > >> path_grouping_policy multibus > >> uid_attribute "ID_SERIAL" > >> rr_min_io 100 > >> failback immediate > >> no_path_retry queue > >> } > >> > >> blacklist { > >> wwid .* > >> devnode "^sda" > >> device { > >> vendor "DELL" > >> product "Universal Xport" > >> } > >> } > >> devices { > >> device { > >> vendor "DELL" > >> product "MD32xxi" > >> path_grouping_policy group_by_prio > >> prio rdac > >> #polling_interval 5 > >> path_checker rdac > >> path_selector "round-robin 0" > >> hardware_handler "1 rdac" > >> failback immediate > >> features "2 pg_init_retries 50" > >> no_path_retry 30 > >> rr_min_io 100 > >> #prio_callout "/sbin/mpath_prio_rdac /dev/%n" > >> } > >> } > >> blacklist_exceptions { > >> wwid "361418770003f0d2800000a6c5df0a959" > >> } > >> multipaths { > >> multipath { > >> wwid "361418770003f0d2800000a6c5df0a959" > >> alias mpath0 > >> } > >> > >> } > >> lsscsi > >> [0:0:0:0] disk HGST HUS726040ALS210 KU27 /dev/sda > >> [14:0:0:0] cd/dvd PLDS DVD+-RW DU-8A5LH 6D5M /dev/sr0 > >> [15:0:0:0] disk DELL MD32xxi 0820 /dev/sdb > >> [15:0:0:31] disk DELL Universal Xport 0820 - > >> [16:0:0:0] disk DELL MD32xxi 0820 /dev/sdc > >> [16:0:0:31] disk DELL Universal Xport 0820 - > >> > >> multipath -v3 > >> Dec 11 15:53:42 | set open fds limit to 1048576/1048576 > >> Dec 11 15:53:42 | loading //lib/multipath/libchecktur.so checker > >> Dec 11 15:53:42 | checker tur: message table size = 3 > >> Dec 11 15:53:42 | loading //lib/multipath/libprioconst.so prioritizer > >> Dec 11 15:53:42 | foreign library "nvme" loaded successfully > >> Dec 11 15:53:42 | sr0: blacklisted, udev property missing > >> Dec 11 15:53:42 | sda: udev property ID_WWN whitelisted > >> Dec 11 15:53:42 | sda: device node name blacklisted > >> Dec 11 15:53:42 | sdb: udev property ID_WWN whitelisted > >> Dec 11 15:53:42 | sdb: mask = 0x1f > >> Dec 11 15:53:42 | sdb: dev_t = 8:16 > >> Dec 11 15:53:42 | sdb: size = 2097152 > >> Dec 11 15:53:42 | sdb: vendor = DELL > >> Dec 11 15:53:42 | sdb: product = MD32xxi > >> Dec 11 15:53:42 | sdb: rev = 0820 > >> Dec 11 15:53:42 | find_hwe: found 2 hwtable matches for > DELL:MD32xxi:0820 > >> Dec 11 15:53:42 | sdb: h:b:t:l = 15:0:0:0 > >> Dec 11 15:53:42 | sdb: tgt_node_name = > >> iqn.1984-05.com.dell:powervault.md3200i.61418770003f0d28000000005bbca139 > >> Dec 11 15:53:42 | sdb: path state = running > >> Dec 11 15:53:42 | sdb: 1011 cyl, 34 heads, 61 sectors/track, start at 0 > >> Dec 11 15:53:42 | sdb: serial = 3CS00II > >> Dec 11 15:53:42 | sdb: get_state > >> Dec 11 15:53:42 | sdb: detect_checker = yes (setting: multipath > internal) > >> Dec 11 15:53:42 | loading //lib/multipath/libcheckrdac.so checker > >> Dec 11 15:53:42 | checker rdac: message table size = 9 > >> Dec 11 15:53:42 | sdb: path_checker = rdac (setting: storage device > >> autodetected) > >> Dec 11 15:53:42 | sdb: checker timeout = 30 s (setting: kernel sysfs) > >> Dec 11 15:53:42 | sdb: rdac state = up > >> Dec 11 15:53:42 | sdb: uid_attribute = ID_SERIAL (setting: > multipath.conf > >> defaults/devices section) > >> Dec 11 15:53:42 | sdb: uid = 361418770003f0d2800000a6c5df0a959 (udev) > >> Dec 11 15:53:42 | sdb: detect_prio = yes (setting: multipath internal) > >> Dec 11 15:53:42 | loading //lib/multipath/libpriordac.so prioritizer > >> Dec 11 15:53:42 | sdb: prio = rdac (setting: storage device > configuration) > >> Dec 11 15:53:42 | sdb: prio args = "" (setting: storage device > >> configuration) > >> Dec 11 15:53:42 | sdb: rdac prio = 11 > >> Dec 11 15:53:42 | sdc: udev property ID_WWN whitelisted > >> Dec 11 15:53:42 | sdc: mask = 0x1f > >> Dec 11 15:53:42 | sdc: dev_t = 8:32 > >> Dec 11 15:53:42 | sdc: size = 2097152 > >> Dec 11 15:53:42 | sdc: vendor = DELL > >> Dec 11 15:53:42 | sdc: product = MD32xxi > >> Dec 11 15:53:42 | sdc: rev = 0820 > >> Dec 11 15:53:42 | find_hwe: found 2 hwtable matches for > DELL:MD32xxi:0820 > >> Dec 11 15:53:42 | sdc: h:b:t:l = 16:0:0:0 > >> Dec 11 15:53:42 | sdc: tgt_node_name = > >> iqn.1984-05.com.dell:powervault.md3200i.61418770003f0d28000000005bbca139 > >> Dec 11 15:53:42 | sdc: path state = running > >> Dec 11 15:53:42 | sdc: 1011 cyl, 34 heads, 61 sectors/track, start at 0 > >> Dec 11 15:53:42 | sdc: serial = 3CS00II > >> Dec 11 15:53:42 | sdc: get_state > >> Dec 11 15:53:42 | sdc: detect_checker = yes (setting: multipath > internal) > >> Dec 11 15:53:42 | sdc: path_checker = rdac (setting: storage device > >> autodetected) > >> Dec 11 15:53:42 | sdc: checker timeout = 30 s (setting: kernel sysfs) > >> Dec 11 15:53:42 | sdc: rdac state = up > >> Dec 11 15:53:42 | sdc: uid_attribute = ID_SERIAL (setting: > multipath.conf > >> defaults/devices section) > >> Dec 11 15:53:42 | sdc: uid = 361418770003f0d2800000a6c5df0a959 (udev) > >> Dec 11 15:53:42 | sdc: detect_prio = yes (setting: multipath internal) > >> Dec 11 15:53:42 | sdc: prio = rdac (setting: storage device > configuration) > >> Dec 11 15:53:42 | sdc: prio args = "" (setting: storage device > >> configuration) > >> Dec 11 15:53:42 | sdc: rdac prio = 11 > >> Dec 11 15:53:42 | loop0: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop1: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop2: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop3: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop4: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop5: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop6: blacklisted, udev property missing > >> Dec 11 15:53:42 | loop7: blacklisted, udev property missing > >> Dec 11 15:53:42 | dm-0: blacklisted, udev property missing > >> Dec 11 15:53:42 | dm-1: blacklisted, udev property missing > >> ===== paths list ===== > >> uuid hcil dev dev_t pri dm_st chk_st > >> vend/pro > >> 361418770003f0d2800000a6c5df0a959 15:0:0:0 sdb 8:16 11 undef undef > >> DELL,MD3 > >> 361418770003f0d2800000a6c5df0a959 16:0:0:0 sdc 8:32 11 undef undef > >> DELL,MD3 > >> Dec 11 15:53:42 | libdevmapper version 1.02.155 (2018-12-18) > >> Dec 11 15:53:42 | DM multipath kernel driver v1.13.0 > >> Dec 11 15:53:42 | sdb: udev property ID_WWN whitelisted > >> Dec 11 15:53:42 | sdb: wwid 361418770003f0d2800000a6c5df0a959 > whitelisted > >> Dec 11 15:53:42 | wwid 361418770003f0d2800000a6c5df0a959 not in wwids > >> file, skipping sdb > >> Dec 11 15:53:42 | sdb: orphan path, only one path > >> Dec 11 15:53:42 | rdac prioritizer refcount 2 > >> Dec 11 15:53:42 | sdc: udev property ID_WWN whitelisted > >> Dec 11 15:53:42 | sdc: wwid 361418770003f0d2800000a6c5df0a959 > whitelisted > >> Dec 11 15:53:42 | wwid 361418770003f0d2800000a6c5df0a959 not in wwids > >> file, skipping sdc > >> Dec 11 15:53:42 | sdc: orphan path, only one path > >> Dec 11 15:53:42 | rdac prioritizer refcount 1 > >> Dec 11 15:53:42 | unloading rdac prioritizer > >> Dec 11 15:53:42 | unloading const prioritizer > >> Dec 11 15:53:42 | unloading rdac checker > >> Dec 11 15:53:42 | unloading tur checker > >> > >> I will appreciated any help. > >> > >> Thanks > >> > >> > >> > >> > >> --- > >> Gilberto Nunes Ferreira > >> > >> (47) 3025-5907 > >> (47) 99676-7530 - Whatsapp / Telegram > >> > >> Skype: gilberto.nunes36 > >> > >> > >> > >> > _______________________________________________ > pve-user mailing list > pve-user@pve.proxmox.com > https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user > _______________________________________________ pve-user mailing list pve-user@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-user