Re: Building a multipath setup with DRBD in dual-Primary
Hi Bart, At the moment this is only a proof of concept wich seems to be working fine. I had some thoughts about it: * The iSCSI target should not run in Write-Back mode * DRBD should run in synchronus mode (protocol C) * I should use the failover policy with dm-multipath instead of round-robin. I got this setup working last night and i started bonnie++ on a lun wich is still running with no errors yet, but ofcourse this should be tested me thurough. As far as my knowlegde goes, this setup should be working as long as i don't start using clustered filesystems since these is only one initiator writing to the LUN? And again, i am only testing this setup, my production envirioment will stick to the Primary/Standby situation with heartbeat. - Met vriendelijke groet, Wido den Hollander Hoofd Systeembeheer / CSO Telefoon Support Nederland: 0900 9633 (45 cpm) Telefoon Support België: 0900 70312 (45 cpm) Telefoon Direct: (+31) (0)20 50 60 104 Fax: +31 (0)20 50 60 111 E-mail: supp...@pcextreme.nl Website: http://www.pcextreme.nl Kennisbank: http://support.pcextreme.nl/ Netwerkstatus: http://nmc.pcextreme.nl On Sat, 2009-05-30 at 07:43 +0200, Bart Van Assche wrote: On Fri, May 29, 2009 at 5:21 PM, Wido wid...@gmail.com wrote: Last night i had the idea of building a multipath iSCSI setup with DRBD in Primary/Primary. [ ... ] Is my setup a possibility or should i stay with the old regular setup of a Primary/Standy with heartbeat? I strongly recommend you to use a primary/standby setup + heartbeat when using dm-multipath instead of a primary/primary setup. With the latter setup you risk to run into data corruption. See also http://lkml.org/lkml/2009/4/13/80. Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~--- signature.asc Description: This is a digitally signed message part
Building a multipath setup with DRBD in dual-Primary
Hello, Last night i had the idea of building a multipath iSCSI setup with DRBD in Primary/Primary. My idea was/is: I have two machines with a 30GB partition wich is synced via DRBD (i assume everybody here is familiar with DRBD). My DRBD is running fine both machines are UpToDate and they are both primary. On top of /dev/drbd0 i made a LVM VG with the name data, in this VG i created 3 LV's with a size of 5GB. For this e-mail i'll call the machines target1, target2 and iscsi- initiator target1: 192.168.5.199 target2: 192.168.5.200 iscsi-initiator: 192.168.5.198 On both the targets i have created the folling ietd.conf: Target iqn.2007-04.nl.pcextreme:ha Lun 0 Path=/dev/data/lun0,Type=fileio Lun 1 Path=/dev/data/lun1,Type=fileio Lun 2 Path=/dev/data/lun2,Type=fileio So both targets have the same IQN and i tried to create a multipath from the client to both tartgets: iscsiadm -m discovery -t sendtargets -p 192.168.5.199 iscsiadm -m discovery -t sendtargets -p 192.168.5.200 iscsiadm -m node -T iqn.2007-04.nl.pcextreme:ha -p 192.168.5.199 -- login iscsiadm -m node -T iqn.2007-04.nl.pcextreme:ha -p 192.168.5.200 -- login This works and i get a total of 6 new SCSI devices, but multipath won't recognize them as a redundant path: r...@iscsi-client:~# multipath -ll 149455400010059580f00dm-3 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 5:0:0:0 sde 8:64 [active][ready] 1494554000100dd3c0e00dm-0 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 4:0:0:1 sdc 8:32 [active][ready] 149455400010092580f00dm-5 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 5:0:0:2 sdg 8:96 [active][ready] 149455400010065580f00dm-4 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 5:0:0:1 sdf 8:80 [active][ready] 1494554000100a03c0e00dm-1 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 4:0:0:0 sdb 8:16 [active][ready] 1494554000100193d0e00dm-2 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 4:0:0:2 sdd 8:48 [active][ready] r...@iscsi-client:~# Is this a target issue because the sessions are not synchronized between both targets? Or is this related to Open-iSCSI and multipath? Is my setup a possibility or should i stay with the old regular setup of a Primary/Standy with heartbeat? Thank you in advance, Wido den Hollander --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
Re: Building a multipath setup with DRBD in dual-Primary
Hi Mike, Yes indeed, that worked! For the record, my ietd.conf Target iqn.2007-04.nl.pcextreme:ha Lun 0 Path=/dev/data/lun0,Type=fileio,ScsiId=nWFKhBCMq56JfSzq Lun 1 Path=/dev/data/lun1,Type=fileio,ScsiId=wrOhZ1hVFANuQOJM Lun 2 Path=/dev/data/lun2,Type=fileio,ScsiId=pW8e21PNf7WR2FDh - Met vriendelijke groet, Wido den Hollander Hoofd Systeembeheer / CSO Telefoon Support Nederland: 0900 9633 (45 cpm) Telefoon Support België: 0900 70312 (45 cpm) Telefoon Direct: (+31) (0)20 50 60 104 Fax: +31 (0)20 50 60 111 E-mail: supp...@pcextreme.nl Website: http://www.pcextreme.nl Kennisbank: http://support.pcextreme.nl/ Netwerkstatus: http://nmc.pcextreme.nl On Fri, 2009-05-29 at 11:13 -0500, Mike Christie wrote: Wido wrote: Hello, Last night i had the idea of building a multipath iSCSI setup with DRBD in Primary/Primary. My idea was/is: I have two machines with a 30GB partition wich is synced via DRBD (i assume everybody here is familiar with DRBD). My DRBD is running fine both machines are UpToDate and they are both primary. On top of /dev/drbd0 i made a LVM VG with the name data, in this VG i created 3 LV's with a size of 5GB. For this e-mail i'll call the machines target1, target2 and iscsi- initiator target1: 192.168.5.199 target2: 192.168.5.200 iscsi-initiator: 192.168.5.198 On both the targets i have created the folling ietd.conf: Target iqn.2007-04.nl.pcextreme:ha Lun 0 Path=/dev/data/lun0,Type=fileio Lun 1 Path=/dev/data/lun1,Type=fileio Lun 2 Path=/dev/data/lun2,Type=fileio So both targets have the same IQN and i tried to create a multipath from the client to both tartgets: iscsiadm -m discovery -t sendtargets -p 192.168.5.199 iscsiadm -m discovery -t sendtargets -p 192.168.5.200 iscsiadm -m node -T iqn.2007-04.nl.pcextreme:ha -p 192.168.5.199 -- login iscsiadm -m node -T iqn.2007-04.nl.pcextreme:ha -p 192.168.5.200 -- login This works and i get a total of 6 new SCSI devices, but multipath won't recognize them as a redundant path: r...@iscsi-client:~# multipath -ll 149455400010059580f00dm-3 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 5:0:0:0 sde 8:64 [active][ready] 1494554000100dd3c0e00dm-0 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 4:0:0:1 sdc 8:32 [active][ready] 149455400010092580f00dm-5 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 5:0:0:2 sdg 8:96 [active][ready] 149455400010065580f00dm-4 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 5:0:0:1 sdf 8:80 [active][ready] 1494554000100a03c0e00dm-1 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 4:0:0:0 sdb 8:16 [active][ready] 1494554000100193d0e00dm-2 IET ,VIRTUAL- DISK [size=5.0G][features=0][hwhandler=0] \_ round-robin 0 [prio=1][active] \_ 4:0:0:2 sdd 8:48 [active][ready] r...@iscsi-client:~# Is this a target issue because the sessions are not synchronized between both targets? Or is this related to Open-iSCSI and multipath? multipath does not use any iscsi info like targetname or portal to build the multipath device. It just uses something like scsi vpd page info. I think in your ietd.conf you will want to use the ScsiId or ScsiSN setting in the Path line. Target iqn.2007-04.nl.pcextreme:ha Lun 0 Path=/dev/data/lun0,Type=fileio,scsi_id=some-uuid1 Lun 1 Path=/dev/data/lun1,Type=fileio,scsi_id=some-uuid2 Lun 2 Path=/dev/data/lun2,Type=fileio,scsi_id=some-uuid3 Then in the other target box have the scsi_id uuids match up. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~--- signature.asc Description: This is a digitally signed message part
Re: Building a multipath setup with DRBD in dual-Primary
On Fri, May 29, 2009 at 5:21 PM, Wido wid...@gmail.com wrote: Last night i had the idea of building a multipath iSCSI setup with DRBD in Primary/Primary. [ ... ] Is my setup a possibility or should i stay with the old regular setup of a Primary/Standy with heartbeat? I strongly recommend you to use a primary/standby setup + heartbeat when using dm-multipath instead of a primary/primary setup. With the latter setup you risk to run into data corruption. See also http://lkml.org/lkml/2009/4/13/80. Bart. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups open-iscsi group. To post to this group, send email to open-iscsi@googlegroups.com To unsubscribe from this group, send email to open-iscsi+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/open-iscsi -~--~~~~--~~--~--~---
