Moving ahead with the assumptions before and I have defined the stacked resource on node "alice" (trk-kvm-01), created the drbd block device:

   root@trk-kvm-01:~# drbdadm create-md --stacked convirt-U
   md_offset 53685411840
   al_offset 53685379072
   bm_offset 53683740672

   Found some data

     ==> This might destroy existing data! <==

   Do you want to proceed?
   [need to type 'yes' to confirm] yes

   initializing activity log
   NOT initializing bitmap
   Writing meta data...
   New drbd meta data block successfully created.

It comes up OK:

   root@trk-kvm-01:~# drbdadm up --stacked convirt-U

   root@trk-kvm-01:~# drbd-overview
     12:convirt/0                Connected  Primary/Secondary
   UpToDate/UpToDate     _convirt           vda virtio
   120:convirt-U/0         ^^12 Connected  Secondary/Secondary

I made it the primary

   root@trk-kvm-01:~# drbdadm primary --force --stacked convirt-U

It seems OK on this node:

   root@trk-kvm-01:~# drbd-overview
     12:convirt/0                Connected Primary/Secondary  
   UpToDate/UpToDate     _convirt vda virtio
   120:convirt-U/0         ^^12 Connected Primary/Secondary  

However when I try to create the drbd block device on the third node I get the following:

   root@trk-kvm-03:~# drbdadm create-md convirt-U
   open(inetrnal) failed: No such file or directory
   open(inetrnal) failed: No such file or directory
   Command 'drbdmeta 120 v08 inetrnal flex-external create-md'
   terminated with exit code 20

What file is it looking for that it fails to find?

The resource is defined with this:

   resource convirt {
            net {
                    protocol C;

            device    /dev/drbd12;
            meta-disk internal;

            on trk-kvm-01 {
            disk      /dev/trk-kvm-01-vg/convirt;
            on trk-kvm-02 {
            disk      /dev/trk-kvm-02-vg/convirt;

   resource convirt-U {
            net {
                    protocol A;

            stacked-on-top-of convirt {
                    device /dev/drbd120;

            on trk-kvm-03 {
                    device          /dev/drbd120;
                    disk /dev/trk-kvm-03-vg/convirt;
                    meta-disk       inetrnal;

The underlying LVM volume does exist:

   root@trk-kvm-03:~# lvs
      LV                VG            Attr       LSize Pool Origin
   Data%  Meta%  Move Log Cpy%Sync Convert
      convirt           trk-kvm-03-vg -wi-a----- 50.00g

I tried removing the volume with lvremove:

   root@trk-kvm-03:~# lvremove /dev/trk-kvm-03-vg/convirt
   Do you really want to remove and DISCARD active logical volume
   convirt? [y/n]: y
      Logical volume "convirt" successfully removed

and creating it again with lvcreate:

   root@trk-kvm-03:~# lvcreate -L50G -n convirt trk-kvm-03-vg
      Logical volume "convirt" created.

it looks OK

   --- Logical volume ---
      LV Path                /dev/trk-kvm-03-vg/convirt
      LV Name                convirt
      VG Name                trk-kvm-03-vg
      LV UUID QT7YNm-zlYi-7S0d-lF26-EEi2-7g69-0iU3aC
      LV Write Access        read/write
      LV Creation host, time trk-kvm-03, 2018-02-05 09:03:56 -0800
      LV Status              available
      # open                 0
      LV Size                50.00 GiB
      Current LE             12800
      Segments               1
      Allocation             inherit
      Read ahead sectors     auto
      - currently set to     256
      Block device           252:13

Why can I not make the DRBD block device?  I see that some required file is missing but I don't understand what:

   root@trk-kvm-03:~# drbd-overview
   120:convirt-U/0  . . .


   root@trk-kvm-03:~# cat /proc/drbd
   version: 8.4.5 (api:1/proto:86-101)
   srcversion: 4B3E2E2CD48CAE5280B5205

   120: cs:Unconfigured

Something simple I have missed/misunderstood?

*Paul O'Rorke*
*Tracker Software Products (Canada) Limited * <>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623


Download latest Releases

On 2018-02-02 11:31 AM, Paul O'Rorke wrote:


I hope this is the right place to ask what may be a silly question.  I have searched the archives and still do not understand properly how to set up "classic 3 node" for DR.

I have been using a simple HA setup for some years now for KVM virtual machines.  I have a DRBD resource for each VM, backed by LVM volumes, each using a unique IP (aliases) via CAT5 directly connecting dedicated NICs on each of "alice" and "bob" (I'm using the example from the docs here for simplicity).  I run half the guests on each host and manually fail over if it is ever needed.  This has been rock solid for me, great performance and has on more than one occasion saved my bacon.  I want to set up a classic 3 node DR with a VPN connected off site 3rd stacked node.

Referring to this 6.18.2. Configuring a stacked resource

I don't understand where to set up the interface for the stacked resource:

|stacked-on-top-of r0 { device /dev/drbd10; address; }|

What interface gets this IP?  I am assuming it goes on the primary node of the lower resource.  So for the example in the docs, if alice is the primary I set up on alice and if I have to switch bob to primary I loose replication to charlie until such time as I either restore alice as the primary or can I set up on bob somehow and continue off site replication while alice is down?

As I am reading it bob has no "knowledge" of charlie and the upper resource can only function while alice is primary.  Is that right?

Sorry if this is explained somewhere and I missed it.



*Paul O'Rorke*
*Tracker Software Products (Canada) Limited * <>
Tel: +1 (250) 324 1621
Fax: +1 (250) 324 1623


Download latest Releases

drbd-user mailing list

drbd-user mailing list

Reply via email to