Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 17/02/2017 14:16, Lars Ellenberg ha scritto: > Thanks, I've updated to 0.99.2 now in repo; much better, most resources > are up at startup. > > drbdmanage restart keeps failing (sometimes shutdown, wait, startup > succeeds) : Just upgraded to python-drbdmanage 0.99.3-1 from pve repo. Resources are coming up succesfully at system starup now. No more errors on "--config-to-test". Nice job. rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On Fri, Feb 17, 2017 at 11:30:03AM +0100, Roberto Resoli wrote: > Il 17/02/2017 09:51, Roland Kammerer ha scritto: > >> does not fixes the situation anymore, the "start" part failing most of > >> times. > > Hi, > > > > yes, there was a regression. Hopefully 0.99.2 fixed that. > > Thanks, I've updated to 0.99.2 now in repo; much better, most resources > are up at startup. > > drbdmanage restart keeps failing (sometimes shutdown, wait, startup > succeeds) : > > # drbdmanage restart > You are going to shut down the drbdmanaged server process on this node. > Please confirm: > yes/no: yes > Attempting to startup the server through D-Bus activation... > ERROR:dbus.proxies:Introspect error on :1.7:/interface: > dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did > not receive a reply. Possible causes include: the remote application did > not send a reply, the message bus security policy blocked the reply, the > reply timeout expired, or the network connection was broken. > D-Bus connection successful, server is running and reachable > > >> The only way to recover I found is to issue a > >> > >> drbdmanage export-res "*" > >> > >> and issuing manually a > >> > >> drbdadm up > >> > >> for every > > You could have done "drbdadm up all" in that case. just do: drbdadm adjust all > Tried, but it was failing leaving disks in "Diskless" state. (may be > because the command finds .drbdctrl already up, giving an exception?) > > It would be handy to have something like a wildcard support: > > drbdadm up vm-* > > I implemented somewhat similar behaviour (provide a list of res to bring > up/down, defaulting to all but .drbdctrl) in a custom script. -- : Lars Ellenberg : LINBIT | Keeping the Digital World Running : DRBD -- Heartbeat -- Corosync -- Pacemaker DRBD® and LINBIT® are registered trademarks of LINBIT __ please don't Cc me, but send to list -- I'm subscribed ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 17/02/2017 09:51, Roland Kammerer ha scritto: >> does not fixes the situation anymore, the "start" part failing most of >> times. > Hi, > > yes, there was a regression. Hopefully 0.99.2 fixed that. Thanks, I've updated to 0.99.2 now in repo; much better, most resources are up at startup. drbdmanage restart keeps failing (sometimes shutdown, wait, startup succeeds) : # drbdmanage restart You are going to shut down the drbdmanaged server process on this node. Please confirm: yes/no: yes Attempting to startup the server through D-Bus activation... ERROR:dbus.proxies:Introspect error on :1.7:/interface: dbus.exceptions.DBusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken. D-Bus connection successful, server is running and reachable >> The only way to recover I found is to issue a >> >> drbdmanage export-res "*" >> >> and issuing manually a >> >> drbdadm up >> >> for every > You could have done "drbdadm up all" in that case. Tried, but it was failing leaving disks in "Diskless" state. (may be because the command finds .drbdctrl already up, giving an exception?) It would be handy to have something like a wildcard support: drbdadm up vm-* I implemented somewhat similar behaviour (provide a list of res to bring up/down, defaulting to all but .drbdctrl) in a custom script. bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On Thu, Feb 16, 2017 at 05:39:42PM +0100, Roberto Resoli wrote: > Il 09/01/2017 10:56, Roberto Resoli ha scritto: > > Il 29/12/2016 15:01, Roberto Resoli ha scritto: > >> Il 28/12/2016 16:00, Roberto Resoli ha scritto: > >>> Il 27/12/2016 10:19, Roberto Resoli ha scritto: > All is up and running nicely, in any case. > >> Another issue: drbdmanaged starts at boot, but most resources remain down; > >> a > >> > >> drbdmanaged restart > >> > >> fixes that. > >> > >> I attach a related fragment from syslog; it seems that some temporary > >> .res files are not generated correctly. > > > > any hint? > > After having upgraded to python-drbdmanage 0.99.1-1 from linbit drbd9 > repo for proxmox, a simple > > drbdmanage restart > > does not fixes the situation anymore, the "start" part failing most of > times. Hi, yes, there was a regression. Hopefully 0.99.2 fixed that. > The only way to recover I found is to issue a > > drbdmanage export-res "*" > > and issuing manually a > > drbdadm up > > for every You could have done "drbdadm up all" in that case. Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 09/01/2017 10:56, Roberto Resoli ha scritto:
> Il 29/12/2016 15:01, Roberto Resoli ha scritto:
>> Il 28/12/2016 16:00, Roberto Resoli ha scritto:
>>> Il 27/12/2016 10:19, Roberto Resoli ha scritto:
All is up and running nicely, in any case.
>> Another issue: drbdmanaged starts at boot, but most resources remain down; a
>>
>> drbdmanaged restart
>>
>> fixes that.
>>
>> I attach a related fragment from syslog; it seems that some temporary
>> .res files are not generated correctly.
>
> any hint?
After having upgraded to python-drbdmanage 0.99.1-1 from linbit drbd9
repo for proxmox, a simple
drbdmanage restart
does not fixes the situation anymore, the "start" part failing most of
times.
The only way to recover I found is to issue a
drbdmanage export-res "*"
and issuing manually a
drbdadm up
for every
I attach a fragment of syslog while a pve node restart; note that at
least one resource ('vm-101-disk-1') is coming up correctly, the other
are failing because (it appears to me) of incorrect temporary resources
definition.
Here two examples of exported defintion ('vm-101-disk-1' is first):
==
# This file was generated by drbdmanage(8), do not edit manually.
#dm-meta:{}
resource vm-101-disk-1 {
template-file "/var/lib/drbd.d/drbdmanage_global_common.conf";
net {
allow-two-primaries yes;
shared-secret "XEDTfmZotteF377OlXOA";
max-epoch-size 8000;
cram-hmac-alg sha1;
max-buffers 8000;
}
connection-mesh {
hosts pve1 pve3 pve2;
}
on pve1 {
node-id 0;
address ipv4 10.1.1.1:7001;
volume 0 {
device minor 101;
disk /dev/drbdpool/vm-101-disk-1_00;
disk {
size 4194304k;
}
meta-disk internal;
}
}
on pve3 {
node-id 2;
address ipv4 10.1.1.3:7001;
volume 0 {
device minor 101;
disk /dev/drbdpool/vm-101-disk-1_00;
disk {
size 4194304k;
}
meta-disk internal;
}
}
on pve2 {
node-id 1;
address ipv4 10.1.1.2:7001;
volume 0 {
device minor 101;
disk /dev/drbdpool/vm-101-disk-1_00;
disk {
size 4194304k;
}
meta-disk internal;
}
}
}
==
# This file was generated by drbdmanage(8), do not edit manually.
#dm-meta:{"create_date": "2017-01-10T09:47:11.089762"}
resource vm-105-disk-1 {
template-file "/var/lib/drbd.d/drbdmanage_global_common.conf";
net {
allow-two-primaries yes;
shared-secret "MYNxm7S4qx3zeoJPTrk9";
cram-hmac-alg sha1;
}
connection-mesh {
hosts pve1 pve3 pve2;
}
on pve1 {
node-id 1;
address ipv4 10.1.1.1:7010;
volume 0 {
device minor 110;
disk /dev/drbdpool/vm-105-disk-1_00;
disk {
size 4194304k;
}
meta-disk internal;
}
}
on pve3 {
node-id 2;
address ipv4 10.1.1.3:7010;
volume 0 {
device minor 110;
disk /dev/drbdpool/vm-105-disk-1_00;
disk {
size 4194304k;
}
meta-disk internal;
}
}
on pve2 {
node-id 0;
address ipv4 10.1.1.2:7010;
volume 0 {
device minor 110;
disk /dev/drbdpool/vm-105-disk-1_00;
disk {
size 4194304k;
}
meta-disk internal;
}
}
}
drbd9_failing_resources.log.gz
Description: application/gzip
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 13/01/2017 19:58, Michele Rossetti ha scritto: > root@mpve1:~# drbdsetup status > .drbdctrl role:Secondary > volume:0 disk:UpToDate > volume:1 disk:UpToDate > mpve2 role:Secondary > volume:0 peer-disk:UpToDate > volume:1 peer-disk:UpToDate > mpve3 role:Primary > volume:0 peer-disk:UpToDate > volume:1 peer-disk:UpToDate This means that all your resources are down, only control volumes are up ... Have you tried a: "drbdmanage restart" ? rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 10/01/2017 11:04, Michele Rossetti ha scritto: > Yes, I confirm that yesterday too, on a 3 server PVE4.3 with DRBD9 > deployed from Proxmox repository, I make a backup and a restore of a KVM > virtual machine; moreover I migrate the VM on the 3 PVE without problem. > > This is the reason of my previous post, to understand if your problem > was created by an update to 4.4 or a DRBD update. See my last messages. After yesterday update all is fine now. By the way, vm migration (live or not) never had problems. rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 10/01/2017 10:14, Roberto Resoli ha scritto: > Il 09/01/2017 19:20, Michele Rossetti ha scritto: >> This means that in PVE cluster of 3 servers with DRBD9 updated isn't >> possible to restore KVM virtual machines? >> Other people on list with the same problem or is only in your >> configuration? >> Just to know before update ;-) > I have just retried today, after having upgraded drbd-utils to > 8.9.10+linbit-1 , apperead yesterday. > > I have successfuly cycled thru > > vm creation -> vm dump -> vm restore > > on a drbd9 (lvm-thin based) storage with only some quirks I will > describe here soon. I think that quirks were entirely related to creation/deletion of lvm volumes (backend of drbd ones). In one case, restore operation resulted in correct creation of new drbd resource, but on one node the assignment was pending, with "drbdmanage resume-all" didn't fixing it. I resolved with a "drbdadm down " on the problematic node, removing (lvremove) the backend lvm volume, and reissuing a "drbdmanage resume-all" that recreated it correctly. Now I can delete the test vm and restore it without any problem. So, my advice in case of problems creating/deleting/restoring vms is to check that creation/deletion of backend lvm volumes is correctly performed as expected. rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 09/01/2017 19:20, Michele Rossetti ha scritto: > This means that in PVE cluster of 3 servers with DRBD9 updated isn't > possible to restore KVM virtual machines? > Other people on list with the same problem or is only in your > configuration? > Just to know before update ;-) I have just retried today, after having upgraded drbd-utils to 8.9.10+linbit-1 , apperead yesterday. I have successfuly cycled thru vm creation -> vm dump -> vm restore on a drbd9 (lvm-thin based) storage with only some quirks I will describe here soon. bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 09/01/2017 19:20, Michele Rossetti ha scritto: > This means that in PVE cluster of 3 servers with DRBD9 updated isn't > possible to restore KVM virtual machines? > Other people on list with the same problem or is only in your > configuration? > Just to know before update ;-) This means that in your actual configuration you can restore to a drbd storage without problems? This would be useful to know, in my case I don't think this had worked even before transitioning to linbit repo. rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
This means that in PVE cluster of 3 servers with DRBD9 updated isn't possible to restore KVM virtual machines? Other people on list with the same problem or is only in your configuration? Just to know before update ;-) Thanks, Michele Il 28/12/2016 16:00, Roberto Resoli ha scritto: Il 27/12/2016 10:19, Roberto Resoli ha scritto: All is up and running nicely, in any case. I said that too early: I tried to restore a kvm virtual machine and it fails, systematically. I attached here the command with relative output, and the fragment of syslog related to restore. It seems that resources (the vm had two disks) are being created correctly, but access to them is failing. I have tried with another vm dump (one disk only); same problem. Any hint? bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 29/12/2016 15:01, Roberto Resoli ha scritto: > Il 28/12/2016 16:00, Roberto Resoli ha scritto: >> Il 27/12/2016 10:19, Roberto Resoli ha scritto: >>> All is up and running nicely, in any case. > Another issue: drbdmanaged starts at boot, but most resources remain down; a > > drbdmanaged restart > > fixes that. > > I attach a related fragment from syslog; it seems that some temporary > .res files are not generated correctly. any hint? Thanks, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 28/12/2016 16:00, Roberto Resoli ha scritto: > Il 27/12/2016 10:19, Roberto Resoli ha scritto: >> All is up and running nicely, in any case. > > I said that too early: I tried to restore a kvm virtual machine and it > fails, systematically. > > I attached here the command with relative output, and the fragment of > syslog related to restore. > > It seems that resources (the vm had two disks) are being created > correctly, but access to them is failing. > > I have tried with another vm dump (one disk only); same problem. Any hint? bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 04/01/2017 13:51, Roland Kammerer ha scritto: > If you are a > company and you provide commercial support/commercial trainings and make > money out of that support business, then I think it is fair to give back > a bit to the people that wrote the stuff you are making money with. I agree, here too. I don't know if that change in license is the better way to achieve that, but it was your decision, and the future will tell if it was a good one. Honestly at the momement I'm interested only in how good DRBD integration in PVE will be, in last months under Proxmox hat it didn't improve much. bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 04/01/2017 12:42, Igor Cicimov ha scritto: > For the "has to remove [...] completely": Puh, dangerous territory to > comment on, but that was the decision of Proxmox. > > > No serious company would integrate drbd in their product and then tell > their customers to talk to Linbit in case of issues. I agree. Proxmox decision was the only feasible one. bye rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Hi Roland I really don't want to blame Linbit or Proxmox and I understand you that you have to protect against some vendors. I hope that Proxmox responsibles are reading all these mails here. I know that it was Proxmox's decision to remove all with DRBD9. But why? I think they don't want to risk violating your new license. If you have a good understanding with Proxmox and vice versa, I really hope that you work together. I want to encourage both companies to clarify all uncertainlies. It's better for PVE/DRBD's future development. Thank you very much. Enrica -Ursprüngliche Nachricht- Von: drbd-user-boun...@lists.linbit.com [mailto:drbd-user-boun...@lists.linbit.com] Im Auftrag von Roland Kammerer Gesendet: Mittwoch, 4. Januar 2017 09:33 An: drbd-user@lists.linbit.com Betreff: Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE On Tue, Jan 03, 2017 at 10:38:38AM +, Enrica Ruedin wrote: > I can't understand Linbit to change license in such a way that PVE > Proxmox has to remove their support completely. This unnecessary > change leads to confusion. But you do understand that we also eat and like to have a roof over our heads? Hey, the Aparthotel Guggach Hotel from which domain you wrote looks nice, that could solve one problem! ;-) In the end the license - and sure IANAL - is a "do whatever you like, but don't interfere with LINBIT's support business". IMO that's it. This was not a move against Proxmox at all, there are other vendors not playing nice... Just writing that because on the ML I have the impression that there is some kind of "Proxmox vs. LINBIT". It isn't. For the "has to remove [...] completely": Puh, dangerous territory to comment on, but that was the decision of Proxmox. They have all the right to do that! No blaming, no nothing. There are other projects/companies that took another road and still ship, integrate, or use drbdmanage with the current license. Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
I am agree. In any environment (lab, production, critical, etc), if between PVE and LINBIT not has a perfect coordination for publish the updates of packages for that together works perfectly, simply all people will stop using DRBD. Also, if the plugin(s) of LINBIT not add features as monitor stats for DRBD, and maybe other functions, of sure that with the time, DRBD will be relegated, and it will not be good for the business of LINBIT. So, I suppose the best thing would be that between PVE and LINBIT talk and study the possibility of has a more close union, remember that the final customer is who determine the tools that be more easy of use and give them more benefits. BR Cesar - Original Message - From: "Enrica Ruedin" <e.rue...@guggach.com> To: <drbd-user@lists.linbit.com> Sent: Wednesday, January 04, 2017 7:02 AM Subject: Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE Hello Dietmar You confirm exactly what I was afraid of. I fully agree that the package of Linbit's drbdmanage shouldn't be in your package but I don't agree concerning the DRBDPlugin.pm in '/usr/share/perl5/PVE/Storage'. This perl script is the interface between PVE and DRBD. A stable or coordinated versioning between 'PVE package - DRBDPlugin.pm - DRBD package' will be critical. A version change on Linbit's side or PVE side without adaption of the plugin could hit a stable environment. I don't know who should be the owner/developer of 'DRBDPlugin.pm', but anyway it needs a coordination especially if PVE/DRBD9 is used in production. So I understand Michele's hesitation. Regards, Enrica -Ursprüngliche Nachricht- Von: drbd-user-boun...@lists.linbit.com [mailto:drbd-user-boun...@lists.linbit.com] Im Auftrag von Dietmar Maurer Gesendet: Mittwoch, 4. Januar 2017 10:20 An: Roland Kammerer <roland.kamme...@linbit.com>; drbd-user@lists.linbit.com Betreff: Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE In the end the license - and sure IANAL - is a "do whatever you like, but don't interfere with LINBIT's support business". IMO that's it. This was not a move against Proxmox at all, there are other vendors not playing nice... Just writing that because on the ML I have the impression that there is some kind of "Proxmox vs. LINBIT". It isn't. I fully agree here. For the "has to remove [...] completely": Puh, dangerous territory to comment on, but that was the decision of Proxmox. No, your new license implies that. It would be illegal to provide support (in any way) for drbdmanage. Besides, it would not make any sense to have two repositories including the same packages - what for? ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Yet some consideration for the interest of all. My age has sadly brought me to see several failures of good initiatives due to pure misunderstandings. We have two products PVE 4 and DRBD9 that seem made for each other, they are absolutely complementary. We have two companies Proxmox and Linbit, both in Vienna, speaking the same language and developing software that do not compete with each other. It is true that the most violent discussions take place between brothers, but I think it's a pity having to give up such an interesting system due to a misunderstanding. One more thing: in the case, fairly widespread, of use of software from the community, there is no economic gain for Proxmox nor Linbit. 163/5000 If instead people want to purchase support, it would be interesting, for potential customers, have a single subscription with the assistance for the two products. So, modify the licence terms, subscribe an agreement between companies, do whatever you want, but please, give us a system (PVE 4 + DBRD9) fully functioning and well serviced. You will see that your business will benefit. Best regards, Michele P.S. In a previous post I made some stupid question, like this: To upgrade to 4.4 can I use the Linbit repository? Which address is to add on apt-get sources.list? I suppose the Proxmox address is to be removed from list, right? (This will be a problem for people using PVE enterprise repository, but it's not my case). Moreover, it's possible, after the sources.list modification, to use update button in PVE GUI? Can someone help me? Thanks On Wed, Jan 04, 2017 at 10:42:54PM +1100, Igor Cicimov wrote: No serious company would integrate drbd in their product and then tell their customers to talk to Linbit in case of issues. The license was only changed for drbdmanage. Kernel and drbd-utils are still under the old license. These companies you are talking about can talk to LINBIT, for example for a partner agreement. Yes, already happened. If you are a private user, in the end nothing changed for you. You can use, patch, package drbdmanage and everything is as before. If you are a company and you provide commercial support/commercial trainings and make money out of that support business, then I think it is fair to give back a bit to the people that wrote the stuff you are making money with. Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user -- """ MICRO srl Informatica e Telecomunicazioni - Web services - Web sites Michele Rossetti sede legale: via Raffa Garzia 7 09126 Cagliari (Italy) sede operativa: viale Marconi 222 09131 Cagliari Ph. +39 070 400240 Fax +39 070 4526207 MKM-REG Web: http://www.microsrl.com http://www.sardi.it E-mail: micro...@microsrl.com """ ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On Wed, Jan 04, 2017 at 10:42:54PM +1100, Igor Cicimov wrote: > No serious company would integrate drbd in their product and then tell > their customers to talk to Linbit in case of issues. The license was only changed for drbdmanage. Kernel and drbd-utils are still under the old license. These companies you are talking about can talk to LINBIT, for example for a partner agreement. Yes, already happened. If you are a private user, in the end nothing changed for you. You can use, patch, package drbdmanage and everything is as before. If you are a company and you provide commercial support/commercial trainings and make money out of that support business, then I think it is fair to give back a bit to the people that wrote the stuff you are making money with. Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Hello Dietmar You confirm exactly what I was afraid of. I fully agree that the package of Linbit's drbdmanage shouldn't be in your package but I don't agree concerning the DRBDPlugin.pm in '/usr/share/perl5/PVE/Storage'. This perl script is the interface between PVE and DRBD. A stable or coordinated versioning between 'PVE package - DRBDPlugin.pm - DRBD package' will be critical. A version change on Linbit's side or PVE side without adaption of the plugin could hit a stable environment. I don't know who should be the owner/developer of 'DRBDPlugin.pm', but anyway it needs a coordination especially if PVE/DRBD9 is used in production. So I understand Michele's hesitation. Regards, Enrica -Ursprüngliche Nachricht- Von: drbd-user-boun...@lists.linbit.com [mailto:drbd-user-boun...@lists.linbit.com] Im Auftrag von Dietmar Maurer Gesendet: Mittwoch, 4. Januar 2017 10:20 An: Roland Kammerer <roland.kamme...@linbit.com>; drbd-user@lists.linbit.com Betreff: Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE > In the end the license - and sure IANAL - is a "do whatever you like, > but don't interfere with LINBIT's support business". IMO that's it. > This was not a move against Proxmox at all, there are other vendors > not playing nice... Just writing that because on the ML I have the > impression that there is some kind of "Proxmox vs. LINBIT". It isn't. I fully agree here. > For the "has to remove [...] completely": Puh, dangerous territory to > comment on, but that was the decision of Proxmox. No, your new license implies that. It would be illegal to provide support (in any way) for drbdmanage. Besides, it would not make any sense to have two repositories including the same packages - what for? ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
> In the end the license - and sure IANAL - is a "do whatever you like, > but don't interfere with LINBIT's support business". IMO that's it. This > was not a move against Proxmox at all, there are other vendors not > playing nice... Just writing that because on the ML I have the > impression that there is some kind of "Proxmox vs. LINBIT". It isn't. I fully agree here. > For the "has to remove [...] completely": Puh, dangerous territory to > comment on, but that was the decision of Proxmox. No, your new license implies that. It would be illegal to provide support (in any way) for drbdmanage. Besides, it would not make any sense to have two repositories including the same packages - what for? ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On Tue, Jan 03, 2017 at 10:38:38AM +, Enrica Ruedin wrote: > I can't understand Linbit to change license in such a way that PVE > Proxmox has to remove their support completely. This unnecessary > change leads to confusion. But you do understand that we also eat and like to have a roof over our heads? Hey, the Aparthotel Guggach Hotel from which domain you wrote looks nice, that could solve one problem! ;-) In the end the license - and sure IANAL - is a "do whatever you like, but don't interfere with LINBIT's support business". IMO that's it. This was not a move against Proxmox at all, there are other vendors not playing nice... Just writing that because on the ML I have the impression that there is some kind of "Proxmox vs. LINBIT". It isn't. For the "has to remove [...] completely": Puh, dangerous territory to comment on, but that was the decision of Proxmox. They have all the right to do that! No blaming, no nothing. There are other projects/companies that took another road and still ship, integrate, or use drbdmanage with the current license. Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
I have the same situation as Michele. I ask me the same questions. I'm following the different mails concerning the change of repo. I can't understand Linbit to change license in such a way that PVE Proxmox has to remove their support completely. This unnecessary change leads to confusion. On the other hand it's great at least that Linbit maintains the storage plugin of PVE in minimum. BUT I think that the new borderline between Proxmox and Linbit isn't good for the future development now. I always hoped that PVE will extend to plugin with GUI components analog Ceph. Such a GUI would help to show/handle resources (incl. status, size and usage), assign resources to hosts etc in PVE. All this isn't possible anymore because Proxmox has stopped their software support with DRBD. I don't think that Linbit will develop new features for PVE/DRBD9. This license change worse the collaboration/integration PVE-DRBD. Further it will boost utilization of Ceph. Until today I thought that a three node cluster PVE/DRBD9 is the perfect solution. BTW my three nodes with PVE 4.3 with DRBD9 work perfectly, but I'm doubt if the combination PVE/DRBD9 has any future. So I appeal to Linbit to work together with Proxmox more closely. Such a strong virtualization environment like PVE together with Linbit's DRBD9 would be a powerful European answer to US software. Thanks Enrica -Ursprüngliche Nachricht- Von: drbd-user-boun...@lists.linbit.com [mailto:drbd-user-boun...@lists.linbit.com] Im Auftrag von Michele Rossetti Gesendet: Montag, 2. Januar 2017 14:53 An: drbd-user@lists.linbit.com Betreff: Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE One advice from who use drbd9 on PVE in production, please: we have a cluster of 3 PVe 4.3 servers with the last DRBD version available prom PVE repository. I read on the list that there are several problems with new DRBD version, not more included on PVE repository, but I don't understand if they are related only with som configuration or with the new changes on drbdmanage. In other words: can we upgrade without problems to PVE 4.4 and to DRBD last software on Linbit repository, or to PVE 4.4 from Proxmox repository and to latest DRBD from Linbit repository? Or it's better to wait some time, to avoid problems on production systems? Thanks and happy New Year, Michele ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 02/01/2017 14:53, Michele Rossetti ha scritto: > One advice from who use drbd9 on PVE in production, please: we have a > cluster of 3 PVe 4.3 servers with the last DRBD version available prom > PVE repository. > > I read on the list that there are several problems with new DRBD > version, I have switched to linbit repo, but I don't think that there are more problems than with pve ones; in my configuration I still have some issues (with qmrestore towards drbd storage, and automatic start of resources at boot, mainly), but I can't say that these are new or specific to linbit provided components. > not more included on PVE repository, but I don't understand if > they are related only with som configuration or with the new changes on > drbdmanage. See my former notes; in my experience I still have many doubts about using DRBD9 with PVE in production, even if I have it running for several months now. In particular I think that PVE DRBD9 storage integration needs refinements (storage occupation still does not correctly accounts for redundancy, and lacks features (such as volume resizing). On DRBD9 side, I still note randomly out-of-sync on some resources after node reboot, which I am able to fix only manually (disconnecting and reconnecting after some time, and similar ...) > In other words: can we upgrade without problems to PVE 4.4 and to DRBD > last software on Linbit repository, or to PVE 4.4 from Proxmox > repository and to latest DRBD from Linbit repository? Or it's better to > wait some time, to avoid problems on production systems? I recommend to upgrade to 4.4, firstly. In any case, given that Proxmox will not provide further support for DRBD9 in PVE, you should consider the switch to linbit repo in any case, if you want to keep using it in the future. > Thanks and happy New Year, Same to you, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
One advice from who use drbd9 on PVE in production, please: we have a cluster of 3 PVe 4.3 servers with the last DRBD version available prom PVE repository. I read on the list that there are several problems with new DRBD version, not more included on PVE repository, but I don't understand if they are related only with som configuration or with the new changes on drbdmanage. In other words: can we upgrade without problems to PVE 4.4 and to DRBD last software on Linbit repository, or to PVE 4.4 from Proxmox repository and to latest DRBD from Linbit repository? Or it's better to wait some time, to avoid problems on production systems? Thanks and happy New Year, Michele ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 28/12/2016 16:00, Roberto Resoli ha scritto:
> Il 27/12/2016 10:19, Roberto Resoli ha scritto:
>> All is up and running nicely, in any case.
Another issue: drbdmanaged starts at boot, but most resources remain down; a
drbdmanaged restart
fixes that.
I attach a related fragment from syslog; it seems that some temporary
.res files are not generated correctly.
rob
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG LvmThinLv: Running external
command: /sbin/lvchange -ay -kn -K drbdpool/vm-102-disk-1_00
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG Entering DrbdManager.run
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG DrbdManager: invoked
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG DrbdManager: hash check:
f76233200229778a7776b4d9aedbc2659b8e706c107dd9b910073a132007eb93
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG DrbdManager: hash unchanged
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG DrbdManager: finished
Dec 29 14:34:21 pve2 drbdmanaged[2295]: DEBUG Exited DrbdManager.run
Dec 29 14:34:22 pve2 org.drbd.drbdmanaged[2056]: Logical volume
"vm-102-disk-1_00" changed.
Dec 29 14:34:22 pve2 drbdmanaged[2295]: DEBUG Entering
DrbdManager.check_res_file
Dec 29 14:34:22 pve2 drbdmanaged[2295]: INFO DrbdAdm: Running external
command: drbdadm --config-to-test
/var/lib/drbd.d/drbdmanage_vm-102-disk-1.res.tmp --config-to-exclude
/var/lib/drbd.d/drbdmanage_vm-102-disk-1.res sh-nop
Dec 29 14:34:22 pve2 drbdmanaged[2295]: INFO DrbdAdm: External command
'drbdadm': Exit code 0
Dec 29 14:34:22 pve2 drbdmanaged[2295]: INFO DrbdAdm: Running external
command: drbdadm -c /var/lib/drbd.d/drbdmanage_vm-102-disk-1.res.tmp -d up
vm-102-disk-1
Dec 29 14:34:22 pve2 drbdmanaged[2295]: ERROR DrbdAdm: External command
'drbdadm': Exit code 10
Dec 29 14:34:22 pve2 drbdmanaged[2295]: ERROR drbdadm/stderr:
/var/lib/drbd.d/drbdmanage_vm-102-disk-1.res.tmp:4: in resource vm-102-disk-1:
Dec 29 14:34:22 pve2 drbdmanaged[2295]: ERROR drbdadm/stderr: #011a host
sections ('on pve2 { ... }') is missing.
Dec 29 14:34:22 pve2 drbdmanaged[2295]: DEBUG Exited
DrbdManager.check_res_file
Dec 29 14:34:22 pve2 drbdmanaged[2295]: INFO Resource file
/var/lib/drbd.d/drbdmanage_vm-102-disk-1.res.tmp not valid
Dec 29 14:34:22 pve2 drbdmanaged[2295]: DEBUG failed to start resource
'vm-102-disk-1', unhandled exception: ResourceFileException#012Exception stack
trace:#012 File
"/usr/lib/python2.7/dist-packages/drbdmanage/drbd/drbdcore.py", line 986, in
initial_up#012self._up_resource(assg)#012 File
"/usr/lib/python2.7/dist-packages/drbdmanage/utils.py", line 1728, in
wrapper#012ret = f(*args, **kwds)#012 File
"/usr/lib/python2.7/dist-packages/drbdmanage/drbd/drbdcore.py", line 1048, in
_up_resource#012self._server.export_assignment_conf(assignment)#012 File
"/usr/lib/python2.7/dist-packages/drbdmanage/server.py", line 5485, in
export_assignment_conf#012self.update_assignment_conf(res_name)#012 File
"/usr/lib/python2.7/dist-packages/drbdmanage/server.py", line 5611, in
update_assignment_conf#012raise update_exception
___
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 28/12/2016 16:00, Roberto Resoli ha scritto: > Il 27/12/2016 10:19, Roberto Resoli ha scritto: >> All is up and running nicely, in any case. > I said that too early: I tried to restore a kvm virtual machine and it > fails, systematically. > > I attached here the command with relative output, and the fragment of > syslog related to restore. > > It seems that resources (the vm had two disks) are being created > correctly, but access to them is failing. > > I have tried with another vm dump (one disk only); same problem. Update: I attach the syslog with drbdmanaged in "loglevel=debug" bye, rob qmrestore_105.tar.gz Description: application/gzip ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 27/12/2016 10:19, Roberto Resoli ha scritto: > All is up and running nicely, in any case. I said that too early: I tried to restore a kvm virtual machine and it fails, systematically. I attached here the command with relative output, and the fragment of syslog related to restore. It seems that resources (the vm had two disks) are being created correctly, but access to them is failing. I have tried with another vm dump (one disk only); same problem. Bye, rob qmrestore-error.tgz Description: application/compressed-tar ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On 27/12/2016 12:08, Roland Kammerer wrote : > On Tue, Dec 27, 2016 at 11:26:45AM +0100, Jean-Daniel Tissot wrote: >> >> >> > http-equiv="Content-Type"> >> >> >> On 27/12/2016 10:19, Roberto Resoli >> wrote : > Please stop sending html-only mail. Sorry for that, I do not see my mail was HTML-only. > >> I used DRBD 9 knowing it is not recommended for production. > Hm, there might be some over interpretation, right? IMO all that was > said is that if you don't need the additional features of DRBD9 (more > nodes, auto-promote, RDMA, resource management with drbdmanage), then > stay with drbd 8.4. IMO drbd9 improved a lot since .0, its simply your > decision. In https://pve.proxmox.com/wiki/DRBD9 they said "DRBD9 integration is introduced in Proxmox VE 4.x as technology preview". I don't know what they mean by preview. I don't have any commercial support for Proxmox or DRBD. Our financial resources are limited. >> Since we are on a production cluster, is it dangerous to switch to >> Linbit repository ? > Switching the repo isn't. Proxmox drbdmange had very minimal > modifications, basically using a different storage plugin as default. We > even ship that modification in our repo. The real thing is that the > drbdmanage *versions* shipped are very different. Get familiar with the > new one, especially the changed startup, try it in some VMs. Know what > the implications are if nodes are mission and you try to start the > cluster. There are many posts about that already, I don't repeat it here > again. > >> When DRBD 9 will be stable and really usable in production? > That's not a serious question, right? When will Linux be stable? For sure, a lot of my servers are on Debian / Jessie, they call it stable. That does not mean there is no bugs, but Jessie can be used in production. What I mean is "Is DRBD 9 in testing mode ? And can we use it in production ?" Sorry for my bad english. Regards, Jean-Daniel > > Regards, rck > ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On Tue, Dec 27, 2016 at 11:26:45AM +0100, Jean-Daniel Tissot wrote: > > >http-equiv="Content-Type"> > > > On 27/12/2016 10:19, Roberto Resoli > wrote : Please stop sending html-only mail. > I used DRBD 9 knowing it is not recommended for production. Hm, there might be some over interpretation, right? IMO all that was said is that if you don't need the additional features of DRBD9 (more nodes, auto-promote, RDMA, resource management with drbdmanage), then stay with drbd 8.4. IMO drbd9 improved a lot since .0, its simply your decision. > Since we are on a production cluster, is it dangerous to switch to > Linbit repository ? Switching the repo isn't. Proxmox drbdmange had very minimal modifications, basically using a different storage plugin as default. We even ship that modification in our repo. The real thing is that the drbdmanage *versions* shipped are very different. Get familiar with the new one, especially the changed startup, try it in some VMs. Know what the implications are if nodes are mission and you try to start the cluster. There are many posts about that already, I don't repeat it here again. > When DRBD 9 will be stable and really usable in production? That's not a serious question, right? When will Linux be stable? Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 27/12/2016 11:26, Jean-Daniel Tissot ha scritto: > On 27/12/2016 10:19, Roberto Resoli wrote : >> I have successfully done the transition to linbit repo drbdmanage. > Hi, > Well, I have done a three nodes PVE cluster. I can't use CEPH or others > storage technologies. > I used DRBD 9 knowing it is not recommended for production. > I had some problems at the beginning, but now it's works well. > HA is working. Live migration is working well. > DRBD sync take quite a long time sometimes but if it's take too long, > rebooting the node correct this problem. > For now, I use Proxmox repository and I don't see DRBD Manage is no more > present on it. It will: https://forum.proxmox.com/threads/drbdmanage-license-change.30404/ On the same forum, Philipp Reisner clarifies: https://forum.proxmox.com/threads/drbdmanage-license-change.30404/#post-152680 > Since we are on a production cluster, is it dangerous to switch to > Linbit repository ? I don't think so, but it is a major switch, so do that with careful planning and make sure to have a backup of every vm before starting. bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On 27/12/2016 10:19, Roberto Resoli wrote : I have successfully done the transition to linbit repo drbdmanage. Hi, Well, I have done a three nodes PVE cluster. I can't use CEPH or others storage technologies. I used DRBD 9 knowing it is not recommended for production. I had some problems at the beginning, but now it's works well. HA is working. Live migration is working well. DRBD sync take quite a long time sometimes but if it's take too long, rebooting the node correct this problem. For now, I use Proxmox repository and I don't see DRBD Manage is no more present on it. Since we are on a production cluster, is it dangerous to switch to Linbit repository ? When DRBD 9 will be stable and really usable in production? Thanks in advance. Bests regards, Jean-Daniel -- Bien cordialement, Jean-Daniel TISSOT Administrateur Systèmes et Réseaux Tel: +33 3 81 666 440 Fax: +33 3 81 666 568 Laboratoire Chrono-environnement 16, Route de Gray 25030 BESANÇON Cédex Plan et Accès ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 23/12/2016 12:22, Roberto Resoli ha scritto: >> So, this is essentially a stop-the-world and then upgrade scenario. > I agree. Will try directly that. I have successfully done the transition to linbit repo drbdmanage. I stopped all vms (but one), upgraded to new components as in the [1] note in http://lists.linbit.com/pipermail/drbd-user/2016-December/023418.html . There are still some problems in shutdown/startup of drbdmanage, in particular at node startup. It is something I am still investigating into. Given that I have a quite particular network setup with full mesh networking without dedicated switch[1], I suspect this is playing a a role. All is up and running nicely, in any case. rob [1] http://lists.linbit.com/pipermail/drbd-user/2016-August/023187.html ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
Il 23/12/2016 09:48, Roland Kammerer ha scritto: ... >>> ...There I would need more information. The version >>> we ship and the one from proxmox are very different. In between these >>> versions the architecture of DM was changed. Just as an example: If you >>> have two nodes in your cluster and you started DM only on one node, it >>> will never start up successfully. Or if only a minority of your overall >>> nodes is operational, drbdmanage will refuse to start, as it waits for a >>> majority. >> For the cluster I will do the test on: it is a three node PVE 4.4-2 >> cluster, vm resides on two main nodes, while the third is dedicated to >> provide quorum and DRBD9 replication only. I will start the test on this >> node. >> >> First question. Is it possible to mix and match versions of drbdmanage >> on the cluster? >> I really would like to update the third node only at first, but now I >> don't think it is possible, given the differences between proxmox >> provided drbdmanage and linbit's one. > The obvious: You could update *all*, there are entries in the control > volume (the .drbdctrl resource acting as cluster DB) that are not used > any more, but they are just ignored on the new version. Ok, nice to know > A mix-and-match isn't a good idea in that scenario. Let's assume you > update the 3rd node only, then this would happen: > > Assuming it is a node that has access to the .drbdctrl, it would try to > become the leader (i.e., switching .drbdctrl to primary). ok > Here it > depends if the resource > is already primary on another node or not. Let's > assume the "good" case, it loses the leader election and decides to > become a satellite. The old nodes would not talk to the new node via > TCP, but the new node expects the leader to talk to it, it would never > receive it's initial configuration. It would just idle around. In order > to remove any potential leftover (autogenerated .res files), the first > thing a node does is to remove all the old autogenerated .res files. So > by starting the new version, you lost all your DM .res files and can not > even manually "drbdadm up" them. For completeness the "bad" case: The > two old nodes idle around and have .drbdctrl as Secondary. The new one > comes and decides to become the leader and switches .drbdctrl to > Primary. The old nodes don't expect that, and would try to switch > .drbdctrl to Primary to store some information (how things worked in the > old version), but a leader never releases the .drbdctrl (new > architecture). Ok, this gives me a good hint about the differences between old and new behaviour. > > So, this is essentially a stop-the-world and then upgrade scenario. I agree. Will try directly that. Bye, rob ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] Testing new DRBD9 dedicated repo for PVE
On Thu, Dec 22, 2016 at 10:48:48PM +0100, Roberto Resoli wrote: > (Bad start, forget the message with the wrong title...) > > I start a new thread, as suggested in [1]; > > I would like to test the new Linbit repositorry for Proxmox Virtual > Environment [2], and I need some preliminary information before starting. > > I refer to these statments from [1]: > > > For the second thing: There I would need more information. The version > > we ship and the one from proxmox are very different. In between these > > versions the architecture of DM was changed. Just as an example: If you > > have two nodes in your cluster and you started DM only on one node, it > > will never start up successfully. Or if only a minority of your overall > > nodes is operational, drbdmanage will refuse to start, as it waits for a > > majority. > > For the cluster I will do the test on: it is a three node PVE 4.4-2 > cluster, vm resides on two main nodes, while the third is dedicated to > provide quorum and DRBD9 replication only. I will start the test on this > node. > > First question. Is it possible to mix and match versions of drbdmanage > on the cluster? > I really would like to update the third node only at first, but now I > don't think it is possible, given the differences between proxmox > provided drbdmanage and linbit's one. The obvious: You could update *all*, there are entries in the control volume (the .drbdctrl resource acting as cluster DB) that are not used any more, but they are just ignored on the new version. A mix-and-match isn't a good idea in that scenario. Let's assume you update the 3rd node only, then this would happen: Assuming it is a node that has access to the .drbdctrl, it would try to become the leader (i.e., switching .drbdctrl to primary). Here it depends if the resource is already primary on another node or not. Let's assume the "good" case, it loses the leader election and decides to become a satellite. The old nodes would not talk to the new node via TCP, but the new node expects the leader to talk to it, it would never receive it's initial configuration. It would just idle around. In order to remove any potential leftover (autogenerated .res files), the first thing a node does is to remove all the old autogenerated .res files. So by starting the new version, you lost all your DM .res files and can not even manually "drbdadm up" them. For completeness the "bad" case: The two old nodes idle around and have .drbdctrl as Secondary. The new one comes and decides to become the leader and switches .drbdctrl to Primary. The old nodes don't expect that, and would try to switch .drbdctrl to Primary to store some information (how things worked in the old version), but a leader never releases the .drbdctrl (new architecture). So, this is essentially a stop-the-world and then upgrade scenario. Regards, rck ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
