Re: [CentOS-virt] CentOS 7.(3) XEN 4.4 install
On 24/01/17 04:03 AM, Günther J. Niederwimmer wrote: > Hello List, > > I have to Install on a older Server Xen but this is not working now ;-( > > I do all I found in the Internet, can you tell me the way to a working XEN ? > > My Problem I mean, is a not working / starting libvirt no socket created ? > ERRORSocket-Erstellung zu '/var/run/libvirt/libvirt-sock' > > Thanks for a answer, Is the 'libvirtd' daemon running? -- Digimer Papers and Projects: https://alteeve.com/w/ "I am, somehow, less interested in the weight and convolutions of Einstein’s brain than in the near certainty that people of equal talent have lived and died in cotton fields and sweatshops." - Stephen Jay Gould ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Networking in KVM
On 17/03/16 04:47 PM, paul.greene...@verizon.net wrote: > Thanks, I followed the 2nd article, and it got the existing virtual > machines communicating with each other. > > However, any new virtual machines I created after making the changes > can't communicate with anything, they can't even get out to the > internet, even if I manually set the IP address info. Any suggestions on > that? > > Paul Check that they're using your static bridge. 'virsh dumpxml ' will have a section like: That tells you that the interface is MAC '52:54:00:71:20:fa' is "plugged in" to the bridge 'bcn_bridge1'. If that doesn't connect to the right bridge, then you need to change it (virt-manager has a simple to use GUI for this, or you can use 'virsh edit ' if you're comfortable editing XML). -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Networking in KVM
On 15/03/16 08:07 PM, paul.greene...@verizon.net wrote: > Hi all, > > New to KVM. Did a group install for "Virtualization Host" on CentOS 7. > Created two virtual machines - one with centos and one with an > evaluation copy of Windows 2012 server. > > Both virtual machines correctly did updates out to their respective > vendors (Centos and MS respectively), but they can't talk to each other > or talk to any other system in my internal network. From what I've found > so far, if I understand correctly, this is the default behaviour for KVM > out of the box. > > What I would like to be able to do is allow the virtual machines to > appear on my internal network just like any other machine, in the same > IP address space, without using dhcp for assigning addresses. The KVM > machines are using 192.168.122.x, my other machines are using 192.168.1.x. > > Any guidance/suggestions greatly appreciated. > > Paul Greene You need a traditional bridge (that acts more like a network switch). A quick google turns up these, which might help: http://jensd.be/207/linux/install-and-use-centos-7-as-kvm-virtualization-host http://unix-linux-server.blogspot.ca/2014/10/centos-7-kvm-installation-and-bridge.html -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] KVM
On 08/02/16 02:20 PM, Gokan Atmaca wrote: >> I'm guessing you're using standard 7,200rpm platter drives? You'll need >> to share more information about your environment in order for us to >> provide useful feedback. Usually though, the answer is 'caching' and/or >> 'faster disks'. > > Yes , 7.2k rpm disks. 2T mirror (soft). In fact, I had such a > preference for slightly more capacity. Those are slow and have poor seek latency. Slow-down of other servers when one hits the disk hard has to be expected. > Unfortunately very expensive SAS drives. But this works only if the > server in question occur. > In this case, about 15 minutes. progress. I don't understand what you are saying/asking, sorry. > On Mon, Feb 8, 2016 at 9:13 PM, Digimer <li...@alteeve.ca> wrote: >> On 08/02/16 02:12 PM, Gokan Atmaca wrote: >>> Hello >>> >>> I use KVM. In a virtual machine "jbd2 dm-0" disk I / O is very >>> increases. It consumes up to 99%. For this reason, slowing down the >>> other virtual machine. What should I do to solve the problem. ? >>> >>> Thanks.. >> >> I'm guessing you're using standard 7,200rpm platter drives? You'll need >> to share more information about your environment in order for us to >> provide useful feedback. Usually though, the answer is 'caching' and/or >> 'faster disks'. >> >> -- >> Digimer >> Papers and Projects: https://alteeve.ca/w/ >> What if the cure for cancer is trapped in the mind of a person without >> access to education? >> ___ >> CentOS-virt mailing list >> CentOS-virt@centos.org >> https://lists.centos.org/mailman/listinfo/centos-virt > ___ > CentOS-virt mailing list > CentOS-virt@centos.org > https://lists.centos.org/mailman/listinfo/centos-virt > -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] KVM
On 08/02/16 02:12 PM, Gokan Atmaca wrote: > Hello > > I use KVM. In a virtual machine "jbd2 dm-0" disk I / O is very > increases. It consumes up to 99%. For this reason, slowing down the > other virtual machine. What should I do to solve the problem. ? > > Thanks.. I'm guessing you're using standard 7,200rpm platter drives? You'll need to share more information about your environment in order for us to provide useful feedback. Usually though, the answer is 'caching' and/or 'faster disks'. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org https://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Video resolution for CentOS guest
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23/02/15 11:11 PM, Robert Nichols wrote: Would someone please point me to some reasonably current instructions for getting greater than 1024x768 video resolution for a CentOS 6 guest on a CentOS 6 KVM/qemu host? When I search online I find stuff from 2009 and 2010 saying, For details see ..., and linking to a URL that no longer exists, or pages that say, You need to switch from VNC to Spice, and giving a long list of out-of-date instructions for doing so. (With virt-manager it takes 2 clicks to do that. Of course it doesn't help -- still maxes out at 1024x768.) I've found that I can just append vga=0x380 to the kernel command line and see Plymouth come up with the full graphical boot screen in the correct 1440x900 resolution, but as soon as gdm starts up, the display scrambles. I find suggestions to generate an xorg.conf file, but no mention of what to put in it. I can run Xorg -configure, but the resulting file contains nothing about video modes, so it's not apparent what needs to be added. I find it particularly annoying that a Windows 7 guest can set any resolution I want up to 2560x1600, but a Linux guest can't go higher than 1024x768. I played with this and found that, in fact, I had to switch the spice / qxl. With that change, I had no trouble pushing EL6 to much higher resolutions. - -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJU7APUAAoJECChztQA3mh0OCoP/jW8jmGWWVdIirc+4G+kvo+S LZrJxIxZVDfJHioICZink8JJOKm9m5k8k0FKB6YonoLNWOgk8HlHfTTzG9dToT/C Rk3oTrI8pDCsMwccngd0VSVR2EQtmzQBp/O/38JHPM0/VjKnum/I1NWli8g5Xoq2 Q1BqAbrYJ2SAhVht2G91DKsP/nPLq93hBU+UrJkhg5bi3aFCw/Da53v5G3oOZTfr 9qS0RoibthrrF2yCIiXW0kdsEtwk8m+RYFroKjGh/PWcYIKhJdU2Rn8a6gDwRYPg 90fVCYwrqir1HChWsYGc0q+p3DNh/0WmPHjbfbs5o66erD2MZkkm7tbyM8gTcl3F 03wKyhO8qoFcCcgbLRBDb/pYKMX3ChOw7b1RFrYah1xWlZCNAWaBCVYm7DUGMfrV zqb70dVEkUch6f+rVxEo6mrWlj4927Pnp9pefTJ6aVibLZZBA040cVCMTpAwpa7H VsR3QhCcwhOpsCdn0WRCZFJyWlGO88Ry5A0RMUHrceBh0wPdTPPDhMiSg1qTlqCT 0VtaBiYeLevEzoMqnqcRI/+wW0/ooJzp5KjkxjFVVVLqwpqNQkHhEMvUBNLuF5Ui 6O5pZSwFk4K1c5sgOfxWkGuvYjKYmUW/BlnsTpulOzHsOVcJwREUC+IrXIftytfI BJ+4n8EX52YmiORmmGRJ =SO1z -END PGP SIGNATURE- ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] C7 with Xen support
On 22/07/14 10:06 AM, Volnei wrote: Thank you Mark. I use F20 with Xen dom0 with much success. I wanted to start using CentOS instead of Fedora but apparently will take a little longer. You can't understand why RH does these things. Back in RHEL 5, Xen was supported. It added a lot of work for Red Hat, maintaining all the kernel patches. This was true for a lot of other programs as well. So to reduce their work load, they thinned down a lot for RHEL 6. They dropped everything they thought was a duplication of another project or which required too much effort for too few users. With KVM, Xen was effectively a duplication of a much simpler to use hypervisor, so it was dropped. No great conspiracy here; Just normal business decisions. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Finally switching from Xen to KVM - question about networking
On 10/06/14 11:46 AM, Steve Campbell wrote: On 6/10/2014 10:46 AM, Digimer wrote: On 10/06/14 10:03 AM, Steve Campbell wrote: I had so much trouble putting Centos 6 guest VMs on a Centos 5 host that I finally switched to a Centos 6 host. I've not needed more that test VMs, so I've used Virtual Machine Manager on the old system, which worked pretty well, so I decided to create my first KVM guest machine. I noticed when I created it, I only had the options of NAT for my network interface, so I used that (obvious). Well, after starting the VM, I find I don't have connectivity with that interface. Reading, I find examples where I need to create bridges perhaps. Xen did most of this for me, so it's a little new to me. Can anyone throw me a clue, please? steve campbell Setting up a bridge is not that hard, and it will give your VMs direct access to the outside world, and host - VM access just fine as well. Here is a link showing how to setup a bridge connected to a bond device. Ignore the bond and pretend it is a straight ethX device: https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces The host has a device named virbr0 that is installed during system installation. It also has a network device vnet0. There are no files in /etc/sysconfig/network-scripts for these. Shouldn't I be able to use the virbr0 virtual bridge for this? I've tried setting up the VM's device with all of the options that is listed, but to no avail. Should I need to set up another bridge for this? And thanks for the link. steve virbr0 is created and managed by libvirtd. If you open Virtual Machine Manager, connect to localhost and then double-click on 'localhost', you will see a tab for creating/managing bridges (NAT'ed, generally). I disable 'virbr0' as NAT'ing is generally not what I want. The 'vnetX' devices are dynamically created to link a VM's interface to a bridge. Think of them as virtual network cables. They get created and destroyed as needed. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Finally switching from Xen to KVM - question about networking
On 10/06/14 12:38 PM, Steve Campbell wrote: On 6/10/2014 12:05 PM, Digimer wrote: On 10/06/14 11:46 AM, Steve Campbell wrote: On 6/10/2014 10:46 AM, Digimer wrote: On 10/06/14 10:03 AM, Steve Campbell wrote: I had so much trouble putting Centos 6 guest VMs on a Centos 5 host that I finally switched to a Centos 6 host. I've not needed more that test VMs, so I've used Virtual Machine Manager on the old system, which worked pretty well, so I decided to create my first KVM guest machine. I noticed when I created it, I only had the options of NAT for my network interface, so I used that (obvious). Well, after starting the VM, I find I don't have connectivity with that interface. Reading, I find examples where I need to create bridges perhaps. Xen did most of this for me, so it's a little new to me. Can anyone throw me a clue, please? steve campbell Setting up a bridge is not that hard, and it will give your VMs direct access to the outside world, and host - VM access just fine as well. Here is a link showing how to setup a bridge connected to a bond device. Ignore the bond and pretend it is a straight ethX device: https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces The host has a device named virbr0 that is installed during system installation. It also has a network device vnet0. There are no files in /etc/sysconfig/network-scripts for these. Shouldn't I be able to use the virbr0 virtual bridge for this? I've tried setting up the VM's device with all of the options that is listed, but to no avail. Should I need to set up another bridge for this? And thanks for the link. steve virbr0 is created and managed by libvirtd. If you open Virtual Machine Manager, connect to localhost and then double-click on 'localhost', you will see a tab for creating/managing bridges (NAT'ed, generally). I disable 'virbr0' as NAT'ing is generally not what I want. The 'vnetX' devices are dynamically created to link a VM's interface to a bridge. Think of them as virtual network cables. They get created and destroyed as needed. Sorry, but I'm confused: My host server has a real NIC and IP address with a real gateway to the outside: virtbr0 IP: 192.168.122.1 Host IP: 10.0.5.16 Gateway IP: 10.0.5.1 on eth0 and this works My VM server has all fake stuff currently: Host IP: 10.0.5.17 Gateway IP: 10.0.5.1 on eth0 and this is like NIC without a cable. So I need to create a bridge device on both the host and VM (lets say I name it br1). I change the eth0 config file on both host and VM to point to br1 and give the br1 config file on both host and VM the correct IP. But won't this just let the two talk to each other. How will the VM server get outside? steve The bridge is created only on the host. Think of the bridge as being like a virtual switch. When a VM is created, you tell it to connect to the bridge, similar to how you would plug a physical wire into a real switch. That provides the link to the network, and then you configure the virtual server's network just as if it was a real network. On the host, you don't set the IP on the ethX device, instead you tell ethX to connect to the bridge (look at 'bond2' in the link above for the 'BRIDGE=...' line). Then you move the host's IP address/config to the bridge itself (look at the 'vbr2' in the link above). If you're still stuck, tell me the IP addresses you want to set in the host and VM and I'll see if I can bang out a couple sample 'ifcfg-X' files. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] [CentOS] KVM and DHCP
On 13/03/14 03:43 PM, James B. Byrne wrote: CentOS-6.5 We have a KVM guest running MS-WinV7pro. This guest is joined to an Active Directory Domain. That domain provides DHCP to the members. The KVM guest does not obtain its IP from the domain but from the local host's qemu hypervisor instead. Is there anyway to get around this and have the guest MS-Win OS get its DHCP from the same place as the rest of the domain members? Yup, build a traditional bridge instead of using qemu's 'default' virbr0 NAT'ed bridge. I do it this way: https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces You can ignore the bonds if you're not using them, and just put the 'BRIDGE=vbrX' in the ethX file. Move the IP from the ethX to vbrX. With that, your VMs will effectively be on the same network, no translation, and the bridge will act just like another switch on your network. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Remove Centos from AWS marketplace
On 09/03/14 11:52 AM, Nico Kadel-Garcia wrote: On Sun, Mar 9, 2014 at 11:28 AM, Digimer li...@alteeve.ca wrote: Would you mind elaborating on this? If a snapshot is a point-in-time image of a VM (or even normal FS), why would DB backups be at risk (assuming things like fsync are used)? I'm asking in general terms... no idea if this is something AWS specific. digimer It's a general issue. If a system snapshot is used to correctly preserve both the disk image, and the state of the VM including memory, well and good. The state is recoverable. There's always a risk that interrupted network transactions left things in an unexpectedly inconsistent state that the VM is not equipped to handle: I'm thinking particularly of wget or other download transactions where the download software was not intelligent enough to verify the download before proceeding. I've been through this a lot lately with chef software. It's compounded by network based filesystem transactions, such as interactions with NFS or CIFS filesystems, which cannot be synchronized with the OS snapshot. But simply relying on the disk image from such an AWS snapshot, without recovering the full system state, is a potential adventure. I've not myself had opportunity to play with this kind of restoration, so I'm uncertain whether AWS allows access to the plain disk image, or automatically would bring the full VM state with it for re-activation of the snapshot. If you're just getting at the disk images, using fsync before the snapshots is helpful, but any atomic transaction that is in progress at the time of the disk image snapshot is not verifiable in the atomicity of that transaction. This particularly includes precisely the sort of page mapped data, sitting in RAM, that the fsync command helps write to disk. And snapshots cheduled from outside controllers, such as automatic snapshots, cannot be reliably synced with system specific fsync database suspension commands without a great deal of integration between the outside system, and the local host, that VM's are not supposed to normally need. I went through great deal of this some years back, shutting down databases, running LVM to get a disk snapshot, then running rsnapshot against the *snapshot* to avoid getting an inconsistent state of the database into the backup system. And there are some *funky* databases out there. Ask sometime about the Use hardlinked RCS files for source control of multiple project branches sometime, if you'd like to wince a lot. This is very useful, thank you kindly for sharing. I suppose I always considered the it's like recovering for the server losing power as usually works and equating that to good enough backup. So I suppose, at best, using snapshot images as a backup ... backup method would be valid... I could see the benefit of recovering the VM, and then if anything wasn't right, using it as the target for restoring data from the proper backup. Thanks again! -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Remove Centos from AWS marketplace
On 09/03/14 11:43 AM, Stephen Harris wrote: On Sun, Mar 09, 2014 at 11:28:07AM -0400, Digimer wrote: Would you mind elaborating on this? If a snapshot is a point-in-time image of a VM (or even normal FS), why would DB backups be at risk (assuming things like fsync are used)? I'm asking in general terms... no idea if this is something AWS specific. Database disk snapshots may include transactions in flight and the on-disk image may not be in a consistent state. Databases such as Oracle try to work around this by ensuring that writes occur in a specific order and have a good recovery process (each data file has a change number; determine the best change number to start from, roll forward from there to recover, then roll back any incomplete transactions) but it's considered crash recovery and shouldn't be part of BAU activity. Other databases may not be so good at recovery (mysql?) and so you run the risk of database corruption if you need to restore the snapshot. If you rely on disk snapshots then it's recommended you do a proper db dump before the snapshot is taken, so that you can recover the database from the dump file and not the snapshot. Thanks for the reply, Stephen. I also replied to Nico, and my comments there can be directed to you, as well. :) -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] Announcing a new HA KVM tutorial!
Almost exactly two years ago, I released the first tutorial for building an HA platform for KVM VMs. In that time, I have learned a lot, created some tools to simplify management and refined the design to handle corner-cases seen in the field. Today, the culmination of that learning is summed up in the 2nd Edition of that tutorial, now called AN!Cluster Tutorial 2. https://alteeve.ca/w/AN!Cluster_Tutorial_2 These HA KVM platforms have been in production for over two years now in facilities all over the world; Universities, municipal governments, corporate DCs, manufacturing facilities, etc. I've gotten wonderful feedback from users and all that real-world experience has been integrated into this new tutorial. As always, everything is 100% open source and free-as-in-beer! The major changes are: * SELinux and iptables are enabled and used. * Numerous slight changes made to the OS and cluster stack configuration to provide better corner-case fault handling. * Architecture refinements; ** Redundant PSUs, UPSes and fence methods emphasized. ** Monitoring multiple UPSes added via modified apcupsd ** Detailed monitoring of LSI-based RAID controllers and drives ** Discussion on hardware considerations for VM performance based on anticipated work loads * Naming convention changes to support the new AN!CDB dashboard[1] ** New alert system covered with fault and notable event alerting * Wider array of guest OSes are covered; ** Windows 7 ** Windows 8 ** Windows 2008 R2 ** Windows 2012 ** Solaris 11 ** FreeBSD 9 ** RHEL 6 ** SLES 11 Beyond that, the formatting of the tutorial itself has been slightly modified. I do think it is the easiest to follow tutorial I have yet been able to produce. I am very proud of this one! :D As always, feedback is always very much appreciated. Everything from typos/grammar mistakes, functional problems or anything else is very valuable. I take all the feedback I get and use it to helping make the tutorials better. Enjoy! Digimer, who now can now start the next tutorial in earnest! 1. https://alteeve.ca/w/AN!CDB -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Announcing a new HA KVM tutorial!
Awesome, I'll be very happy for your feedback! On 06/01/14 11:20 AM, Aly Khimji wrote: Thank you very much for this, looks like a good read. Will provide feedback :) Aly On Mon, Jan 6, 2014 at 11:11 AM, Digimer li...@alteeve.ca mailto:li...@alteeve.ca wrote: Almost exactly two years ago, I released the first tutorial for building an HA platform for KVM VMs. In that time, I have learned a lot, created some tools to simplify management and refined the design to handle corner-cases seen in the field. Today, the culmination of that learning is summed up in the 2nd Edition of that tutorial, now called AN!Cluster Tutorial 2. https://alteeve.ca/w/AN!Cluster_Tutorial_2 These HA KVM platforms have been in production for over two years now in facilities all over the world; Universities, municipal governments, corporate DCs, manufacturing facilities, etc. I've gotten wonderful feedback from users and all that real-world experience has been integrated into this new tutorial. As always, everything is 100% open source and free-as-in-beer! The major changes are: * SELinux and iptables are enabled and used. * Numerous slight changes made to the OS and cluster stack configuration to provide better corner-case fault handling. * Architecture refinements; ** Redundant PSUs, UPSes and fence methods emphasized. ** Monitoring multiple UPSes added via modified apcupsd ** Detailed monitoring of LSI-based RAID controllers and drives ** Discussion on hardware considerations for VM performance based on anticipated work loads * Naming convention changes to support the new AN!CDB dashboard[1] ** New alert system covered with fault and notable event alerting * Wider array of guest OSes are covered; ** Windows 7 ** Windows 8 ** Windows 2008 R2 ** Windows 2012 ** Solaris 11 ** FreeBSD 9 ** RHEL 6 ** SLES 11 Beyond that, the formatting of the tutorial itself has been slightly modified. I do think it is the easiest to follow tutorial I have yet been able to produce. I am very proud of this one! :D As always, feedback is always very much appreciated. Everything from typos/grammar mistakes, functional problems or anything else is very valuable. I take all the feedback I get and use it to helping make the tutorials better. Enjoy! Digimer, who now can now start the next tutorial in earnest! 1. https://alteeve.ca/w/AN!CDB -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org mailto:CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
On 22/11/13 17:11, aurfalien wrote: Sorry guys, I've tried and tried, no dice. Seems like I am missing missing a vent1, vnet2, etc... to br0 association. I can see were the vnet# gets created upon VM startup. And based on how my VM xml file is set, will go to either br0, br1. br2, etc... But in my case, the only interface that works is vnet0 for all my VMs. In the CentOS virtual machine manager for whatever NIC you choose, there is a drop down option for virtual network interface. For source device, I only ever see a vnet0 to br0. For my other bridges, there is only eth# to vnet#. The configs for this are rather simple and I don't know were else to look; various /etc/sysconfig/network* files and the VM xml config. Everythings is set to the same MTU wether standard or jumbo, but no matter what, my VMs network interfaces work when set to vnet0 as its connected to br0. I cannot get br6 to show with vnet2 for example. Not even my vnet1 is connected to br1 but rather br0. However in the UI as mentioned before, i do not see a vnet1 to br1 relationship. Are there any other config files I can look at? - aurf Why do you have so many bridges? In almost all cases, only one bridge is needed. The bridge should connect to a real interface to get to the outside world. Then all VMs should point to that bridge. I think you might be over-complicating things. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
default NAT'ed bridge. I don't recommend using those. I usually destroy them, personally. So to fix your problem, you need to tell the VMs to use br6. If you want to use jumbo frames on br0, you need to increase the MTU of eth0. Remember that the bridge will use the MTU of the lowest connected device. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
On 21/11/13 17:32, aurfalien wrote: On Nov 21, 2013, at 2:24 PM, Digimer wrote: I'm not sure what you are asking. You should not see the vnetX devices from the VM (or even the VM's definition file). They're created as needed to link the VM's interface to the bridge. Think of them as simple network cables. Some of the formatting isn't showing well on my mail client (text only), so I am having a little trouble parsing some of the data... If the VMs are using br6, then you see that it's already at 9000, so you should be able to use 9000 from inside the VM as well. Trick is, the vnetX devices are connected to the br0 bridge instead, which is set to 1500 because eth0 is still 1500. So at this point, the VMs are traversing br0, not br6. As for 'virbr0', that is libvirtd's default NAT'ed bridge. I don't recommend using those. I usually destroy them, personally. So to fix your problem, you need to tell the VMs to use br6. If you want to use jumbo frames on br0, you need to increase the MTU of eth0. Remember that the bridge will use the MTU of the lowest connected device. So far my current VMs that work, I see there network as; Host device vnet 0 (Bridge 'br0') I do not see a; Host device vnet# (Bridge 'br6') My ints of interest are set to jumbo so thats not a prob. I think the prob is that I am missing the vnet bridge device for eth6. So I'm curious why its not there and how do I create it? - aurf I can't speak to the tools you are using, but I can say that this is where the bridge is defined in the VM's XML definition file: [root@an-c05n01 ~]# cat /shared/definitions/vm01-win2008.xml |grep vbr -B 2 -A 5 interface type='bridge' mac address='52:54:00:8e:67:32'/ source bridge='vbr2'/ target dev='vnet0'/ model type='virtio'/ alias name='net0'/ address type='pci' domain='0x' bus='0x00' slot='0x03' function='0x0'/ /interface Try changing: source bridge='br0'/ - source bridge='br6'/ and restart the VM. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
On 21/11/13 17:42, aurfalien wrote: On Nov 21, 2013, at 2:36 PM, Digimer wrote: On 21/11/13 17:32, aurfalien wrote: On Nov 21, 2013, at 2:24 PM, Digimer wrote: I'm not sure what you are asking. You should not see the vnetX devices from the VM (or even the VM's definition file). They're created as needed to link the VM's interface to the bridge. Think of them as simple network cables. Some of the formatting isn't showing well on my mail client (text only), so I am having a little trouble parsing some of the data... If the VMs are using br6, then you see that it's already at 9000, so you should be able to use 9000 from inside the VM as well. Trick is, the vnetX devices are connected to the br0 bridge instead, which is set to 1500 because eth0 is still 1500. So at this point, the VMs are traversing br0, not br6. As for 'virbr0', that is libvirtd's default NAT'ed bridge. I don't recommend using those. I usually destroy them, personally. So to fix your problem, you need to tell the VMs to use br6. If you want to use jumbo frames on br0, you need to increase the MTU of eth0. Remember that the bridge will use the MTU of the lowest connected device. So far my current VMs that work, I see there network as; Host device vnet 0 (Bridge 'br0') I do not see a; Host device vnet# (Bridge 'br6') My ints of interest are set to jumbo so thats not a prob. I think the prob is that I am missing the vnet bridge device for eth6. So I'm curious why its not there and how do I create it? - aurf I can't speak to the tools you are using, but I can say that this is where the bridge is defined in the VM's XML definition file: [root@an-c05n01 ~]# cat /shared/definitions/vm01-win2008.xml |grep vbr -B 2 -A 5 interface type='bridge' mac address='52:54:00:8e:67:32'/ source bridge='vbr2'/ target dev='vnet0'/ model type='virtio'/ alias name='net0'/ address type='pci' domain='0x' bus='0x00' slot='0x03' function='0x0'/ /interface Try changing: source bridge='br0'/ - source bridge='br6'/ and restart the VM. It already has the source bridge as br6. But I think I need to have a; vnet6 to br6 relationship defined somewhere. Right now I only see; Host device eth6 to br6 but I need vnet6 to br6 or something like that. Currently, while my guest VM sees its int via ifconfig, it cannot get an packets to/from. This is why I feel the need for a vnet to br6. - aurf The 'vnetX' number doesn't relate to the interface, bridge or anything else. The vnetX number is a simple sequence that increments each time a VM is started. So don't think that you need 'vnet6'... it can be anything. The 'brctl show' output from earlier showed that both vnet0 and vnet1 were connected to br0. You can try using the bridge utils to remove them from br0 and connect them to br6 as a test. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
On 21/11/13 18:20, aurfalien wrote: On Nov 21, 2013, at 2:45 PM, Digimer wrote: The 'vnetX' number doesn't relate to the interface, bridge or anything else. The vnetX number is a simple sequence that increments each time a VM is started. So don't think that you need 'vnet6'... it can be anything. The 'brctl show' output from earlier showed that both vnet0 and vnet1 were connected to br0. You can try using the bridge utils to remove them from br0 and connect them to br6 as a test. -- Digimer Well, when I remove vnet1 from br0 and add vnet1 to br1, I loose connectivity with my VMs. No biggy so I reboot my entire host. Then vnet1 show back under br0. I just don't understand enough about this to get a clue, depressing. - aurf Think of each bridge as if it were a physical switch. When you detached vnet1 from br0, you unplugged it from a switch. When you attached it to br1, you plugged it into another switch. If there is no connection out to your network/internet on a given switch, then anything plugged into that switch will go nowhere. Same with bridges. You seemed to indicate earlier that the main connection was on br6. Is this true? If so, then switch br6 is the switch with the uplink to your network. Plug a VM into it and you can route out through it. When you rebooted the VM, the hypervisor read the definition file. That definition file says to plug in the server to br0. So it makes sense that the reboot reconnected it to br0. If you want to use jumbo frames on the br0 switch, you need to set the larger MTU on the interfaces are all set to your desired MTU size. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
It's not so much hard as it is knowing all the hops in your network. If anything along the chain has a low MTU, the whole route is effectively reduced. On 21/11/13 20:20, Nico Kadel-Garcia wrote: This is int4eresting stuff. I do note that the virt-manager tool, and NetworkManager, give *no* insight and detailed management sufficient to resolve this stuff. Note also that dancing through all the hoops to get this working, end-to-end, is one of the big reasons that most environments refuse to even *try* to use jumbo frames, as helpful as they sometimes are to heavy data transfers. On Thu, Nov 21, 2013 at 6:58 PM, Digimer li...@alteeve.ca wrote: On 21/11/13 18:20, aurfalien wrote: On Nov 21, 2013, at 2:45 PM, Digimer wrote: The 'vnetX' number doesn't relate to the interface, bridge or anything else. The vnetX number is a simple sequence that increments each time a VM is started. So don't think that you need 'vnet6'... it can be anything. The 'brctl show' output from earlier showed that both vnet0 and vnet1 were connected to br0. You can try using the bridge utils to remove them from br0 and connect them to br6 as a test. -- Digimer Well, when I remove vnet1 from br0 and add vnet1 to br1, I loose connectivity with my VMs. No biggy so I reboot my entire host. Then vnet1 show back under br0. I just don't understand enough about this to get a clue, depressing. - aurf Think of each bridge as if it were a physical switch. When you detached vnet1 from br0, you unplugged it from a switch. When you attached it to br1, you plugged it into another switch. If there is no connection out to your network/internet on a given switch, then anything plugged into that switch will go nowhere. Same with bridges. You seemed to indicate earlier that the main connection was on br6. Is this true? If so, then switch br6 is the switch with the uplink to your network. Plug a VM into it and you can route out through it. When you rebooted the VM, the hypervisor read the definition file. That definition file says to plug in the server to br0. So it makes sense that the reboot reconnected it to br0. If you want to use jumbo frames on the br0 switch, you need to set the larger MTU on the interfaces are all set to your desired MTU size. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
What you do in the VMs does not impact the hosts, so I didn't speak to
that. Having the bridge, interfaces, switches and vnets at 9000 (for
example) doesn't immediately enable large frames in the virtual servers.
It simply means that all of the links between the VM and other devices
on the network are ready for JFs.
Imagine this;
{real switch}
|
{ethX + ethY}
|
{bondX}
|
{vbr0}
|
{vnetX}
|
{VM's eth0}
All of these devices need to have their MTU set to your desires value.
If any one of these is still 1500, then only standard frames will be
able to traverse them.
* real switch; Log into it and make sure jumbo frames are enabled
* ethX + ethY; If you are using bonding, be sure both/all slaved
interfaces are set to use a large frame.
* bondX; Again if you use a bond, make sure the bondX interface has a
large frame.
* vbr0; The bridge can not be set to a specific MTU size. It will use
the lowest MTU of the various devices / interfaces connected to it.
* vnetX; These are the virtual network cables that are used to plug
in a VM's interface to the bridge. This is not new by any means. In the
real world, network cables don't have setable MTUs of course. In the
virtual world though, they do. These interfaces are spontaneously
created and destroyed as VMs come and go. This is what the udev rule is
for because these virtual network cables don't have traditional
ifcfg-X files.
* VM's eth0; This is the (emulated) network card in your virtual server.
If you told the hypervisor to replicate an e1000 intel card or use the
virtio-net driver, you can set a large MTU. However, if you used
something like an emulated realtek card, those don't support jumbo
frames, so their emulated counterparts will not support large frames either.
hth
digimer
On 21/11/13 13:32, Nico Kadel-Garcia wrote:
I was under the impression that the relevant MTU settings were on the
*node's* local ifcfg-eth* configurations. Did something change with
KVM internal networking in the last year?
On Thu, Nov 21, 2013 at 1:03 PM, Digimer li...@alteeve.ca wrote:
The problem is that there are no ifcfg-vnetX config files. They are
dynamically created as VMs are created or migrated to a node. You could
manually (or via script) change the MTU, but that would mean that the
MTU on the bridge would drop momentarily when new VMs start. This could
break network traffic for any existing VMs (or real devices) using large
frames.
I'm not a fan of udev either, but in this case, it is the best option.
Of course, I am certainly open to hearing alternative methods if they exist.
On 21/11/13 08:39, Nico Kadel-Garcia wrote:
Stay out of udev if you can. It's often overwritten by component
addition and manipulation MTU is parsed, and overridden, by options in
/etc/sysconfig/network-scripts/ifcfg-[device]. I find it much safer to
read and manage there, and if new devices are added or replaced, the
behavior is dominated by the HWADDR associated config files there,
no matter what udev thinks the device number or name should be..
snip
Another update;
To make sure the VMs' vnetX devices are created with a larger MTU, you
*sill* need to update udev[1].
Append to /etc/udev/rules.d/70-persistent-net.rules;
# Make all VMs' vnetX devices come up with an MTU of 9000.
SUBSYSTEM==net, ACTION==add, KERNEL==vnet*, ATTR{mtu}=9000
Assuming you find that you can use an MTU of '9000', of course. No
need to reboot or even restart networking. Just add that line and then
provision/boot your VMs. If the VMs are already running, you can adjust
the MTU of the existing 'vnetX' devices with:
ifconfig vnetX mtu 9000
Cheers!
PS - Credit for the udev rule:
http://linuxaleph.blogspot.ca/2013/01/how-to-network-jumbo-frames-to-kvm-guest.html
--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos-virt
--
Digimer
Papers and Projects: https://alteeve.ca/w/
What if the cure for cancer is trapped in the mind of a person without
access to education?
___
CentOS-virt mailing list
CentOS-virt@centos.org
http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS-virt] proper bridging technoque
On 20/11/13 19:04, aurfalien wrote: Hi, Wondering if this is the proper bridging technique to use for Centos6+KVM; http://wiki.centos.org/HowTos/KVM Before I embark on this again, I would like to do it by the book. Thanks in advance, - aurf Personally, I do this: https://alteeve.ca/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Configuring_The_Bridge It gives the VMs direct access to the outside network, as if they were normal servers. I've used this setup for years without issue under many different VMs with various OSes. cheers -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] proper bridging technoque
On 20/11/13 20:49, aurfalien wrote: On Nov 20, 2013, at 4:47 PM, Digimer wrote: On 20/11/13 19:47, aurfalien wrote: On Nov 20, 2013, at 4:44 PM, Digimer wrote: On 20/11/13 19:25, aurfalien wrote: On Nov 20, 2013, at 4:13 PM, Digimer wrote: On 20/11/13 19:04, aurfalien wrote: Hi, Wondering if this is the proper bridging technique to use for Centos6+KVM; http://wiki.centos.org/HowTos/KVM Before I embark on this again, I would like to do it by the book. Thanks in advance, - aurf Personally, I do this: https://alteeve.ca/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Configuring_The_Bridge It gives the VMs direct access to the outside network, as if they were normal servers. I've used this setup for years without issue under many different VMs with various OSes. cheers Many many thanks, will use it. Sounds like it will bode well concerning jumbo frames. - aurf Jumbo frames should be fine. I don't generally use it myself, but I have tested it with success. Just be sure to enable it on the bridge and slaved devices. Simply adding 'MTU=' to each ifcfg-x file should be sufficient. -- Digimer Man, really sorry to bug as this seems begnin as I've done this numerous time but on non bridged ints. When I add MTU=9000 to the bridged int, I get; RNETLINK answers invalid argument My phys int is showing jumbo but the bridged int is showing standard. No bother at all. It has been a bit since I tested it though, so I will have to experiment a bit myself Done! I remember the trick now; The bridge will take the MTU of the _lowest_ MTU device connected to it. So in my case here, I up'ed the MTU of the backing ethX and bondY devices, but the bridge stayed at 1500. Trying to adjust it failed with 'SIOCSIFMTU: Invalid argument', which is the kernel's way of saying that the MTU is too large for the device (usually hit when surpassing the hardwares real MTU). Being a bridge though, this didn't make sense. When I up'ed the MTU of the vnetX devices though, the bridge jumped up on its own. So I suspect that if you do 'brctl show' and then check the MTU of the connected devices, one of them will still have a low MTU. Push it up and then do a non-fragmenting ping 28 bytes smaller than your MTU size. If the ping works, you know the MTU is increased. All this said, my experience with realtek NICs left me detesting them. I've seen cards advertised as supporting jumbo frames going up to silly sizes like 7200 only. Further, in benchmarks, the performance dropped over something like an MTU of 4000. If you want to determine the actual maximum MTU of a given interface, this might help; https://github.com/digimer/network_profiler/blob/master/network_profiler It's a little script that uses passwordless SSH between two nodes and automatically determines the maximum MTU between the two machines and then benchmarks at 100 byte intervals. When it's done, it spits out a graph showing the full and half-duplex results so you can see which MTU was the best to use. Once you've profiled the real devices, you can then work on the MTU of the higher-layer devices like bonds, bridges and virtual interfaces. hth -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Would a hardware TRNG benefit VMs?
On 16/07/13 18:01, Pete Travis wrote: On Jul 16, 2013 10:20 AM, Digimer li...@alteeve.ca mailto:li...@alteeve.ca wrote: On 16/07/13 06:30, Stefano Stabellini wrote: On Mon, 15 Jul 2013, Digimer wrote: On 15/07/13 07:04, Stefano Stabellini wrote: On Sat, 13 Jul 2013, Digimer wrote: Hi all, I've been curious about using hardware true random number generators in my VM clusters. This got me wondering about the VM's /dev/random source... If I setup a hardware TRNG on the host, would the VM's also benefit from it? Xen VMs would not be able to benefit from it. I don't know about KVM. Would you be able to elaborate on why not? I know Xen and KVM differ, but this is not a topic I have seen discussed at all before, so any insight would be helpful and appreciated! :) Because we don't have a paravirtualized interface to export the randomness to the guest. It would be a nice small little project to do that though. Ah, ok. I'm an admin, not a programmer, so I am not sure what would be needed. Given how popular VMs are now, I can imagine it would be a nice feature to have. If it's a small project, then all the better! :) Thanks! -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? There is such a thing, see https://fedoraproject.org/wiki/Features/Virtio_RNG --Pete Ok, that is _awesome_! I hope it gets into RHEL 7. Thank you kindly for the link! -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] Would a hardware TRNG benefit VMs?
Hi all, I've been curious about using hardware true random number generators in my VM clusters. This got me wondering about the VM's /dev/random source... If I setup a hardware TRNG on the host, would the VM's also benefit from it? Thanks! -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] GFS2 hangs after one node going down
On 03/25/2013 08:44 AM, Maurizio Giungato wrote: Il 22/03/2013 16:27, Digimer ha scritto: On 03/22/2013 11:21 AM, Maurizio Giungato wrote: Il 22/03/2013 00:34, Digimer ha scritto: On 03/21/2013 02:09 PM, Maurizio Giungato wrote: Il 21/03/2013 18:48, Maurizio Giungato ha scritto: Il 21/03/2013 18:14, Digimer ha scritto: On 03/21/2013 01:11 PM, Maurizio Giungato wrote: Hi guys, my goal is to create a reliable virtualization environment using CentOS 6.4 and KVM, I've three nodes and a clustered GFS2. The enviroment is up and working, but I'm worry for the reliability, if I turn the network interface down on one node to simulate a crash (for example on the node node6.blade): 1) GFS2 hangs (processes go in D state) until node6.blade get fenced 2) not only node6.blade get fenced, but also node5.blade! Help me to save my last neurons! Thanks Maurizio DLM, the distributed lock manager provided by the cluster, is designed to block when a known goes into an unknown state. It does not unblock until that node is confirmed to be fenced. This is by design. GFS2, rgmanager and clustered LVM all use DLM, so they will all block as well. As for why two nodes get fenced, you will need to share more about your configuration. My configuration is very simple I attached cluster.conf and hosts files. This is the row I added in /etc/fstab: /dev/mapper/KVM_IMAGES-VL_KVM_IMAGES /var/lib/libvirt/images gfs2 defaults,noatime,nodiratime 0 0 I set also fallback_to_local_locking = 0 in lvm.conf (but nothing change) PS: I had two virtualization enviroments working like a charm on OCFS2, but since Centos 6.x I'm not able to install it, there is same way to achieve the same results with GFS2 (with GFS2 sometime I've a crash after only a service network restart [I've many interfaces then this operation takes more than 10 seconds], with OCFS2 I've never had this problem. Thanks I attached my logs from /var/log/cluster/* The configuration itself seems ok, though I think you can safely take qdisk out to simplify things. That's neither here nor there though. This concerns me: Mar 21 19:00:14 fenced fence lama6.blade dev 0.0 agent fence_bladecenter result: error from agent Mar 21 19:00:14 fenced fence lama6.blade failed How are you triggering the failure(s)? The failed fence would certainly help explain the delays. As I mentioned earlier, DLM is designed to block when a node is in an unknowned state (failed but not yet successfully fenced). As an aside; I do my HA VMs using clustered LVM LVs as the backing storage behind the VMs. GFS2 is an excellent file system, but it is expensive. Putting your VMs directly on the LV takes them out of the equation I used 'service network stop' to simulate the failure, the node get fenced through fence_bladecenter (BladeCenter HW) Anyway, I took qdisk out and put GFS2 aside and now I've my VM on LVM LVs, I'm trying for many hours to reproduce the issue - only the node where I execute 'service network stop' get fenced - using fallback_to_local_locking = 0 in lvm.conf LVM LVs remain writable also while fencing take place All seems to work like a charm now. I'd like to understand what was happening. I'll try for same day before trusting it. Thank you so much. Maurizio Testing testing testing. It's good that you plan to test before trusting. I wish everyone had that philosophy! The clustered locking for LVM comes into play for activating/inactivating, creating, deleting, resizing and so on. It does not affect what happens in an LV. That's why an LV remains writeable when a fence is pending. However, I feel this is safe because rgmanager won't recover a VM on another node until the lost node is fenced. Cheers Thank you very much! The cluster continue working like a charm. Failure after failure I mean :) We are not using rgmanager fault management because doesn't have a check about the memory availability on the destination node, then we prefer to manage this situation with custom script we wrote. last questions: - have you any advice to improve the tollerance against network failures? - to avoid having a gfs2 only for VM's xml, I've thought to keep them on each node synced with rsync. Any alternatives? - If I want to have only the clustered LVM without no other functions, can you advice about a minimal configuration? (for example I think that rgmanager is not necessary) Thank you in advance For network redundancy, I use two switches and bonded (mode=1) links with one link going to either switch. This way, losing a NIC or a switch won't break the cluster. Details here: https://alteeve.ca/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Network Using rsync to keep the XML files in sync is fine, if you really don't want to use GFS2. You do not need rgmanager for clvmd to work. All you need is the base cluster.conf (and working fencing, as you've seen). If you are over-provisioning VMs and need
Re: [CentOS-virt] GFS2 hangs after one node going down
On 03/22/2013 11:21 AM, Maurizio Giungato wrote: Il 22/03/2013 00:34, Digimer ha scritto: On 03/21/2013 02:09 PM, Maurizio Giungato wrote: Il 21/03/2013 18:48, Maurizio Giungato ha scritto: Il 21/03/2013 18:14, Digimer ha scritto: On 03/21/2013 01:11 PM, Maurizio Giungato wrote: Hi guys, my goal is to create a reliable virtualization environment using CentOS 6.4 and KVM, I've three nodes and a clustered GFS2. The enviroment is up and working, but I'm worry for the reliability, if I turn the network interface down on one node to simulate a crash (for example on the node node6.blade): 1) GFS2 hangs (processes go in D state) until node6.blade get fenced 2) not only node6.blade get fenced, but also node5.blade! Help me to save my last neurons! Thanks Maurizio DLM, the distributed lock manager provided by the cluster, is designed to block when a known goes into an unknown state. It does not unblock until that node is confirmed to be fenced. This is by design. GFS2, rgmanager and clustered LVM all use DLM, so they will all block as well. As for why two nodes get fenced, you will need to share more about your configuration. My configuration is very simple I attached cluster.conf and hosts files. This is the row I added in /etc/fstab: /dev/mapper/KVM_IMAGES-VL_KVM_IMAGES /var/lib/libvirt/images gfs2 defaults,noatime,nodiratime 0 0 I set also fallback_to_local_locking = 0 in lvm.conf (but nothing change) PS: I had two virtualization enviroments working like a charm on OCFS2, but since Centos 6.x I'm not able to install it, there is same way to achieve the same results with GFS2 (with GFS2 sometime I've a crash after only a service network restart [I've many interfaces then this operation takes more than 10 seconds], with OCFS2 I've never had this problem. Thanks I attached my logs from /var/log/cluster/* The configuration itself seems ok, though I think you can safely take qdisk out to simplify things. That's neither here nor there though. This concerns me: Mar 21 19:00:14 fenced fence lama6.blade dev 0.0 agent fence_bladecenter result: error from agent Mar 21 19:00:14 fenced fence lama6.blade failed How are you triggering the failure(s)? The failed fence would certainly help explain the delays. As I mentioned earlier, DLM is designed to block when a node is in an unknowned state (failed but not yet successfully fenced). As an aside; I do my HA VMs using clustered LVM LVs as the backing storage behind the VMs. GFS2 is an excellent file system, but it is expensive. Putting your VMs directly on the LV takes them out of the equation I used 'service network stop' to simulate the failure, the node get fenced through fence_bladecenter (BladeCenter HW) Anyway, I took qdisk out and put GFS2 aside and now I've my VM on LVM LVs, I'm trying for many hours to reproduce the issue - only the node where I execute 'service network stop' get fenced - using fallback_to_local_locking = 0 in lvm.conf LVM LVs remain writable also while fencing take place All seems to work like a charm now. I'd like to understand what was happening. I'll try for same day before trusting it. Thank you so much. Maurizio Testing testing testing. It's good that you plan to test before trusting. I wish everyone had that philosophy! The clustered locking for LVM comes into play for activating/inactivating, creating, deleting, resizing and so on. It does not affect what happens in an LV. That's why an LV remains writeable when a fence is pending. However, I feel this is safe because rgmanager won't recover a VM on another node until the lost node is fenced. Cheers -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] GFS2 hangs after one node going down
On 03/21/2013 01:11 PM, Maurizio Giungato wrote: Hi guys, my goal is to create a reliable virtualization environment using CentOS 6.4 and KVM, I've three nodes and a clustered GFS2. The enviroment is up and working, but I'm worry for the reliability, if I turn the network interface down on one node to simulate a crash (for example on the node node6.blade): 1) GFS2 hangs (processes go in D state) until node6.blade get fenced 2) not only node6.blade get fenced, but also node5.blade! Help me to save my last neurons! Thanks Maurizio DLM, the distributed lock manager provided by the cluster, is designed to block when a known goes into an unknown state. It does not unblock until that node is confirmed to be fenced. This is by design. GFS2, rgmanager and clustered LVM all use DLM, so they will all block as well. As for why two nodes get fenced, you will need to share more about your configuration. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] GFS2 hangs after one node going down
On 03/21/2013 02:09 PM, Maurizio Giungato wrote: Il 21/03/2013 18:48, Maurizio Giungato ha scritto: Il 21/03/2013 18:14, Digimer ha scritto: On 03/21/2013 01:11 PM, Maurizio Giungato wrote: Hi guys, my goal is to create a reliable virtualization environment using CentOS 6.4 and KVM, I've three nodes and a clustered GFS2. The enviroment is up and working, but I'm worry for the reliability, if I turn the network interface down on one node to simulate a crash (for example on the node node6.blade): 1) GFS2 hangs (processes go in D state) until node6.blade get fenced 2) not only node6.blade get fenced, but also node5.blade! Help me to save my last neurons! Thanks Maurizio DLM, the distributed lock manager provided by the cluster, is designed to block when a known goes into an unknown state. It does not unblock until that node is confirmed to be fenced. This is by design. GFS2, rgmanager and clustered LVM all use DLM, so they will all block as well. As for why two nodes get fenced, you will need to share more about your configuration. My configuration is very simple I attached cluster.conf and hosts files. This is the row I added in /etc/fstab: /dev/mapper/KVM_IMAGES-VL_KVM_IMAGES /var/lib/libvirt/images gfs2 defaults,noatime,nodiratime 0 0 I set also fallback_to_local_locking = 0 in lvm.conf (but nothing change) PS: I had two virtualization enviroments working like a charm on OCFS2, but since Centos 6.x I'm not able to install it, there is same way to achieve the same results with GFS2 (with GFS2 sometime I've a crash after only a service network restart [I've many interfaces then this operation takes more than 10 seconds], with OCFS2 I've never had this problem. Thanks I attached my logs from /var/log/cluster/* The configuration itself seems ok, though I think you can safely take qdisk out to simplify things. That's neither here nor there though. This concerns me: Mar 21 19:00:14 fenced fence lama6.blade dev 0.0 agent fence_bladecenter result: error from agent Mar 21 19:00:14 fenced fence lama6.blade failed How are you triggering the failure(s)? The failed fence would certainly help explain the delays. As I mentioned earlier, DLM is designed to block when a node is in an unknowned state (failed but not yet successfully fenced). As an aside; I do my HA VMs using clustered LVM LVs as the backing storage behind the VMs. GFS2 is an excellent file system, but it is expensive. Putting your VMs directly on the LV takes them out of the equation. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] create a guest
On 01/16/2013 12:45 PM, mattias wrote: can i create a guest with libvirt and use an existing disc with freebsd? Yes. The 'virt-manager' application will let you use a disc in a drive as installation media. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] create a guest
Then you can use virt-install, but you will need to setup a VNC connection to the host in order to see the new VM and finish the install. If you're new to this, I'd strongly recommend installing the CentOS desktop and using virt-manager until you get used to it though. On 01/16/2013 12:49 PM, mattias wrote: i not use a desktop os windows on my desktop centos minimal on the server - Original Message - From: Digimer li...@alteeve.ca To: Discussion about the virtualization on CentOS centos-virt@centos.org Sent: Wednesday, January 16, 2013 6:48 PM Subject: Re: [CentOS-virt] create a guest On 01/16/2013 12:45 PM, mattias wrote: can i create a guest with libvirt and use an existing disc with freebsd? Yes. The 'virt-manager' application will let you use a disc in a drive as installation media. -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt -- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] Fencing question(s)
On 06/21/2012 01:01 PM, Steve Campbell wrote: I've got a Centos 5.8 box that I use for testing Xen VMs. I'm trying to setup some Centos 6.2 hosts in a high availability situation. I've tried the pacemaker route and couldn't get the connection established, so thought I'd try the RH-preferred methods. Mind you, all of these VMs are on the same host. Using Conga, it seems that I need some sort of fencing device, which doesn't really work in a VM environment. I'm guessing it's the same with command line configuration. Firstly, is it really possible to set up two VMs on the same machine to do HA? I can't see why not, but I'm not the best source of knowledge on any of this. Secondly, is there a useable fencing device that would work in a virtual environment for me to test with? I'd really like to avoid fencing altogether, even once in production, but every thing I read suggests against that. Any type of reply would be appreciated. steve campbell HA on the same node is kind of contradictory. However, yes, you can build (and fence) a cluster of VMs on the same host. Look at fence_xvm or fence_virsh. Can you use two servers? If so, you can build a truly HA cluster for your VMs; https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial -- Digimer Papers and Projects: https://alteeve.com ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] New Tutorial - RHCS + DRBD + KVM; 2-Node HA on EL6
On 01/04/2012 11:08 AM, Karanbir Singh wrote: hi, On 01/03/2012 02:54 PM, Digimer wrote: On 01/03/2012 09:43 AM, Tom Bishop wrote: This is sweet, I am in need for doing something for a SMB and nothing is out there that is affordable for small busineesses, will look into this. Feel free to ask if you have any questions. :) Just wondering if you really want to retain the 'stick with 6.0 endorsement' - KB Doh! I meant to remove that with 6.2's release... Fixed. Thank you for catching that! :) -- Digimer E-Mail: digi...@alteeve.com Freenode handle: digimer Papers and Projects: http://alteeve.com Node Assassin: http://nodeassassin.org omg my singularity battery is dead again. stupid hawking radiation. - epitron ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] New Tutorial - RHCS + DRBD + KVM; 2-Node HA on EL6
On 01/04/2012 04:06 PM, Fernando Martinez wrote: El 03/01/2012 11:29 a.m., Digimer escribió: I'm happy to announce a new tutorial! https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial Thanks for this excellent tutorial! It's a great job. I will test soon. Fernando M. Please let me know how it works out. :) -- Digimer E-Mail: digi...@alteeve.com Freenode handle: digimer Papers and Projects: http://alteeve.com Node Assassin: http://nodeassassin.org omg my singularity battery is dead again. stupid hawking radiation. - epitron ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
[CentOS-virt] New Tutorial - RHCS + DRBD + KVM; 2-Node HA on EL6
Hi all, I'm happy to announce a new tutorial! https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial This tutorial walks a user through the entire process of building a 2-Node cluster for making KVM virtual machines highly available. It uses Red Hat Cluster services v3 and DRBD 8.3.12. It is written such that you can use entirely free or fully Red Hat supported environments. Highlights; * Full network and power redundancy; no single-points of failure. * All off-the-shelf hardware; Storage via DRBD. * Starts with base OS install, no clustering experience required. * All software components explained. * Includes all testing steps covered. * Configuration is used in production environments! This tutorial is totally free (no ads, no registration) and released under the Creative Common 3.0 Share-Alike Non-Commercial license. Feedback is always appreciated! -- Digimer E-Mail: digi...@alteeve.com Freenode handle: digimer Papers and Projects: http://alteeve.com Node Assassin: http://nodeassassin.org omg my singularity battery is dead again. stupid hawking radiation. - epitron ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] New Tutorial - RHCS + DRBD + KVM; 2-Node HA on EL6
On 01/03/2012 09:43 AM, Tom Bishop wrote: This is sweet, I am in need for doing something for a SMB and nothing is out there that is affordable for small busineesses, will look into this. Feel free to ask if you have any questions. :) -- Digimer E-Mail: digi...@alteeve.com Freenode handle: digimer Papers and Projects: http://alteeve.com Node Assassin: http://nodeassassin.org omg my singularity battery is dead again. stupid hawking radiation. - epitron ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
Re: [CentOS-virt] New Tutorial - RHCS + DRBD + KVM; 2-Node HA on EL6
On 01/03/2012 10:20 AM, Clint Redwood wrote: Thanks! This is great - I've been planning and am half-way though creating such a cluster, but I've been using Fedora15/16 as Centos6 wasn't out when I started. Any idea if this will work with Fedora as a host OS, or does it have to be RHEL/Centos? It should work, more or less, as-is on Fedora. Do note though that things are changing rapidly and that Fedora is already at the end of the 3.1 version, about to go 3.2, where EL6 is (and will remain) on 3.0. Also, I can not recommend ever using Fedora in production as a server. The support cycle is far too short and the testing not nearly as extensive as EL6 proper. I've tested several times on Fedora, and inevitably run into gotchas. So in short; I *strongly* recommend using an EL6 distro. Cheers! -- Digimer E-Mail: digi...@alteeve.com Freenode handle: digimer Papers and Projects: http://alteeve.com Node Assassin: http://nodeassassin.org omg my singularity battery is dead again. stupid hawking radiation. - epitron ___ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
