[ovirt-users] Re: owner of vm paused/unpaused operation
On 10/10/19 10:44 AM, Gianluca Cecchi wrote: On Thu, Oct 10, 2019 at 9:56 AM Francesco Romani <mailto:from...@redhat.com>> wrote: The only way Vdsm will not pause the VM is if libvirt+qemu never reports any ioerror, which is something I'm not sure is possible and that I'd never recommend anyway. Vdsm always tries hard to be super-careful with respect possible data corruption. OK. In case of storage not accessible for a bunch of seconds is more a matter of I/O blocked than data corruption. True, but we can know only ex-poste that the storage was just temporarily unavailable, don't we? If no other host powers on the VM I think there is no risk of data corruption itself, or at least no more than when you have a physical server and for some reason the I/O operations to its physical disks (local or on a SAN) are blocked for some tens of seconds. IMO, a storage unresponsive for tens of seconds is something which should be uncommon and very alarming in every circumstances, especially for physical servers. What i'm trying to say is that yes, there probabily are ways to sidestep this behaviour, but I think this is the wrong direction and adds fragility rather than convenience to the system. The host could ever do a poweroff of the VM itself, instead of leaving control to the underlying libvirt+qemu I see that by default the qemu-kvm process in my oVirt 4.3.6 is spawned for every disk with the options: ...,werror=stop,rerror=stop,... Only for the ide channel of the CD device I have: ...,werror=report,rerror=report,readonly=on and the manual page for qemu-kvm tells: werror=action,rerror=action Specify which action to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), "report" (report the error to the guest), "enospc" (pause QEMU only if the host disk is full; report the error to the guest otherwise). The default setting is werror=enospc and rerror=report. So I think that if I want in any way to modify behavior I have to change the options so that I keep "report" for both write and read errors on virtual disks. Yep. I don't remember what Engine allows. Worst case you can use an hook, but once again this is making things a bit more fragile. I'm only experimenting to see possible different options to manage "temporary" problems at storage level, that often resolve without manual actions in tens of seconds, sometimes due to uncorrect operations at levels managed by other teams (network, storage, ecc). I think the best option is improve the current behaviour: learn why Vdsm fails to unpause the VM and improve here. -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UZ742LYQZMYQY7ZTE5ZGMYMLRS433A5T/
[ovirt-users] Re: owner of vm paused/unpaused operation
On 10/10/19 9:07 AM, Gianluca Cecchi wrote: > How is determined the timeout to use to put the VM in pause mode? The VM is paused immediately as soon as libvirt, through QEMU, reports IOError, to avoid data corruption. Now, when libvirt reports this error depends laregly on the timeout set for the storage configuration, which is done at host level, using system tools (e.g. it is not a Vdsm tunable) For test I have set this in multipath.conf of host: devices { device { all_devs yes # Set timeout of queuing of 5*28 = 140 seconds # similar to vSphere APD timeout # no_path_retry fail no_path_retry 28 polling_interval 5 } So it should wait at least 140 seconds before passing error to upper layer correct? AFAICT yes > Sometimes I see after clearing the problems that the VM is > automatically un-paused, sometimes no: how is this managed? I noticed that if I set disk as virtio-scsi (it seems virtio has no timeout definable and passes suddenly the error to upper layer) and disk timeout of vm disk (through udev rule) to 180 seconds, I can block access to the storage for example for 100 seconds and the host is able to reinstate paths and then vm is always unpaused. But I would like to prevent VM from pausing at all What else to tweak? The only way Vdsm will not pause the VM is if libvirt+qemu never reports any ioerror, which is something I'm not sure is possible and that I'd never recommend anyway. Vdsm always tries hard to be super-careful with respect possible data corruption. Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7QGJTJ5DSD5ZKDGV6KD7TRSRRNUJFCDO/
[ovirt-users] Re: owner of vm paused/unpaused operation
On 10/8/19 4:06 PM, Gianluca Cecchi wrote: Hi Gianluca Hello, I'm doing some tests related to storage latency or problems manually created to debug and manage reactions of hosts and VMs. What is the subsystem/process/daemon responsible to pause a VM when problems arise on storage for the host where the VM is running? It's Vdsm itself. How is determined the timeout to use to put the VM in pause mode? The VM is paused immediately as soon as libvirt, through QEMU, reports IOError, to avoid data corruption. Now, when libvirt reports this error depends laregly on the timeout set for the storage configuration, which is done at host level, using system tools (e.g. it is not a Vdsm tunable) Sometimes I see after clearing the problems that the VM is automatically un-paused, sometimes no: how is this managed? It depends on the error condition that happens. Vdsm tries to recovery automatically when it is safe to do so. When in doubt, Vdsm always plays it safe wrt user data Are there any counters so that if VM has been paused and and problems are not solved in a certain timeframe the unpause can be done only manually by the sysadmin? AFAIR no, because if Vdsm can't be sure, the only real option is to let the sysadmin check and decide. Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FX5PLYPI6BRELZETPZRO3FGMMJEPV2QL/
[ovirt-users] Re: Ovirt attach serial port on VM
On 9/27/18 7:08 AM, Alex K wrote: Hi Brad, On Wed, Sep 26, 2018, 22:25 <mailto:brad.riem...@cloud5.com>> wrote: Alex, Have you developed a solution for this? I find myself in a similar situation.. No, I have not. Although I understand what is needed (amend xml of vm with custom pre-start hooks) I need some python know-how. Hi, You may find this patch useful: https://gerrit.ovirt.org/#/c/91499/ it is mostly untested, but should be a good starting point. HTH, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/E7VLDOUFZ23WVHFKMGADCD7OMKJF6H6Y/
[ovirt-users] Re: Docker best practices
On 07/22/2018 11:01 AM, Barak Korren wrote: On 19 July 2018 at 17:41, Николаев Алексей mailto:alexeynikolaev.p...@yandex.ru>> wrote: Thx for all. 18.07.2018, 10:37, "Petr Kotas" mailto:pko...@redhat.com>>: Hi Alex, would you mind being more specific with your question? What are you trying to achieve? According to documentation https://ovirt.org/feature/container-support/ <https://ovirt.org/feature/container-support/> there are two different approaches to working with Docker: 1. run the container on hosts virtualization, along with the VM; 2. run containers inside VMS managed by oVirt. I would like to hear the pros and cons of each method. Which one is preferable to use in the oVirt infrastructure. Please don't take the following statement as being an official statement of any sort as I'm not personally involved with any of the ongoing development of the features mentioned above, but from my POV being involved in the oVirt CI/release process it seems that: As far as #1 goes, it seems to have never made it past an initial POC + a blog post. Yep. We made a PoC but it never gained traction. The code is still there, but it is pretty much unsupported and it is going to be removed in future releases. Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2LIOUNUCX5RBUH7AICNFJZGPZGANJUI5/
[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error
Thanks! On 07/18/2018 02:11 PM, Maton, Brett wrote: Sure no problem, mom log attached. It seems to me that MOM is restarting *very* often: 2018-07-18 06:07:04,909 - mom - INFO - MOM starting 2018-07-18 06:07:20,407 - mom - INFO - MOM starting 2018-07-18 06:07:30,658 - mom - INFO - MOM starting 2018-07-18 06:07:30,658 - mom - INFO - MOM starting And I totally not expect this. Martin, Andrej, is this the expected behaviour? Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/YOD45VBKT6BFGGABV4XUYEL6IQSJIMRL/
[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error
On 07/18/2018 07:24 AM, Maton, Brett wrote: Thanks Francesco, Log attached. Interestings, it seems the fault comes from MOM: 2018-07-18 06:07:22,836+0100 INFO (jsonrpc/3) [api.host] FINISH getStats error=:'NoneType' object has no attribute 'statistics'"> from=:::10.11.30.253,58840 (api:50) 2018-07-18 06:07:22,836+0100 ERROR (jsonrpc/3) [jsonrpc.JsonRpcServer] Internal server error (__init__:611) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request res = method(**params) File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, in _dynamicMethod result = fn(*methodArgs) File "", line 2, in getStats File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1402, in getStats sampling.host_samples.stats())} File "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 75, in get_stats ret.update(cif.mom.getKsmStats()) File "/usr/lib/python2.7/site-packages/vdsm/momIF.py", line 75, in getKsmStats stats = self._mom.getStatistics()['host'] File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__ return self.__send(self.__name, args) File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request verbose=self.__verbose File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request return self.single_request(host, handler, request_body, verbose) File "/usr/lib64/python2.7/xmlrpclib.py", line 1306, in single_request return self.parse_response(response) File "/usr/lib64/python2.7/xmlrpclib.py", line 1482, in parse_response return u.close() File "/usr/lib64/python2.7/xmlrpclib.py", line 794, in close raise Fault(**self._stack[0]) Fault: :'NoneType' object has no attribute 'statistics'"> 2018-07-18 06:07:22,836+0100 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call Host.getStats failed (error -32603) in 0.01 seconds (__init__:573) 2018-07-18 06:07:22,909+0100 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573) 2018-07-18 06:07:22,912+0100 INFO (jsonrpc/4) [api.virt] START getStats() from=::1,60058, vmId=a1af259b-7917-49ac-8bcc-eb4cc302e723 (api:46) 2018-07-18 06:07:22,912+0100 INFO (jsonrpc/4) [api] FINISH getStats error=Virtual machine does not exist: {'vmId': u'a1af259b-7917-49ac-8bcc-eb4cc302e723'} (api:127) 2018-07-18 06:07:22,912+0100 INFO (jsonrpc/4) [api.virt] FINISH getStats return={'status': {'message': "Virtual machine does not exist: {'vmId': u'a1af259b-7917-49ac-8bcc-eb4cc302e723'}", 'code': 1}} from=::1,60058, vmId=a1af259b-7917-49ac-8bcc-eb4cc302e723 (api:52) 2018-07-18 06:07:22,912+0100 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:573) 2018-07-18 06:07:23,043+0100 INFO (jsonrpc/6) [root] /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= (hooks:110) It could be a (benign) race on shutdown, but better continue the investigation. Could you please also chare the mom.log corresponding to the timeframe you already provided? Thanks, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZSEGPBX5D7CK2HSH3B5VM2Y35CTV4ZUN/
[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error
On 07/17/2018 07:30 AM, Maton, Brett wrote: I've got one physical host in a 3 host CentOS7.5 cluster that reports the following error several times a day VDSM node3.example.com <http://node3.example.com> command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '":\'NoneType\' object has no attribute \'statistics\'">'} Any ideas what the problem might be? Hi, I can make a wild guess[1], but let's try educated guesses first: could you please share the Vdsm logs around the time on which you see this error? Thanks, +++ [1] It seems to me that a NIC card was removed from a VM during a statistic reporting cycle, or somehow failed to report stats. But really, is a wild guess.Let's look at the logs first. -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/JSRLKAZSSPXGUBDE7CXRQDXZTQJBY5YY/
[ovirt-users] Re: After minor upgrade oVirt Node to 4.2.3 can't start Virtual machine
On 06/07/2018 11:55 AM, stefan.pale...@x112.cz wrote: Dear all, after upgrade oVirt Node to latest 4.2.3 I can't start any virtual machine with network interface. VM machine without network interface starts normally. Exactly the same situation is on freshly installed oVirt node machine with different hardware. I use openvswitch on both machines. After some searching I found out that there are the same version of openvswitch(2.9.0-3) on old and new version of oVirt node but different versions of libvirt(3.9.0-14 vs 3.2.0-14) Error message of libvirtd when the VM machine fail to start: VM OPNsense is down with error. Exit message: Unable to add bridge ovirtmgmt port vnet0: Operation not supported. Hi, could you please share the Vdsm and Engine logs? Thanks, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/URUT3SIKW5AP6LRNL5ANRAZMIRMXN2E7/
[ovirt-users] Re: Serial pass-through on oVirt VMs
On 05/22/2018 04:15 PM, Joshua Blake wrote: > > Hello Tomas, > > Thank you for coming back to me. I don't suppose you have any examples > or guides you could point me in the direction of please? > > Kind Regards, > Joshua Blake > You may want to start from there (warning: NOT tested on real hw or with real VMs) https://gerrit.ovirt.org/91499 bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org
[ovirt-users] Re: vGPU VM not starting
On 05/21/2018 02:42 PM, Callum Smith wrote: > Dear Ales, > > 4.2.3,5-1 > > Through extensive testing done with the help of Martin Polednik the > issues with the vGPU startup appear to be within the nvidia drivers, > so continuation of that issue is now going through nvidia. > > The issue with the nics MTU appears to have gone away with the upgrade > of host to the version above. Nice to know! If the MTU issue reappears, please file a bug -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org
[ovirt-users] Re: vGPU VM not starting
On 05/17/2018 12:01 AM, Callum Smith wrote: > Dear All, > > Our vGPU installation is progressing, though the VM is failing to start. > > 2018-05-16 22:57:34,328+0100 ERROR (vm/1bc9dae8) [virt.vm] > (vmId='1bc9dae8-a0ea-44b3-9103-5805100648d0') The vm start process > failed (vm:943) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 872, > in _startUnderlyingVm > self._run() > File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2872, > in _run > dom.createWithFlags(flags) > File > "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", > line 130, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", > line 92, in wrapper > return func(inst, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1099, in > createWithFlags > if ret == -1: raise libvirtError ('virDomainCreateWithFlags() > failed', dom=self) > libvirtError: Cannot get interface MTU on '': No such device This is another bug, related to https://bugzilla.redhat.com/show_bug.cgi?id=1561010. The proper fix is on Engine side, even though we can fix it on Vdsm side too Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org
[ovirt-users] Re: Unable to start VM after 4.2 upgrade
Hi again, On 05/18/2018 11:55 AM, Francesco Romani wrote: > Hi, > > > On 05/17/2018 02:58 PM, Ernest Beinrohr wrote: >> Hi, I updated my engine and 1 host to 4.2 from 4.1 and now I cannot >> start a VM and get this error: >> >> "UnsupportedType: Unsupported {} for ioTune". >> >> My other 6 4.1 hosts are able to start VM normally with the new 4.2 >> engine. > Please file a bug about this, make sure to report the error below. Also, please make sure to report the steps you did to get this error. Are you just starting a new VM using 4.2 Engine and 4.2 host? Or are you migrating an old VM created with Engine 4.1? Just for the sake of completeness (not sure it applies here), this flow is NOT supported: 1. have a VM happily run on 4.1 host 2. upgrade Vdsm on that host from 4.1 to 4.2 while the VM is running 3. restart Vdsm Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org
[ovirt-users] Re: Unable to start VM after 4.2 upgrade
Hi, On 05/17/2018 02:58 PM, Ernest Beinrohr wrote: > Hi, I updated my engine and 1 host to 4.2 from 4.1 and now I cannot > start a VM and get this error: > > "UnsupportedType: Unsupported {} for ioTune". > > My other 6 4.1 hosts are able to start VM normally with the new 4.2 > engine. Please file a bug about this, make sure to report the error below. > > > My VM has this inside: total_bytes_sec="0" total_iops_sec="0" write_bytes_sec="0" > write_iops_sec="100"/> Where is this exactly? bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org
Re: [ovirt-users] Unable to get volume size for domain
On 04/19/2018 09:44 AM, nico...@devels.es wrote: > Hi, > > We're running oVirt 4.1.9 and recently we've experienced a corruption > of a lot of VMs... I tried to investigate the reasons but I simply > can't find an explaination of why this happens. > > I'd really appreciate if someone could take a look at these logs and > shed some light on what happened and if this can be fixed somehow, > because these are really a lot of VMs affected... I'm attaching both > engine and SPM's vdsm logs. > > These storage domains are all iSCSI-based. This is pretty frequent on your vdsm logs and alarming: VolumeGroupSizeError: Volume Group not big enough: ('Not enough free extents for extending LV f3bb30a0-4745-4c77-aaca-e5f385905c1e/5c1ad23a-e418-4bb2-86b7-af4b14079224 (free=6, needed=8)',) The fix depends on how you set up your ISCSI storage -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Network and disk inactive after 4.2.1 upgrade
On 02/13/2018 04:05 PM, Chris Adams wrote: > I upgraded my dev cluster from 4.2.0 to 4.2.1 yesterday, and I noticed > that all my VMs show the network interfaces unplugged and disks inactive > (despite the VMs being up and running just fine). This includes the > hosted engine. > > I had not rebooted VMs after upgrading, so I tried powering one off and > on; it would not start until I manually activated the disk. > > I haven't seen a problem like this before (although it usually means > that I did something wrong :) ) - what should I look at? Hi, you may have hit this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1542117 it could affect all VMs started with Engine <= 4.1 and later imported in 4.2, or Hosted Engine. If your VM was first created under oVirt 4.2, please file a new bug. HTH, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Multiple 'scsi' controllers with index '0'.
Hi, could you please file a bug? Please attach the failing XML, you should find it pretty easily in the Vdsm logs. Thanks, On 02/09/2018 12:08 PM, spfma.t...@e.mail.fr wrote: > > Hi, > > I just wanted to increase the number of CPUs for a VM and after > validating, I got the following error when I try to start it: > > VM vm-test is down with error. Exit message: XML error: Multiple > 'scsi' controllers with index '0'. > > I am sure it is a bug, but for now, what can I do in order to remove > or edit conflicting devices definitions ? I need to be able to start > this machine. > > 4.2.0.2-1.el7.centos (as I still don't manage to update the hosted > engine to something newer) > > Regards > > > > FreeMail powered by mail.fr <https://mail.fr> > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] self hosted engine vm won't start
On 10/30/2017 03:56 PM, Hristo Pavlov wrote: > Thank you all! > > Francesco, > > [root@alpha ~]# journalctl -u libvirtd > -- Logs begin at Sun 2017-10-29 09:39:58 EET, end at Mon 2017-10-30 > 16:35:36 EET. -- > Oct 29 09:41:24 alpha.datamax.bg systemd[1]: Starting Virtualization > daemon... > Oct 29 09:41:26 alpha.datamax.bg systemd[1]: Started Virtualization > daemon. > Oct 29 22:30:26 alpha.datamax.bg libvirtd[4207]: libvirt version: > 2.0.0, package: 10.el7_3.9 (CentOS BuildSystem > <http://bugs.centos.org>, 2017-05-25-20:52:28, c1bm.rdu2.centos.org) > Oct 29 22:30:26 alpha.datamax.bg libvirtd[4207]: hostname: > alpha.datamax.bg > Oct 29 22:30:26 alpha.datamax.bg libvirtd[4207]: XML error: Attempted > double use of PCI slot :00:06.0 (may need "multifunction='on'" for > device on function 0) This error will prevent the VM to start. Now, we need to see why it happens in the first place. Will look at the vdsm logs as soon as possible. Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] self hosted engine vm won't start
Hi, anything in the journal about libvirt? (journalctl -u libvirtd) could you share a bigger chunk of the vdsm log, demonstrating the failed VM start? Bests, On 10/30/2017 03:28 PM, Hristo Pavlov wrote: > I tried it already, doesn't start on any of the nodes. > > > In a log /var/log/libvirt/qemu/HostedEngine.log at allthree nodes has > nothing to do with starting, as if it had not started. > > > Понедельник, 30 октября 2017, 16:11 +02:00 от Nathanaël Blanchet > <blanc...@abes.fr>: > > Hi, > > It happened to me too, after a live migration of it, I shut down > the hosted engine on the targeted host, and I couldn't restart it > anymore on this specific host. But I was able to start it on the > initial one, where I initially where I deployed the HE. It was > like the lease and the libvirt host definition staid on the first > host after migration. > > So try you may try hosted-engine --vm-start on one of other hosts... > > > Le 30/10/2017 à 12:28, Hristo Pavlov a écrit : >> Hi All, >> >> Our oVirt cluster is with 3 nodes with shared fibre channel >> storage, the engine virtual machine is self hosted. >> >> Hypervisors OS: CentOS Linux release 7.3 / x86_64, oVirt version >> is 4.1.2.2. The environment has been working for about a year >> without any problems. >> >> Aftershutdown of the hosted engine virtual machine, it doesn't >> start. >> >> Тhese commands that were executed: >> >> hosted-engine --set-maintenance --mode=global >> hosted-engine --vm-shutdown >> >> after the status of engine vm was down, we executed start. >> >> [root@alpha] hosted-engine --vm-start >> VM exists and is down, destroying it >> Exception in thread Client localhost:54321 (most likely raised >> during interpreter shutdown): >> >> we noticied that at vdsm.log >> >> 017-10-30 13:11:04,863+0200 INFO (jsonrpc/1) >> [jsonrpc.JsonRpcServer] RPC call StorageDomain.getStats succeeded >> in 0.26 seconds (__init__:533) >> 2017-10-30 13:11:05,802+0200 INFO (jsonrpc/6) >> [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in >> 0.01 seconds (__init__:533) >> 2017-10-30 13:11:05,825+0200 WARN (jsonrpc/2) [virt.vm] >> (vmId='da98112d-b9fb-4098-93fa-1f1374b41e46') Failed to get >> metadata, domain not connected. (vm:2765) >> 2017-10-30 13:11:05,825+0200 ERROR (jsonrpc/2) >> [jsonrpc.JsonRpcServer] Internal server error (__init__:570) >> Traceback (most recent call last): >> File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", >> line 565, in _handle_request >> res = method(**params) >> File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", >> line 202, in _dynamicMethod >> result = fn(*methodArgs) >> File "/usr/share/vdsm/API.py", line 1454, in getAllVmIoTunePolicies >> io_tune_policies_dict = self._cif.getAllVmIoTunePolicies() >> File "/usr/share/vdsm/clientIF.py", line 448, in >> getAllVmIoTunePolicies >> 'current_values': v.getIoTune()} >> File "/usr/share/vdsm/virt/vm.py", line 2803, in getIoTune >> result = self.getIoTuneResponse() >> File "/usr/share/vdsm/virt/vm.py", line 2816, in getIoTuneResponse >> res = self._dom.blockIoTune( >> File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", >> line 47, in __getattr__ >> % self.vmid) >> NotConnectedError: VM u'da98112d-b9fb-4098-93fa-1f1374b41e46' was >> not started yet or was shut down >> >> >> The storage of self hosted engine multipath, pvs, lvs, seems ok... >> >> At the moment of the three nodes there is a working about 100 >> virtual machines and we can't manage them. >> >> Does anyone have any ideas, what can be done то recover self >> hosted engine virtual machine? >> >> Thahk You! >> Have a nice day! >> >> >> >> >> >> >> >> >> >> >> >> >> >> ___ >> Users mailing list >> Users@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users > > -- > Nathanaël Blanchet > > Supervision réseau > Pôle Infrastrutures Informatiques > 227 avenue Professeur-Jean-Louis-Viala > 34193 MONTPELLIER CEDEX 5 > Tél. 33 (0)4 67 54 84 55 > Fax 33 (0)4 67 54 84 14 > blanc...@abes.fr > > > > > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Live migration error in 4.1.2
Hi, On 05/30/2017 02:25 PM, Vadim wrote: > Hi, > > Ovirt 4.1.1 clean install ugraded to 4.1.2 > > > I'm having trouble migrating VMs in a 2-node cluster. VM can run on both hosts > > I tried different migration policies but all of them ended by error. > > vdsm and qemu logs with post-copy policy of source and destination attached. > > #rpm- qa | grep -e libvirt -e qemu | sort > > centos-release-qemu-ev-1.0-1.el7.noarch > ipxe-roms-qemu-20160127-5.git6366fa7a.el7.noarch > libvirt-client-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-interface-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-network-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-qemu-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-secret-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-driver-storage-2.0.0-10.el7_3.9.x86_64 > libvirt-daemon-kvm-2.0.0-10.el7_3.9.x86_64 > libvirt-lock-sanlock-2.0.0-10.el7_3.9.x86_64 > libvirt-python-2.0.0-2.el7.x86_64 > qemu-guest-agent-2.5.0-3.el7.x86_64 > qemu-img-ev-2.6.0-28.el7_3.9.1.x86_64 > qemu-kvm-common-ev-2.6.0-28.el7_3.9.1.x86_64 > qemu-kvm-ev-2.6.0-28.el7_3.9.1.x86_64 > qemu-kvm-tools-ev-2.6.0-28.el7_3.9.1.x86_64 This: 2017-05-30T10:14:34.426783Z qemu-kvm: warning: All CPU(s) up to maxcpus should be described in NUMA config 2017-05-30 10:40:06.805+: initiating migration qemu-kvm: hw/display/qxl.c:2133: qxl_pre_save: Assertion `d->last_release_offset < d->vga.vram_size' failed. 2017-05-30 10:46:44.664+: shutting down is a QEMU issue. Please file a bug[1] against qemu +++ [1] As usual, it is advised to check on bugzilla first of the same issue was already reporte Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] migration of a VM: fails
On 05/18/2017 04:53 PM, cmc wrote: > I have to shut the VM down to stop it trying to repeatedly trying to > migrate the problematic host (always the same one). If I take it out > of maintenance, it will move the VMs back to balance (as per policy), > so this is rather inconvenient. It took leaving it overnight and > letting oVirt try repeatedly every few minutes to get it to migrate > the VM (I can't wait that long, so I've had to shut that VM down for > now) Hi, do you always have the same error? This: 2017-05-16T10:12:23.206731Z qemu-kvm: socket_writev_buffer: Got err=32 for (69776/18446744073709551615) is most likely found when QEMU is (failing to) transferring the VM state during the migration. >From https://bugzilla.redhat.com/show_bug.cgi?id=1355662 , however, we learn that this message could be a red herring - if we see it, doesn't mean something's wrong. >From oVirt perspective, it seems all good. We need to investigate the lower layers: libvirt, qemu. Let's start. Are you by any chance using the post copy migration mode? Could you please share the libvirt debug logs, at least on the source side? https://wiki.libvirt.org/page/DebugLogs You may want to do a test run with the debug logs turned on and disable them just after, those are VERY verbose. Thanks, > > On Wed, May 17, 2017 at 4:13 PM, cmc <iuco...@gmail.com> wrote: >> Just a note on this: a similar thing is now happening with the same VM >> when I upgrade the other node, i.e., it can't move this one VM over >> (so far) from one host to another. I will leave it trying overnight to >> see if it succeeds. >> >> Thanks, >> >> Cam >> >> On Wed, May 17, 2017 at 11:40 AM, cmc <iuco...@gmail.com> wrote: >>> Hi Francesco, >>> >>> I left it running after I posted to the list, and it eventually (after >>> many failed attempts) moved the VM without any intervention by me, and >>> then updated the host, so that explains the differences in the >>> versions of qemu between the hosts (they probably would have been the >>> same when I tried the move first). The xml is attached. >>> >>> qemu and libvirt versions on the source host: >>> >>> ipxe-roms-qemu-20160127-5.git6366fa7a.el7.noarch >>> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.5.x86_64 >>> qemu-img-ev-2.6.0-28.el7_3.9.1.x86_64 >>> qemu-kvm-common-ev-2.6.0-28.el7_3.9.1.x86_64 >>> qemu-kvm-ev-2.6.0-28.el7_3.9.1.x86_64 >>> qemu-kvm-tools-ev-2.6.0-28.el7_3.3.1.x86_64 >>> >>> libvirt-2.0.0-10.el7_3.5.x86_64 >>> libvirt-client-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-config-network-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-interface-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-lxc-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-network-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-secret-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-storage-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-kvm-2.0.0-10.el7_3.5.x86_64 >>> libvirt-lock-sanlock-2.0.0-10.el7_3.5.x86_64 >>> libvirt-python-2.0.0-2.el7.x86_64 >>> >>> qemu and libvirt versions on the dest host: >>> >>> ipxe-roms-qemu-20160127-5.git6366fa7a.el7.noarch >>> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.4.x86_64 >>> qemu-img-ev-2.6.0-28.el7_3.3.1.x86_64 >>> qemu-kvm-common-ev-2.6.0-28.el7_3.3.1.x86_64 >>> qemu-kvm-ev-2.6.0-28.el7_3.3.1.x86_64 >>> qemu-kvm-tools-ev-2.6.0-28.el7_3.3.1.x86_64 >>> >>> libvirt-client-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-interface-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-network-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-secret-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-driver-storage-2.0.0-10.el7_3.4.x86_64 >>> libvirt-daemon-kvm-2.0.0-10.el7_3.4.x86_64 >>> libvirt-lock-sanlock-2.0.0-10.el7_3.4.x86_64 >>> libvirt-python-2.0.0-2.el7.x86_64 >>> >>> >>> Thanks, >>> >>> Cam >>> >&
Re: [ovirt-users] migration of a VM: fails
On 05/16/2017 01:06 PM, cmc wrote: > Hi, > > Just trying to place in maintenance mode for a version upgrade, and > one VM fails to migrate. The other 20-odd move over successfully. In > /var/log/libvirt/qemu/, the VM's log on the source reports: > > 2017-05-16 09:48:06.339+: initiating migration > 2017-05-16T09:52:25.498932Z qemu-kvm: socket_writev_buffer: Got err=32 > for (131328/18446744073709551615) > 2017-05-16 09:52:47.311+: initiating migration > 2017-05-16T09:57:06.755402Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69776/18446744073709551615) > 2017-05-16 09:57:55.109+: initiating migration > 2017-05-16T10:02:14.143221Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69776/18446744073709551615) > 2017-05-16 10:02:59.497+: initiating migration > 2017-05-16T10:07:18.542872Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69776/18446744073709551615) > 2017-05-16 10:08:03.896+: initiating migration > 2017-05-16T10:12:23.206731Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69776/18446744073709551615) > 2017-05-16 10:13:08.941+: initiating migration > 2017-05-16T10:17:27.861843Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69776/18446744073709551615) > 2017-05-16 10:18:13.690+: initiating migration > 2017-05-16T10:22:32.929689Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69803/18446744073709551615) > 2017-05-16 10:23:19.846+: initiating migration > 2017-05-16T10:27:39.175724Z qemu-kvm: socket_writev_buffer: Got err=32 > for (69776/18446744073709551615) > 2017-05-16 10:28:25.141+: initiating migration > 2017-05-16T10:28:35.620070Z qemu-kvm: socket_writev_buffer: Got err=32 > for (65753/18446744073709551615) > 2017-05-16 10:29:10.678+: initiating migration > 2017-05-16T10:33:29.718527Z qemu-kvm: socket_writev_buffer: Got err=32 > for (53477/18446744073709551615) > 2017-05-16 10:38:35.517+: initiating migration > Hi, it seems either qemu issue or misconfiguration. To investigate, we need more data; so could you please share: 1. the domain XML (virtsh -r dumpxml ...) and/or the qemu command line of the affected VM, on the source side 2. the version of QEMU and libvirt that you are running Thanks and bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Ovirt 4.0 to 4.1 CentOS7.3 and libvirtd 2.0.0 segfault issue
On 04/18/2017 08:09 AM, Rafał Wojciechowski wrote: > > hello, > > I made comparison(+diff) between xml passing through vdsm which is > working and another one which cause libvirtd segfault > > https://paste.fedoraproject.org/paste/eqpe8Byu2l-3SRdXc6LTLl5M1UNdIGYhyRLivL9gydE= > > > I am not sure if below setting are fine but I dont know how to change them > > > (I dont have so much ram and vgamem) > those are kibibytes though (https://libvirt.org/formatdomain.html#elementsVideo), are pretty conservarvative settings > > passwdValidTo="1970-01-01T00:00:01" port="-1" tlsPort="-1" type="spice"> > (ports with "-"? maybe it is fine because of autoport settings...) > Yes, "-1" means "autoallocation from libvirt". I don't see obvious issues in this XML, and, most importantly, one invalid XML should never cause libvirtd to segfault. I'd file a libvirt bug. -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirtvm-console : Failed to execute login on behalf - for user
Hi, On 03/02/2017 12:21 PM, Eduardo Mayoral wrote: > > Hi, > > I am getting exactly the same issue here with 4.1 , when trying to > log in to the serial console over SSH. > > > The user with domain is "emayoral_...@arsyslan.es" (please note > mailman may translate the "at" character to a textual "_at_"). The > First name and last name as read from active directory is "Eduardo > Mayoral" (with no quotes) > > The password is: 08.HJYqoce,nrW (OK, this is not the real password, > but it has the same special characters and approximate structure and > length) > > This is the engine.log output. > > 2017-03-02 11:13:31,917Z INFO > [org.ovirt.engine.core.bll.aaa.LoginOnBehalfCommand] (default task-25) > [5d9b7d18] Running command: LoginOnBehalfCommand internal: true. > 2017-03-02 11:13:31,938Z ERROR > [org.ovirt.engine.core.sso.utils.SsoUtils] (default task-33) [] > OAuthException server_error: java.text.ParseException: Invalid > character ' ' encountered. > 2017-03-02 11:13:31,939Z ERROR > [org.ovirt.engine.core.bll.aaa.LoginOnBehalfCommand] (default task-25) > [5d9b7d18] Unable to create engine session: EngineException: user > emayoral_...@arsyslan.es in domain 'arsyslan.es-authz (Failed with > error PRINCIPAL_NOT_FOUND and code 5200) > 2017-03-02 11:13:31,945Z ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-25) [5d9b7d18] EVENT_ID: > USER_LOGIN_ON_BEHALF_FAILED(1,402), Correlation ID: 5d9b7d18, Call > Stack: null, Custom Event ID: -1, Message: Failed to execute login on > behalf - for user emayoral_...@arsyslan.es. > 2017-03-02 11:13:31,945Z ERROR > [org.ovirt.engine.core.services.VMConsoleProxyServlet] (default > task-25) [5d9b7d18] Error processing request: : > java.lang.RuntimeException: Unable to create session using LoginOnBehalf > This smells like one engine internal bug. Please make sure to file one bugzilla entry. Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] problems with serial console in 4.1.1
On 03/28/2017 11:35 AM, Gianluca Cecchi wrote: > Hello, Ciao Gianluca :) > trying serial console in 4.1.1 I have some problems. > I have not understood if it depends on my client, or the guest or oVirt... > > My client is Fedora 25 updated > My guest is a CentOS 7.3 VM updated (my latest successful attempts > were when in 7.2) > My oVirt is 4.1.1 (my latest successful attempts were when in 4.0.x) > > Some tests > > $ ssh -i .ssh/id_dsa.pub ovirt-vmconsole@ovmgr1 -p > Available Serial Consoles: > 00 c7service[2e571c77-bae1-4c1c-bf98-effaf9fed741] > 01 c7testovn1[63818176-cfb5-4a00-912d-ce0e812be76c] > 02 c7testovn2[f00c8f30-a658-4a18-bf85-3b523de93183] > 03 dbatest6[633b58b6-5e99-491b-a312-0976d4e8d979] > 04 nas4free[6b9013d1-1837-4b0f-aa34-c9130d4f1229] > 05 raclab1[1775d180-288d-4116-af1e-d90b2989cb8a] > 06 raclab2[852aa8b1-a3e2-4658-be91-0ab51b6b7260] > SELECT> 00 > > and it stays here without giving the os prompt. Sometimes it happen that the *getty in the guest is actually ready, but you just don't get the prompt. To check that, the easiest way is to just hit enter a few times to see if there is any echo/prompt showing up. > > In 7.3 guest I have VirtIO serial console enabled and this situation > > [root@c7service ~]# cat /proc/cmdline > BOOT_IMAGE=/vmlinuz-3.10.0-514.6.1.el7.x86_64 root=/dev/mapper/cl-root > ro rd.lvm.lv <http://rd.lvm.lv>=cl/root rd.lvm.lv > <http://rd.lvm.lv>=cl/swap rhgb quiet console=ttyS0 > [root@c7service ~]# > > [root@c7service ~]# ps -ef|grep agetty > root 718 1 0 11:15 ttyS000:00:00 /sbin/agetty > --keep-baud 115200 38400 9600 ttyS0 vt220 > root 1760 1541 0 11:29 pts/000:00:00 grep --color=auto agetty > [root@c7service ~]# > > So it seems ok. Yes, looks good > I have not understood the error I get connecting this way: > > $ ssh -i .ssh/id_dsa.pub ovirt-vmconsole@ovmgr1 -p connect > --vm-name c7service > ERROR: No pty support, please enable at client side > $ The error message is a bit cryptic. It means you haven't given the "-t" ssh options. Please retry with: ssh -t -i .ssh/id_dsa.pub ovirt-vmconsole@ovmgr1 -p connect --vm-name c7service > What is "client" in the error message, my Fedora 25..? Yep, it is referring to the ssh client. -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Details about why a live migration failed
On 03/24/2017 04:01 PM, Davide Ferrari wrote: > Source: CentOS 7.2 - qemu-kvm-ev-2.3.0-31.el7.16.1 > Dest: CentOS 7.3 - qemu-kvm-ev-2.6.0-28.el7_3.3.1 > > To be fair I'm trying to migrate away that VM so I can install updates > on the source host. Another, hopefully less likely, case is a QEMu bug. If QEMU cannot correctly restore the VM state from the migration data stream, you'll get the same error. Anyway, this definitely looks like libvirt/qemu issue; better be prepared to file a libvirt bug. does it happen migrating to *any* CentOS7.3 box? Bests, -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Details about why a live migration failed
On 03/24/2017 11:58 AM, Davide Ferrari wrote: > And this is the vdsm log from vmhost04: > > Thread-6320717::INFO::2017-03-24 > 11:41:13,019::migration::712::virt.vm::(monitor_migration) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::Migration Progress: 190 > seconds elapsed, 98% of data processed, total data: 16456MB, processed > data: 9842MB, remaining data: 386MB, transfer speed 52MBps, zero > pages: 1718676MB, compressed: 0MB, dirty rate: -1, memory iteration: -1 > libvirtEventLoop::DEBUG::2017-03-24 > 11:41:21,007::vm::4291::virt.vm::(onLibvirtLifecycleEvent) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::event Suspended detail 0 > opaque None > libvirtEventLoop::INFO::2017-03-24 > 11:41:21,025::vm::4815::virt.vm::(_logGuestCpuStatus) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::CPU stopped: onSuspend > libvirtEventLoop::DEBUG::2017-03-24 > 11:41:21,069::vm::4291::virt.vm::(onLibvirtLifecycleEvent) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::event Resumed detail 0 > opaque None > libvirtEventLoop::INFO::2017-03-24 > 11:41:21,069::vm::4815::virt.vm::(_logGuestCpuStatus) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::CPU running: onResume > Thread-6320715::DEBUG::2017-03-24 > 11:41:21,224::migration::715::virt.vm::(stop) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::stopping migration > monitor thread > Thread-6320715::ERROR::2017-03-24 > 11:41:21,225::migration::252::virt.vm::(_recover) > vmId=`4f627cc1-9b52-4eef-bf3a-c02e8a6303b8`::operation failed: > migration job: unexpectedly failed This is surprising (no pun intended) With a pretty high chance this comes from libvirt, I'm afraid you need to dig in the libvirt logs/journal entries to learn more. Vdsm could unfortunately do better than what it is already doing :\ -- Francesco Romani Senior SW Eng., Virtualization R Red Hat IRC: fromani github: @fromanirh ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] How to enable tunnel migration
Hi again, please also share the version of the software stack you are using besides oVirt (e.g. Operating System, libvirt, qemu...) On 03/07/2017 04:34 AM, like...@cs2c.com.cn wrote: > Hi, > > I'm using oVirt 3.6.7 now. I want to enable tunnel migration feature. > But when i set tunnel_migration value of vm as Ture through python > SDK(I can't find any place to enable tunnel migration on the web ui, > so i use SDK), then migrate the vm i got the following error message: > > Thread-9417628::ERROR::2017-03-07 > 11:25:48,538::migration::310::virt.vm::(run) > vmId=`08b4ccc1-c508-439b-a9ba-ca0d42a7072c`::Failed to migrate > Traceback (most recent call last): > File "/usr/share/vdsm/virt/migration.py", line 298, in run > self._startUnderlyingMigration(time.time()) > File "/usr/share/vdsm/virt/migration.py", line 364, in > _startUnderlyingMigration > self._perform_migration(duri, muri) > File "/usr/share/vdsm/virt/migration.py", line 403, in _perform_migration > self._vm._dom.migrateToURI3(duri, params, flags) > File "/usr/share/vdsm/virt/virdomain.py", line 68, in f > ret = attr(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line > 124, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper > return func(inst, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1836, in > migrateToURI3 > if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', > dom=self) > libvirtError: 不支持的参数:migration URI is not supported by tunnelled migration > > So, i want to know how should i enable the tunnel migration feature. > Can anyone help me? > Thanks > > Regards. > > ---- > like...@cs2c.com.cn > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- Francesco Romani Red Hat Engineering Virtualization R & D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] How to enable tunnel migration
Hi, This smells like a bug. Please file a bug entry for this. On 03/07/2017 04:34 AM, like...@cs2c.com.cn wrote: > Hi, > > I'm using oVirt 3.6.7 now. I want to enable tunnel migration feature. > But when i set tunnel_migration value of vm as Ture through python > SDK(I can't find any place to enable tunnel migration on the web ui, > so i use SDK), then migrate the vm i got the following error message: > > Thread-9417628::ERROR::2017-03-07 > 11:25:48,538::migration::310::virt.vm::(run) > vmId=`08b4ccc1-c508-439b-a9ba-ca0d42a7072c`::Failed to migrate > Traceback (most recent call last): > File "/usr/share/vdsm/virt/migration.py", line 298, in run > self._startUnderlyingMigration(time.time()) > File "/usr/share/vdsm/virt/migration.py", line 364, in > _startUnderlyingMigration > self._perform_migration(duri, muri) > File "/usr/share/vdsm/virt/migration.py", line 403, in _perform_migration > self._vm._dom.migrateToURI3(duri, params, flags) > File "/usr/share/vdsm/virt/virdomain.py", line 68, in f > ret = attr(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line > 124, in wrapper > ret = f(*args, **kwargs) > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper > return func(inst, *args, **kwargs) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1836, in > migrateToURI3 > if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', > dom=self) > libvirtError: 不支持的参数:migration URI is not supported by tunnelled migration > > So, i want to know how should i enable the tunnel migration feature. > Can anyone help me? > Thanks > > Regards. > > ---- > like...@cs2c.com.cn > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- Francesco Romani Red Hat Engineering Virtualization R & D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] migration failures
On 02/23/2017 04:20 PM, Michael Watters wrote: > I have an ovirt cluster running ovirt 4.0 and I am seeing several errors > when I attempt to put one of our nodes into maintenance mode. The logs > on the source server show errors as follows. > > Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: operation > aborted: migration job: canceled by client > Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: internal > error: qemu unexpectedly closed the monitor: 2017-02-23T15:12:58.289459Z > qemu-kvm: warning: CPU(s) not present in any NUMA nodes: 2 3 4 5 6 7 8 9 10 > 11 12 13 14 15 > > > 2017-02-23T15:12:58.289684Z qemu-kvm: warning: All CPU(s) up to maxcpus > should be described in NUMA config > > > > 2017-02-23T15:15:07.889891Z qemu-kvm: Unknown combination of migration > flags: 0 > > 2017-02-23T15:15:07.890821Z qemu-kvm: error while loading state section id > 2(ram) > > 2017-02-23T15:15:07.892357Z qemu-kvm: load of migration failed: Invalid > argument > > This cluster does *not* have NUMA enabled so I am not sure why this > error is happening. It's one implementation detail. NUMA is enabled transparently because it is required for memory hotplug support. It should be fully transparent. > Some migrations did succeed after being restarted > on a different host however I have two VMs that appear to be stuck. Is > there a way to resolve this? The load/save state errors are most often found when the two sides of the migration have different and incompatible version of QEMU. In turn, this is quite often a bug, because forward migrations (e.g. from 2.3.0 to 2.4.0) are always supported, for obvious upgrade needs. So, which version of libvirt and qemu do you have on the sides of failing migration paths? Bests, -- Francesco Romani Red Hat Engineering Virtualization R & D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] missing hvc0 device on hosted engine
On 02/06/2017 08:14 PM, Berger, Sandy wrote: > > I’ve been running oVirt 4.0.x for a while now and upgraded the engine > and hosts to 4.1. Sometimes, on both 4.0 and 4.1, trying to run the > “hosted-engine –console” command gives the following results: > > > > The engine VM is running on this host > > Connected to domain HostedEngine > > Escape character is ^] > > error: internal error: cannot find character device > > > > Once this starts occurring, it never works again. > > > > When this happens I note that the hvc0 device does not exist in /dev > and therefore there is no getty process running for that device. I > notice that the “Enable VirtIO serial console” box is not checked on > the hosted engine VM but when I check it and click OK it’s unchecked > again when I open up the edit window again. > > > > How do I get console control back from my oVirt hosts? > Hi, it is strongly recommended to use ttyS0 (legacy serial driver emulation) and not hvc0 (newer virtio serial driver) because we had reports of kernel panics () using the new driver. We are going to re-evaluate this setting for 4.2.0, and, perhaps, later 4.1.z, but for 4.1.0 this is still valid. In terms of Vm configuration, the console device should have the "consoleType" = "serial" setting. This is the default Engine sends. Not sure what HE is doing here, not if you fix it manually. -- Francesco Romani Red Hat Engineering Virtualization R & D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] [oVirt 4.1.0] experimental container support
Hi all, In case you missed it, oVirt 4.1.0 gained some support to run container side to side to VMs. Read the full story here: https://fromanirh.github.io/containers-in-ovirt.html I'll be happy to answer any question - make sure to CC me to have a faster response time! Bests, -- Francesco Romani Red Hat Engineering Virtualization R & D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Ovirt Hypervisor vdsm.Scheduler logs fill partition
- Original Message - > From: "Simone Tiraboschi" <stira...@redhat.com> > To: "Steve Dainard" <sdain...@spd1.com>, "Francesco Romani" > <from...@redhat.com> > Cc: "users" <users@ovirt.org> > Sent: Friday, October 14, 2016 9:59:49 AM > Subject: Re: [ovirt-users] Ovirt Hypervisor vdsm.Scheduler logs fill partition > > On Fri, Oct 14, 2016 at 1:12 AM, Steve Dainard <sdain...@spd1.com> wrote: > > > Hello, > > > > I had a hypervisor semi-crash this week, 4 of ~10 VM's continued to run, > > but the others were killed off somehow and all VM's running on this host > > had '?' status in the ovirt UI. > > > > This appears to have been caused by vdsm logs filling up disk space on the > > logging partition. > > > > I've attached the log file vdsm.log.27.xz which shows this error: > > > > vdsm.Scheduler::DEBUG::2016-10-11 > > 16:42:09,318::executor::216::Executor::(_discard) > > Worker discarded: > action= > 'virt.periodic.DriveWatermarkMonitor'> > > at 0x7f8e90021210> at 0x7f8e90021250> discarded at 0x7f8dd123e850> > > > > which happens more and more frequently throughout the log. > > > > It was a bit difficult to understand what caused the failure, but the logs > > were getting really large, then being xz'd which compressed 11G+ into a few > > MB. Once this happened the disk space would be freed, and nagios wouldn't > > hit the 3rd check to throw a warning, until pretty much right at the crash. > > > > I was able to restart vdsmd to resolve the issue, but I still need to know > > why these logs started to stack up so I can avoid this issue in the future. > > > > We had this one: https://bugzilla.redhat.com/show_bug.cgi?id=1383259 > but in your case the logs are rotating. > Francesco? Hi, yes, it is a different issue. Here the log messages are caused by the Worker threads of the periodic subsystem, which are leaking[1]. This was a bug in Vdsm (insufficient protection against rogue domains), but the real problem is that some of your domain are being unresponsive at hypervisor level. The most likely cause is in turn unresponsive storages. Fixes are been committed and shipped with Vdsm 4.17.34. See: ttps://bugzilla.redhat.com/1364925 HTH, +++ [1] actually, they are replaced too quickly, leading to unbound growth. So those aren't actually "leaking", Vdsm is just overzealous handling one error condition, making things worse than before. Still serious issue, no doubt, but quite different cause. -- Francesco Romani Red Hat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] [vdsm] status update: running containers alongside VMs
edVMProperties='volumeMap=^[a-zA-Z_-]+:[a-zA-Z_-]+$;containerImage=^[a-zA-Z]+(://|)[a-zA-Z]+$;containerType=^(docker|rkt)$' --cver=4.0 It is worth stressing that while the variables are container-specific, the VM custom properties are totally inuntrusive and old concept in oVirt, so this step is totally safe. Now restart Engine to let it use the new variables: # systemctl restart ovirt-engine The next step is actually configure one "container VM" and run it. ## 4. Create the container "VM" To finally run a container, you start creating a VM much like you always did, with few changes 1. most of the hardware-related configuration isn't relevant for container "VMs", besides cpu share and memory limits; this will be better documented in the future; unneeded configuration will just be ignored 2. You need to set some custom properties for your container "VM". Those are actually needed to enable the container flow, and they are documented in the next section. You *need* to set at least `containerType` and `containerImage`. ### 4.2. Custom variables for container support The container support needs some custom properties to be properly configured: 1. `containerImage` (*needed* to enable the container system). Just select the target image you want to run. You can use the standard syntax of the container runtimes. 2. `containerType` (*needed* to enable the container system). Selects the container runtime you want to use. All the available options are always showed. Please note that unavailable container options are not yet grayed out. If you *do not* have rkt support on your host, you still can select it, but it won't work. 3. `volumeMap` key:value like. You can map one "VM" disk (key) to one container volume (value), to have persistent storage. Only file-based storage is supported. Example configuration: `containerImage = redis` `containerType = docker` `volumeMap = vda:data` (this may not be needed, and the volume label is just for illustrative purposes) ### 4.2. A little bit of extra work: preload the images on the Vdsm host This step is not needed by the flow, and will be handled by oVirt in the future. The issue is how the container image are handled. They are stored by the container management system (rkt, docker) on each host, and they are not pre-downloaded. To shorten the duration of the first boot, you are advised to pre-download the image(s) you want to run. For example ## on the Vdsm host you want to use with containers # docker pull redis ## 5. Run the container "VM" You are now all set to run your "VM" using oVirt Engine, just like any existing VM. Some actions doesn't make sense for a container "VM", like live migration. Engine won't stop you to try to do those actions, but they will fail gracefully using the standard errors. ## 6. Next steps What to expect from this project in the future? For the integration with Vdsm, we want to fix the existing known issues, most notably: * add proper monitoring/reporting of the container health * ensure proper integration of the container image store with oVirt storage management * streamline the network configuration What is explicitely excluded yet is any Engine change. This is a Vdsm-only change at the moment, so fixing the following is currently unplanned: * First and foremost, Engine will not distinguish between real VMs and container VMs. Actions unavailable to container will not be hidden from UI. Same for monitoring and configuration data, which will be ignored. * Engine is NOT aware of the volumes one container can use. You must inspect and do the mapping manually. * Engine is NOT aware of the available container runtimes. You must select it carefully Proper integration with Engine may be added in the future once this feature exits from the experimental/provisional stage. Thanks for reading, make sure to share your thoughts on the oVirt mailing lists! +++ [1] we keep calling it that way _only_ internally, because it's a short name we are used to. After the merge/once we release it, we will use a different name, like "vdsm-containers" or something like it. -- Francesco Romani Red Hat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] how to create vm from snapshot via ovirt-shell (ovirt 4.0)
- Original Message - > From: "csyeow" <csy...@gmail.com> > To: users@ovirt.org > Cc: "csyeow ( gmail )" <csy...@gmail.com> > Sent: Monday, September 19, 2016 1:11:42 PM > Subject: [ovirt-users] how to create vm from snapshot via ovirt-shell (ovirt > 4.0) > Hi ovirt expertise > i am new to ovirt, i now start install and play around. > i unable find any doc from web site to create vm from snapshot via > ovirt-shell (ovirt 4.0) , any help on this. > i mange to create vm from snapshot via webui > i would like to perform backup vm via api by using ovirt-shell. i have > totally no knowledge on python > ideal as below: > create snapshot > #add snapshot --parent-vm-name "rhel6.8_server02" --description 'my snapshot' > create vm from snapshot > ## missing, need help Hi, you mean you want to start a VM with the snapshot attached? Perhaps this page could help https://www.ovirt.org/develop/release-management/features/storage/backup-restore-api-integration/ Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Can't add logical network
- Original Message - > From: "Maxence Sartiaux" <cont...@makz.me> > To: "users" <users@ovirt.org> > Sent: Monday, September 19, 2016 11:53:12 AM > Subject: [ovirt-users] Can't add logical network > Hello, > On a clean install ovirt 4.0.3-1, i've configured my bond on my hypervisor, > created a new network on the engine and now i try to assign the logical > network to my bond in the host network settings. > When i save the configuration, the engine throw me an error and the following > event > VDSM hypervisor1 command failed: [Errno 2] No such file or directory: > u'/etc/sysconfig/network-scripts/ifcfg-lan' > if i "touch" the file manually i don't have this error but my file only > contain "MTU=1500" nothing more. > Engine bug or i miss something ? How you configured the bond on your hypervisor? Seems like Vdsm doesn't like your changes, perhaps it cannot just identify the interface configuration Edward, do you have any recommendations about this flow? -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] intel skylake oVirt 4.0
- Original Message - > From: "Roman Nikolayevich Drovalev" <drova...@kaluga-gov.ru> > To: Users@ovirt.org > Sent: Thursday, September 8, 2016 7:04:37 AM > Subject: [ovirt-users] intel skylake oVirt 4.0 > > Hello all! > > Currently, the oVirt 4.0 not on the list of supported processors for the > cluster "Intel Skylake" ! > Is it possible to add hosts to the new processor in the cluster pool or you > must wait for the update oVirt ? Hi, the proper support requires one update of the low level stack, like libvirt and qemu, so the best way is to wait for the updated packages to come; unfortunately, this may require a long cycle, since those packages are most often provided by your distribution. HTH, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVirt updates available on host due to libvirt missing
- Original Message - > From: "Sandro Bonazzola" <sbona...@redhat.com> > To: "Gianluca Cecchi" <gianluca.cec...@gmail.com>, "Michal Skrivanek" > <mskri...@redhat.com>, "Francesco Romani" > <from...@redhat.com> > Cc: "users" <users@ovirt.org> > Sent: Friday, August 12, 2016 2:01:54 PM > Subject: Re: [ovirt-users] oVirt updates available on host due to libvirt > missing > > On Fri, Aug 12, 2016 at 12:25 PM, Gianluca Cecchi <gianluca.cec...@gmail.com > > wrote: > > > Hello, > > My self hosted engine environment was born in 3.6.0 in November with > > CentOS 7 on host and CentOS 7 appliance. > > I managed several updates applying 3.6.2, then 3.6.5, then 4.0. > > Now I'm at 4.0.2 final and in web admin gui I see a message regarding > > updates available on host (hosted_engine_1) that doesn't go away. > > The host usually was updated through "yum update" and not from the gui > > during updates described above. > > > > In events pane it seems the problem is related with libvirt package > > > > Host hosted_engine_1 has available updates: libvirt. > > > > Actually at this moment the libvirt package (that seems actually a sort of > > meta-package) is not installed and I think has been never here. Situation > > is: > > > > [root@ractor log]# rpm -qa|grep libvirt > > libvirt-daemon-driver-network-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-driver-interface-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-kvm-1.2.17-13.el7_2.5.x86_64 > > libvirt-client-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-1.2.17-13.el7_2.5.x86_64 > > libvirt-lock-sanlock-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-driver-storage-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-driver-qemu-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.5.x86_64 > > libvirt-python-1.2.17-2.el7.x86_64 > > libvirt-daemon-config-nwfilter-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.5.x86_64 > > libvirt-daemon-driver-secret-1.2.17-13.el7_2.5.x86_64 > > [root@ractor log]# > > > > If I run "yum install libvirt" I'm proposed: > > > > Dependencies Resolved > > > > > > > > Package Arch Version > > Repository Size > > > > > > Installing: > > libvirt x86_641.2.17-13.el7_2.5 > > updates119 k > > Installing for dependencies: > > libvirt-daemon-config-networkx86_641.2.17-13.el7_2.5 > > updates120 k > > libvirt-daemon-driver-lxcx86_641.2.17-13.el7_2.5 > > updates747 k > > > > Transaction Summary > > > > ======== > > Install 1 Package (+2 Dependent packages) > > > > How to proceed? [...] > I t should be safe to install the dependencies. Adding Michal and Francesco > to confirm. Not sure why it is proposed just now and not before. The -lxc package is not used by oVirt, so it will just sit there. The libvirt-daemon-config-network package should be harmless, it just contains the configuration fir libvirt's default bridge 'virbr0' Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] messages file filled by vdsm logs in 4.0
- Original Message - > From: "Gianluca Cecchi" <gianluca.cec...@gmail.com> > To: "users" <users@ovirt.org> > Sent: Tuesday, July 5, 2016 11:16:54 AM > Subject: [ovirt-users] messages file filled by vdsm logs in 4.0 > Hello, > since updating to 4.0 yesterday, my /var/log/messages on hypervisor seem > filled by vdsm messages Ciao Gianluca, please try the following: 1. edit /etc/vdsm/vdsm.conf 2. in the [devel] section, set "python_warnings_enable=false" it should read like [devel] python_warnings_enable = false 3. changes will take place after a Vdsm restart HTH, Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Is it possible to disable qxl video?
- Original Message - > From: "Arman Khalatyan" <arm2...@gmail.com> > To: "users" <users@ovirt.org> > Sent: Monday, July 4, 2016 11:52:24 AM > Subject: [ovirt-users] Is it possible to disable qxl video? > Hi, > I am doing PCI Passthrough for GPUs. > Is it possible somehow to disable/remove default video qxl? > thanks, > Arman,. Hi, The 4.0 Vdsm allows you to run a headless VM, meaning a VM without graphic device/frontend. Not sure Engine allows you that however. I'm afraid you will need to write a Vdsm hook. There could be simpler solutions depending on the Engine Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] disk not bootable
- Original Message - > From: "Nir Soffer" <nsof...@redhat.com> > To: "Fernando Fuentes" <ffuen...@darktcp.net> > Cc: "Francesco Romani" <from...@redhat.com>, "users" <users@ovirt.org> > Sent: Saturday, July 2, 2016 11:18:01 AM > Subject: Re: [ovirt-users] disk not bootable > > On Sat, Jul 2, 2016 at 1:33 AM, Fernando Fuentes <ffuen...@darktcp.net> > wrote: > > Nir, > > > > Ok I ran another test and this one I moved from NFS domain to iSCSI and > > stop working than I moved it back and still unable to run... Windows VM > > is saying "no available boot disk" > > VM: Win7-Test > > Host: Zeta > > Info as requested: http://pastebin.com/1fSi3auz > > We need a working xml to compare to. [snip expected changes] > oVirt > oVirt Node > 6-5.el6.centos.11.2 > - C938F077-55E2-3E50-A694-9FCB7661FD89 > + 735C7A01-1F16-3CF0-AF8C-A99823E95AC0 > > Not expected - maybe this is confusing windows? > > Francesco, why vm serial has changed after moving disks from one storage > domain > to another? We put in serial either 1. the UUID Engine send to us 2. the host UUID as returned by our getHostUUID utility function the latter is unlikely to change, even after this disk move. So the first suspect in line is Engine Arik, do you know if Engine is indeed supposed to change the UUID in this flow? That seems very surprising. Thanks and bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Problem with Getty and Serial Consoles
- Original Message - > From: "Christophe TREFOIS" <christophe.tref...@uni.lu> > To: "users" <users@ovirt.org> > Sent: Monday, June 13, 2016 5:05:28 PM > Subject: [ovirt-users] Problem with Getty and Serial Consoles > > Dear all, > > I am running 3.6.6 and am able to select a Console, however, the screen is > black. > > On the hypervisor, I try to start getty service manually, and end up with > following error in journal > > Jun 13 17:01:01 elephant-server.lcsb.uni.lu systemd[1]: Stopping > user-0.slice. > -- Subject: Unit user-0.slice has begun shutting down > -- Defined-By: systemd > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel > -- > -- Unit user-0.slice has begun shutting down. > Jun 13 17:01:37 elephant-server.lcsb.uni.lu systemd[1]: Job > dev-hvc0.device/start timed out. > Jun 13 17:01:37 elephant-server.lcsb.uni.lu systemd[1]: Timed out waiting for > device dev-hvc0.device. > -- Subject: Unit dev-hvc0.device has failed > -- Defined-By: systemd > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel > -- > -- Unit dev-hvc0.device has failed. > -- > -- The result is timeout. > Jun 13 17:01:37 elephant-server.lcsb.uni.lu systemd[1]: Dependency failed for > Serial Getty on hvc0. > -- Subject: Unit serial-getty@hvc0.service has failed > -- Defined-By: systemd > -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel > -- > -- Unit serial-getty@hvc0.service has failed. > > I am running CentOS 7.2. > > Does anybody have some pointers on what could be the issue here? Hi! Make sure you added the right parameter to the kernel command line, as per """ TL;DR: To make use of a serial console, just use console=ttyS0 on the kernel command line, and systemd will automatically start a getty on it for you. """ nice full tutorial on http://0pointer.de/blog/projects/serial-console.html Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ssh -t serial connection on Widows
- Original Message - > From: "Nathanaël Blanchet" <blanc...@abes.fr> > To: "Francesco Romani" <from...@redhat.com> > Cc: users@ovirt.org > Sent: Friday, April 22, 2016 1:34:59 PM > Subject: Re: [ovirt-users] ssh -t serial connection on Widows > > Thank you for your answer. > > Le 22/04/2016 12:39, Francesco Romani a écrit : > > - Original Message - > >> From: "Nathanaël Blanchet" <blanc...@abes.fr> > >> To: users@ovirt.org > >> Sent: Friday, April 22, 2016 11:26:42 AM > >> Subject: [ovirt-users] ssh -t serial connection on Widows > >> > >> Concerning ovirt-serial connection : > >> > >> ssh -t -p ovirt-vmconsole@engine > >> > >> Waiting for the closed officiel bash implementation on W10, does it > >> exist a way to do the same with putty or other ssh client? > > Hi, > > > > I believe the only real issue here is the -t option equivalent. > > Regarding putty, IIUC the -t behaviour is the default: > > > > https://the.earth.li/~sgtatham/putty/0.60/htmldoc/Chapter4.html#config-ssh-pty > > > >> As the spice/vnc console, could it be a good idea to get the same on the > >> UI so as to be OS independant? > > Sorry, I don't understand this question. Could you please rephrase it? > Sure, what about introducing a new button next to the console one, to > benefit of the serial console into the webadmin? One of the key ideas for this feature was to avoid the need to log into the webadmin to access the VMs, so I think this new button would not be very helpful :\ Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ssh -t serial connection on Widows
- Original Message - > From: "Nathanaël Blanchet" <blanc...@abes.fr> > To: users@ovirt.org > Sent: Friday, April 22, 2016 11:26:42 AM > Subject: [ovirt-users] ssh -t serial connection on Widows > > Concerning ovirt-serial connection : > > ssh -t -p ovirt-vmconsole@engine > > Waiting for the closed officiel bash implementation on W10, does it > exist a way to do the same with putty or other ssh client? Hi, I believe the only real issue here is the -t option equivalent. Regarding putty, IIUC the -t behaviour is the default: https://the.earth.li/~sgtatham/putty/0.60/htmldoc/Chapter4.html#config-ssh-pty > As the spice/vnc console, could it be a good idea to get the same on the > UI so as to be OS independant? Sorry, I don't understand this question. Could you please rephrase it? Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] payload device serial
Thanks for the report! Could you please file a Vdsm bug? Bests, - Original Message - > From: "Pavel Gashev" <p...@acronis.com> > To: users@ovirt.org > Sent: Wednesday, March 30, 2016 4:39:14 PM > Subject: [ovirt-users] payload device serial > Hello, > I hit a bug, and just want to share a solution. > VM with a payload (Initial run) do not start with libvirt >= 1.3.2. VDSM log > says: "libvirtError: unsupported configuration: Disks 'hdc' and 'hdd' have > identical serial". > Yes, both cdrom devices have the same serial. Empty serial: > > file="/var/run/vdsm/payload/2eaf9c8e-2123-4b48-9b62-f96168ac7a36.41f22cb1676858ad4e22e8440519032d.img" > startupPolicy="optional"/> > > > > > > > > > > > I don't know where is the issue. Either libvirt should work with empty > serials, or VDSM should generate serial at least for payload device. > Related bug - https://bugzilla.redhat.com/show_bug.cgi?id=1245013 > Quick fix is to install a VDSM hook to > /usr/libexec/vdsm/hooks/before_vm_start: > -- cut here -- > #!/usr/bin/python > import hooking > import uuid > domxml = hooking.read_domxml() > for disk in domxml.getElementsByTagName('disk'): > if disk.getAttribute('device') == 'cdrom': > for source in disk.getElementsByTagName('source'): > if source.getAttribute('file').find('/payload/') > 0: > for serial in disk.getElementsByTagName('serial'): > if not serial.hasChildNodes(): > serial.appendChild(domxml.createTextNode(str(uuid.uuid4( > hooking.write_domxml(domxml) > ------ cut here -- > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Serial Console not running on ovirt-engine
- Original Message - > From: "Christophe TREFOIS" <christophe.tref...@uni.lu> > To: "users" <users@ovirt.org> > Cc: "lcsb-sysadmins" <lcsb-sysadm...@uni.lu> > Sent: Wednesday, March 30, 2016 6:31:48 PM > Subject: [ovirt-users] Serial Console not running on ovirt-engine > Dear all, > So am running 3.6.3.4-1 on CentOS 7.2, and did a engine-setup. > I checked now, and see that ovirt-vmconsole-proxy was in fact installed > (before, during or after, I don’t know). > [root@bio2-engine-server ~]# yum install ovirt-vmconsole-proxy > Loaded plugins: fastestmirror, versionlock > Loading mirror speeds from cached hostfile > * base: centos.mirror.root.lu > * epel: mirror.imt-systems.com > * extras: centos.mirror.root.lu > * ovirt-3.5: ftp.nluug.nl > * ovirt-3.5-epel: mirror.imt-systems.com > * ovirt-3.6: ftp.nluug.nl > * ovirt-3.6-epel: mirror.imt-systems.com > * updates: centos.mirror.root.lu > Package ovirt-vmconsole-proxy-1.0.0-1.el7.centos.noarch already installed and > latest version > Nothing to do > But I don’t see any process running on port . > I couldn’t find in the docs, how to start the listener. It is a regular systemd unit, so # systemctl status ovirt-vmconsole-proxy-sshd and # systemctl start ovirt-vmconsole-proxy-sshd If you want it at every boot (most likely yes) # systemctl enable ovirt-vmconsole-proxy-sshd -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] seria consol setup
- Original Message - > From: "Yedidyah Bar David" <d...@redhat.com> > To: "Christophe TREFOIS" <christophe.tref...@uni.lu>, "Francesco Romani" > <from...@redhat.com> > Cc: "Fabrice Bacchella" <fabrice.bacche...@orange.fr>, "users" > <users@ovirt.org> > Sent: Thursday, March 31, 2016 8:00:04 AM > Subject: Re: [ovirt-users] seria consol setup > > On Wed, Mar 30, 2016 at 7:28 PM, Christophe TREFOIS > <christophe.tref...@uni.lu> wrote: > > Hi, > > > > I have a question on this. > > > > Can there be multiple SSH keys in that box in the GUI? > > > > For instance, we might have 2 keys for our “Admin” account? > > Not sure, Francesco? Yes, you can paste multiple new-line separated public keys in the same box. -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] How to connect a VM with a port = -1
- Original Message - > From: "zhangjian" <zhangjian2...@cn.fujitsu.com> > To: users@ovirt.org > Sent: Friday, March 25, 2016 5:13:28 AM > Subject: [ovirt-users] How to connect a VM with a port = -1 > Hi guys, > I created a VM in ovirt, and I found it a port = -1, How can I connect to it > using like remote-viewer. > -- > console.vv > [virt-viewer] > type=spice > host=XXX.XXX.XXX.XXX > port=-1 > password=J4xu1swd59A5 > # Password is valid for 120 seconds. > delete-this-file=1 > fullscreen=0 > title=test:%d > toggle-fullscreen=shift+f11 > ... > ... > ... > -- > I usually use the following command to connect to my VM when it has a > positive value。 > remote-viewer spice://XXX.XXX.XXX.XXX:590X Hi, -1 is what Engine sends to trigger auto allocation of spice ports from libvirt on the virtualization Host. Once the VM is booted, Vdsm should report the port allocated by libvirt to Engine. Does this happen consistently or just sporadically? Which version of packages are you using? Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] seria consol setup
- Original Message - > From: "Fabrice Bacchella" <fabrice.bacche...@orange.fr> > To: "Francesco Romani" <from...@redhat.com> > Cc: "Yedidyah Bar David" <d...@redhat.com>, "users" <users@ovirt.org> > Sent: Wednesday, March 23, 2016 4:29:15 PM > Subject: Re: [ovirt-users] seria consol setup > > I'm trying, my configuration is still incomplete, I added in my httpd.conf: > > > ServerName XXX > DocumentRoot htdocs > > RedirectMatch ^/$ /ovirt-engine/ > > SSLEngine on > SSLCertificateFile /etc/pki/ovirt-engine/certs/apache.cer > SSLCertificateKeyFile /etc/pki/ovirt-engine/keys/apache.key.nopass > SSLCACertificateFile /etc/pki/ovirt-engine/apache-ca.pem > > RequestHeader unset Expect early > > > ^/(ovirt-engine($|/)|api($|/)|RHEVManagerWeb/|OvirtEngineWeb/|ca.crt$|engine.ssh.key.txt$|rhevm.ssh.key.txt$)> > ProxyPassMatch ajp://127.0.0.1:8702 timeout=3600 retry=5 > > AddOutputFilterByType DEFLATE text/javascript text/css text/html > text/xml text/json application/xml application/json > application/x-yaml > > > > > and in /etc/ovirt-engine/ovirt-vmconsole-proxy-helper.conf.d/99-my.conf > ENGINE_BASE_URL=https://localhost:1443/ovirt-engine/ > > but no progress : > > su - ovirt-vmconsole -c '/usr/libexec/ovirt-vmconsole-proxy-keys --debug > list' > ERROR: Internal error > > --debug don't provide any help You should find them in the journal/system logger; otherwise it is a {different,new} bug. Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] seria consol setup
- Original Message - > From: "Fabrice Bacchella" <fabrice.bacche...@orange.fr> > To: "Francesco Romani" <from...@redhat.com> > Cc: "Yedidyah Bar David" <d...@redhat.com>, "users" <users@ovirt.org> > Sent: Wednesday, March 23, 2016 1:21:11 PM > Subject: Re: [ovirt-users] seria consol setup > > > > Le 23 mars 2016 à 12:32, Francesco Romani <from...@redhat.com> a écrit : > > > > - Original Message - > >> From: "Yedidyah Bar David" <d...@redhat.com> > >> To: "Fabrice Bacchella" <fabrice.bacche...@orange.fr>, "Francesco Romani" > >> <from...@redhat.com> > >> Cc: "users" <users@ovirt.org> > >> Sent: Wednesday, March 23, 2016 12:28:52 PM > >> Subject: Re: [ovirt-users] seria consol setup > > > >>> I can always use puppet to modify just this line, it will be fine for me. > >>> > >>> The point 4 in Automatic Setup is not very helpfull: > >>> " • once the setup succesfully run, and once ovirt-engine is > >>> running, > >>> you can log in and register a SSH key. (TODO: add picture)" > >>> > >>> what does it mean ? > > > > It just means that you need to add SSH public keys for the users which want > > to use > > the serial console. > > > > E.g. log in user portal > > in the top right corner there is the $user drop down menu, click on it > > select "options" > > paste public key here > > > > HTH, > > It tried that, I didn't work. What didn't work? Adding the keys or -AFAIK- the full authentication? > By digging in log and configuration, I think > it's because I have an Apache server in front of ovirt-engine, using a > specific SSO authentication module (using CAS), so the certificate-base > authentication is failing, if my comprehension is good. So you should add a > few line about that in the documentation. Will improve in this regard > Should I make the proxy helper > talks directly to tomcat by playing with ENGINE_BASE_URL in > /etc/ovirt-engine/ovirt-vmconsole-proxy-helper.conf.d ? Yes, the proxy helper is supposed to talk directly with the Engine. > There is also a small glitch in the documentation: > su - ovirt-vmconsole -c 'ovirt-vmconsole-proxy-keys list' > but it should be: > su - ovirt-vmconsole -c '/usr/libexec/ovirt-vmconsole-proxy-keys list' Thanks, will fix. Bests, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] seria consol setup
- Original Message - > From: "Yedidyah Bar David" <d...@redhat.com> > To: "Fabrice Bacchella" <fabrice.bacche...@orange.fr>, "Francesco Romani" > <from...@redhat.com> > Cc: "users" <users@ovirt.org> > Sent: Wednesday, March 23, 2016 12:28:52 PM > Subject: Re: [ovirt-users] seria consol setup > > I can always use puppet to modify just this line, it will be fine for me. > > > > The point 4 in Automatic Setup is not very helpfull: > > " • once the setup succesfully run, and once ovirt-engine is running, > > you can log in and register a SSH key. (TODO: add picture)" > > > > what does it mean ? It just means that you need to add SSH public keys for the users which want to use the serial console. E.g. log in user portal in the top right corner there is the $user drop down menu, click on it select "options" paste public key here HTH, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Logs Error
- Original Message - > From: "Francesco Romani" <from...@redhat.com> > To: "Budur Nagaraju" <nbud...@gmail.com> > Cc: "users" <users@ovirt.org> > Sent: Monday, February 8, 2016 11:43:49 AM > Subject: Re: [ovirt-users] Logs Error > > - Original Message - > > From: "Budur Nagaraju" <nbud...@gmail.com> > > To: "users" <users@ovirt.org> > > Sent: Monday, February 8, 2016 10:00:36 AM > > Subject: [ovirt-users] Logs Error > > > > > > > > Getting below logs for some reason, in the engine > > Hi! > Can please file a bug about it? Hi again, sorry for the quite long delay; meantime I managed to file a bug, https://bugzilla.redhat.com/show_bug.cgi?id=1307048 and work on a candidate patch. https://gerrit.ovirt.org/#/c/53341/ But I can't reproduce this on 3.6.5 the way I thought, it seems something's missing. Could you please add some details to help me reproduce this issue? Thanks! -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Going crazy with emory hotplug on 3.6
- Original Message - > From: "Markus Stockhausen" <stockhau...@collogia.de> > To: "users" <users@ovirt.org> > Sent: Sunday, February 21, 2016 1:19:10 PM > Subject: [ovirt-users] Going crazy with emory hotplug on 3.6 > > Hi there, Hi Markus, [...] > To make a long story short: I manually compiled OVirt/Centos qemu 2.3 and > enabled > the old Fedora machine type pc-1.0. Can you please elaborate on this? Which package and version of qemu were you running? We recommend to use the -ev variant (e.g. qemu-kvm-ev). The -ev variants provide in turn the *rhel* machine types (qemu -machine ?), and they have much better compatibility guarantees and an upgrade path. If you run the pc-* machines, you can easily run into troubles like this. > Question: What is the desired way to disable the hotplug qemu parametrization > for single VMs in 3.6 cluster? It MUST be possible otherwise it would not > make sense to choos older machine types. Good point, but this should be true for rhel* machine types, and I didn't expect you had pc-* machine in the first place! Could you please share the package list and versions? Qemu, libvirt, vdsm, ovirt-engine. Please be prepared to file a oVirt bug, on Engine, because Vdsm is just the humble servant here, doing exactly what Engine orders. -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Logs Error
- Original Message - > From: "Budur Nagaraju" <nbud...@gmail.com> > To: "users" <users@ovirt.org> > Sent: Monday, February 8, 2016 10:00:36 AM > Subject: [ovirt-users] Logs Error > > > > Getting below logs for some reason, in the engine Hi! Can please file a bug about it? Thanks, -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] console ports
Hi Budur, this has not changed since last release. The range is [5900-6923], extremes included. HTH, - Original Message - > From: "Budur Nagaraju" <nbud...@gmail.com> > To: "users" <users@ovirt.org> > Sent: Friday, November 20, 2015 5:58:19 AM > Subject: [ovirt-users] console ports > > HI > > can someone pls update the default vm console ports which needs to be open in > firewall ? > > Regards, > Nagaraju > > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Ovirt 3.6 Beta testing hosted-engine : no VM engine on Webui
- Original Message - > From: "Simone Tiraboschi" <stira...@redhat.com> > To: "wodel youchi" <wodel.you...@gmail.com> > Cc: "users" <users@ovirt.org>, "Roy Golan" <rgo...@redhat.com>, "Francesco > Romani" <from...@redhat.com> > Sent: Monday, September 7, 2015 4:55:41 PM > Subject: Re: [ovirt-users] Ovirt 3.6 Beta testing hosted-engine : no VM > engine on Webui > > > Another thing > > > > the engine-setup fails when ovirt-vmconsole is configured, whith this error > > [ INFO ] Restarting ovirt-vmconsole proxy service > > *[ ERROR ] Failed to execute stage 'Closing up': Failed to stop service > > 'ovirt-vmconsole-proxy-sshd'* > > [ INFO ] Stage: Clean up > > Log file is located at > > /var/log/ovirt-engine/setup/ovirt-engine-setup-20150907143356-3a4lap.log Please share this (or any equivalent) log to learn why the service failed to stop > > [ INFO ] Generating answer file > > '/var/lib/ovirt-engine/setup/answers/20150907145116-setup.conf' > > [ INFO ] Stage: Pre-termination > > [ INFO ] Stage: Termination > > [ ERROR ] Execution of setup failed > > > > > > [root@engine ~]# *dnf whatprovides */ovirt-vmconsole-proxy-sshd* > > Last metadata expiration check performed 0:44:12 ago on Mon Sep 7 > > 14:34:22 2015. > > ovirt-vmconsole-proxy-1.0.0-0.0.master.20150821105438.gite14b2f0.fc22.noarch > > : oVirt VM console proxy components > > Dépôt : ovirt-3.6-snapshot > > > > ovirt-vmconsole-proxy-1.0.0-0.0.master.20150616120457.gitc1fb2bd.fc22.noarch > > : oVirt VM console proxy components > > Dépôt : ovirt-3.6 > > > > > > *ovirt-vmconsole-proxy-1.0.0-0.0.master.20150821105438.gite14b2f0.fc22.noarch > > : oVirt VM console proxy componentsDépôt : ovirt-3.6* > > > > ovirt-vmconsole-proxy-1.0.0-0.0.master.20150730112609.git3126bf1.fc22.noarch > > : oVirt VM console proxy components > > Dépôt : ovirt-3.6 > > > > I am using the latest package available 2015 08 21 > > > > but: > > > > [root@engine ~]# rpm -ql > > ovirt-vmconsole-1.0.0-0.0.master.20150821105438.gite14b2f0.fc22.noarch/etc/ovirt-vmconsole > > /etc/pki/ovirt-vmconsole > > /usr/lib/python2.7/site-packages/ovirt_vmconsole > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/__init__.py > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/__init__.pyc > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/__init__.pyo > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/__init__.py > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/__init__.pyc > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/__init__.pyo > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/base.py > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/base.pyc > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/base.pyo > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/config.py > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/config.pyc > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/config.pyo > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/utils.py > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/utils.pyc > > /usr/lib/python2.7/site-packages/ovirt_vmconsole/common/utils.pyo > > /usr/share/doc/ovirt-vmconsole > > /usr/share/doc/ovirt-vmconsole/README > > /usr/share/doc/ovirt-vmconsole/README.API > > /usr/share/ovirt-vmconsole > > /usr/share/ovirt-vmconsole/empty > > /usr/share/selinux/packages/ovirt-vmconsole > > /usr/share/selinux/packages/ovirt-vmconsole/ovirt_vmconsole.pp > > > > > > As you can see there is no *ovirt-vmconsole-proxy-sshd *file setup is referring to ovirt-vmconsole-proxy, more specifically to the unit file: $ rpm -ql ovirt-vmconsole-proxy | grep service /usr/lib/systemd/system/ovirt-vmconsole-proxy-sshd.service -- Francesco Romani RedHat Engineering Virtualization R & D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] messages from journal in centos7 hosts
Hi, - Original Message - From: Nicolas Ecarnot nico...@ecarnot.net To: users@ovirt.org Sent: Wednesday, May 13, 2015 9:30:22 AM Subject: Re: [ovirt-users] messages from journal in centos7 hosts Hello list, Coming from : https://www.mail-archive.com/users@ovirt.org/msg24878.html I'm also being disturb by this ennoyance. Is there a workaround? (oVirt 3.5.1, CentOS 7 hosts, iSCSI storage) This message should have been removed in recent VDSM versions. Which version of VDSM and libvirt are you running? Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirt 3.5.2 cannot start windows vms
- Original Message - From: Michal Skrivanek mskri...@redhat.com To: Francesco Romani from...@redhat.com Cc: Wolfgang Bucher wolfgang.buc...@netland-mn.de, users@ovirt.org (users@ovirt.org) users@ovirt.org Sent: Tuesday, May 5, 2015 9:33:03 AM Subject: Re: [ovirt-users] ovirt 3.5.2 cannot start windows vms here are the logfiles from libvirt May 4 19:08:22 ovirt kernel: ovirtmgmt: port 2(vnet0) entered forwarding state May 4 19:08:22 ovirt kernel: ovirtmgmt: port 2(vnet0) entered forwarding state May 4 19:08:23 ovirt sanlock[638]: 2015-05-04 19:08:23+0200 5077 [638]: cmd 9 target pid 10182 not found May 4 19:08:23 ovirt systemd: Starting Virtual Machine qemu-testwin. May 4 19:08:23 ovirt systemd-machined: New machine qemu-testwin. May 4 19:08:23 ovirt systemd: Started Virtual Machine qemu-testwin. May 4 19:08:23 ovirt kvm: 1 guest now active May 4 19:08:23 ovirt kernel: ovirtmgmt: port 2(vnet0) entered disabled state May 4 19:08:23 ovirt kernel: device vnet0 left promiscuous mode May 4 19:08:23 ovirt kernel: ovirtmgmt: port 2(vnet0) entered disabled state May 4 19:08:23 ovirt kvm: 0 guests now active May 4 19:08:23 ovirt systemd-machined: Machine qemu-testwin terminated. May 4 19:08:23 ovirt libvirtd: 9183: error : qemuMonitorOpenUnix:309 : Verbindung mit Monitor-Socket gescheitert: Kein passender Prozess gefunden May 4 19:08:23 ovirt libvirtd: 9183: error : qemuProcessWaitForMonitor:2131 : Interner Fehler: Prozess während der Verbindungsaufnahme zum Monitor beendet This is the translation of the well-known 'internal error: monitor disconbected' QEMU error (not literal transaltion)! which is in turn caused by... :2015-05-04T17:08:23.274206Z qemu-kvm: -drive file=/var/run/vdsm/payload/c07772b8-6369-44cf-b554-b8dcb0e0e09b.0a41ac3e81bce0429e32b725fbf3ba5d.img,if=none,id=drive-fdc0-0-0,format=raw,serial=: could not open disk image /var/run/vdsm/payload/c07772b8-6369-44cf-b554-b8dcb0e0e09b.0a41ac3e81bce0429e32b725fbf3ba5d.img: Could not open file: Permission denied ... this error, and so the root cause seems indeed very much the same of https://bugzilla.redhat.com/show_bug.cgi?id=1213410#c7 because of this: May 4 19:08:23 ovirt libvirtd: 9183: warning : virSecuritySELinuxRestoreSecurityFileLabel:1034 : cannot lookup default selinux label for /rhev/data-center/d5e8a32f-35ed-4dec-bf9d-3c818c2780a4/66f8876c-0898-4ff2-9325-a14835f2a872/images/b329d34e-78b3-46a5-9df8-00b83c2c982a/c79a596b-5701-4afd-a5b5-d37cf412095c From the data gathered so far, it seems a selinux issue. This is from Wolfgang's issue or the bug 1213410 or is it the same thing? From Wolfgang's issue, this thread In the bug the floppy creation failed In logs attached earlier to this thread it seems the floppy was created but libvirt access failed I believe is the same root cause for bz1213410, it was just more hidden Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirt 3.5.2 cannot start windows vms
Hi, - Original Message - From: Wolfgang Bucher wolfgang.buc...@netland-mn.de To: Michal Skrivanek mskri...@redhat.com Cc: users@ovirt.org (users@ovirt.org) users@ovirt.org Sent: Monday, May 4, 2015 7:12:14 PM Subject: Re: [ovirt-users] ovirt 3.5.2 cannot start windows vms AW: [ovirt-users] ovirt 3.5.2 cannot start windows vms Hello, here are the logfiles from libvirt May 4 19:08:22 ovirt kernel: ovirtmgmt: port 2(vnet0) entered forwarding state May 4 19:08:22 ovirt kernel: ovirtmgmt: port 2(vnet0) entered forwarding state May 4 19:08:23 ovirt sanlock[638]: 2015-05-04 19:08:23+0200 5077 [638]: cmd 9 target pid 10182 not found May 4 19:08:23 ovirt systemd: Starting Virtual Machine qemu-testwin. May 4 19:08:23 ovirt systemd-machined: New machine qemu-testwin. May 4 19:08:23 ovirt systemd: Started Virtual Machine qemu-testwin. May 4 19:08:23 ovirt kvm: 1 guest now active May 4 19:08:23 ovirt kernel: ovirtmgmt: port 2(vnet0) entered disabled state May 4 19:08:23 ovirt kernel: device vnet0 left promiscuous mode May 4 19:08:23 ovirt kernel: ovirtmgmt: port 2(vnet0) entered disabled state May 4 19:08:23 ovirt kvm: 0 guests now active May 4 19:08:23 ovirt systemd-machined: Machine qemu-testwin terminated. May 4 19:08:23 ovirt libvirtd: 9183: error : qemuMonitorOpenUnix:309 : Verbindung mit Monitor-Socket gescheitert: Kein passender Prozess gefunden May 4 19:08:23 ovirt libvirtd: 9183: error : qemuProcessWaitForMonitor:2131 : Interner Fehler: Prozess während der Verbindungsaufnahme zum Monitor beendet This is the translation of the well-known 'internal error: monitor disconbected' QEMU error (not literal transaltion)! which is in turn caused by... :2015-05-04T17:08:23.274206Z qemu-kvm: -drive file=/var/run/vdsm/payload/c07772b8-6369-44cf-b554-b8dcb0e0e09b.0a41ac3e81bce0429e32b725fbf3ba5d.img,if=none,id=drive-fdc0-0-0,format=raw,serial=: could not open disk image /var/run/vdsm/payload/c07772b8-6369-44cf-b554-b8dcb0e0e09b.0a41ac3e81bce0429e32b725fbf3ba5d.img: Could not open file: Permission denied ... this error, and so the root cause seems indeed very much the same of https://bugzilla.redhat.com/show_bug.cgi?id=1213410#c7 because of this: May 4 19:08:23 ovirt libvirtd: 9183: warning : virSecuritySELinuxRestoreSecurityFileLabel:1034 : cannot lookup default selinux label for /rhev/data-center/d5e8a32f-35ed-4dec-bf9d-3c818c2780a4/66f8876c-0898-4ff2-9325-a14835f2a872/images/b329d34e-78b3-46a5-9df8-00b83c2c982a/c79a596b-5701-4afd-a5b5-d37cf412095c From the data gathered so far, it seems a selinux issue. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirt 3.5.2 cannot start windows vms
Hi - Original Message - From: Wolfgang Bucher wolfgang.buc...@netland-mn.de To: users@ovirt.org Sent: Sunday, May 3, 2015 1:53:35 PM Subject: [ovirt-users] ovirt 3.5.2 cannot start windows vms ovirt 3.5.2 cannot start windows vms Hello, i have a new and one updated ovirt 3.5.2 and cannot start windows vms. here is the vdsm.log part Thread-137::DEBUG::2015-05-03 13:46:22,725::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 1 edom: 10 level: 2 message: Interner Fehler: Prozess während der Verbindungsaufnahme zum Monitor beendet :2015-05-03T11:46:22.396113Z qemu-kvm: -drive file=/var/run/vdsm/payload/8801126b-6fc8-4d6f-af48-b0be9fdd2c83.0a41ac3e81bce0429e32b725fbf3ba5d.img,if=none,id=drive-fdc0-0-0,format=raw,serial=: could not open disk image /var/run/vdsm/payload/8801126b-6fc8-4d6f-af48-b0be9fdd2c83.0a41ac3e81bce0429e32b725fbf3ba5d.img: Could not open file: Permission denied with runonce and sysprep floppy attached the vms are starting. It seems ovirt now tries to attach a floppy always. Maybe this BZ is related: https://bugzilla.redhat.com/show_bug.cgi?id=1213410 can you share the hypervisor configuration? Is it running on oVirt Node maybe? Or CentOS anything else? Can you follow the steps outlined in https://bugzilla.redhat.com/show_bug.cgi?id=1213410#c2 and https://bugzilla.redhat.com/show_bug.cgi?id=1213410#c5 ? To troubleshoot this issue, we need more context from vdsm.log and from supervdsm.log. Thanks, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] [ACTION REQUIRED] oVirt 3.5.2 and 3.5.3 status (building RC2 today!)
- Original Message - From: Sandro Bonazzola sbona...@redhat.com To: Eli Mesika emes...@redhat.com, Ido Barkan ibar...@redhat.com, Adam Litke ali...@redhat.com, Francesco Romani from...@redhat.com, Maor Lipchuk mlipc...@redhat.com, Vered Volansky vvola...@redhat.com, Eli Mesika emes...@redhat.com, Users@ovirt.org, de...@ovirt.org Sent: Wednesday, March 18, 2015 8:56:17 AM Subject: [ACTION REQUIRED] oVirt 3.5.2 and 3.5.3 status (building RC2 today!) Hi, we still have 5 open blockers for 3.5.2[1]: Bug IDWhiteboard Status Summary 1161012 infra POSTtask cleaning utility should erase commands that have running tasks 1187244 network POST[RHEL 7.0 + 7.1] Host configure with DHCP is losing connectivity after some time - dhclient is not running 1177220 storage ASSIGNED[BLOCKED] Failed to Delete First snapshot with live merge 1196327 virtASSIGNED[performance] bad getVMList output creates unnecessary calls from Engine 1202360 virtPOST[performance] bad getVMList output creates unnecessary calls from Engine For both virth bugs above we need both these patches: 38679 and 38805 (!) Circumstances mandate extremely careful verification in master and 3.5 branch. Verification on master is almost done (very last check pending), verification on 3.5 branch is halfway done with candidate patches. I'm aiming for merge in master later today and for merge on 3.5 worst case tomorrow morning, with a bit of luck within the day. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] VDSM tests not running on RHEL 7 (on make rpm)
- Original Message - From: Liron Aravot lara...@redhat.com To: users users@ovirt.org Sent: Thursday, March 12, 2015 8:46:08 AM Subject: [ovirt-users] VDSM tests not running on RHEL 7 (on make rpm) Hi, haven't investigated that more deeply- I fetched the latest master code on a RHEL7 machine - when i run 'make rpm' it does work but the tests aren't running. on f20 machine that seems to work fine. any idea on that? Yes, 'make check' is intentionally skipped on RHEL. in vdsm.spec: === cut here === # Skips check since rhel default repos lack pep8 and pyflakes %if ! 0%{?rhel} %{!?with_check:%global with_check 1} %else %{!?with_check:%global with_check 0} %endif === cut here === Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] 3.5.1 not detecting Haswell CPU
- Original Message - From: Blaster blas...@556nato.com To: users@ovirt.org Sent: Monday, February 23, 2015 5:43:17 AM Subject: [ovirt-users] 3.5.1 not detecting Haswell CPU Hi, I just upgraded from a working 3.5.0 on F20 to 3.5.1. After the upgrade, 3.5.1 started detecting my i7-4790k as a Sandy Bridge, and not the Haswell it properly detected with 3.5.0. I had to force the CPU type to Sandy Bridge in order to bring the node up. vdsClient -s 0 getVdsCaps | grep cpu cpuCores = '4' cpuFlags = 'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,pdpe1gb,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,nopl,xtopology,nonstop_tsc,aperfmperf,eagerfpu,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,est,tm2,ssse3,fma,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,movbe,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,abm,ida,arat,pln,pts,dtherm,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,tsc_adjust,bmi1,hle,avx2,smep,bmi2,erms,invpcid,rtm,xsaveopt,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge' cpuModel = 'Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz' cpuSockets = '1' cpuSpeed = '4300.000' cpuThreads = '8' numaNodes = {'0': {'cpus': [0, 1, 2, 3, 4, 5, 6, 7], 'totalMemory': '32022'}} # cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz stepping: 3 microcode : 0x1c cpu MHz : 4299.843 cache size : 8192 KB physical id : 0 siblings: 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm xsaveopt bugs: bogomips: 7999.90 clflush size: 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: I have vague memories of similar bugs in the past. Can you please share the libvirt debug logs of one hypervisor host which has this misbehaviour? Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Ovirt 3.5.1, Windows 2008 hangs
Hi, - Original Message - From: wodel youchi wodel_d...@yahoo.fr To: users@ovirt.org Sent: Wednesday, February 11, 2015 5:21:05 PM Subject: [ovirt-users] Ovirt 3.5.1, Windows 2008 hangs Hi, I've a two nodes hosted-engine ovirt 3.5 used storage is NFS4 for the: engine, vms, iso and export I have many Linux VMs and one Windows 2008 VM. The windows 2008 VMs is acting as a second AD, it's the second time in a month where the VM hangs and the cpu usage is 100%. I have this problem only with this VM only. Can you provide the versions of the missing parts of your stack? libvirt (on the hypervisor host) qemu (same) OS (same) I tried to look into vdsm logs, but no luck so far, all I found is GuestMonitor-Win2k8x64-AD2::DEBUG::2015-02-11 12:01:02,158::vm::486::vm.Vm::(_getUserCpuTuneInfo) vmId=`ce0bcb92-90e8-45a7-9b9f-cfed10e920d6`::Domain Metadata is not set This is not relevant to your issue Is there a log file where I can follow the behaviour of the VM, because for now I have no clue about what is causing this problem. emulation is done by QEMU, which has VERY terse logging. Anyway, its logs are in /var/log/libvirt/qemu/$VM_NAME.log -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Ovirt 3.5.1, Windows 2008 hangs
- Original Message - From: wodel youchi wodel_d...@yahoo.fr To: Vinzenz Feenstra vfeen...@redhat.com, Sandro Bonazzola sbona...@redhat.com, users@ovirt.org, Lev Veyde lve...@redhat.com Sent: Thursday, February 19, 2015 3:16:44 PM Subject: Re: [ovirt-users] Ovirt 3.5.1, Windows 2008 hangs Hi, The OS for all systems (hypervisors and the engine) is Centos 6.6 x64 with latest updates Kernel 2.6.32-504.8.1.el6.x86_64 Engine version 3.5.1 Libvirt: libvirt-0.10.2-46.el6_6.3.x86_64 qemu: qemu-kvm-rhev-0.12.1.2-2.448.el6_6.x86_64 qemu-img-rhev-0.12.1.2-2.448.el6_6.x86_64 qemu-kvm-rhev-tools-0.12.1.2-2.448.el6_6.x86_64 Vdsm: vdsm-4.16.10-8.gitc937927.el6.x86_64 I am using NFS4 for my shared storage I've attached the log file of the VM from /var/log/libvirt/qemu Indeed nothing really evident from qemu log, as unfortunately was expected. We heard about problems on qemu 0.12.x and recent windowses. This bug may be relevant: https://bugzilla.redhat.com/show_bug.cgi?id=1163828 You are not facing the same issue but something underneath can be related. I'd suggest you to disable the hyperV optimization and see if that helps. To do so, you need to edit the osinfo properties (should be in /usr/share/ovirt-engine/conf/osinfo-defaults.properties ) and change the line os.windows_xp.devices.hyperv.enabled.value = true to os.windows_xp.devices.hyperv.enabled.value = false After a restart of Engine, you'll need to reboot the affected VM to let its configuration change. To check that the configuration changed, you should NOT see anymore this option hv_relaxed in the qemu commandline/logs If this doesn't help, I'm afraid you'll need to file a QEMU bug. HTH, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] why is it called vdsm?
- Original Message - From: Greg Sheremeta gsher...@redhat.com To: de...@ovirt.org, users users@ovirt.org Sent: Friday, February 13, 2015 3:04:51 PM Subject: [ovirt-users] why is it called vdsm? Why is it called vdsm and not something a little more descriptive and project-related, like ovirt-agent? Well, we could think of a rename for oVirt 4.0 :) Not 100% joking: besides being a bit more google-friendlier, that could solve the packaging troubles and make old-vdsm and new-vdsm-or-whaterver-we-may-call-it trivially parallel installable on a given host. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] restore state is very long after suspending
Hi, - Original Message - From: Nathanaël Blanchet blanc...@abes.fr To: Oved Ourfali oourf...@redhat.com Cc: users@ovirt.org, Michal Skrivanek mskri...@redhat.com, from...@redhat.com Sent: Friday, January 30, 2015 2:36:49 PM Subject: Re: [ovirt-users] restore state is very long after suspending not very convinient to read the log file... The vm name is garbin. Thank you for your help First thing, sorry for long delay, I was traveling last week. Indeed the VDSM log file seems messed up and somehow tampered (maybe bad copy/paste?) I can tell that - reconstructing from the VDSM log, nothing obviously wrong seems there. But we need a complete clean log to be sure. - if the operation eventually succeeds, to understand the slowness we must inspect the QEMU logs (most likely will be unhelpfully terse) and/or investigate QEMU to understand the behaviour Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] vdsm losing connection to libvirt
- Original Message - From: Dan Kenigsberg dan...@redhat.com To: users@ovirt.org, ybron...@redhat.com Sent: Tuesday, December 16, 2014 11:29:36 PM Subject: Re: [ovirt-users] vdsm losing connection to libvirt 20:56:38,754::libvirtconnection::135::root::(wrapper) connection to libvirt broken. ecode: 1 edom: 7 [...] 20:56:38,755::BindingXMLRPC::1142::vds::(wrapper) libvirt error Traceback (most recent call last): File /usr/share/vdsm/rpc/BindingXMLRPC.py, line 1135, in wrapper res = f(*args, **kwargs) File /usr/share/vdsm/rpc/BindingXMLRPC.py, line 463, in getCapabilities ret = api.getCapabilities() File /usr/share/vdsm/API.py, line 1245, in getCapabilities c = caps.get() File /usr/share/vdsm/caps.py, line 615, in get caps.update(netinfo.get()) File /usr/lib/python2.7/site-packages/vdsm/netinfo.py, line 812, in get nets = networks() File /usr/lib/python2.7/site-packages/vdsm/netinfo.py, line 119, in networks allNets = ((net, net.name()) for net in conn.listAllNetworks(0)) File /usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py, line 129, in wrapper __connections.get(id(target)).pingLibvirt() File /usr/lib64/python2.7/site-packages/libvirt.py, line 3642, in getLibVersion if ret == -1: raise libvirtError ('virConnectGetLibVersion() failed', conn=self) libvirtError: internal error: client socket is closed Does anything suspecious show up on libvirtd.log? I heard a report about a libvirt segmentation fault on centos7 - could this be it, too? Does `ps -o stime `pgrep libvirtd`` match with reported libvirt internal error? Would you provide the precise version of libvirt and vdsm? Confirmed. I'm aware of these bugs in particular, which could be related to a fair amount of mysterious libvirtd crashes: https://bugzilla.redhat.com/show_bug.cgi?id=1162208 https://bugzilla.redhat.com/show_bug.cgi?id=1171124 https://bugzilla.redhat.com/show_bug.cgi?id=1141159 It seems *more frequent* when migrations are involved, but as last bug tells us, it may happen on other cases as well. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] [QE][ACTION REQUIRED] oVirt 3.5.1 RC status
- Original Message - From: Sandro Bonazzola sbona...@redhat.com To: Users@ovirt.org, de...@ovirt.org Sent: Wednesday, December 10, 2014 9:07:54 AM Subject: [ovirt-users] [QE][ACTION REQUIRED] oVirt 3.5.1 RC status Hi, We have still blockers for oVirt 3.5.1 RC release so we need to postpone it until they'll be fixed. ACTION: Being so near to winter's holidays we need to discuss the new tentative date for RC in today sync meeting. The bug tracker [1] shows 2 open blocker: Bug IDWhiteboard Status Summary 1160846 sla NEW Can't add disk to VM without specifying disk profile when the storage domain has more than one disk profile 1168709 virtNEW Hosted Engine VM is listed as paused after upgrading from 3.4.4 to 3.5.1 snapshot In order to stabilize the release a new branch ovirt-engine-3.5.1 will be created from the same git hash used for composing the RC. - ACTION: assignee please provide ETA on above blockers 1168709 - patch seems simple, posted and verified (BZ entry just updated to POST). needs review. ETA for merge: definitely within the week, worst case. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Out of the box experience on Fedora - was: Re: [QE][ACTION REQUIRED] oVirt 3.5.1 RC status - postponed
- Original Message - From: Sandro Bonazzola sbona...@redhat.com To: Francesco Romani from...@redhat.com, Users@ovirt.org, de...@ovirt.org Sent: Tuesday, November 25, 2014 9:28:25 AM Subject: Re: [ovirt-users] Out of the box experience on Fedora - was: Re: [QE][ACTION REQUIRED] oVirt 3.5.1 RC status - postponed However, the reporter *has* a very valid point, which motivated me to write me this mail: a. F20 is a supported platform b. I *guess* Fedora is the platform of choice to try out QEMU and to initially play with it c. out-of-the box experience with oVirt and Fedora is cumbersome, many steps and tunings are needed. This may annoy users - without a valid reason! Can you detail the many steps and tunings are needed? I just install Fedora 20 and oVirt 3.5 snapshot and it usually just works... You are right. That was an overstatement from me (apologies, lack of caffeine hurts). Once setting up oVirt repos everything goes fine on my setups. This is definitely good enough for me. It is good enough for other people as well? I just believe that the mentioned BZ provides us -as project- an chance to check that, and check the Fedora support status. Hopefully the answer would be yes, it is streamlined and good enough, so we can move out and go ahead :) Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] Out of the box experience on Fedora - was: Re: [QE][ACTION REQUIRED] oVirt 3.5.1 RC status - postponed
- Original Message - From: Sandro Bonazzola sbona...@redhat.com To: Users@ovirt.org, de...@ovirt.org Sent: Tuesday, November 25, 2014 7:07:14 AM Subject: [ovirt-users] [QE][ACTION REQUIRED] oVirt 3.5.1 RC status - postponed [...] The following bugs have been keyworded as Regression and not marked as blockers: 1165336 virtASSIGNEDFC20 qemu needs kvmclock bugfixes Hi, I'd like to elaborate a bit more here for the sake of the openness. TL;DR version - no actual regression in oVirt, meaning 'we applied a patch and we broke something'. - fix is underway - to propose the relevant patches to Fedora QEMU package Long(ish) version 1 some time ago we adhered to QEMU/KVM clock settings recommendations (see https://bugzilla.redhat.com/show_bug.cgi?id=1053846) 2 these recommendations are *still* valid as today - I just checked with upstream developers while investigating bz1165336 3 these recommendations may have surprising effects like disabling HPET clock 4 on some old(ish) upstream QEMUs, disabling HPET may hurt migrations - hence bz1165336 5 only *very* recent QEMUs (2.2.0rc0!) have the fixes, which are about improving kvm clock, while HPET clock is still not recommended (see #2 above) 6 if the qemu-kvm-rhev is used (available in the oVirt repo), the experience is significantly better However, the reporter *has* a very valid point, which motivated me to write me this mail: a. F20 is a supported platform b. I *guess* Fedora is the platform of choice to try out QEMU and to initially play with it c. out-of-the box experience with oVirt and Fedora is cumbersome, many steps and tunings are needed. This may annoy users - without a valid reason! d. hence there is an unneededly high first step to try out oVirt, and this is hurting the project Now, I'd like to raise this question * it is true that Fedora is the platform of choice to try out and evaluate oVirt? * if so, is the experience on Fedora streamlined enough or could it made simple, hence we could have a better vector to spread oVirt? * what we could do, as oVirt project, to improve the above? Feedback welcome -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] VM migration/creation on host problem ...
- Original Message - From: James James jre...@gmail.com To: Francesco Romani from...@redhat.com Cc: users users@ovirt.org Sent: Friday, 14 November, 2014 4:53:22 PM Subject: Re: [ovirt-users] VM migration/creation on host problem ... In /var/log/libvirt/libvirt.log : [root@ovirt-node2 ~]# tail -f /var/log/libvirt/libvirtd.log 2014-11-14 15:45:03.770+: 13769: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present 2014-11-14 15:45:18.791+: 13772: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present 2014-11-14 15:45:33.812+: 13772: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present 2014-11-14 15:45:48.867+: 13768: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present 2014-11-14 15:46:03.889+: 13772: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present 2014-11-14 15:46:18.911+: 13768: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present 2014-11-14 15:46:21.595+: 13767: error : virNetSocketReadWire:1194 : End of file while reading data: Input/output error 2014-11-14 15:46:33.931+: 13770: error : virDomainObjGetMetadata:16650 : metadata not found: Requested metadata element is not present Does anybody knows what that means ? That and the selinux issue you reported previously reminded me of one bug: https://bugzilla.redhat.com/show_bug.cgi?id=1157185 further information: https://bugzilla.redhat.com/show_bug.cgi?id=1150243#c17 Can't find more precise information at the moment :( -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] VM migration/creation on host problem ...
- Original Message - From: James James jre...@gmail.com To: users users@ovirt.org Sent: Thursday, November 13, 2014 5:41:21 PM Subject: [ovirt-users] VM migration/creation on host problem ... Hello, I've just reinstall a host (ovirt-node2) and now I am unable to create/migrate vm on this host. In attachment, the log from the destination server (ovirt-node2). Any help will be appreciated. James Hi James, can you please add the libvirt logs? depending on the version of oVirt VDSM is either /var/log/libvirt/libvirtd.log or multiplexed into /var/log/messages please also add QEMU logs as found in /var/log/libvirt/qemu/$VM_NAME.log where VM_NAME is puppetdb here thanks, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Live migration on el6?
- Original Message - From: Demeter Tibor tdeme...@itsmart.hu To: users users@ovirt.org Sent: Friday, November 7, 2014 10:23:26 AM Subject: [ovirt-users] Live migration on el6? Hi, I have a question. Need I rebuild and install the redhat's qemu-kvm for ovirt 3.5 if I wanna working live migration function? At this moment it is not working for me. Live VM migration should work out of the box. Please add more details if it does'nt. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVirt 3.5 on RHEL 6.6 hyperv enabled breaks Windows VMs
- Original Message - From: Charles Gruener cgrue...@gruener.us To: Michal Skrivanek michal.skriva...@redhat.com Cc: users@ovirt.org Users users@ovirt.org Sent: Monday, October 27, 2014 1:32:50 PM Subject: Re: [ovirt-users] oVirt 3.5 on RHEL 6.6 hyperv enabled breaks Windows VMs The three Windows VMs I tried were Windows 7, Windows 8.1, and Windows Server 2012 R2. All experienced the stop error. This is quite interesting because the hv_relaxed thing was supposed to _improve_ stability for windows 7 onwards. I was expecting troubles with older windows (XP), not with newer ones. Can you share what is running on your hypervisors? versions of libvirt and qemu. Moreover, qemu and libvirt logs would help. Thanks, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] oVirt 3.5 on RHEL 6.6 hyperv enabled breaks Windows VMs
- Original Message - From: Charles Gruener cgrue...@gruener.us To: Francesco Romani from...@redhat.com Cc: Michal Skrivanek michal.skriva...@redhat.com, users@ovirt.org Users users@ovirt.org Sent: Monday, October 27, 2014 3:56:00 PM Subject: Re: [ovirt-users] oVirt 3.5 on RHEL 6.6 hyperv enabled breaks Windows VMs Here's what's installed on one of my hypervisors: gpxe-roms-qemu.noarch0.9.7-6.12.el6 @rhel-x86_64-server-6 libvirt.x86_64 0.10.2-46.el6 @rhel-x86_64-server-6 libvirt-client.x86_640.10.2-46.el6 @rhel-x86_64-server-6 libvirt-lock-sanlock.x86_64 0.10.2-46.el6 @rhel-x86_64-server-optional-6 libvirt-python.x86_640.10.2-46.el6 @rhel-x86_64-server-6 ovirt-release35.noarch 001-1 @/ovirt-release35 qemu-img-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 @ovirt-3.5 qemu-kvm-rhev.x86_64 2:0.12.1.2-2.415.el6_5.14 @ovirt-3.5 qemu-kvm-rhev-tools.x86_64 2:0.12.1.2-2.415.el6_5.14 @ovirt-3.5 vdsm.x86_64 4.16.7-1.gitdb83943.el6 @ovirt-3.5 vdsm-cli.noarch 4.16.7-1.gitdb83943.el6 @ovirt-3.5 vdsm-jsonrpc.noarch 4.16.7-1.gitdb83943.el6 @ovirt-3.5 vdsm-python.noarch 4.16.7-1.gitdb83943.el6 @ovirt-3.5 vdsm-python-zombiereaper.noarch 4.16.7-1.gitdb83943.el6 @ovirt-3.5 vdsm-xmlrpc.noarch 4.16.7-1.gitdb83943.el6 @ovirt-3.5 vdsm-yajsonrpc.noarch4.16.7-1.gitdb83943.el6 @ovirt-3.5 virt-what.x86_64 1.11-1.2.el6 @anaconda-RedHatEnterpriseLinux-201301301459.x86_64/6.4 OK, seems a stock RHEL 6.6. I wonder if the recommended hv settings are really supported on old QEMUs. Need to check about this: will do and will report. If you'll be the most welcome to file a bug against oVirt for this issue, and set target release to 3.5.1, so we can track this issue. I noticed the machines do not have the qemu-kvm package installed, only qemu-kvm-rhev. What's the difference in those two packages? When I try to manually install qemu-kvm, it won't and lets me know that: Package 2:qemu-kvm-0.12.1.2-2.445.el6.x86_64 is obsoleted by 2:qemu-kvm-rhev-0.12.1.2-2.415.el6_5.14.x86_64 which is already installed This is totally expected. qemu-kvm-rhev provides additional facilities and it is better to use it. I've also noticed that I can't live snapshot machines because it says my underlying qemu doesn't support it and thought that might be a related issue. This is known and different issue: https://bugzilla.redhat.com/show_bug.cgi?id=1149667 hopefully the fix will be available soon The machines have Intel(R) Xeon(R) CPU E5-2670 CPUs, if it makes any difference. Charles Thanks! -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Nesting Hyper-V
- Original Message - From: Sven Kieske s.kie...@mittwald.de To: users@ovirt.org Sent: Wednesday, October 8, 2014 8:02:29 AM Subject: Re: [ovirt-users] Nesting Hyper-V On 08/10/14 00:39, Nathan Llaneza wrote: Hello All, I work for a educational institution, and we currently use oVirt in our datacenter. We are on the latest stable build of oVirt 3.4.4. Most of our hypervisors run CentOS 6.5; however, we have recently been asked if we could assign virtual hypervisors (Server 2012 R2) to some of our classes for learning purposes. I have been doing some research and found that I needed atleast kernel 3.10; therefore, I took one hypervisors and converted it to CentOS 7 fully patched. I also installed the nestedvt hook as described here http://community.redhat.com/blog/2013/08/testing-ovirt-3-3-with-nested-kvm/. I am all researched/googled out, and I am having no luck. After I do a typical install of Server 2012 R2 it will not let me install the Hyper-V role. I quote Hyper-V cannot be installed: A hypervisor is already running. When I run the command cat /sys/module/kvm_intel/parameters/nested it comes back with a Y. If I look in the host hooks it shows the nestedvt and macspoof. The VM has a custom property of macspoof = true. Did I miss something? Is this possible? Has anyone done this before? I guess you need the cpu param hv_relaxed there is a BZ open on it, see this blog for all cpu flags: http://blog.wikichoon.com/2014/07/enabling-hyper-v-enlightenments-with-kvm.html -cpu ...,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time there are plenty of bugs / improvements open, here is the list: https://bugzilla.redhat.com/buglist.cgi?quicksearch=hv_relaxedlist_id=2902293 Initial HyperV enlightnement support was added to oVirt 3.5.0: (e.g. : http://gerrit.ovirt.org/#/c/30254/ http://gerrit.ovirt.org/#/c/30255/ ) More optimization will come soon, depending on libvirt availability. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] 3.4: VDSM Memory consumption
- Original Message - From: Daniel Helgenberger daniel.helgenber...@m-box.de To: Dan Kenigsberg dan...@redhat.com Cc: users@ovirt.org Sent: Monday, September 29, 2014 12:25:22 PM Subject: Re: [ovirt-users]3.4: VDSM Memory consumption Dan, I just reply to the list since I do not want to clutter BZ: While migrating VMs is easy (and the sampling is already running), can someone tell me the correct polling port to block with iptables? Thanks, Hi Daniel, there is indeed a memory profiling patch under discussion: http://gerrit.ovirt.org/#/c/32019/ but for your case we'll need a backport to 3.4.x and clearer install instructions, which I'll prepare as soon as possible. As for your question: if I understood correctly what you are asking (still catching up the thread), if you are trying to rule out the stats polling made by Engine to this bad leak, one simple way to test is just to shutdown Engine, and let VDSMs run unguarded on hypervisors. You'll be able to command these VDSMs using vdsClient or restarting Engine. In absence of commands from Engine VDSM will just sit idle and it will not do anything to VMs except it's own sampling. If you are on IRC, on freenode/#vdsm or OFTC/#ovirt, please ping me anytime - see my sign below for my IRC nickname. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] 3.4: VDSM Memory consumption
- Original Message - From: Daniel Helgenberger daniel.helgenber...@m-box.de To: Francesco Romani from...@redhat.com Cc: Dan Kenigsberg dan...@redhat.com, users@ovirt.org Sent: Monday, September 29, 2014 2:54:13 PM Subject: Re: [ovirt-users]3.4: VDSM Memory consumption Hello Francesco, On 29.09.2014 13:55, Francesco Romani wrote: - Original Message - From: Daniel Helgenberger daniel.helgenber...@m-box.de To: Dan Kenigsberg dan...@redhat.com Cc: users@ovirt.org Sent: Monday, September 29, 2014 12:25:22 PM Subject: Re: [ovirt-users] 3.4: VDSM Memory consumption Dan, I just reply to the list since I do not want to clutter BZ: While migrating VMs is easy (and the sampling is already running), can someone tell me the correct polling port to block with iptables? Thanks, Hi Daniel, there is indeed a memory profiling patch under discussion: http://gerrit.ovirt.org/#/c/32019/ but for your case we'll need a backport to 3.4.x and clearer install instructions, which I'll prepare as soon as possible. I updated the BZ (and are now blocking 54321/tcp on one of my hosts). and verified it is not reachable. As general info: This system I am using is my LAB / Test / eval setup for a final deployment for ovirt (then 3.5) in production; so it will go away some time in the future (a few weeks / months). If I am the only one experiencing this problem then you might be better of allocating resources elsewhere ;) Thanks for your understanding :) Unfortunately it is true that developer resources aren't so abundant, but it is also true that memleaks should never be discarded easily and without due investigation, considering the nature and the role of VDSM. So, I'm all in for further investigation regarding this issue. As for your question: if I understood correctly what you are asking (still catching up the thread), if you are trying to rule out the stats polling made by Engine to this bad leak, one simple way to test is just to shutdown Engine, and let VDSMs run unguarded on hypervisors. You'll be able to command these VDSMs using vdsClient or restarting Engine. As I said in my BZ comment this is not an option right now, but if understand the matter correctly IPTABLES reject should ultimately do the same? Definitely yes! Just do whatever it is more convenient for you. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] OVIRT-3.5-TEST-DAY-3: replace XML-rpc with JSON-rpc
Hi everyone of this test day I played with the new JSON-rpc transport, which is enabled by default. I set up from scratch two hypervisors running on RHEL 6.5, while Engine was running on F19. I created everything from scratch and tested all the flows I known good enough, including less-then-common stuff. The list includes creating storage domains, VMs, hotplug/unplug disks, snapshots, migration, hibernation and dehibernation. On top of that, I prepared an environment to run more tests, that included create 200 VMs, started/stopped 100 of them to test. I kept an eye on the logs other than Engine. Everything I tried went OK, and logs look good to me. I run in a few hiccups, which I mention for the sake of completeness: - VDSM refused to start or run VMs initially: libvirt config included relics from past environment on the same box, not JSON-rpc fault. Fixed with new config and (later) a reboot. - Trying recovery, Engine took longer than expected to sync up with VDSM. I have not hard data and feeling is not enough to file a BZ, so I didn't. - Still trying recovery, one and just one time Engine had stale data from VDSM (reported two VMs as present which actually aren't). Not sure it was related to JSON-rpc, can't reproduce, so not filed a BZ. Since I need to continue profiling anyway (better late than never I guess), I include a couple of test to see if JSON-rpc makes any difference to JSON-rpc. Results will be posted on de...@ovirt.org and/or on the oVirt wiki. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] [QA] [ACTION REQUIRED] oVirt 3.5.0 RC2 status
- Original Message - From: Sandro Bonazzola sbona...@redhat.com To: Users@ovirt.org, de...@ovirt.org Cc: Doron Fediuck dfedi...@redhat.com, Eyal Edri ee...@redhat.com, Itamar Heim ih...@redhat.com, David Caro dcaro...@redhat.com, Yair Zaslavsky yzasl...@redhat.com, ykap...@redhat.com, Tal Nisan tni...@redhat.com, Mooli Tayer mta...@redhat.com, Francesco Romani from...@redhat.com, Adam Litke ali...@redhat.com Sent: Tuesday, August 12, 2014 5:43:40 PM Subject: [QA] [ACTION REQUIRED] oVirt 3.5.0 RC2 status Hi, tomorrow we should compose oVirt 3.5.0 RC2 starting at 08:00 UTC We still have the following blockers list: 1127460 virtNEW VM abnormal stop after extending when using thin provisioning on block storage [...] Analysis doesn't show yet a bug on VDSM, which seems to behave correctly https://bugzilla.redhat.com/show_bug.cgi?id=1127460#c6 https://bugzilla.redhat.com/show_bug.cgi?id=1127460#c7 NEEDINFO requests to QEMU and LVM maintainers are been sent, no answer yet. Since this was marked as blocker, I'll invest more time on this and will look for a workaround, if possible, within the end of the week. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] [QA] [ACTION REQUIRED] oVirt 3.5.0 RC2 status
- Original Message - From: Adam Litke ali...@redhat.com To: Sandro Bonazzola sbona...@redhat.com Cc: Users@ovirt.org, de...@ovirt.org, Doron Fediuck dfedi...@redhat.com, Eyal Edri ee...@redhat.com, Itamar Heim ih...@redhat.com, David Caro dcaro...@redhat.com, Yair Zaslavsky yzasl...@redhat.com, ykap...@redhat.com, Tal Nisan tni...@redhat.com, Mooli Tayer mta...@redhat.com, Francesco Romani from...@redhat.com Sent: Tuesday, August 12, 2014 7:36:52 PM Subject: Re: [QA] [ACTION REQUIRED] oVirt 3.5.0 RC2 status On 12/08/14 17:43 +0200, Sandro Bonazzola wrote: Hi, tomorrow we should compose oVirt 3.5.0 RC2 starting at 08:00 UTC We still have the following blockers list: Bug ID Whiteboard Status Summary 1127294 storage POSTLive Merge: Resolve unknown merge status in vdsm after host crash 1109920 storage POSTLive Merge: Extend internal block volumes during merge There are several patches for master (6) that must be merged and backported to 3.5. Thanks Francesco for your reviews (I will repost the series this afternoon for followup review). I would appreciate a look by those I've included as reviewers (you received a separate email from me) so we can converge on these ASAP. No problem, let me just reiterate that except for the patch I already +1-ed, it was a very initial review, I just need a bit of time to fully grok them. More deep review will come ASAP. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Testday: json rpc
- Original Message - From: Antoni Segura Puimedon asegu...@redhat.com To: de...@ovirt.org, users users@ovirt.org Sent: Wednesday, July 30, 2014 2:15:36 PM Subject: [ovirt-users] Testday: json rpc [...] **Logging** When doing a network operation with xmlrpc we'd get the following in vdsm.log Thread-21::DEBUG::2014-07-30 13:38:11,414::BindingXMLRPC::1127::vds::(wrapper) client [10.34.61.242]::call setupNetworks with ({'10': {'nic': 'em2', 'vlan': '10', 'STP': 'no', 'bridged': 'true', 'mtu': '1500'}}, {}, {'connectivityCheck': 'true', 'connectivityTimeout': 120}) {} flowID [686033d4] Thread-21::DEBUG::2014-07-30 13:38:32,689::BindingXMLRPC::1134::vds::(wrapper) return setupNetworks with {'status': {'message': 'Done', 'code': 0}} As you can see, we get the bare minimum logging one could ask for, an entry with the command called and the data it received and another entry with the return result data. Doing the same with jsonrpc (and ignoring the excessive IOProcess) if I search for setupNetworks the only thing I get is: Thread-23057::DEBUG::2014-07-30 13:32:44,126::__init__::462::jsonrpc.JsonRpcServer::(_serveRequest) Looking for method 'Host_setupNetworks' in bridge And if I search for the data received, like 'STP', there is nothing whatsoever. As I said, unless this is fixed and we get entries with the same amount of data as before, it can't be used in production nor in development. https://bugzilla.redhat.com/1124813 Can you share some VDSM logs? Attaching them to BZ would be super. I'd like to check if that applies to virt flows as well - as I actually tend to believe. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] oVirt 3.5 test day 2 results
Hi everyone, Hi tested again http://www.ovirt.org/Features/WebSocketProxy_on_a_separate_host What happened on tast day 1 * found minor packaging issues * stopped earlier facing SSL issues, had a followup the day after an managed to have the feature working. This time things got better, and again the feature works as expected. The packaging issues are gone, but I still had UX annoyances along the way. I followed instructions on the wiki page above. Platform: F20 hypervisor host F20 engine host F19 websocket proxy (Didn't had time to test on different platforms because local bandwith issues eat lot of time just to install things) Installation went fine. websocket proxy setup is maybe a bit clumsy (I mean the text mode wizard), but it is bearable (I don't mind at all, but someone else can...); for some reasons (I cannot exclude an error from mine) engine got configured to use localhost as websocket proxy. To fix this I edited the engine config (update on DBMS), but then faced this error on proxy side: Jul 29 17:13:14 shinji ovirt-websocket-proxy.py[17004]: 1: handler exception: [Errno 1] _ssl.c:504: error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher to redo the websocket setup I removed (actually renamed) /etc/pki/ovirt-engine and rerun setup. After that everything worked fine Jul 29 17:26:52 shinji ovirt-websocket-proxy.py[17180]: 8: connecting to: 192.168.1.53:5900 (using SSL) Jul 29 17:26:52 shinji ovirt-websocket-proxy.py[17180]: 5: 192.168.1.177: SSL/TLS (wss://) WebSocket connection Jul 29 17:26:52 shinji ovirt-websocket-proxy.py[17180]: 5: 192.168.1.177: Version hybi-13, base64: 'False' Jul 29 17:26:52 shinji ovirt-websocket-proxy.py[17180]: 5: 192.168.1.177: Path: '/eyJ2YWxpZFRvIjoiMjAxNDA3MjkxNTIx [...] 192.168.1.53 is the hypervisor host I used Now the point is maybe I did some mistakes or overlooked some configuration steps (maybe blindly hit return instead of changing a default), but I suggest to improve the docs/wiki to document how to fix common gotchas and/or to reconfigure things. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] [test day] day 1 results
[as sent to devel-list] Hi, During the day 1 I tested http://www.ovirt.org/Features/WebSocketProxy_on_a_separate_host Found a couple of minor issues, due to packaging glitches and wiki page not up to date. https://bugzilla.redhat.com/show_bug.cgi?id=1114920 https://bugzilla.redhat.com/show_bug.cgi?id=1114978 besides that the feature seems to work nicely -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] experience with AMD Kabini (Jaguar) CPUs
- Original Message - From: Doron Fediuck dfedi...@redhat.com To: i iordanov iiorda...@gmail.com Cc: users@ovirt.org Sent: Thursday, June 5, 2014 6:50:58 PM Subject: Re: [ovirt-users] experience with AMD Kabini (Jaguar) CPUs - Original Message - From: i iordanov iiorda...@gmail.com To: Doron Fediuck dfedi...@redhat.com Cc: users@ovirt.org, Yedidyah Bar David d...@redhat.com, Sandro Bonazzola sbona...@redhat.com Sent: Thursday, June 5, 2014 7:46:24 PM Subject: Re: [ovirt-users] experience with AMD Kabini (Jaguar) CPUs Hi Doron, I haven't purchased the Kabini-based system. I hoping that I would be able to learn in advance whether an all-in-one installation of oVirt would work on it. Unfortunately, returning it would be a hassle and I would not get a full refund for it. If there is no way to discover in advance whether oVirt will work on the system, I can take a risk. Is an all-in-one oVirt installation known to work on any of the modern AMD offerings? Thanks! iordan Hi everyone, I'm playing with a new AMD Kabini host, to see how it behaves as very low-cost[1], low-consumption hypervisor host[2]. I'm still installing the node, so I don't have actual data yet. According to libvirt, the host present itself as Opteron_G3. Plese find attached the output from virsh (1.2.5) capabilities (caps.txt) lscpu (lscpu.txt) lspci (lspci.txt) +++ [1] You can get a CPU + mainboard with ~85 EURO or even less, and it will consume 25W fully loaded, much less when idle. The fan is close to inaudible. [2] I'll put together a document/wiki page with my findings and experience afterwards. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani 00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Root Complex 00:01.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Kabini [Radeon HD 8400 / R3 Series] 00:01.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Kabini HDMI/DP Audio 00:02.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 0 00:02.4 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Functions 5:1 00:02.5 PCI bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Functions 5:1 00:10.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB XHCI Controller (rev 01) 00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] (rev 40) 00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller (rev 39) 00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller (rev 39) 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB OHCI Controller (rev 39) 00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller (rev 39) 00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller (rev 3a) 00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD] FCH Azalia Controller (rev 02) 00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge (rev 11) 00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 0 00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 1 00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 2 00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 3 00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 4 00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 16h Processor Function 5 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 11) 02:00.0 USB controller: ASMedia Technology Inc. ASM1042A USB 3.0 Host Controller Architecture: x86_64 CPU op-mode(s):32-bit, 64-bit Byte Order:Little Endian CPU(s):4 On-line CPU(s) list: 0-3 Thread(s) per core:1 Core(s) per socket:4 Socket(s): 1 NUMA node(s): 1 Vendor ID: AuthenticAMD CPU family:22 Model: 0 Model name:AMD Athlon(tm) 5350 APU with Radeon(tm) R3 Stepping: 1 CPU MHz: 800.000 CPU max MHz: 2050. CPU min MHz: 800. BogoMIPS: 4092.62 Virtualization:AMD-V L1d cache: 32K L1i cache: 32K L2 cache: 2048K NUMA node0 CPU(s): 0-3 capabilities host uuid0e989c0a-956b-343d-c5ac-40167eb07d87/uuid cpu archx86_64/arch modelOpteron_G3/model vendorAMD/vendor topology sockets='1' cores='4' threads='1'/ feature name='bmi1'/ feature name='perfctr_nb'/ feature name='topoext'/ feature name='wdt'/ feature name='skinit'/ feature name='ibs'/ feature name='osvw'/ feature name
Re: [ovirt-users] getVdsCapabilites unexpected exception [was: Re: AIO 3.4 on fedora 19 initial errors before coming up]
Hi, - Original Message - From: Gianluca Cecchi gianluca.cec...@gmail.com To: Roy Golan rgo...@redhat.com Cc: users users@ovirt.org Sent: Sunday, May 11, 2014 11:49:06 PM Subject: Re: [ovirt-users] getVdsCapabilites unexpected exception [was: Re: AIO 3.4 on fedora 19 initial errors before coming up] [...] it seems the error in vdsm.log when I run the command above is of this type: Thread-25::ERROR::2014-05-11 20:18:02,202::BindingXMLRPC::1086::vds::(wrapper) unexpected error Traceback (most recent call last): File /usr/share/vdsm/BindingXMLRPC.py, line 1070, in wrapper res = f(*args, **kwargs) File /usr/share/vdsm/BindingXMLRPC.py, line 393, in getCapabilities ret = api.getCapabilities() File /usr/share/vdsm/API.py, line 1185, in getCapabilities c = caps.get() File /usr/share/vdsm/caps.py, line 369, in get caps.update(netinfo.get()) File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 557, in get netAttr.get('qosOutbound')) File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 487, in _getNetInfo ipv4addr, ipv4netmask, ipv6addrs = getIpInfo(iface) File /usr/lib64/python2.7/site-packages/vdsm/netinfo.py, line 317, in getIpInfo ipv6addrs = devInfo.get_ipv6_addresses() SystemError: error return without exception set Based on above errors, I think that for some reason these two python related packages that were updated yesterday are causing some problems with vdsm. Can you confirm that you can run ok the 3.4 vdsm with those? vdsm-4.14.6-0.fc19.x86_64 May 10 21:24:23 Updated: python-ethtool-0.9-2.fc19.x86_64 May 10 21:24:23 Updated: python-lxml-3.3.5-1.fc19.x86_64 I can also try to rollback and see... I was right. Against what to bugzilla? This is a show stopper for fedora 19 ovirt users... Unfortunately, you are been hit by https://bugzilla.redhat.com/show_bug.cgi?id=1078312 It is fixed on gerrit, but you'll need VDSM = 4.14.8.1 Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] getVdsCapabilites unexpected exception [was: Re: AIO 3.4 on fedora 19 initial errors before coming up]
Hi, - Original Message - From: Gianluca Cecchi gianluca.cec...@gmail.com To: Francesco Romani from...@redhat.com Cc: users users@ovirt.org, Roy Golan rgo...@redhat.com Sent: Monday, May 12, 2014 9:56:33 AM Subject: Re: [ovirt-users] getVdsCapabilites unexpected exception [was: Re: AIO 3.4 on fedora 19 initial errors before coming up] Il 12/mag/2014 08:33 Francesco Romani from...@redhat.com ha scritto: Unfortunately, you are been hit by https://bugzilla.redhat.com/show_bug.cgi?id=1078312 It is fixed on gerrit, but you'll need VDSM = 4.14.8.1 Bests, So you are telling I can try to update my AIO to to 3.4.1 and as it will provide (after updating vdsm too) vdsm-4.14.8.1-0.fc19.x86_64.rpm the problem should go away? Yes, it should. Just replacing VDSM should be enough, however. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] Live migration failing
- Original Message - From: Steve Dainard sdain...@miovision.com To: users users@ovirt.org Sent: Tuesday, April 29, 2014 4:32:08 AM Subject: Re: [ovirt-users] Live migration failing Another error on migration. Hi, in both cases the core issue is ibvirtError: Unable to read from monitor: Connection reset by peer can you share the libvirtd and qemu logs? Hopefully we can find some more information on those logs. Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] VM satelit is down. Exit message: 'int' object has no attribute 'replace'.
Hi, - Original Message - From: Gabi C gab...@gmail.com To: Joop jvdw...@xs4all.nl Cc: users@ovirt.org Sent: Monday, March 31, 2014 3:08:47 PM Subject: Re: [Users] VM satelit is down. Exit message: 'int' object has no attribute 'replace'. Hello! Do I have enable kvm and libvirt also on engine? I think not Note: On Fedora 19, you'll need to enable fedora-virt-preview repository for using Fedora 19 as node on 3.4 cluster .the problem still persist You've very likely experienced the same problem reported here https://bugzilla.redhat.com/show_bug.cgi?id=1067064 A fix was already been committed. About kvm and libvirt, you only need them on the nodes running hypervisors; engine itself does not depend on them. -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] vdsm python error
- Original Message - From: jacek burghardt jaceksburgha...@gmail.com To: users@ovirt.org Sent: Saturday, March 8, 2014 3:57:15 AM Subject: [Users] vdsm python error I am attempting to use vdsm with arch linux and I am getting this error when I run vdsm-tool Traceback (most recent call last): [...] File /usr/lib/python2.7/site-packages/vdsm/tool/service.py, line 36, in execCmd return _execCmd(argv, raw=raw, *args, **kwargs) File /usr/lib/python2.7/site-packages/vdsm/utils.py, line 682, in execCmd deathSignal=deathSignal, childUmask=childUmask) TypeError: __init__() got an unexpected keyword argument 'childUmask' How I can solve it ? Hi, please upgrade python-cpopen to version 1.3 Bests, -- Francesco Romani RedHat Engineering Virtualization R D Phone: 8261328 IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [Users] Host Non-Operational from sanlock and VM fails to migrate
- Original Message - From: Nir Soffer nsof...@redhat.com To: Trey Dockendorf treyd...@gmail.com Cc: users users@ovirt.org, Michal Skrivanek mskri...@redhat.com Sent: Wednesday, February 12, 2014 10:04:04 AM Subject: Re: [Users] Host Non-Operational from sanlock and VM fails to migrate [...] The vm was starting a migration to the other host: Thread-26::DEBUG::2014-02-03 07:49:18,067::BindingXMLRPC::965::vds::(wrapper) client [192.168.202.99]::call vmMigrate with ({'tunneled': 'false', 'dstqemu': '192.168.202.103', 'src': 'vm01.brazos.tamu.edu', 'dst': 'vm02.brazos.tamu.edu:54321', 'vmId': '741f9811-db68-4dc4-a88a-7cb9be576e57', 'method': 'online'},) {} flowID [7829ae2a] Thread-26::DEBUG::2014-02-03 07:49:18,067::API::463::vds::(migrate) {'tunneled': 'false', 'dstqemu': '192.168.202.103', 'src': 'vm01.brazos.tamu.edu', 'dst': 'vm02.brazos.tamu. edu:54321', 'vmId': '741f9811-db68-4dc4-a88a-7cb9be576e57', 'method': 'online'} Thread-26::DEBUG::2014-02-03 07:49:18,068::BindingXMLRPC::972::vds::(wrapper) return vmMigrate with {'status': {'message': 'Migration in progress', 'code': 0}, 'progress': 0} The migration was almost complete after 20 seconds: Thread-29::INFO::2014-02-03 07:49:38,329::vm::815::vm.Vm::(run) vmId=`741f9811-db68-4dc4-a88a-7cb9be576e57`::Migration Progress: 20 seconds elapsed, 99% of data processed, 99% of mem processed But it never completed: Thread-29::WARNING::2014-02-03 07:54:38,383::vm::792::vm.Vm::(run) vmId=`741f9811-db68-4dc4-a88a-7cb9be576e57`::Migration is stuck: Hasn't progressed in 300.054134846 seconds. Aborting. CCing Michal to inspect why the migration has failed. Hi, I had a look at the logs, and this looks like another libvirt/qemu I/O related issue. If QEMU on the src hosts cannot reliably access storage, migration may get stuck; this seems the case given the information provided; VDSM detected the migration was not progressing and aborted it. libvirt has an option (which we already use) to detect those scenarios, the VIR_MIGRATE_ABORT_ON_ERROR flag, but unfortunately this is not 100% reliable yet, for reasons outlined below in this mail. We are aware of this issue and we are actively working to improve the handling of such scenarios, but actually this is mostly on QEMU. The core issue here is that when we use NFS (or ISCSI), and there is an I/O error, QEMU can get blocked inside the kernel, waiting for the faulty I/O operation to complete, and thus fail to report an I/O error. It really depends on what specific operation fails, and there are many possible cases and error scenarios. Of course, if QEMU is blocked and fails to report the I/O error, libvirt can do nothing to report/recover error, so VDSM can do even less. This is known and acknowledged both by libvirt and QEMU developers. But there are some good news, because newer versions of QEMU have improvements on this field: QEMU recently gained native block devices[1], which, among other things, will make it more robust in presence of I/O errors, and should improve the error reporting as well. RHEL7 should have a version of QEMU with native ISCSI; hopefully NFS will follow soon enough. HTH, +++ [1] for example, ISCSI, recently merged: http://comments.gmane.org/gmane.comp.emulators.qemu/92599 work on NFS is ongoing. -- Francesco Romani RedHat Engineering Virtualization R D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[Users] oVirt 3.4 test day: iscsi multipath
Hi everyone, During the last oVirt 3.4 test day I played a bit with ISCSI multipath: http://www.ovirt.org/Feature/iSCSI-Multipath#Configure_iSCSI_Multipathing It took me longer than expected to properly set up things but, once done, everything looked fine at the config level (both seen from engine and the actual OS configuration) and worked well on my tests, including physical disconnections mid-I/O. Well done! :) -- Francesco Romani RedHat Engineering Virtualization R D IRC: fromani ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users