Re: [ovirt-users] I broke my ovirt real good....
On 04/16/2018 10:02 AM, Alexander Wels wrote: > On Friday, April 13, 2018 6:48:31 PM EDT ~Stack~ wrote: [sni] >> It just sits there and in the log files there is the below messages >> repeating. It's like it doesn't care for the fact that this was an >> imported domain or something. >> >> Thoughts? >> >> Thanks! >> ~Stack~ >> > > Don't know too much about the VDSM side of things. But obviously its looking > for a storage domain it can't find anymore. You can try restarting VDSM > (won't > affect running VMs) and see if rescans the available storage domains and > won't > try to access it during the migration of the VMs. Other than that I don't > know. No worries. Thanks for responding. One of my hosts has gone berserk anyway so I'm just going to do a complete fresh reinstall tomorrow. The host says "Host has no default route" which is a load of bull. There's nothing wrong with the default route or network connectivity. However, oVirt puts it into non-opperational where it will sit for about 20 minutes. When it finally actually stops that process, it immediately (milliseconds later) puts it into "activating" but then complain about the default route and the whole process starts over again. There's something wrong with this install so I'm just going to take the nuke-it-from-orbit-and-start-over approach tomorrow morning. ~Stack~ signature.asc Description: OpenPGP digital signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] I broke my ovirt real good....
On Friday, April 13, 2018 6:48:31 PM EDT ~Stack~ wrote: > On 04/13/2018 07:16 AM, Alexander Wels wrote: > > On Thursday, April 12, 2018 6:26:07 PM EDT ~Stack~ wrote: > >> Greetings, > >> > >> So I did a over-confident-admin-makes-rookie-mistake. I changed a bunch > >> of things all back-to-back and thus don't actually know what broke. :-D > >> > >> The only two real "big" changes were: > >> * Upgrade from 4.2.1 to 4.2.2 > >> * change my ovirtmgmt network > >> > >> The update I followed the upgrade procedures and I thought it all went > >> pretty well. Because I am moving it from a testing into what I hope will > >> be a more heavily used environment, I changed my ovirtmgmt network from > >> 192.168.100.0/24 to 192.168.101.0/24 via the web-gui. > >> > >> That was a touch tricker than just a change as I had to poke the > >> management engine host to be reachable on both network for a while, then > >> it just seemed OK. > >> > >> What's happening is: > >> * I can no longer migrate a vm from one host to the other. > >> * If I try to do a "reinstall" it dies. > >> * There is some serious network lag between my hosts on a 10Gb network. > >> * I've got all kinds of python2.4 failures in my vdsm and mom logs. > >> > >> Those are least the biggies. > >> > >> So while I was planning on moving this to a more active use case, right > >> now - it is all still my play ground. I would REALLY hate to lose the > >> VM's but everything else can go and be rebuilt. > >> > >> Given that I've somehow really broke this system pretty good, would it > >> be more advisable to blow away and rebuild it ALL or can I simply delete > >> the hypervisor hosts and rebuild them? > >> > >> Thoughts? > >> > >> Thanks! > >> ~Stack~ > > > > As long as you don't destroy the data on your data domain you can rebuild > > the engine and hosts and then import the existing data domain without too > > many issues. I have destroyed my engine database many times, and I am > > still using the same VMs from the same data domain. > > > > Here is what I do when I mess up my database to the point I have to make a > > new one: > > > > 1. Recreate the engine and database, so that I have basically have an > > empty > > engine with no hosts and VMs. > > 1.1 (Optional) make a new DC that is not default. and add a cluster. > > 2. Add my hosts (I only have 2 so that is quick and easy). > > 3. Add a throw away data domain (This is needed to get the DC up so I can > > import the existing data domain). > > 4. Import (NOT new, import) the existing data domain. > > 5. Do to Storage->Storage Domains->VM import and import the VMs I want. > > 6. Same for templates and disks if needed. > > 7. After you have imported the VMs/Templates/Disks you can detach and > > remove the throw away data domain and the one you imported becomes the > > master domain. > > > > Note if you want to move VMs between your play ground and more serious > > system you can simply detach your data domain from the play ground, then > > attach it to the serious engine (so you have 2 engines, one play ground > > and one serious) and import which VMs you want. That way you won't run > > into issues with configuring networks and stuff like you experienced. > > Thanks for that help. I did that and everything looks fantastic...except > I can't migrate VM's. :-/ > > It just sits there and in the log files there is the below messages > repeating. It's like it doesn't care for the fact that this was an > imported domain or something. > > Thoughts? > > Thanks! > ~Stack~ > Don't know too much about the VDSM side of things. But obviously its looking for a storage domain it can't find anymore. You can try restarting VDSM (won't affect running VMs) and see if rescans the available storage domains and won't try to access it during the migration of the VMs. Other than that I don't know. > > 2018-04-13 16:58:59,920-0500 ERROR (monitor/232975a) [storage.Monitor] > Setting up monitor for 232975ad-1771-4b6b-afda-958f7b745867 failed > (monitor:329) > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line > 326, in _setupLoop > self._setupMonitor() > File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line > 348, in _setupMonitor > self._produceDomain() > File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 158, in > wrapper > value = meth(self, *a, **kw) > File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line > 366, in _produceDomain > self.domain = sdCache.produce(self.sdUUID) > File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, > in produce > domain.getRealDomain() > File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, > in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, > in _realProduce > domain = self._findDomain(sdUUID) > File
Re: [ovirt-users] I broke my ovirt real good....
On 04/13/2018 07:16 AM, Alexander Wels wrote: > On Thursday, April 12, 2018 6:26:07 PM EDT ~Stack~ wrote: >> Greetings, >> >> So I did a over-confident-admin-makes-rookie-mistake. I changed a bunch >> of things all back-to-back and thus don't actually know what broke. :-D >> >> The only two real "big" changes were: >> * Upgrade from 4.2.1 to 4.2.2 >> * change my ovirtmgmt network >> >> The update I followed the upgrade procedures and I thought it all went >> pretty well. Because I am moving it from a testing into what I hope will >> be a more heavily used environment, I changed my ovirtmgmt network from >> 192.168.100.0/24 to 192.168.101.0/24 via the web-gui. >> >> That was a touch tricker than just a change as I had to poke the >> management engine host to be reachable on both network for a while, then >> it just seemed OK. >> >> What's happening is: >> * I can no longer migrate a vm from one host to the other. >> * If I try to do a "reinstall" it dies. >> * There is some serious network lag between my hosts on a 10Gb network. >> * I've got all kinds of python2.4 failures in my vdsm and mom logs. >> >> Those are least the biggies. >> >> So while I was planning on moving this to a more active use case, right >> now - it is all still my play ground. I would REALLY hate to lose the >> VM's but everything else can go and be rebuilt. >> >> Given that I've somehow really broke this system pretty good, would it >> be more advisable to blow away and rebuild it ALL or can I simply delete >> the hypervisor hosts and rebuild them? >> >> Thoughts? >> >> Thanks! >> ~Stack~ > > As long as you don't destroy the data on your data domain you can rebuild the > engine and hosts and then import the existing data domain without too many > issues. I have destroyed my engine database many times, and I am still using > the same VMs from the same data domain. > > Here is what I do when I mess up my database to the point I have to make a > new > one: > > 1. Recreate the engine and database, so that I have basically have an empty > engine with no hosts and VMs. > 1.1 (Optional) make a new DC that is not default. and add a cluster. > 2. Add my hosts (I only have 2 so that is quick and easy). > 3. Add a throw away data domain (This is needed to get the DC up so I can > import the existing data domain). > 4. Import (NOT new, import) the existing data domain. > 5. Do to Storage->Storage Domains->VM import and import the VMs I want. > 6. Same for templates and disks if needed. > 7. After you have imported the VMs/Templates/Disks you can detach and remove > the throw away data domain and the one you imported becomes the master domain. > > Note if you want to move VMs between your play ground and more serious system > you can simply detach your data domain from the play ground, then attach it > to > the serious engine (so you have 2 engines, one play ground and one serious) > and import which VMs you want. That way you won't run into issues with > configuring networks and stuff like you experienced. > Thanks for that help. I did that and everything looks fantastic...except I can't migrate VM's. :-/ It just sits there and in the log files there is the below messages repeating. It's like it doesn't care for the fact that this was an imported domain or something. Thoughts? Thanks! ~Stack~ 2018-04-13 16:58:59,920-0500 ERROR (monitor/232975a) [storage.Monitor] Setting up monitor for 232975ad-1771-4b6b-afda-958f7b745867 failed (monitor:329) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 326, in _setupLoop self._setupMonitor() File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 348, in _setupMonitor self._produceDomain() File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 158, in wrapper value = meth(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/monitor.py", line 366, in _produceDomain self.domain = sdCache.produce(self.sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in produce domain.getRealDomain() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in _findDomain return findMethod(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 176, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'232975ad-1771-4b6b-afda-958f7b745867',) 2018-04-13 16:58:59,923-0500 ERROR (monitor/bc975a4) [storage.Monitor] Setting up monitor for bc975a4c-6c38-4248-b3f7-a26945f23693 failed (monitor:329) Traceback (most recent call last): File
Re: [ovirt-users] I broke my ovirt real good....
On Thursday, April 12, 2018 6:26:07 PM EDT ~Stack~ wrote: > Greetings, > > So I did a over-confident-admin-makes-rookie-mistake. I changed a bunch > of things all back-to-back and thus don't actually know what broke. :-D > > The only two real "big" changes were: > * Upgrade from 4.2.1 to 4.2.2 > * change my ovirtmgmt network > > The update I followed the upgrade procedures and I thought it all went > pretty well. Because I am moving it from a testing into what I hope will > be a more heavily used environment, I changed my ovirtmgmt network from > 192.168.100.0/24 to 192.168.101.0/24 via the web-gui. > > That was a touch tricker than just a change as I had to poke the > management engine host to be reachable on both network for a while, then > it just seemed OK. > > What's happening is: > * I can no longer migrate a vm from one host to the other. > * If I try to do a "reinstall" it dies. > * There is some serious network lag between my hosts on a 10Gb network. > * I've got all kinds of python2.4 failures in my vdsm and mom logs. > > Those are least the biggies. > > So while I was planning on moving this to a more active use case, right > now - it is all still my play ground. I would REALLY hate to lose the > VM's but everything else can go and be rebuilt. > > Given that I've somehow really broke this system pretty good, would it > be more advisable to blow away and rebuild it ALL or can I simply delete > the hypervisor hosts and rebuild them? > > Thoughts? > > Thanks! > ~Stack~ As long as you don't destroy the data on your data domain you can rebuild the engine and hosts and then import the existing data domain without too many issues. I have destroyed my engine database many times, and I am still using the same VMs from the same data domain. Here is what I do when I mess up my database to the point I have to make a new one: 1. Recreate the engine and database, so that I have basically have an empty engine with no hosts and VMs. 1.1 (Optional) make a new DC that is not default. and add a cluster. 2. Add my hosts (I only have 2 so that is quick and easy). 3. Add a throw away data domain (This is needed to get the DC up so I can import the existing data domain). 4. Import (NOT new, import) the existing data domain. 5. Do to Storage->Storage Domains->VM import and import the VMs I want. 6. Same for templates and disks if needed. 7. After you have imported the VMs/Templates/Disks you can detach and remove the throw away data domain and the one you imported becomes the master domain. Note if you want to move VMs between your play ground and more serious system you can simply detach your data domain from the play ground, then attach it to the serious engine (so you have 2 engines, one play ground and one serious) and import which VMs you want. That way you won't run into issues with configuring networks and stuff like you experienced. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] I broke my ovirt real good....
On 04/13/2018 03:02 AM, Michael Mortensen (MCMR) wrote: > Hi Stack, > > Do you use FQDN? Did you perhaps hit this one > https://www.ovirt.org/blog/2016/05/modify-ifcfg-files/ ? The discussion in > this bug report may be of assistance in that case: > https://bugzilla.redhat.com/show_bug.cgi?id=1252534 That looks very interesting! I will investigate that. > If you've stored the VM disks and templates and whatnot on a network share > like NFS, you should be able to start all over and import your old (current) > storage domains and start using your templates etc. I am currently using NFS. I will see how this networking issue you pointed me to works out, then maybe rebuild. Thank you for the assistance! ~Stack~ signature.asc Description: OpenPGP digital signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] I broke my ovirt real good....
Hi Stack, Do you use FQDN? Did you perhaps hit this one https://www.ovirt.org/blog/2016/05/modify-ifcfg-files/ ? The discussion in this bug report may be of assistance in that case: https://bugzilla.redhat.com/show_bug.cgi?id=1252534 If you've stored the VM disks and templates and whatnot on a network share like NFS, you should be able to start all over and import your old (current) storage domains and start using your templates etc. Best wishes // Mike -Original Message- From: users-boun...@ovirt.org [mailto:users-boun...@ovirt.org] On Behalf Of ~Stack~ Sent: 13. april 2018 00:26 To: users <users@ovirt.org> Subject: [ovirt-users] I broke my ovirt real good Greetings, So I did a over-confident-admin-makes-rookie-mistake. I changed a bunch of things all back-to-back and thus don't actually know what broke. :-D The only two real "big" changes were: * Upgrade from 4.2.1 to 4.2.2 * change my ovirtmgmt network The update I followed the upgrade procedures and I thought it all went pretty well. Because I am moving it from a testing into what I hope will be a more heavily used environment, I changed my ovirtmgmt network from 192.168.100.0/24 to 192.168.101.0/24 via the web-gui. That was a touch tricker than just a change as I had to poke the management engine host to be reachable on both network for a while, then it just seemed OK. What's happening is: * I can no longer migrate a vm from one host to the other. * If I try to do a "reinstall" it dies. * There is some serious network lag between my hosts on a 10Gb network. * I've got all kinds of python2.4 failures in my vdsm and mom logs. Those are least the biggies. So while I was planning on moving this to a more active use case, right now - it is all still my play ground. I would REALLY hate to lose the VM's but everything else can go and be rebuilt. Given that I've somehow really broke this system pretty good, would it be more advisable to blow away and rebuild it ALL or can I simply delete the hypervisor hosts and rebuild them? Thoughts? Thanks! ~Stack~ ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] I broke my ovirt real good....
Greetings, So I did a over-confident-admin-makes-rookie-mistake. I changed a bunch of things all back-to-back and thus don't actually know what broke. :-D The only two real "big" changes were: * Upgrade from 4.2.1 to 4.2.2 * change my ovirtmgmt network The update I followed the upgrade procedures and I thought it all went pretty well. Because I am moving it from a testing into what I hope will be a more heavily used environment, I changed my ovirtmgmt network from 192.168.100.0/24 to 192.168.101.0/24 via the web-gui. That was a touch tricker than just a change as I had to poke the management engine host to be reachable on both network for a while, then it just seemed OK. What's happening is: * I can no longer migrate a vm from one host to the other. * If I try to do a "reinstall" it dies. * There is some serious network lag between my hosts on a 10Gb network. * I've got all kinds of python2.4 failures in my vdsm and mom logs. Those are least the biggies. So while I was planning on moving this to a more active use case, right now - it is all still my play ground. I would REALLY hate to lose the VM's but everything else can go and be rebuilt. Given that I've somehow really broke this system pretty good, would it be more advisable to blow away and rebuild it ALL or can I simply delete the hypervisor hosts and rebuild them? Thoughts? Thanks! ~Stack~ signature.asc Description: OpenPGP digital signature ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
