Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network Outage

2017-07-19 Thread Anthony . Fillmore
All services active and running except the vdsm-network.service which last 
entry is “activating”:

[root@t0894bmh1001 vdsm.conf.d]# systemctl status -l vdsm-network.service -l
● vdsm-network.service - Virtual Desktop Server Manager network restoration
   Loaded: loaded (/usr/lib/systemd/system/vdsm-network.service; enabled; 
vendor preset: enabled)
   Active: activating (start) since Tue 2017-07-18 10:42:57 CDT; 23h ago
  Process: 8216 ExecStartPre=/usr/bin/vdsm-tool --vvverbose --append 
--logfile=/var/log/vdsm/upgrade.log upgrade-unified-persistence (code=exited, 
status=0/SUCCESS)
Main PID: 8231 (vdsm-tool)
   CGroup: /system.slice/vdsm-network.service
   ├─8231 /usr/bin/python /usr/bin/vdsm-tool restore-nets
   └─8240 /usr/bin/python /usr/share/vdsm/vdsm-restore-net-config
From: Alan Griffiths [mailto:apgriffith...@gmail.com]
Sent: Wednesday, July 19, 2017 10:13 AM
To: Anthony.Fillmore 
Cc: Pavel Gashev ; users@ovirt.org; Brandon.Markgraf 
; Sandeep.Mendiratta 

Subject: Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network 
Outage

Looking at vdsmd.service on one of my 4.0 hosts.

Requires=multipathd.service libvirtd.service time-sync.target \
 iscsid.service rpcbind.service supervdsmd.service sanlock.service \
 vdsm-network.service

Are all these services present and running?


On 19 July 2017 at 16:05, Anthony.Fillmore 
> wrote:
Are the vdsm.conf or mom.conf file in /etc/vdsm of note in this situation?

From: Anthony.Fillmore
Sent: Wednesday, July 19, 2017 9:57 AM
To: 'Alan Griffiths' >
Cc: Pavel Gashev >; 
users@ovirt.org; Brandon.Markgraf 
>; 
Sandeep.Mendiratta 
>
Subject: RE: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network 
Outage

[boxname ~]# systemctl | grep -i dead
mom-vdsm.service

   start MOM instance configured for VDSM purposes
vdsmd.service   

   start Virtual Desktop Server Manager


[ boxname ~]# systemctl | grep -i exited
blk-availability.service

 Availability of block devices
iptables.service

 IPv4 firewall with iptables
kdump.service   

 Crash recovery kernel arming
kmod-static-nodes.service   

 Create list of required static device nodes for the 
current kernel
lvm2-monitor.service

 Monitoring of LVM2 mirrors, snapshots etc. using dmeventd 
or progress polling
lvm2-pvscan@253:3.service 

   LVM2 PV scan on device 
253:3
lvm2-pvscan@253:4.service 

   LVM2 PV scan on device 
253:4
lvm2-pvscan@8:3.service 

 LVM2 PV scan on device 8:3
network.service 

 LSB: Bring up/down networking
openvswitch-nonetwork.service   

 Open vSwitch Internal Unit

Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network Outage

2017-07-19 Thread Anthony . Fillmore
Are the vdsm.conf or mom.conf file in /etc/vdsm of note in this situation?

From: Anthony.Fillmore
Sent: Wednesday, July 19, 2017 9:57 AM
To: 'Alan Griffiths' 
Cc: Pavel Gashev ; users@ovirt.org; Brandon.Markgraf 
; Sandeep.Mendiratta 

Subject: RE: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network 
Outage

[boxname ~]# systemctl | grep -i dead
mom-vdsm.service

   start MOM instance configured for VDSM purposes
vdsmd.service   

   start Virtual Desktop Server Manager


[ boxname ~]# systemctl | grep -i exited
blk-availability.service

 Availability of block devices
iptables.service

 IPv4 firewall with iptables
kdump.service   

 Crash recovery kernel arming
kmod-static-nodes.service   

 Create list of required static device nodes for the 
current kernel
lvm2-monitor.service

 Monitoring of LVM2 mirrors, snapshots etc. using dmeventd 
or progress polling
lvm2-pvscan@253:3.service 

   LVM2 PV scan on device 
253:3
lvm2-pvscan@253:4.service 

   LVM2 PV scan on device 
253:4
lvm2-pvscan@8:3.service 

 LVM2 PV scan on device 8:3
network.service 

 LSB: Bring up/down networking
openvswitch-nonetwork.service   

 Open vSwitch Internal Unit
openvswitch.service 

 Open vSwitch
rhel-dmesg.service  

 Dump dmesg to /var/log/dmesg
rhel-import-state.service   

 Import network configuration from initramfs
rhel-readonly.service   

 Configure read-only root support
systemd-journal-flush.service   

 Flush Journal to Persistent Storage
systemd-modules-load.service

 Load Kernel Modules
systemd-random-seed.service 

 Load/Save Random Seed
systemd-readahead-collect.service   

 Collect Read-Ahead Data
systemd-readahead-replay.service 

Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network Outage

2017-07-19 Thread Anthony . Fillmore
[boxname ~]# systemctl | grep -i dead
mom-vdsm.service

   start MOM instance configured for VDSM purposes
vdsmd.service   

   start Virtual Desktop Server Manager


[ boxname ~]# systemctl | grep -i exited
blk-availability.service

 Availability of block devices
iptables.service

 IPv4 firewall with iptables
kdump.service   

 Crash recovery kernel arming
kmod-static-nodes.service   

 Create list of required static device nodes for the 
current kernel
lvm2-monitor.service

 Monitoring of LVM2 mirrors, snapshots etc. using dmeventd 
or progress polling
lvm2-pvscan@253:3.service   

 LVM2 PV scan on device 253:3
lvm2-pvscan@253:4.service   

 LVM2 PV scan on device 253:4
lvm2-pvscan@8:3.service 

 LVM2 PV scan on device 8:3
network.service 

 LSB: Bring up/down networking
openvswitch-nonetwork.service   

 Open vSwitch Internal Unit
openvswitch.service 

 Open vSwitch
rhel-dmesg.service  

 Dump dmesg to /var/log/dmesg
rhel-import-state.service   

 Import network configuration from initramfs
rhel-readonly.service   

 Configure read-only root support
systemd-journal-flush.service   

 Flush Journal to Persistent Storage
systemd-modules-load.service

 Load Kernel Modules
systemd-random-seed.service 

 Load/Save Random Seed
systemd-readahead-collect.service   

 Collect Read-Ahead Data
systemd-readahead-replay.service

 Replay Read-Ahead Data
systemd-remount-fs.service  

 Remount Root and Kernel File Systems
systemd-sysctl.service  


Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network Outage

2017-07-19 Thread Anthony . Fillmore
Hey Alan,

Rpcbind is running on my box, looks like no issue there.  Any other ideas on 
what could be keeping vdsmd dead?  I even uninstalled all Ovirt related 
components from the host and went for a reinstall of the host through Ovirt 
(just short of actually fully removing the host from ovirt and re-adding, which 
I want to avoid) and the reinstall ends up timing out when it attempts to start 
VDSM (checking logs can see the service is dead when it gets here).

Thanks,
Tony

From: Alan Griffiths [mailto:apgriffith...@gmail.com]
Sent: Wednesday, July 19, 2017 4:14 AM
To: Anthony.Fillmore 
Cc: Pavel Gashev ; users@ovirt.org; Brandon.Markgraf 
; Sandeep.Mendiratta 

Subject: Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network 
Outage

Is rpcbind running? This is a dependency for vdsmd.

I've seen issues where rpcbind will not start on boot if IPv6 is disabled. The 
solution for me was to rebuild the initramfs, aka "dracut -f"

On 18 July 2017 at 18:13, Anthony.Fillmore 
> wrote:
[boxname ~]# systemctl status -l vdsm-network
● vdsm-network.service - Virtual Desktop Server Manager network restoration
   Loaded: loaded (/usr/lib/systemd/system/vdsm-network.service; enabled; 
vendor preset: enabled)
   Active: activating (start) since Tue 2017-07-18 10:42:57 CDT; 1h 29min ago
  Process: 8216 ExecStartPre=/usr/bin/vdsm-tool --vvverbose --append 
--logfile=/var/log/vdsm/upgrade.log upgrade-unified-persistence (code=exited, 
status=0/SUCCESS)
Main PID: 8231 (vdsm-tool)
   CGroup: /system.slice/vdsm-network.service
   ├─8231 /usr/bin/python /usr/bin/vdsm-tool restore-nets
   └─8240 /usr/bin/python /usr/share/vdsm/vdsm-restore-net-config

Jul 18 10:42:57 
t0894bmh1001.stores.target.com 
systemd[1]: Starting Virtual Desktop Server Manager network restoration...

Thanks,
Tony
From: Pavel Gashev [mailto:p...@acronis.com]
Sent: Tuesday, July 18, 2017 11:17 AM
To: Anthony.Fillmore 
>; 
users@ovirt.org
Cc: Brandon.Markgraf 
>; 
Sandeep.Mendiratta 
>
Subject: [EXTERNAL] Re: [ovirt-users] Host stuck unresponsive after Network 
Outage

Anthony,

Output of “systemctl status -l vdsm-network” would help.


From: > on behalf of 
"Anthony.Fillmore" 
>
Date: Tuesday, 18 July 2017 at 18:13
To: "users@ovirt.org" 
>
Cc: "Brandon.Markgraf" 
>, 
"Sandeep.Mendiratta" 
>
Subject: [ovirt-users] Host stuck unresponsive after Network Outage

Hey Ovirt Users and Team,

I have a host that I am unable to recover post a network outage.  The host is 
stuck in unresponsive mode, even though the host is on the network, able to SSH 
and seems to be healthy.  I’ve tried several things to recover the host in 
Ovirt, but have had no success so far.  I’d like to reach out to the community 
before blowing away and rebuilding the host.

Environment: I have an Ovengine server with about 26 Datacenters, with 2 to 3 
hosts per Datacenter.  My Ovengine server is hosted centrally, with my hosts 
being bare-metal and distributed throughout my environment.Ovengine is 
version 4.0.6.

What I’ve tried: put into maintenance mode, rebooted the host.  Confirmed host 
was rebooted and tried to active, goes back to unresponsive.   Attempted a 
reinstall, which fails.

Checking from the host perspective, I can see the following problems:

[boxname~]# systemctl status vdsmd
● vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor 
preset: enabled)
   Active: inactive (dead)

Jul 14 12:34:28 boxname systemd[1]: Dependency failed for Virtual Desktop 
Server Manager.
Jul 14 12:34:28 boxname systemd[1]: Job vdsmd.service/start failed with result 
'dependency'.

Going a bit deeper, the results of journalctl –xe:

[root@boxname ~]# journalctl -xe
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun shutting down.
Jul 18 09:07:31 boxname systemd[1]: Stopped Virtualization daemon.
-- Subject: Unit libvirtd.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has finished shutting down.
Jul 18 09:07:31 boxname systemd[1]: Reloading.

Re: [ovirt-users] [EXTERNAL] Re: Host stuck unresponsive after Network Outage

2017-07-18 Thread Anthony . Fillmore
[boxname ~]# systemctl status -l vdsm-network
● vdsm-network.service - Virtual Desktop Server Manager network restoration
   Loaded: loaded (/usr/lib/systemd/system/vdsm-network.service; enabled; 
vendor preset: enabled)
   Active: activating (start) since Tue 2017-07-18 10:42:57 CDT; 1h 29min ago
  Process: 8216 ExecStartPre=/usr/bin/vdsm-tool --vvverbose --append 
--logfile=/var/log/vdsm/upgrade.log upgrade-unified-persistence (code=exited, 
status=0/SUCCESS)
Main PID: 8231 (vdsm-tool)
   CGroup: /system.slice/vdsm-network.service
   ├─8231 /usr/bin/python /usr/bin/vdsm-tool restore-nets
   └─8240 /usr/bin/python /usr/share/vdsm/vdsm-restore-net-config

Jul 18 10:42:57 t0894bmh1001.stores.target.com systemd[1]: Starting Virtual 
Desktop Server Manager network restoration...

Thanks,
Tony
From: Pavel Gashev [mailto:p...@acronis.com]
Sent: Tuesday, July 18, 2017 11:17 AM
To: Anthony.Fillmore ; users@ovirt.org
Cc: Brandon.Markgraf ; Sandeep.Mendiratta 

Subject: [EXTERNAL] Re: [ovirt-users] Host stuck unresponsive after Network 
Outage

Anthony,

Output of “systemctl status -l vdsm-network” would help.


From: > on behalf of 
"Anthony.Fillmore" 
>
Date: Tuesday, 18 July 2017 at 18:13
To: "users@ovirt.org" 
>
Cc: "Brandon.Markgraf" 
>, 
"Sandeep.Mendiratta" 
>
Subject: [ovirt-users] Host stuck unresponsive after Network Outage

Hey Ovirt Users and Team,

I have a host that I am unable to recover post a network outage.  The host is 
stuck in unresponsive mode, even though the host is on the network, able to SSH 
and seems to be healthy.  I’ve tried several things to recover the host in 
Ovirt, but have had no success so far.  I’d like to reach out to the community 
before blowing away and rebuilding the host.

Environment: I have an Ovengine server with about 26 Datacenters, with 2 to 3 
hosts per Datacenter.  My Ovengine server is hosted centrally, with my hosts 
being bare-metal and distributed throughout my environment.Ovengine is 
version 4.0.6.

What I’ve tried: put into maintenance mode, rebooted the host.  Confirmed host 
was rebooted and tried to active, goes back to unresponsive.   Attempted a 
reinstall, which fails.

Checking from the host perspective, I can see the following problems:

[boxname~]# systemctl status vdsmd
● vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor 
preset: enabled)
   Active: inactive (dead)

Jul 14 12:34:28 boxname systemd[1]: Dependency failed for Virtual Desktop 
Server Manager.
Jul 14 12:34:28 boxname systemd[1]: Job vdsmd.service/start failed with result 
'dependency'.

Going a bit deeper, the results of journalctl –xe:

[root@boxname ~]# journalctl -xe
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun shutting down.
Jul 18 09:07:31 boxname systemd[1]: Stopped Virtualization daemon.
-- Subject: Unit libvirtd.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has finished shutting down.
Jul 18 09:07:31 boxname systemd[1]: Reloading.
Jul 18 09:07:31 boxname systemd[1]: Binding to IPv6 address not available since 
kernel does not support IPv6.
Jul 18 09:07:31 boxname systemd[1]: [/usr/lib/systemd/system/rpcbind.socket:6] 
Failed to parse address value, ignoring: [::
Jul 18 09:07:31 boxname systemd[1]: Started Auxiliary vdsm service for running 
helper functions as root.
-- Subject: Unit supervdsmd.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit supervdsmd.service has finished starting up.
--
-- The start-up result is done.
Jul 18 09:07:31 boxname systemd[1]: Starting Auxiliary vdsm service for running 
helper functions as root...
-- Subject: Unit supervdsmd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit supervdsmd.service has begun starting up.
Jul 18 09:07:31 boxname systemd[1]: Starting Virtualization daemon...
-- Subject: Unit libvirtd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun starting up.
Jul 18 09:07:32 boxname systemd[1]: Started Virtualization daemon.
-- Subject: Unit libvirtd.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- 

[ovirt-users] Host stuck unresponsive after Network Outage

2017-07-18 Thread Anthony . Fillmore
Hey Ovirt Users and Team,

I have a host that I am unable to recover post a network outage.  The host is 
stuck in unresponsive mode, even though the host is on the network, able to SSH 
and seems to be healthy.  I’ve tried several things to recover the host in 
Ovirt, but have had no success so far.  I’d like to reach out to the community 
before blowing away and rebuilding the host.

Environment: I have an Ovengine server with about 26 Datacenters, with 2 to 3 
hosts per Datacenter.  My Ovengine server is hosted centrally, with my hosts 
being bare-metal and distributed throughout my environment.Ovengine is 
version 4.0.6.

What I’ve tried: put into maintenance mode, rebooted the host.  Confirmed host 
was rebooted and tried to active, goes back to unresponsive.   Attempted a 
reinstall, which fails.

Checking from the host perspective, I can see the following problems:

[boxname~]# systemctl status vdsmd
● vdsmd.service - Virtual Desktop Server Manager
   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor 
preset: enabled)
   Active: inactive (dead)

Jul 14 12:34:28 boxname systemd[1]: Dependency failed for Virtual Desktop 
Server Manager.
Jul 14 12:34:28 boxname systemd[1]: Job vdsmd.service/start failed with result 
'dependency'.

Going a bit deeper, the results of journalctl -xe:

[root@boxname ~]# journalctl -xe
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun shutting down.
Jul 18 09:07:31 boxname systemd[1]: Stopped Virtualization daemon.
-- Subject: Unit libvirtd.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has finished shutting down.
Jul 18 09:07:31 boxname systemd[1]: Reloading.
Jul 18 09:07:31 boxname systemd[1]: Binding to IPv6 address not available since 
kernel does not support IPv6.
Jul 18 09:07:31 boxname systemd[1]: [/usr/lib/systemd/system/rpcbind.socket:6] 
Failed to parse address value, ignoring: [::
Jul 18 09:07:31 boxname systemd[1]: Started Auxiliary vdsm service for running 
helper functions as root.
-- Subject: Unit supervdsmd.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit supervdsmd.service has finished starting up.
--
-- The start-up result is done.
Jul 18 09:07:31 boxname systemd[1]: Starting Auxiliary vdsm service for running 
helper functions as root...
-- Subject: Unit supervdsmd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit supervdsmd.service has begun starting up.
Jul 18 09:07:31 boxname systemd[1]: Starting Virtualization daemon...
-- Subject: Unit libvirtd.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has begun starting up.
Jul 18 09:07:32 boxname systemd[1]: Started Virtualization daemon.
-- Subject: Unit libvirtd.service has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit libvirtd.service has finished starting up.
--
-- The start-up result is done.
Jul 18 09:07:32 boxname systemd[1]: Starting Virtual Desktop Server Manager 
network restoration...
-- Subject: Unit vdsm-network.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit vdsm-network.service has begun starting up.
lines 2751-2797/2797 (END)

Does the community have suggestions on what can be done next to recover this 
host within Ovirt?  I can provide additional log dumps as needed, please inform 
with what you need to assist further.

Thank you,
Tony

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] [EXTERNAL] Re: oVirt: Host Removal Question

2017-06-07 Thread Anthony . Fillmore
Ok - I was able to get taskcleaner.sh working.  I stopped ovirt-engine service, 
ran the script and cleared out active and zombie tasks.  Restarted the engine, 
put the host into maintenance mode and was FINALLY able to remove it.

Thanks a ton for your assistance, it is truly appreciated.

Thanks,
Tony

-Original Message-
From: Alexander Wels [mailto:aw...@redhat.com] 
Sent: Wednesday, June 7, 2017 10:00 AM
To: Anthony.Fillmore <anthony.fillm...@target.com>
Cc: users@ovirt.org; Brandon.Markgraf <brandon.markg...@target.com>
Subject: Re: [EXTERNAL] Re: [ovirt-users] oVirt: Host Removal Question

On Wednesday, June 7, 2017 10:50:56 AM EDT Anthony. Fillmore wrote:
> Hey Alexander,
> 
> The query pieces you gave me allowed me to successfully set the host in
> maintenance mode.   When I go to remove the host, I get the following
> error:  'Cannot remove host.  Related operation is currently in progress.' 
>  The host is not part of a datacenter, but is part of a cluster.  Any 
> other thoughts as to how to get this host removed?
> 
> Thanks,
> Tony
> 

If there is a task still running (well the database thinks a task is running as 
obviously the host is gone), you maybe be able to use the taskcleaner.sh 
utility to clear out the running task.

taskcleaner.sh -h will give you all the options.

it should be in /share/ovirt-engine/setup/dbutils/taskcleaner.sh

> -Original Message-
> From: Alexander Wels [mailto:aw...@redhat.com]
> Sent: Tuesday, June 6, 2017 10:36 AM
> To: Anthony.Fillmore <anthony.fillm...@target.com>
> Cc: users@ovirt.org; Brandon.Markgraf <brandon.markg...@target.com>
> Subject: Re: [EXTERNAL] Re: [ovirt-users] oVirt: Host Removal Question
> 
> On Tuesday, June 6, 2017 11:18:35 AM EDT Anthony. Fillmore wrote:
> > Hey Alexander,
> > 
> > I did those exact steps roughly two days ago...the host is still 
> > stuck in preparing for maintenance mode.  Confirming the host has 
> > been rebooted seems to have no effect.
> > 
> > Any other ideas?  Some way to hit the Ovirt Database and manipulate 
> > the value for what state the host is in?  Remove the host entirely 
> > from the back-end database?
> 
> That should have worked, but if all else fails you can always set the 
> host in maintaince in the database doing this in the engine database:
> 
> update vds_dynamic set status=2 where vds_id = 
> 
> you can figure out the id of the host by doing this
> 
> select vds_id from vds_static where vds_name=''
> 
> If you are feeling brave you can combine the two into a single 
> statement, but I would be sure you have the right host before you do that.
> 
> Then once you have updated the status, you can refresh the webadmin 
> and the host should be in maintenaince mode and you should be able to 
> remove it normally. I would not attempt more manual manipulation of 
> the database than setting the status to 2 (which is maintenaince).
> > -Original Message-
> > From: Alexander Wels [mailto:aw...@redhat.com]
> > Sent: Tuesday, June 6, 2017 9:29 AM
> > To: users@ovirt.org
> > Cc: Brandon.Markgraf <brandon.markg...@target.com>; Anthony.Fillmore 
> > <anthony.fillm...@target.com> Subject: [EXTERNAL] Re: [ovirt-users] oVirt:
> > Host Removal Question
> > 
> > On Monday, June 5, 2017 4:10:54 PM EDT Brandon. Markgraf wrote:
> > > Hello oVirt Users,
> > > We have a cluster that has been decommissioned and we are trying 
> > > to remove the hosts from the oVirt Engine but one host is stuck in 
> > > "Preparing for Maintenance".  It's preventing me from removing 
> > > that host and the associated cluster.
> > > 
> > > The physical server has been shut down and is no longer accessible.
> > > What's the best way to take this server in oVirt Engine out of 
> > > this status so we can remove the host and the cluster from oVirt Engine?
> > > 
> > > Thanks so much in advance and please let me know if there are any 
> > > questions.
> > > 
> > > Brandon Markgraf | *Target |
> > 
> > IIRC when I had that issue (I removed the host physically before 
> > removing it from oVirt) I had to do the following:
> > 
> > - Right click in the grid on the host, and select confirm host has 
> > been rebooted (this is basically you telling oVirt the host has been 
> > fenced). - After a while the host should end up as down (or unknown 
> > state). - You should be able to select maintaince at that point, 
> > this might take a while (timeouts etc). - This should put the host 
> > in maintaince and you should be able to remove it then.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] [EXTERNAL] Re: oVirt: Host Removal Question

2017-06-07 Thread Anthony . Fillmore
Awesome, this is exactly what I was looking for!  Thank you!  

One last thing - Is there a data dictionary available somewhere for the Ovirt 
Postgresql DB tables and views?  Some way I can view the full schema and 
understand what data is located where?  Documentation online seems very scarce 
in this regard.

Thanks again, I truly appreciate the help!

Tony

-Original Message-
From: Alexander Wels [mailto:aw...@redhat.com] 
Sent: Tuesday, June 6, 2017 10:36 AM
To: Anthony.Fillmore <anthony.fillm...@target.com>
Cc: users@ovirt.org; Brandon.Markgraf <brandon.markg...@target.com>
Subject: Re: [EXTERNAL] Re: [ovirt-users] oVirt: Host Removal Question

On Tuesday, June 6, 2017 11:18:35 AM EDT Anthony. Fillmore wrote:
> Hey Alexander,
> 
> I did those exact steps roughly two days ago...the host is still stuck 
> in preparing for maintenance mode.  Confirming the host has been 
> rebooted seems to have no effect.
> 
> Any other ideas?  Some way to hit the Ovirt Database and manipulate 
> the value for what state the host is in?  Remove the host entirely 
> from the back-end database?
> 

That should have worked, but if all else fails you can always set the host in 
maintaince in the database doing this in the engine database:

update vds_dynamic set status=2 where vds_id = 

you can figure out the id of the host by doing this

select vds_id from vds_static where vds_name=''

If you are feeling brave you can combine the two into a single statement, but I 
would be sure you have the right host before you do that.

Then once you have updated the status, you can refresh the webadmin and the 
host should be in maintenaince mode and you should be able to remove it 
normally. I would not attempt more manual manipulation of the database than 
setting the status to 2 (which is maintenaince).

> -Original Message-
> From: Alexander Wels [mailto:aw...@redhat.com]
> Sent: Tuesday, June 6, 2017 9:29 AM
> To: users@ovirt.org
> Cc: Brandon.Markgraf <brandon.markg...@target.com>; Anthony.Fillmore 
> <anthony.fillm...@target.com> Subject: [EXTERNAL] Re: [ovirt-users] oVirt:
> Host Removal Question
> On Monday, June 5, 2017 4:10:54 PM EDT Brandon. Markgraf wrote:
> > Hello oVirt Users,
> > We have a cluster that has been decommissioned and we are trying to 
> > remove the hosts from the oVirt Engine but one host is stuck in 
> > "Preparing for Maintenance".  It's preventing me from removing that 
> > host and the associated cluster.
> > 
> > The physical server has been shut down and is no longer accessible.
> > What's the best way to take this server in oVirt Engine out of this 
> > status so we can remove the host and the cluster from oVirt Engine?
> > 
> > Thanks so much in advance and please let me know if there are any 
> > questions.
> > 
> > Brandon Markgraf | *Target |
> 
> IIRC when I had that issue (I removed the host physically before 
> removing it from oVirt) I had to do the following:
> 
> - Right click in the grid on the host, and select confirm host has 
> been rebooted (this is basically you telling oVirt the host has been 
> fenced). - After a while the host should end up as down (or unknown 
> state). - You should be able to select maintaince at that point, this 
> might take a while (timeouts etc). - This should put the host in 
> maintaince and you should be able to remove it then.



___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] [EXTERNAL] Re: oVirt: Host Removal Question

2017-06-07 Thread Anthony . Fillmore
Hey Alexander,

I did those exact steps roughly two days ago...the host is still stuck in 
preparing for maintenance mode.  Confirming the host has been rebooted seems to 
have no effect.

Any other ideas?  Some way to hit the Ovirt Database and manipulate the value 
for what state the host is in?  Remove the host entirely from the back-end 
database?

-Original Message-
From: Alexander Wels [mailto:aw...@redhat.com] 
Sent: Tuesday, June 6, 2017 9:29 AM
To: users@ovirt.org
Cc: Brandon.Markgraf ; Anthony.Fillmore 

Subject: [EXTERNAL] Re: [ovirt-users] oVirt: Host Removal Question

On Monday, June 5, 2017 4:10:54 PM EDT Brandon. Markgraf wrote:
> Hello oVirt Users,
> We have a cluster that has been decommissioned and we are trying to 
> remove the hosts from the oVirt Engine but one host is stuck in 
> "Preparing for Maintenance".  It's preventing me from removing that 
> host and the associated cluster.
> 
> The physical server has been shut down and is no longer accessible.  
> What's the best way to take this server in oVirt Engine out of this 
> status so we can remove the host and the cluster from oVirt Engine?
> 
> Thanks so much in advance and please let me know if there are any questions.
> 
> Brandon Markgraf | *Target |

IIRC when I had that issue (I removed the host physically before removing it 
from oVirt) I had to do the following:

- Right click in the grid on the host, and select confirm host has been 
rebooted (this is basically you telling oVirt the host has been fenced).
- After a while the host should end up as down (or unknown state).
- You should be able to select maintaince at that point, this might take a 
while (timeouts etc).
- This should put the host in maintaince and you should be able to remove it 
then.


___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users