Re: [ovirt-users] HA agent fails to start

2016-04-14 Thread Simone Tiraboschi 
On Thu, Apr 14, 2016 at 6:53 PM, Richard Neuboeck  wrote:
> On 14.04.16 18:46, Simone Tiraboschi wrote:
>> On Thu, Apr 14, 2016 at 4:04 PM, Richard Neuboeck  
>> wrote:
>>> On 04/14/2016 02:14 PM, Simone Tiraboschi wrote:
 On Thu, Apr 14, 2016 at 12:51 PM, Richard Neuboeck
  wrote:
> On 04/13/2016 10:00 AM, Simone Tiraboschi wrote:
>> On Wed, Apr 13, 2016 at 9:38 AM, Richard Neuboeck 
>>  wrote:
>>> The answers file shows the setup time of both machines.
>>>
>>> On both machines hosted-engine.conf got rotated right before I wrote
>>> this mail. Is it possible that I managed to interrupt the rotation with
>>> the reboot so the backup was accurate but the update not yet written to
>>> hosted-engine.conf?
>>
>> AFAIK we don't have any rotation mechanism for that file; something
>> else you have in place on that host?
>
> Those machines are all CentOS 7.2 minimal installs. The only
> adaptation I do is installing vim, removing postfix and installing
> exim, removing firewalld and installing iptables-service. Then I add
> the oVirt repos (3.6 and 3.6-snapshot) and deploy the host.
>
> But checking lsof shows that 'ovirt-ha-agent --no-daemon' has access
> to the config file (and the one ending with ~):
>
> # lsof | grep 'hosted-engine.conf~'
> ovirt-ha- 193446   vdsm  351u  REG
> 253,01021135070683
> /etc/ovirt-hosted-engine/hosted-engine.conf~

 This is not that much relevant if the file was renamed after
 ovirt-ha-agent opened it.
 Try this:

 [root@c72he20160405h1 ovirt-hosted-engine-setup]# tail -n1 -f
 /etc/ovirt-hosted-engine/hosted-engine.conf &
 [1] 28866
 [root@c72he20160405h1 ovirt-hosted-engine-setup]# port=

 [root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep 
 hosted-engine.conf
 tail  28866  root3r  REG
 253,0  10141595898 /etc/ovirt-hosted-engine/hosted-engine.conf
 [root@c72he20160405h1 ovirt-hosted-engine-setup]# mv
 /etc/ovirt-hosted-engine/hosted-engine.conf
 /etc/ovirt-hosted-engine/hosted-engine.conf_123
 [root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep 
 hosted-engine.conf
 tail  28866  root3r  REG
 253,0  10141595898
 /etc/ovirt-hosted-engine/hosted-engine.conf_123
 [root@c72he20160405h1 ovirt-hosted-engine-setup]#

>>>
>>> I've issued the commands you suggested but I don't know how that
>>> helps to find the process accessing the config files.
>>>
>>> After moving the hosted-engine.conf file the HA agent crashed
>>> logging the information that the config file is not available.
>>>
>>> Here is the output from every command:
>>>
>>> # tail -n1 -f /etc/ovirt-hosted-engine/hosted-engine.conf &
>>> [1] 167865
>>> [root@cube-two ~]# port=
>>> # lsof | grep hosted-engine.conf
>>> ovirt-ha- 166609   vdsm5u  REG
>>> 253,01021134433491
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm7u  REG
>>> 253,01021134433453
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm8u  REG
>>> 253,01021134433489
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm9u  REG
>>> 253,01021134433493
>>> /etc/ovirt-hosted-engine/hosted-engine.conf~
>>> ovirt-ha- 166609   vdsm   10u  REG
>>> 253,01021134433495
>>> /etc/ovirt-hosted-engine/hosted-engine.conf
>>> tail  167865   root3r  REG
>>> 253,01021134433493
>>> /etc/ovirt-hosted-engine/hosted-engine.conf~
>>> # mv /etc/ovirt-hosted-engine/hosted-engine.conf
>>> /etc/ovirt-hosted-engine/hosted-engine.conf_123
>>> # lsof | grep hosted-engine.conf
>>> ovirt-ha- 166609   vdsm5u  REG
>>> 253,01021134433491
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm7u  REG
>>> 253,01021134433453
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm8u  REG
>>> 253,01021134433489
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm9u  REG
>>> 253,01021134433493
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm   10u  REG
>>> 253,01021134433495
>>> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
>>> ovirt-ha- 166609   vdsm   12u

Re: [ovirt-users] HA agent fails to start

2016-04-14 Thread Simone Tiraboschi 
On Thu, Apr 14, 2016 at 4:04 PM, Richard Neuboeck  wrote:
> On 04/14/2016 02:14 PM, Simone Tiraboschi wrote:
>> On Thu, Apr 14, 2016 at 12:51 PM, Richard Neuboeck
>>  wrote:
>>> On 04/13/2016 10:00 AM, Simone Tiraboschi wrote:
 On Wed, Apr 13, 2016 at 9:38 AM, Richard Neuboeck  
 wrote:
> The answers file shows the setup time of both machines.
>
> On both machines hosted-engine.conf got rotated right before I wrote
> this mail. Is it possible that I managed to interrupt the rotation with
> the reboot so the backup was accurate but the update not yet written to
> hosted-engine.conf?

 AFAIK we don't have any rotation mechanism for that file; something
 else you have in place on that host?
>>>
>>> Those machines are all CentOS 7.2 minimal installs. The only
>>> adaptation I do is installing vim, removing postfix and installing
>>> exim, removing firewalld and installing iptables-service. Then I add
>>> the oVirt repos (3.6 and 3.6-snapshot) and deploy the host.
>>>
>>> But checking lsof shows that 'ovirt-ha-agent --no-daemon' has access
>>> to the config file (and the one ending with ~):
>>>
>>> # lsof | grep 'hosted-engine.conf~'
>>> ovirt-ha- 193446   vdsm  351u  REG
>>> 253,01021135070683
>>> /etc/ovirt-hosted-engine/hosted-engine.conf~
>>
>> This is not that much relevant if the file was renamed after
>> ovirt-ha-agent opened it.
>> Try this:
>>
>> [root@c72he20160405h1 ovirt-hosted-engine-setup]# tail -n1 -f
>> /etc/ovirt-hosted-engine/hosted-engine.conf &
>> [1] 28866
>> [root@c72he20160405h1 ovirt-hosted-engine-setup]# port=
>>
>> [root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep 
>> hosted-engine.conf
>> tail  28866  root3r  REG
>> 253,0  10141595898 /etc/ovirt-hosted-engine/hosted-engine.conf
>> [root@c72he20160405h1 ovirt-hosted-engine-setup]# mv
>> /etc/ovirt-hosted-engine/hosted-engine.conf
>> /etc/ovirt-hosted-engine/hosted-engine.conf_123
>> [root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep 
>> hosted-engine.conf
>> tail  28866  root3r  REG
>> 253,0  10141595898
>> /etc/ovirt-hosted-engine/hosted-engine.conf_123
>> [root@c72he20160405h1 ovirt-hosted-engine-setup]#
>>
>
> I've issued the commands you suggested but I don't know how that
> helps to find the process accessing the config files.
>
> After moving the hosted-engine.conf file the HA agent crashed
> logging the information that the config file is not available.
>
> Here is the output from every command:
>
> # tail -n1 -f /etc/ovirt-hosted-engine/hosted-engine.conf &
> [1] 167865
> [root@cube-two ~]# port=
> # lsof | grep hosted-engine.conf
> ovirt-ha- 166609   vdsm5u  REG
> 253,01021134433491
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm7u  REG
> 253,01021134433453
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm8u  REG
> 253,01021134433489
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm9u  REG
> 253,01021134433493
> /etc/ovirt-hosted-engine/hosted-engine.conf~
> ovirt-ha- 166609   vdsm   10u  REG
> 253,01021134433495
> /etc/ovirt-hosted-engine/hosted-engine.conf
> tail  167865   root3r  REG
> 253,01021134433493
> /etc/ovirt-hosted-engine/hosted-engine.conf~
> # mv /etc/ovirt-hosted-engine/hosted-engine.conf
> /etc/ovirt-hosted-engine/hosted-engine.conf_123
> # lsof | grep hosted-engine.conf
> ovirt-ha- 166609   vdsm5u  REG
> 253,01021134433491
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm7u  REG
> 253,01021134433453
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm8u  REG
> 253,01021134433489
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm9u  REG
> 253,01021134433493
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm   10u  REG
> 253,01021134433495
> /etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
> ovirt-ha- 166609   vdsm   12u  REG
> 253,01021134433498
> /etc/ovirt-hosted-engine/hosted-engine.conf~
> ovirt-ha- 166609   vdsm   13u  REG
> 253,01021134433499
> /etc/ovirt-hosted-engine/hosted-engine.conf_123
> tail  167865   root3r  REG
> 253,01021134433493
>

Re: [ovirt-users] HA agent fails to start

2016-04-14 Thread Richard Neuboeck 
On 04/14/2016 02:14 PM, Simone Tiraboschi wrote:
> On Thu, Apr 14, 2016 at 12:51 PM, Richard Neuboeck
>  wrote:
>> On 04/13/2016 10:00 AM, Simone Tiraboschi wrote:
>>> On Wed, Apr 13, 2016 at 9:38 AM, Richard Neuboeck  
>>> wrote:
 The answers file shows the setup time of both machines.

 On both machines hosted-engine.conf got rotated right before I wrote
 this mail. Is it possible that I managed to interrupt the rotation with
 the reboot so the backup was accurate but the update not yet written to
 hosted-engine.conf?
>>>
>>> AFAIK we don't have any rotation mechanism for that file; something
>>> else you have in place on that host?
>>
>> Those machines are all CentOS 7.2 minimal installs. The only
>> adaptation I do is installing vim, removing postfix and installing
>> exim, removing firewalld and installing iptables-service. Then I add
>> the oVirt repos (3.6 and 3.6-snapshot) and deploy the host.
>>
>> But checking lsof shows that 'ovirt-ha-agent --no-daemon' has access
>> to the config file (and the one ending with ~):
>>
>> # lsof | grep 'hosted-engine.conf~'
>> ovirt-ha- 193446   vdsm  351u  REG
>> 253,01021135070683
>> /etc/ovirt-hosted-engine/hosted-engine.conf~
> 
> This is not that much relevant if the file was renamed after
> ovirt-ha-agent opened it.
> Try this:
> 
> [root@c72he20160405h1 ovirt-hosted-engine-setup]# tail -n1 -f
> /etc/ovirt-hosted-engine/hosted-engine.conf &
> [1] 28866
> [root@c72he20160405h1 ovirt-hosted-engine-setup]# port=
> 
> [root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep 
> hosted-engine.conf
> tail  28866  root3r  REG
> 253,0  10141595898 /etc/ovirt-hosted-engine/hosted-engine.conf
> [root@c72he20160405h1 ovirt-hosted-engine-setup]# mv
> /etc/ovirt-hosted-engine/hosted-engine.conf
> /etc/ovirt-hosted-engine/hosted-engine.conf_123
> [root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep 
> hosted-engine.conf
> tail  28866  root3r  REG
> 253,0  10141595898
> /etc/ovirt-hosted-engine/hosted-engine.conf_123
> [root@c72he20160405h1 ovirt-hosted-engine-setup]#
> 

I've issued the commands you suggested but I don't know how that
helps to find the process accessing the config files.

After moving the hosted-engine.conf file the HA agent crashed
logging the information that the config file is not available.

Here is the output from every command:

# tail -n1 -f /etc/ovirt-hosted-engine/hosted-engine.conf &
[1] 167865
[root@cube-two ~]# port=
# lsof | grep hosted-engine.conf
ovirt-ha- 166609   vdsm5u  REG
253,01021134433491
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm7u  REG
253,01021134433453
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm8u  REG
253,01021134433489
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm9u  REG
253,01021134433493
/etc/ovirt-hosted-engine/hosted-engine.conf~
ovirt-ha- 166609   vdsm   10u  REG
253,01021134433495
/etc/ovirt-hosted-engine/hosted-engine.conf
tail  167865   root3r  REG
253,01021134433493
/etc/ovirt-hosted-engine/hosted-engine.conf~
# mv /etc/ovirt-hosted-engine/hosted-engine.conf
/etc/ovirt-hosted-engine/hosted-engine.conf_123
# lsof | grep hosted-engine.conf
ovirt-ha- 166609   vdsm5u  REG
253,01021134433491
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm7u  REG
253,01021134433453
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm8u  REG
253,01021134433489
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm9u  REG
253,01021134433493
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm   10u  REG
253,01021134433495
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)
ovirt-ha- 166609   vdsm   12u  REG
253,01021134433498
/etc/ovirt-hosted-engine/hosted-engine.conf~
ovirt-ha- 166609   vdsm   13u  REG
253,01021134433499
/etc/ovirt-hosted-engine/hosted-engine.conf_123
tail  167865   root3r  REG
253,01021134433493
/etc/ovirt-hosted-engine/hosted-engine.conf (deleted)


> The issue is understanding who renames that file on your host.

From what I've seen so far it looks like a child of vdsm accesses
/etc/ovirt-hosted-engine/hosted-engine.conf

Re: [ovirt-users] HA agent fails to start

2016-04-14 Thread Simone Tiraboschi 
On Thu, Apr 14, 2016 at 12:51 PM, Richard Neuboeck
 wrote:
> On 04/13/2016 10:00 AM, Simone Tiraboschi wrote:
>> On Wed, Apr 13, 2016 at 9:38 AM, Richard Neuboeck  
>> wrote:
>>> The answers file shows the setup time of both machines.
>>>
>>> On both machines hosted-engine.conf got rotated right before I wrote
>>> this mail. Is it possible that I managed to interrupt the rotation with
>>> the reboot so the backup was accurate but the update not yet written to
>>> hosted-engine.conf?
>>
>> AFAIK we don't have any rotation mechanism for that file; something
>> else you have in place on that host?
>
> Those machines are all CentOS 7.2 minimal installs. The only
> adaptation I do is installing vim, removing postfix and installing
> exim, removing firewalld and installing iptables-service. Then I add
> the oVirt repos (3.6 and 3.6-snapshot) and deploy the host.
>
> But checking lsof shows that 'ovirt-ha-agent --no-daemon' has access
> to the config file (and the one ending with ~):
>
> # lsof | grep 'hosted-engine.conf~'
> ovirt-ha- 193446   vdsm  351u  REG
> 253,01021135070683
> /etc/ovirt-hosted-engine/hosted-engine.conf~

This is not that much relevant if the file was renamed after
ovirt-ha-agent opened it.
Try this:

[root@c72he20160405h1 ovirt-hosted-engine-setup]# tail -n1 -f
/etc/ovirt-hosted-engine/hosted-engine.conf &
[1] 28866
[root@c72he20160405h1 ovirt-hosted-engine-setup]# port=

[root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep hosted-engine.conf
tail  28866  root3r  REG
253,0  10141595898 /etc/ovirt-hosted-engine/hosted-engine.conf
[root@c72he20160405h1 ovirt-hosted-engine-setup]# mv
/etc/ovirt-hosted-engine/hosted-engine.conf
/etc/ovirt-hosted-engine/hosted-engine.conf_123
[root@c72he20160405h1 ovirt-hosted-engine-setup]# lsof | grep hosted-engine.conf
tail  28866  root3r  REG
253,0  10141595898
/etc/ovirt-hosted-engine/hosted-engine.conf_123
[root@c72he20160405h1 ovirt-hosted-engine-setup]#

The issue is understanding who renames that file on your host.
As a thumb rule, if a file name is appended with a tilde~, it only
means that it is a backup created by a text editor or similar program.


>>> [root@cube-two ~]# ls -l /etc/ovirt-hosted-engine
>>> total 16
>>> -rw-r--r--. 1 root root 3252 Apr  8 10:35 answers.conf
>>> -rw-r--r--. 1 root root 1021 Apr 13 09:31 hosted-engine.conf
>>> -rw-r--r--. 1 root root 1021 Apr 13 09:30 hosted-engine.conf~
>>>
>>> [root@cube-three ~]# ls -l /etc/ovirt-hosted-engine
>>> total 16
>>> -rw-r--r--. 1 root root 3233 Apr 11 08:02 answers.conf
>>> -rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf
>>> -rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf~
>>>
>>> On 12.04.16 16:01, Simone Tiraboschi wrote:
 Everything seams fine here,
 /etc/ovirt-hosted-engine/hosted-engine.conf seams to be correctly
 created with the right name.
 Can you please check the latest modification time of your
 /etc/ovirt-hosted-engine/hosted-engine.conf~ and compare it with the
 setup time?

 On Tue, Apr 12, 2016 at 2:34 PM, Richard Neuboeck  
 wrote:
> On 04/12/2016 11:32 AM, Simone Tiraboschi wrote:
>> On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck 
>>  wrote:
>>> Hi oVirt Group,
>>>
>>> in my attempts to get all aspects of oVirt 3.6 up and running I
>>> stumbled upon something I'm not sure how to fix:
>>>
>>> Initially I installed a hosted engine setup. After that I added
>>> another HA host (with hosted-engine --deploy). The host was
>>> registered in the Engine correctly and HA agent came up as expected.
>>>
>>> However if I reboot the second host (through the Engine UI or
>>> manually) HA agent fails to start. The reason seems to be that
>>> /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
>>> file ending with ~ exists though.
>>
>> Can you please attach hosted-engine-setup logs from your additional 
>> hosts?
>> AFAIK our code will never take a ~ ending backup of that file.
>
> ovirt-hosted-engine-setup logs from both additional hosts are
> attached to this mail.
>
>>
>>> Here are the log messages from the journal:
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
>>> Hosted Engine High Availability Monitoring Agent...
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
>>> agent 1.3.5.3-0.0.master started
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
>>> certificate common name: cube-two.tbi.univie.ac.at
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at

Re: [ovirt-users] HA agent fails to start

2016-04-14 Thread Richard Neuboeck 
On 04/13/2016 10:00 AM, Simone Tiraboschi wrote:
> On Wed, Apr 13, 2016 at 9:38 AM, Richard Neuboeck  
> wrote:
>> The answers file shows the setup time of both machines.
>>
>> On both machines hosted-engine.conf got rotated right before I wrote
>> this mail. Is it possible that I managed to interrupt the rotation with
>> the reboot so the backup was accurate but the update not yet written to
>> hosted-engine.conf?
> 
> AFAIK we don't have any rotation mechanism for that file; something
> else you have in place on that host?

Those machines are all CentOS 7.2 minimal installs. The only
adaptation I do is installing vim, removing postfix and installing
exim, removing firewalld and installing iptables-service. Then I add
the oVirt repos (3.6 and 3.6-snapshot) and deploy the host.

But checking lsof shows that 'ovirt-ha-agent --no-daemon' has access
to the config file (and the one ending with ~):

# lsof | grep 'hosted-engine.conf~'
ovirt-ha- 193446   vdsm  351u  REG
253,01021135070683
/etc/ovirt-hosted-engine/hosted-engine.conf~


>> [root@cube-two ~]# ls -l /etc/ovirt-hosted-engine
>> total 16
>> -rw-r--r--. 1 root root 3252 Apr  8 10:35 answers.conf
>> -rw-r--r--. 1 root root 1021 Apr 13 09:31 hosted-engine.conf
>> -rw-r--r--. 1 root root 1021 Apr 13 09:30 hosted-engine.conf~
>>
>> [root@cube-three ~]# ls -l /etc/ovirt-hosted-engine
>> total 16
>> -rw-r--r--. 1 root root 3233 Apr 11 08:02 answers.conf
>> -rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf
>> -rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf~
>>
>> On 12.04.16 16:01, Simone Tiraboschi wrote:
>>> Everything seams fine here,
>>> /etc/ovirt-hosted-engine/hosted-engine.conf seams to be correctly
>>> created with the right name.
>>> Can you please check the latest modification time of your
>>> /etc/ovirt-hosted-engine/hosted-engine.conf~ and compare it with the
>>> setup time?
>>>
>>> On Tue, Apr 12, 2016 at 2:34 PM, Richard Neuboeck  
>>> wrote:
 On 04/12/2016 11:32 AM, Simone Tiraboschi wrote:
> On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck  
> wrote:
>> Hi oVirt Group,
>>
>> in my attempts to get all aspects of oVirt 3.6 up and running I
>> stumbled upon something I'm not sure how to fix:
>>
>> Initially I installed a hosted engine setup. After that I added
>> another HA host (with hosted-engine --deploy). The host was
>> registered in the Engine correctly and HA agent came up as expected.
>>
>> However if I reboot the second host (through the Engine UI or
>> manually) HA agent fails to start. The reason seems to be that
>> /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
>> file ending with ~ exists though.
>
> Can you please attach hosted-engine-setup logs from your additional hosts?
> AFAIK our code will never take a ~ ending backup of that file.

 ovirt-hosted-engine-setup logs from both additional hosts are
 attached to this mail.

>
>> Here are the log messages from the journal:
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
>> Hosted Engine High Availability Monitoring Agent...
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
>> agent 1.3.5.3-0.0.master started
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
>> certificate common name: cube-two.tbi.univie.ac.at
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>> ovirt-ha-agent
>> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
>> Engine is not configured. Shutting down.
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>> ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
>> Engine is not configured. Shutting down.
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
>> ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a
>>
>> If I restore the configuration from the backup file and manually
>> restart the HA agent it's working properly.
>>
>> For testing purposes I added a third HA host which turn out to
>> behave exactly the same.
>>
>> Any help would be appreciated!
>> Thanks
>> Cheers
>> Richard
>>
>> --
>> /dev/null
>>
>>
>> ___
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>


 --
 /dev/null
>>
>>
>>
>>

Re: [ovirt-users] HA agent fails to start

2016-04-13 Thread Simone Tiraboschi 
On Wed, Apr 13, 2016 at 9:38 AM, Richard Neuboeck  wrote:
> The answers file shows the setup time of both machines.
>
> On both machines hosted-engine.conf got rotated right before I wrote
> this mail. Is it possible that I managed to interrupt the rotation with
> the reboot so the backup was accurate but the update not yet written to
> hosted-engine.conf?

AFAIK we don't have any rotation mechanism for that file; something
else you have in place on that host?

> [root@cube-two ~]# ls -l /etc/ovirt-hosted-engine
> total 16
> -rw-r--r--. 1 root root 3252 Apr  8 10:35 answers.conf
> -rw-r--r--. 1 root root 1021 Apr 13 09:31 hosted-engine.conf
> -rw-r--r--. 1 root root 1021 Apr 13 09:30 hosted-engine.conf~
>
> [root@cube-three ~]# ls -l /etc/ovirt-hosted-engine
> total 16
> -rw-r--r--. 1 root root 3233 Apr 11 08:02 answers.conf
> -rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf
> -rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf~
>
> On 12.04.16 16:01, Simone Tiraboschi wrote:
>> Everything seams fine here,
>> /etc/ovirt-hosted-engine/hosted-engine.conf seams to be correctly
>> created with the right name.
>> Can you please check the latest modification time of your
>> /etc/ovirt-hosted-engine/hosted-engine.conf~ and compare it with the
>> setup time?
>>
>> On Tue, Apr 12, 2016 at 2:34 PM, Richard Neuboeck  
>> wrote:
>>> On 04/12/2016 11:32 AM, Simone Tiraboschi wrote:
 On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck  
 wrote:
> Hi oVirt Group,
>
> in my attempts to get all aspects of oVirt 3.6 up and running I
> stumbled upon something I'm not sure how to fix:
>
> Initially I installed a hosted engine setup. After that I added
> another HA host (with hosted-engine --deploy). The host was
> registered in the Engine correctly and HA agent came up as expected.
>
> However if I reboot the second host (through the Engine UI or
> manually) HA agent fails to start. The reason seems to be that
> /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
> file ending with ~ exists though.

 Can you please attach hosted-engine-setup logs from your additional hosts?
 AFAIK our code will never take a ~ ending backup of that file.
>>>
>>> ovirt-hosted-engine-setup logs from both additional hosts are
>>> attached to this mail.
>>>

> Here are the log messages from the journal:
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
> Hosted Engine High Availability Monitoring Agent...
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
> agent 1.3.5.3-0.0.master started
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
> certificate common name: cube-two.tbi.univie.ac.at
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> ovirt-ha-agent
> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
> Engine is not configured. Shutting down.
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
> Engine is not configured. Shutting down.
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
> ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a
>
> If I restore the configuration from the backup file and manually
> restart the HA agent it's working properly.
>
> For testing purposes I added a third HA host which turn out to
> behave exactly the same.
>
> Any help would be appreciated!
> Thanks
> Cheers
> Richard
>
> --
> /dev/null
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>>>
>>>
>>> --
>>> /dev/null
>
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] HA agent fails to start

2016-04-13 Thread Richard Neuboeck 
The answers file shows the setup time of both machines.

On both machines hosted-engine.conf got rotated right before I wrote
this mail. Is it possible that I managed to interrupt the rotation with
the reboot so the backup was accurate but the update not yet written to
hosted-engine.conf?

[root@cube-two ~]# ls -l /etc/ovirt-hosted-engine
total 16
-rw-r--r--. 1 root root 3252 Apr  8 10:35 answers.conf
-rw-r--r--. 1 root root 1021 Apr 13 09:31 hosted-engine.conf
-rw-r--r--. 1 root root 1021 Apr 13 09:30 hosted-engine.conf~

[root@cube-three ~]# ls -l /etc/ovirt-hosted-engine
total 16
-rw-r--r--. 1 root root 3233 Apr 11 08:02 answers.conf
-rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf
-rw-r--r--. 1 root root 1002 Apr 13 09:31 hosted-engine.conf~

On 12.04.16 16:01, Simone Tiraboschi wrote:
> Everything seams fine here,
> /etc/ovirt-hosted-engine/hosted-engine.conf seams to be correctly
> created with the right name.
> Can you please check the latest modification time of your
> /etc/ovirt-hosted-engine/hosted-engine.conf~ and compare it with the
> setup time?
> 
> On Tue, Apr 12, 2016 at 2:34 PM, Richard Neuboeck  
> wrote:
>> On 04/12/2016 11:32 AM, Simone Tiraboschi wrote:
>>> On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck  
>>> wrote:
 Hi oVirt Group,

 in my attempts to get all aspects of oVirt 3.6 up and running I
 stumbled upon something I'm not sure how to fix:

 Initially I installed a hosted engine setup. After that I added
 another HA host (with hosted-engine --deploy). The host was
 registered in the Engine correctly and HA agent came up as expected.

 However if I reboot the second host (through the Engine UI or
 manually) HA agent fails to start. The reason seems to be that
 /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
 file ending with ~ exists though.
>>>
>>> Can you please attach hosted-engine-setup logs from your additional hosts?
>>> AFAIK our code will never take a ~ ending backup of that file.
>>
>> ovirt-hosted-engine-setup logs from both additional hosts are
>> attached to this mail.
>>
>>>
 Here are the log messages from the journal:
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
 Hosted Engine High Availability Monitoring Agent...
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
 INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
 agent 1.3.5.3-0.0.master started
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
 INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
 certificate common name: cube-two.tbi.univie.ac.at
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
 ovirt-ha-agent
 ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
 Engine is not configured. Shutting down.
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
 ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
 Engine is not configured. Shutting down.
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
 INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
 Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
 ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a

 If I restore the configuration from the backup file and manually
 restart the HA agent it's working properly.

 For testing purposes I added a third HA host which turn out to
 behave exactly the same.

 Any help would be appreciated!
 Thanks
 Cheers
 Richard

 --
 /dev/null


 ___
 Users mailing list
 Users@ovirt.org
 http://lists.ovirt.org/mailman/listinfo/users

>>
>>
>> --
>> /dev/null




signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] HA agent fails to start

2016-04-12 Thread Simone Tiraboschi 
Everything seams fine here,
/etc/ovirt-hosted-engine/hosted-engine.conf seams to be correctly
created with the right name.
Can you please check the latest modification time of your
/etc/ovirt-hosted-engine/hosted-engine.conf~ and compare it with the
setup time?

On Tue, Apr 12, 2016 at 2:34 PM, Richard Neuboeck  wrote:
> On 04/12/2016 11:32 AM, Simone Tiraboschi wrote:
>> On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck  
>> wrote:
>>> Hi oVirt Group,
>>>
>>> in my attempts to get all aspects of oVirt 3.6 up and running I
>>> stumbled upon something I'm not sure how to fix:
>>>
>>> Initially I installed a hosted engine setup. After that I added
>>> another HA host (with hosted-engine --deploy). The host was
>>> registered in the Engine correctly and HA agent came up as expected.
>>>
>>> However if I reboot the second host (through the Engine UI or
>>> manually) HA agent fails to start. The reason seems to be that
>>> /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
>>> file ending with ~ exists though.
>>
>> Can you please attach hosted-engine-setup logs from your additional hosts?
>> AFAIK our code will never take a ~ ending backup of that file.
>
> ovirt-hosted-engine-setup logs from both additional hosts are
> attached to this mail.
>
>>
>>> Here are the log messages from the journal:
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
>>> Hosted Engine High Availability Monitoring Agent...
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
>>> agent 1.3.5.3-0.0.master started
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
>>> certificate common name: cube-two.tbi.univie.ac.at
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> ovirt-ha-agent
>>> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
>>> Engine is not configured. Shutting down.
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
>>> Engine is not configured. Shutting down.
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
>>> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
>>> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
>>> ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a
>>>
>>> If I restore the configuration from the backup file and manually
>>> restart the HA agent it's working properly.
>>>
>>> For testing purposes I added a third HA host which turn out to
>>> behave exactly the same.
>>>
>>> Any help would be appreciated!
>>> Thanks
>>> Cheers
>>> Richard
>>>
>>> --
>>> /dev/null
>>>
>>>
>>> ___
>>> Users mailing list
>>> Users@ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>
>
> --
> /dev/null
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] HA agent fails to start

2016-04-12 Thread Simone Tiraboschi 
On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck  wrote:
> Hi oVirt Group,
>
> in my attempts to get all aspects of oVirt 3.6 up and running I
> stumbled upon something I'm not sure how to fix:
>
> Initially I installed a hosted engine setup. After that I added
> another HA host (with hosted-engine --deploy). The host was
> registered in the Engine correctly and HA agent came up as expected.
>
> However if I reboot the second host (through the Engine UI or
> manually) HA agent fails to start. The reason seems to be that
> /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
> file ending with ~ exists though.

Can you please attach hosted-engine-setup logs from your additional hosts?
AFAIK our code will never take a ~ ending backup of that file.

> Here are the log messages from the journal:
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
> Hosted Engine High Availability Monitoring Agent...
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
> agent 1.3.5.3-0.0.master started
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
> certificate common name: cube-two.tbi.univie.ac.at
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> ovirt-ha-agent
> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
> Engine is not configured. Shutting down.
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
> Engine is not configured. Shutting down.
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
> ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a
>
> If I restore the configuration from the backup file and manually
> restart the HA agent it's working properly.
>
> For testing purposes I added a third HA host which turn out to
> behave exactly the same.
>
> Any help would be appreciated!
> Thanks
> Cheers
> Richard
>
> --
> /dev/null
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

Re: [ovirt-users] HA agent fails to start

2016-04-11 Thread Sandro Bonazzola 
On Mon, Apr 11, 2016 at 8:11 AM, Richard Neuboeck 
wrote:

> Hi oVirt Group,
>
> in my attempts to get all aspects of oVirt 3.6 up and running I
> stumbled upon something I'm not sure how to fix:
>
> Initially I installed a hosted engine setup. After that I added
> another HA host (with hosted-engine --deploy). The host was
> registered in the Engine correctly and HA agent came up as expected.
>
> However if I reboot the second host (through the Engine UI or
> manually) HA agent fails to start. The reason seems to be that
> /etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
> file ending with ~ exists though.
>

Simone, Martin, can you please investigate on this?



>
> Here are the log messages from the journal:
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
> Hosted Engine High Availability Monitoring Agent...
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
> agent 1.3.5.3-0.0.master started
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
> certificate common name: cube-two.tbi.univie.ac.at
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> ovirt-ha-agent
> ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
> Engine is not configured. Shutting down.
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
> Engine is not configured. Shutting down.
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
> INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
> Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
> ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a
>
> If I restore the configuration from the backup file and manually
> restart the HA agent it's working properly.
>
> For testing purposes I added a third HA host which turn out to
> behave exactly the same.
>
> Any help would be appreciated!
> Thanks
> Cheers
> Richard
>
> --
> /dev/null
>
>
> ___
> Users mailing list
> Users@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>


-- 
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] HA agent fails to start

2016-04-11 Thread Richard Neuboeck 
Hi oVirt Group,

in my attempts to get all aspects of oVirt 3.6 up and running I
stumbled upon something I'm not sure how to fix:

Initially I installed a hosted engine setup. After that I added
another HA host (with hosted-engine --deploy). The host was
registered in the Engine correctly and HA agent came up as expected.

However if I reboot the second host (through the Engine UI or
manually) HA agent fails to start. The reason seems to be that
/etc/ovirt-hosted-engine/hosted-engine.conf is empty. The backup
file ending with ~ exists though.

Here are the log messages from the journal:
Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]: Starting oVirt
Hosted Engine High Availability Monitoring Agent...
Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
INFO:ovirt_hosted_engine_ha.agent.agent.Agent:ovirt-hosted-engine-ha
agent 1.3.5.3-0.0.master started
Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
certificate common name: cube-two.tbi.univie.ac.at
Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
ovirt-ha-agent
ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine ERROR Hosted
Engine is not configured. Shutting down.
Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
ERROR:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Hosted
Engine is not configured. Shutting down.
Apr 11 07:29:39 cube-two.tbi.univie.ac.at ovirt-ha-agent[3747]:
INFO:ovirt_hosted_engine_ha.agent.agent.Agent:Agent shutting down
Apr 11 07:29:39 cube-two.tbi.univie.ac.at systemd[1]:
ovirt-ha-agent.service: main process exited, code=exited, status=255/n/a

If I restore the configuration from the backup file and manually
restart the HA agent it's working properly.

For testing purposes I added a third HA host which turn out to
behave exactly the same.

Any help would be appreciated!
Thanks
Cheers
Richard

-- 
/dev/null



signature.asc
Description: OpenPGP digital signature
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users