Re: [ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-05-03 Thread Simone Tiraboschi
On Mon, May 2, 2016 at 9:15 PM, Gianluca Cecchi
 wrote:
>
>
> On Mon, May 2, 2016 at 8:39 PM, Gianluca Cecchi 
> wrote:
>>
>> On Mon, May 2, 2016 at 11:14 AM, Simone Tiraboschi 
>> wrote:
>>>
>>>
>>> >>>
>>> >>> Can you please check the entropy value on your host?
>>> >>>  cat /proc/sys/kernel/random/entropy_avail
>>> >>>
>>> >>
>>> >> I have not at hand now the server. I'll check soon and report
>>> >> Do you mean entropy of the physical server that will operate as
>>> >> hypervisor?
>>>
>>> On the hypervisor
>>>
>>> > That's a good question. Simone - do you know if we start the guest with
>>> > virtio-rng?
>>>
>>> AFAIK we are not.
>>>
>>
>> On the only existing hypervisor, just after booting and exiting global
>> maintenance, causing hosted engine to start, I have
>>
>> [root@ovirt01 ~]# uptime
>>  20:34:17 up 6 min,  1 user,  load average: 0.23, 0.20, 0.11
>>
>> [root@ovirt01 ~]# cat /proc/sys/kernel/random/entropy_avail
>> 3084
>>
>> BTW on the self hosted engine VM:
>> [root@ovirt ~]# uptime
>>  18:35:33 up 4 min,  1 user,  load average: 0.06, 0.25, 0.13
>>
>> [root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail
>> 14
>>
>> On the hypervisor:
>> [root@ovirt01 ~]# ps -ef | grep [q]emu | grep virtio-rng
>> [root@ovirt01 ~]#
>>
>> On engine VM:
>> [root@ovirt ~]# ll /dev/hwrng
>> ls: cannot access /dev/hwrng: No such file or directory
>> [root@ovirt ~]#
>>
>> [root@ovirt ~]# lsmod | grep virtio_rng
>> [root@ovirt ~]#
>>
>> May I change anything so that engine VM has virtio-rng enabled?
>>
>> Gianluca
>>
>>
>
> I verified very slow login time in webadmin after welcome page, with my
> configuration that is for now based on /etc/hosts.
> After reading a previous post, and having after about 30 minutes only 114 as
> entropy in hosted engine vm, I made this in engine VM:

Thanks for your report Gianluca,
adding virtio-rng or adding haveged daemon to the appliance is indeed
a good idea: could you please fill an RFE on bugzilla for that?

> yum install haveged
> systemctl enable haveged
>
> put host in global maintenance
> shutdown engine VM
> exit from maintenance
>
> engine VM starts and immediately I have:
>
> [root@ovirt ~]# uptime
>  19:05:10 up 0 min,  1 user,  load average: 0.68, 0.20, 0.07
>
> [root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail
> 1369
>
> And login in web admin page now almost immediate
>
> Inside the thread I read:
> http://lists.ovirt.org/pipermail/users/2016-April/038805.html
>
> it wasn't clear if I can edit the engine VM in webadmin (or other mean) and
> enable the random generator option or if the haveged way is the one to go
> with in case of self hosted engine
> Is there a list of what I can change (if any) and what not for the engine
> VM?
> For example I would like to change the time zone that is GMT now (I think
> inherited from the OVF of the appliance?)
>
> Thanks,
> Gianluca
>
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-05-02 Thread Gianluca Cecchi
On Mon, May 2, 2016 at 8:39 PM, Gianluca Cecchi 
wrote:

> On Mon, May 2, 2016 at 11:14 AM, Simone Tiraboschi 
> wrote:
>
>>
>> >>>
>> >>> Can you please check the entropy value on your host?
>> >>>  cat /proc/sys/kernel/random/entropy_avail
>> >>>
>> >>
>> >> I have not at hand now the server. I'll check soon and report
>> >> Do you mean entropy of the physical server that will operate as
>> hypervisor?
>>
>> On the hypervisor
>>
>> > That's a good question. Simone - do you know if we start the guest with
>> > virtio-rng?
>>
>> AFAIK we are not.
>>
>>
> On the only existing hypervisor, just after booting and exiting global
> maintenance, causing hosted engine to start, I have
>
> [root@ovirt01 ~]# uptime
>  20:34:17 up 6 min,  1 user,  load average: 0.23, 0.20, 0.11
>
> [root@ovirt01 ~]# cat /proc/sys/kernel/random/entropy_avail
> 3084
>
> BTW on the self hosted engine VM:
> [root@ovirt ~]# uptime
>  18:35:33 up 4 min,  1 user,  load average: 0.06, 0.25, 0.13
>
> [root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail
> 14
>
> On the hypervisor:
> [root@ovirt01 ~]# ps -ef | grep [q]emu | grep virtio-rng
> [root@ovirt01 ~]#
>
> On engine VM:
> [root@ovirt ~]# ll /dev/hwrng
> ls: cannot access /dev/hwrng: No such file or directory
> [root@ovirt ~]#
>
> [root@ovirt ~]# lsmod | grep virtio_rng
> [root@ovirt ~]#
>
> May I change anything so that engine VM has virtio-rng enabled?
>
> Gianluca
>
>
>
I verified very slow login time in webadmin after welcome page, with my
configuration that is for now based on /etc/hosts.
After reading a previous post, and having after about 30 minutes only 114
as entropy in hosted engine vm, I made this in engine VM:

yum install haveged
systemctl enable haveged

put host in global maintenance
shutdown engine VM
exit from maintenance

engine VM starts and immediately I have:

[root@ovirt ~]# uptime
 19:05:10 up 0 min,  1 user,  load average: 0.68, 0.20, 0.07

[root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail
1369

And login in web admin page now almost immediate

Inside the thread I read:
http://lists.ovirt.org/pipermail/users/2016-April/038805.html

it wasn't clear if I can edit the engine VM in webadmin (or other mean) and
enable the random generator option or if the haveged way is the one to go
with in case of self hosted engine
Is there a list of what I can change (if any) and what not for the engine
VM?
For example I would like to change the time zone that is GMT now (I think
inherited from the OVF of the appliance?)

Thanks,
Gianluca
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-05-02 Thread Gianluca Cecchi
On Mon, May 2, 2016 at 11:14 AM, Simone Tiraboschi 
wrote:

>
> >>>
> >>> Can you please check the entropy value on your host?
> >>>  cat /proc/sys/kernel/random/entropy_avail
> >>>
> >>
> >> I have not at hand now the server. I'll check soon and report
> >> Do you mean entropy of the physical server that will operate as
> hypervisor?
>
> On the hypervisor
>
> > That's a good question. Simone - do you know if we start the guest with
> > virtio-rng?
>
> AFAIK we are not.
>
>
On the only existing hypervisor, just after booting and exiting global
maintenance, causing hosted engine to start, I have

[root@ovirt01 ~]# uptime
 20:34:17 up 6 min,  1 user,  load average: 0.23, 0.20, 0.11

[root@ovirt01 ~]# cat /proc/sys/kernel/random/entropy_avail
3084

BTW on the self hosted engine VM:
[root@ovirt ~]# uptime
 18:35:33 up 4 min,  1 user,  load average: 0.06, 0.25, 0.13

[root@ovirt ~]# cat /proc/sys/kernel/random/entropy_avail
14

On the hypervisor:
[root@ovirt01 ~]# ps -ef | grep [q]emu | grep virtio-rng
[root@ovirt01 ~]#

On engine VM:
[root@ovirt ~]# ll /dev/hwrng
ls: cannot access /dev/hwrng: No such file or directory
[root@ovirt ~]#

[root@ovirt ~]# lsmod | grep virtio_rng
[root@ovirt ~]#

May I change anything so that engine VM has virtio-rng enabled?

Gianluca
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users


Re: [ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-05-02 Thread Simone Tiraboschi
On Mon, May 2, 2016 at 11:06 AM, Yedidyah Bar David  wrote:
> On Mon, May 2, 2016 at 11:48 AM, Gianluca Cecchi
>  wrote:
>> On Mon, May 2, 2016 at 9:58 AM, Simone Tiraboschi wrote:
>>>
>>>
>>>
>>> hosted-engine-setup creates a fresh VM and inject a cloud-init script
>>> to configure it and execute there engine-setup to configure the engine
>>> as needed.
>>> Since engine-setup is running on the engine VM triggered by
>>> cloud-init, hosted-engine-setup has no way to really control its
>>> process status so we simply gather its output with a timeout of 10
>>> minutes between each single output line.
>>> In nothing happens within 10 minutes (the value is easily
>>> customizable), hosted-engine-setup thinks that engine-setup is stuck.
>>
>>
>>
>> How can one customize the pre-set timeout?

To set 20 minutes you can pass this
OVEHOSTED_ENGINE/engineSetupTimeout=int:1200


>> Could it be better to ask the user at the end of timeout if he/she wants to
>> wait again, instead of directly fail?
>
> Perhaps, can you please open a bz?

+1

>>> So the issue we have to understood is why this simple command took
>>> more than 10 minutes in your env:
>>> 2016-04-30 17:56:57 DEBUG
>>> otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
>>> plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool',
>>> '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
>>> 'password-reset', 'admin', '--password=env:pass', '--force',
>>> '--password-valid-to=2216-03-13 17:56:57Z'), executable='None',
>>> cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH':
>>> '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**',
>>> 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/',
>>> 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH':
>>> '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
>>>
>>> '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log',
>>> 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly',
>>> 'OTOPI_EXECDIR': '/'}
>>
>>
>>
>>
>> It seemed quite strange to me too (see below further info on this)
>>
>>>
>>> Can you please check the entropy value on your host?
>>>  cat /proc/sys/kernel/random/entropy_avail
>>>
>>
>> I have not at hand now the server. I'll check soon and report
>> Do you mean entropy of the physical server that will operate as hypervisor?

On the hypervisor

> That's a good question. Simone - do you know if we start the guest with
> virtio-rng?

AFAIK we are not.

> This is another case of [1], perhaps we should reopen it.
>
> [1] https://bugzilla.redhat.com/show_bug.cgi?id=1319827
>
>>
>>
>>>
>>> > As a last question how to clean up things in case I have to start from
>>> > scratch.
>>>
>>> I'd recommend to redeploy from scratch instead of trying fixing it
>>> but, before that, we need to understand the root issue.
>>>
>>
>> So, trying restart the setup with generated answer file I got:
>> 1) if VM still powered on, an error about this condition
>> 2) if VM powered down, an error abut storage domain already in place and
>> restart not supported in this condition.
>>
>> I was able to continue with these steps:
>>
>> a) remove what inside the partially setup self hosted engine storage domain
>> rm -rf /SHE_DOMAIN/*
>> cd SHE_DOMAIN
>> mklost+found
>>
>> b) reboot the hypervisor
>>
>> c) stop vdsmd
>>
>> d) start the setup again with the answer file
>> It seems all went well and this time strangely the step that took more than
>> 10 minutes before lasted less than 2 seconds
>>
>> I was then able to deploy storage and iso domains without problems and self
>> hosted engine domain correctly detected and imported too.
>> Created two CentOS VMs without problems (6.7 and 7.2).
>>
>> See below the full output of deploy command
>>
>>
>> [root@ovirt01 ~]# hosted-engine --deploy
>> --config-append=/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf
>> [ INFO  ] Stage: Initializing
>> [ INFO  ] Generating a temporary VNC password.
>> [ INFO  ] Stage: Environment setup
>>   Configuration files:
>> ['/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf']
>>   Log file:
>> /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160501014326-8frbxk.log
>>   Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos)
>> [ INFO  ] Hardware supports virtualization
>> [ INFO  ] Bridge ovirtmgmt already created
>> [ INFO  ] Stage: Environment packages setup
>> [ INFO  ] Stage: Programs detection
>> [ INFO  ] Stage: Environment setup
>> [ INFO  ] Stage: Environment customization
>>
>>   --== STORAGE CONFIGURATION ==--
>>
>>   During customization use CTRL-D to abort.
>> [ INFO  ] Installing on first host
>>
>>   --== SYSTEM CONFIGURATION ==--
>>
>>
>>   --== NETWORK CONFIGURATION ==--
>>
>>
>>   --== VM CONFIGURATION ==--
>>
>> [ INFO  ] Checking OVF archive content (could take a few minutes depending
>> on 

Re: [ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-05-02 Thread Yedidyah Bar David
On Mon, May 2, 2016 at 11:48 AM, Gianluca Cecchi
 wrote:
> On Mon, May 2, 2016 at 9:58 AM, Simone Tiraboschi wrote:
>>
>>
>>
>> hosted-engine-setup creates a fresh VM and inject a cloud-init script
>> to configure it and execute there engine-setup to configure the engine
>> as needed.
>> Since engine-setup is running on the engine VM triggered by
>> cloud-init, hosted-engine-setup has no way to really control its
>> process status so we simply gather its output with a timeout of 10
>> minutes between each single output line.
>> In nothing happens within 10 minutes (the value is easily
>> customizable), hosted-engine-setup thinks that engine-setup is stuck.
>
>
>
> How can one customize the pre-set timeout?
> Could it be better to ask the user at the end of timeout if he/she wants to
> wait again, instead of directly fail?

Perhaps, can you please open a bz?

>
>
>>
>> So the issue we have to understood is why this simple command took
>> more than 10 minutes in your env:
>> 2016-04-30 17:56:57 DEBUG
>> otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
>> plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool',
>> '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
>> 'password-reset', 'admin', '--password=env:pass', '--force',
>> '--password-valid-to=2216-03-13 17:56:57Z'), executable='None',
>> cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH':
>> '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**',
>> 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/',
>> 'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH':
>> '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
>>
>> '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log',
>> 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly',
>> 'OTOPI_EXECDIR': '/'}
>
>
>
>
> It seemed quite strange to me too (see below further info on this)
>
>>
>> Can you please check the entropy value on your host?
>>  cat /proc/sys/kernel/random/entropy_avail
>>
>
> I have not at hand now the server. I'll check soon and report
> Do you mean entropy of the physical server that will operate as hypervisor?

That's a good question. Simone - do you know if we start the guest with
virtio-rng?

This is another case of [1], perhaps we should reopen it.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1319827

>
>
>>
>> > As a last question how to clean up things in case I have to start from
>> > scratch.
>>
>> I'd recommend to redeploy from scratch instead of trying fixing it
>> but, before that, we need to understand the root issue.
>>
>
> So, trying restart the setup with generated answer file I got:
> 1) if VM still powered on, an error about this condition
> 2) if VM powered down, an error abut storage domain already in place and
> restart not supported in this condition.
>
> I was able to continue with these steps:
>
> a) remove what inside the partially setup self hosted engine storage domain
> rm -rf /SHE_DOMAIN/*
> cd SHE_DOMAIN
> mklost+found
>
> b) reboot the hypervisor
>
> c) stop vdsmd
>
> d) start the setup again with the answer file
> It seems all went well and this time strangely the step that took more than
> 10 minutes before lasted less than 2 seconds
>
> I was then able to deploy storage and iso domains without problems and self
> hosted engine domain correctly detected and imported too.
> Created two CentOS VMs without problems (6.7 and 7.2).
>
> See below the full output of deploy command
>
>
> [root@ovirt01 ~]# hosted-engine --deploy
> --config-append=/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf
> [ INFO  ] Stage: Initializing
> [ INFO  ] Generating a temporary VNC password.
> [ INFO  ] Stage: Environment setup
>   Configuration files:
> ['/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf']
>   Log file:
> /var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160501014326-8frbxk.log
>   Version: otopi-1.4.1 (otopi-1.4.1-1.el7.centos)
> [ INFO  ] Hardware supports virtualization
> [ INFO  ] Bridge ovirtmgmt already created
> [ INFO  ] Stage: Environment packages setup
> [ INFO  ] Stage: Programs detection
> [ INFO  ] Stage: Environment setup
> [ INFO  ] Stage: Environment customization
>
>   --== STORAGE CONFIGURATION ==--
>
>   During customization use CTRL-D to abort.
> [ INFO  ] Installing on first host
>
>   --== SYSTEM CONFIGURATION ==--
>
>
>   --== NETWORK CONFIGURATION ==--
>
>
>   --== VM CONFIGURATION ==--
>
> [ INFO  ] Checking OVF archive content (could take a few minutes depending
> on archive size)
> [ INFO  ] Checking OVF XML content (could take a few minutes depending on
> archive size)
> [WARNING] OVF does not contain a valid image description, using default.
>   Enter root password that will be used for the engine appliance
> (leave it empty to skip):
>   Confirm appliance root password:
> 

Re: [ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-05-02 Thread Simone Tiraboschi
On Sat, Apr 30, 2016 at 10:59 PM, Gianluca Cecchi
 wrote:
> Hello,
> trying to deploy a self hosted engine on an Intel NUC6i5SYB with CentOS 7.2
> using oVirt 3.6.5 and appliance (picked up rpm is
> ovirt-engine-appliance-3.6-20160420.1.el7.centos.noarch)
>
> Near the end of the command
> hosted-engine --deploy
>
> I get
> ...
>   |- [ INFO  ] Initializing PostgreSQL
>   |- [ INFO  ] Creating PostgreSQL 'engine' database
>   |- [ INFO  ] Configuring PostgreSQL
>   |- [ INFO  ] Creating/refreshing Engine database schema
>   |- [ INFO  ] Creating/refreshing Engine 'internal' domain database
> schema
> [ ERROR ] Engine setup got stuck on the appliance
> [ ERROR ] Failed to execute stage 'Closing up': Engine setup is stalled on
> the appliance since 600 seconds ago. Please check its log on the appliance.
> [ INFO  ] Stage: Clean up
> [ INFO  ] Generating answer file
> '/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf'
> [ INFO  ] Stage: Pre-termination
> [ INFO  ] Stage: Termination
> [ ERROR ] Hosted Engine deployment failed: this system is not reliable,
> please check the issue, fix and redeploy
>
> On host log I indeed see the 10 minutes timeout:
>
> 2016-04-30 19:56:52 DEBUG otopi.plugins.otopi.dialog.human
> dialog.__logString:219 DIALOG:SEND |- [ INFO  ]
> Creating/refreshing Engine 'internal' domain database schema
> 2016-04-30 20:06:53 ERROR
> otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:140
> Engine setup got stuck on the appliance
>
> On engine I don't see any particular problem but a ten minutes delay in its
> log:
>
> 2016-04-30 17:56:57 DEBUG otopi.context context.dumpEnvironment:514
> ENVIRONMENT DUMP - END
> 2016-04-30 17:56:57 DEBUG otopi.context context._executeMethod:142 Stage
> misc METHOD
> otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc.Plugin._setupAdminPassword
> 2016-04-30 17:56:57 DEBUG
> otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
> plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool',
> '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
> 'password-reset', 'admin', '--password=env:pass', '--force',
> '--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None',
> env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH':
> '/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**',
> 'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME':
> u'/usr/lib/jvm/jre', 'PATH':
> '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
> '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log',
> 'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'}
> 2016-04-30 18:07:06 DEBUG
> otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
> plugin.executeRaw:878 execute-result: ('/usr/bin/ovirt-aaa-jdbc-tool',
> '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
> 'password-reset', 'admin', '--password=env:pass', '--force',
> '--password-valid-to=2216-03-13 17:56:57Z'), rc=0
>
> and its last lines are:
>
> 2016-04-30 18:07:06 DEBUG
> otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
> plugin.execute:936 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool',
> '--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
> 'password-reset', 'admin', '--password=env:pass', '--force',
> '--password-valid-to=2216-03-13 17:56:57Z') stdout:
> updating user admin...
> user updated successfully

hosted-engine-setup creates a fresh VM and inject a cloud-init script
to configure it and execute there engine-setup to configure the engine
as needed.
Since engine-setup is running on the engine VM triggered by
cloud-init, hosted-engine-setup has no way to really control its
process status so we simply gather its output with a timeout of 10
minutes between each single output line.
In nothing happens within 10 minutes (the value is easily
customizable), hosted-engine-setup thinks that engine-setup is stuck.

So the issue we have to understood is why this simple command took
more than 10 minutes in your env:
2016-04-30 17:56:57 DEBUG
otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool',
'--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
'password-reset', 'admin', '--password=env:pass', '--force',
'--password-valid-to=2216-03-13 17:56:57Z'), executable='None',
cwd='None', env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH':
'/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**',
'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/',
'OVIRT_ENGINE_JAVA_HOME': u'/usr/lib/jvm/jre', 'PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
'/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log',
'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly',
'OTOPI_EXECDIR': '/'}

Can you please check 

[ovirt-users] hosted engine setup failed for 10 minutes delay.. engine seems alive

2016-04-30 Thread Gianluca Cecchi
Hello,
trying to deploy a self hosted engine on an Intel NUC6i5SYB with CentOS 7.2
using oVirt 3.6.5 and appliance (picked up rpm
is ovirt-engine-appliance-3.6-20160420.1.el7.centos.noarch)

Near the end of the command
hosted-engine --deploy

I get
...
  |- [ INFO  ] Initializing PostgreSQL
  |- [ INFO  ] Creating PostgreSQL 'engine' database
  |- [ INFO  ] Configuring PostgreSQL
  |- [ INFO  ] Creating/refreshing Engine database schema
  |- [ INFO  ] Creating/refreshing Engine 'internal' domain
database schema
[ ERROR ] Engine setup got stuck on the appliance
[ ERROR ] Failed to execute stage 'Closing up': Engine setup is stalled on
the appliance since 600 seconds ago. Please check its log on the appliance.
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file
'/var/lib/ovirt-hosted-engine-setup/answers/answers-20160430200654.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination
[ ERROR ] Hosted Engine deployment failed: this system is not reliable,
please check the issue, fix and redeploy

On host log I indeed see the 10 minutes timeout:

2016-04-30 19:56:52 DEBUG otopi.plugins.otopi.dialog.human
dialog.__logString:219 DIALOG:SEND |- [ INFO  ]
Creating/refreshing Engine 'internal' domain database schema
2016-04-30 20:06:53 ERROR
otopi.plugins.ovirt_hosted_engine_setup.engine.health health._closeup:140
Engine setup got stuck on the appliance

On engine I don't see any particular problem but a ten minutes delay in its
log:

2016-04-30 17:56:57 DEBUG otopi.context context.dumpEnvironment:514
ENVIRONMENT DUMP - END
2016-04-30 17:56:57 DEBUG otopi.context context._executeMethod:142 Stage
misc METHOD
otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc.Plugin._setupAdminPassword
2016-04-30 17:56:57 DEBUG
otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
plugin.executeRaw:828 execute: ('/usr/bin/ovirt-aaa-jdbc-tool',
'--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
'password-reset', 'admin', '--password=env:pass', '--force',
'--password-valid-to=2216-03-13 17:56:57Z'), executable='None', cwd='None',
env={'LANG': 'en_US.UTF-8', 'SHLVL': '1', 'PYTHONPATH':
'/usr/share/ovirt-engine/setup/bin/..::', 'pass': '**FILTERED**',
'OVIRT_ENGINE_JAVA_HOME_FORCE': '1', 'PWD': '/', 'OVIRT_ENGINE_JAVA_HOME':
u'/usr/lib/jvm/jre', 'PATH':
'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin', 'OTOPI_LOGFILE':
'/var/log/ovirt-engine/setup/ovirt-engine-setup-20160430175551-dttt2p.log',
'OVIRT_JBOSS_HOME': '/usr/share/ovirt-engine-wildfly', 'OTOPI_EXECDIR': '/'}
2016-04-30 18:07:06 DEBUG
otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
plugin.executeRaw:878 execute-result: ('/usr/bin/ovirt-aaa-jdbc-tool',
'--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
'password-reset', 'admin', '--password=env:pass', '--force',
'--password-valid-to=2216-03-13 17:56:57Z'), rc=0

and its last lines are:

2016-04-30 18:07:06 DEBUG
otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
plugin.execute:936 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool',
'--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
'password-reset', 'admin', '--password=env:pass', '--force',
'--password-valid-to=2216-03-13 17:56:57Z') stdout:
updating user admin...
user updated successfully

2016-04-30 18:07:06 DEBUG
otopi.plugins.ovirt_engine_setup.ovirt_engine.config.aaajdbc
plugin.execute:941 execute-output: ('/usr/bin/ovirt-aaa-jdbc-tool',
'--db-config=/etc/ovirt-engine/aaa/internal.properties', 'user',
'password-reset', 'admin', '--password=env:pass', '--force',
'--password-valid-to=2216-03-13 17:56:57Z') stderr:


2016-04-30 18:07:06 DEBUG otopi.context context._executeMethod:142 Stage
misc METHOD
otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca.Plugin._miscUpgrade
2016-04-30 18:07:06 INFO
otopi.plugins.ovirt_engine_setup.ovirt_engine.pki.ca ca._miscUpgrade:510
Upgrading CA

Full logs of host and engine here:
https://drive.google.com/file/d/0BwoPbcrMv8mvQm9jeDhpZEdRUjg/view?usp=sharing

I can connect via vnc to the engine and see 277 tables in engine database
(277 rows in output of "\d" command)

Can anyone tell me if I can follow up without starting from scratch and how
in case?
Also understand the reason of this delay, as the NUC is a physical host
with 32Gb of ram and SSD disks and should be quite fast... faster than a VM
non my laptop where I had no problems in similar setup...

As a last question how to clean up things in case I have to start from
scratch.

I can leave the situation as it is in the moment, so I can work on the live
environment before power off

Thanks in advance,
Gianluca
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users