[ovirt-users] Re: Unable to get HE up after update
On Mon, Oct 10, 2016 at 11:27 AM, Susinthiran Sithamparanathan <
chesu...@gmail.com> wrote:
> Hi,
> all the logs are now at https://my.owndrive.com/index.
> php/s/3Dcyho9bqo7oZs8
>
> I did a quick debug in the VM and i think we are getting closer to the
> root cause:
> https://paste.fedoraproject.org/447579/14760912/
>
> It seems the SSL/TLS certs are all missing. Now i wonder which RPM package
> does contain these so that i can try to reinstall it.
>
> Appreciate your help so far!
>
>
OK, the issue on the host is just here:
MainThread::DEBUG::2016-10-10
11:18:34,169::brokerlink::282::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(_communicate)
Full response: success {"reason": "failed liveliness check", "health":
"bad", "vm": "up", "detail": "up"}
MainThread::DEBUG::2016-10-10
11:18:34,169::brokerlink::255::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(_checked_communicate)
Successful response from socket
MainThread::DEBUG::2016-10-10
11:18:34,170::brokerlink::151::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(get_monitor_status)
Success, status {"reason": "failed liveliness check", "health": "bad",
"vm": "up", "detail": "up"}
the engine VM goes up but the engine no and so after a certain amount of
time it tries again with a reboot.
We should definitively add a more explicit log entry there!
Now the point is just why your engine is not starting.
>
>
>
> On Mon, Oct 10, 2016 at 10:17 AM, Simone Tiraboschi
> wrote:
>
>>
>>
>> On Mon, Oct 10, 2016 at 10:13 AM, Yedidyah Bar David
>> wrote:
>>
>>> On Mon, Oct 10, 2016 at 10:56 AM, Simone Tiraboschi
>>> wrote:
>>> >
>>> >
>>> > On Sun, Oct 9, 2016 at 5:13 PM, Susinthiran Sithamparanathan
>>> > wrote:
>>> >>
>>> >> Sure, here it is: https://my.owndrive.com/index.php/s/MFoFyKJVLjzezey
>>> >>
>>> >
>>> > The agent is periodically restarting the engine VM but from the logs I
>>> don't
>>> > see why.
>>>
>>> Also it keeps doing:
>>>
>>>
>> Yes, this is fine: by design ovirt-ha-agent periodically (about 30-40
>> seconds) reconnects the hosted-engine storage domain.
>>
>>
>>> MainThread::INFO::2016-10-09
>>> 17:06:01,025::hosted_engine::612::ovirt_hosted_engine_ha.age
>>> nt.hosted_engine.HostedEngine::(_initialize_vdsm)
>>> Initializing VDSM
>>> MainThread::INFO::2016-10-09
>>> 17:06:05,118::hosted_engine::639::ovirt_hosted_engine_ha.age
>>> nt.hosted_engine.HostedEngine::(_initialize_storage_images)
>>> Connecting the storage
>>> MainThread::INFO::2016-10-09
>>> 17:06:05,131::storage_server::218::ovirt_hosted_engine_ha.li
>>> b.storage_server.StorageServer::(connect_storage_server)
>>> Connecting storage server
>>> MainThread::INFO::2016-10-09
>>> 17:06:13,459::storage_server::225::ovirt_hosted_engine_ha.li
>>> b.storage_server.StorageServer::(connect_storage_server)
>>> Connecting storage server
>>> MainThread::INFO::2016-10-09
>>> 17:06:13,496::storage_server::232::ovirt_hosted_engine_ha.li
>>> b.storage_server.StorageServer::(connect_storage_server)
>>> Refreshing the storage domain
>>> MainThread::INFO::2016-10-09
>>> 17:06:13,737::hosted_engine::666::ovirt_hosted_engine_ha.age
>>> nt.hosted_engine.HostedEngine::(_initialize_storage_images)
>>> Preparing images
>>> MainThread::INFO::2016-10-09
>>> 17:06:13,737::image::126::ovirt_hosted_engine_ha.lib.image.I
>>> mage::(prepare_images)
>>> Preparing images
>>>
>>> Does this make sense, Simone?
>>>
>>> Please check/share also /var/log/vdsm/* . Thanks.
>>>
>>> > Can you please set the agent in debug mode and share again its logs?
>>> >
>>> > You have to edit /etc/ovirt-hosted-engine-ha/agent-log.conf changing
>>> from
>>> >
>>> > [logger_root]
>>> > level=INFO
>>> >
>>> > to
>>> > [logger_root]
>>> > level=DEBUG
>>> >
>>> > and then restart ovirt-ha-agent.
>>> >
>>> >
>>> >>
>>> >> On Sun, Oct 9, 2016 at 3:19 PM, Doron Fediuck
>>> wrote:
>>> >>>
>>> >>> Can you please provide the HA agent logs?
>>> >>>
>>> >>> On Sun, Oct 9, 2016 at 1:57 PM, Susinthiran Sithamparanathan
>>> >>> wrote:
>>>
>>> Hi,
>>> i've been discussing this issue over at IRC (#ovirt on irc.oftc.net
>>> ))
>>> and was suggested to send an email with logs the ML.
>>>
>>> The problem i have is that the HE is a loop getting up and down ( (
>>> i
>>> assume it's the HA trying to get it up) after upgrade to oVirt 4. I
>>> have a
>>> couple of VMs i use daily with SPICE and remoteviewer and i hope to
>>> get them
>>> up and running again. Due to this, i've put it in global
>>> maintenance mode.
>>> I'm attaching the server log in the hope of getting some help and
>>> can
>>> provide more details if needed to debug the issue further.
>>>
>>> The log is too big for fpaste so it's to be found here:
>>> https://my.owndrive.com/index.php/s/Gk4De1jy9fWmFa8 as well as
>>> attached
>>> here.
>>>
>>>
>>> Thanks!
>>>
>>>
>>> --
>>>
>>> Susinthiran Sithamparanathan
>>>
>>>
[ovirt-users] Re: Unable to get HE up after update
On Mon, Oct 10, 2016 at 11:40 AM, Simone Tiraboschi
wrote:
>
>
> On Mon, Oct 10, 2016 at 11:27 AM, Susinthiran Sithamparanathan <
> chesu...@gmail.com> wrote:
>
>> Hi,
>> all the logs are now at https://my.owndrive.com/index.
>> php/s/3Dcyho9bqo7oZs8
>>
>> I did a quick debug in the VM and i think we are getting closer to the
>> root cause:
>> https://paste.fedoraproject.org/447579/14760912/
>>
>> It seems the SSL/TLS certs are all missing. Now i wonder which RPM
>> package does contain these so that i can try to reinstall it.
>>
>> Appreciate your help so far!
>>
>>
> OK, the issue on the host is just here:
> MainThread::DEBUG::2016-10-10 11:18:34,169::brokerlink::282:
> :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(_communicate) Full
> response: success {"reason": "failed liveliness check", "health": "bad",
> "vm": "up", "detail": "up"}
> MainThread::DEBUG::2016-10-10 11:18:34,169::brokerlink::255:
> :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(_checked_communicate)
> Successful response from socket
> MainThread::DEBUG::2016-10-10 11:18:34,170::brokerlink::151:
> :ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(get_monitor_status)
> Success, status {"reason": "failed liveliness check", "health": "bad",
> "vm": "up", "detail": "up"}
>
> the engine VM goes up but the engine no and so after a certain amount of
> time it tries again with a reboot.
> We should definitively add a more explicit log entry there!
>
> Now the point is just why your engine is not starting.
>
Can you please upload you engine-setup logs from the engine VM?
>
>
>
>>
>>
>>
>> On Mon, Oct 10, 2016 at 10:17 AM, Simone Tiraboschi
>> wrote:
>>
>>>
>>>
>>> On Mon, Oct 10, 2016 at 10:13 AM, Yedidyah Bar David
>>> wrote:
>>>
On Mon, Oct 10, 2016 at 10:56 AM, Simone Tiraboschi <
stira...@redhat.com> wrote:
>
>
> On Sun, Oct 9, 2016 at 5:13 PM, Susinthiran Sithamparanathan
> wrote:
>>
>> Sure, here it is: https://my.owndrive.com/index.
php/s/MFoFyKJVLjzezey
>>
>
> The agent is periodically restarting the engine VM but from the logs
I don't
> see why.
Also it keeps doing:
>>> Yes, this is fine: by design ovirt-ha-agent periodically (about 30-40
>>> seconds) reconnects the hosted-engine storage domain.
>>>
>>>
MainThread::INFO::2016-10-09
17:06:01,025::hosted_engine::612::ovirt_hosted_engine_ha.age
nt.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM
MainThread::INFO::2016-10-09
17:06:05,118::hosted_engine::639::ovirt_hosted_engine_ha.age
nt.hosted_engine.HostedEngine::(_initialize_storage_images)
Connecting the storage
MainThread::INFO::2016-10-09
17:06:05,131::storage_server::218::ovirt_hosted_engine_ha.li
b.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2016-10-09
17:06:13,459::storage_server::225::ovirt_hosted_engine_ha.li
b.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2016-10-09
17:06:13,496::storage_server::232::ovirt_hosted_engine_ha.li
b.storage_server.StorageServer::(connect_storage_server)
Refreshing the storage domain
MainThread::INFO::2016-10-09
17:06:13,737::hosted_engine::666::ovirt_hosted_engine_ha.age
nt.hosted_engine.HostedEngine::(_initialize_storage_images)
Preparing images
MainThread::INFO::2016-10-09
17:06:13,737::image::126::ovirt_hosted_engine_ha.lib.image.I
mage::(prepare_images)
Preparing images
Does this make sense, Simone?
Please check/share also /var/log/vdsm/* . Thanks.
> Can you please set the agent in debug mode and share again its logs?
>
> You have to edit /etc/ovirt-hosted-engine-ha/agent-log.conf changing
from
>
> [logger_root]
> level=INFO
>
> to
> [logger_root]
> level=DEBUG
>
> and then restart ovirt-ha-agent.
>
>
>>
>> On Sun, Oct 9, 2016 at 3:19 PM, Doron Fediuck
wrote:
>>>
>>> Can you please provide the HA agent logs?
>>>
>>> On Sun, Oct 9, 2016 at 1:57 PM, Susinthiran Sithamparanathan
>>> wrote:
Hi,
i've been discussing this issue over at IRC (#ovirt on
irc.oftc.net))
and was suggested to send an email with logs the ML.
The problem i have is that the HE is a loop getting up and down (
( i
assume it's the HA trying to get it up) after upgrade to oVirt 4.
I have a
couple of VMs i use daily with SPICE and remoteviewer and i hope
to get them
up and running again. Due to this, i've put it in global
maintenance mode.
I'm attaching the server log in the hope of getting some help and
can
provide more details if needed to debug the issue further.
[ovirt-users] Re: Unable to get HE up after update
Sure, here it is: https://my.owndrive.com/index.php/s/MFoFyKJVLjzezey On Sun, Oct 9, 2016 at 3:19 PM, Doron Fediuck wrote: > Can you please provide the HA agent logs? > > On Sun, Oct 9, 2016 at 1:57 PM, Susinthiran Sithamparanathan < > chesu...@gmail.com> wrote: > >> Hi, >> i've been discussing this issue over at IRC (#ovirt on irc.oftc.net)) >> and was suggested to send an email with logs the ML. >> >> The problem i have is that the HE is a loop getting up and down ( ( i >> assume it's the HA trying to get it up) after upgrade to oVirt 4. I have a >> couple of VMs i use daily with SPICE and remoteviewer and i hope to get >> them up and running again. Due to this, i've put it in global maintenance >> mode. >> I'm attaching the server log in the hope of getting some help and can >> provide more details if needed to debug the issue further. >> >> The log is too big for fpaste so it's to be found here: >> https://my.owndrive.com/index.php/s/Gk4De1jy9fWmFa8 as well as attached >> here. >> >> >> Thanks! >> >> >> -- >> >> Susinthiran Sithamparanathan >> >> ___ >> Users mailing list >> Users@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> >> > -- Susinthiran Sithamparanathan -- IMPORTANT! This message has been scanned for viruses and phishing links. However, it is your responsibility to evaluate the links and attachments you choose to click. If you are uncertain, we always try to help. Greetings helpd...@actnet.se -- IMPORTANT! This message has been scanned for viruses and phishing links. However, it is your responsibility to evaluate the links and attachments you choose to click. If you are uncertain, we always try to help. Greetings helpd...@actnet.se ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/PMLC6ZBGDFYOGLWNEEJV26M7O63GNEPQ/
[ovirt-users] Re: Unable to get HE up after update
Hi, all the logs are now at https://my.owndrive.com/index.php/s/3Dcyho9bqo7oZs8 I did a quick debug in the VM and i think we are getting closer to the root cause: https://paste.fedoraproject.org/447579/14760912/ It seems the SSL/TLS certs are all missing. Now i wonder which RPM package does contain these so that i can try to reinstall it. Appreciate your help so far! On Mon, Oct 10, 2016 at 10:17 AM, Simone Tiraboschi wrote: > > > On Mon, Oct 10, 2016 at 10:13 AM, Yedidyah Bar David > wrote: > >> On Mon, Oct 10, 2016 at 10:56 AM, Simone Tiraboschi >> wrote: >> > >> > >> > On Sun, Oct 9, 2016 at 5:13 PM, Susinthiran Sithamparanathan >> > wrote: >> >> >> >> Sure, here it is: https://my.owndrive.com/index.php/s/MFoFyKJVLjzezey >> >> >> > >> > The agent is periodically restarting the engine VM but from the logs I >> don't >> > see why. >> >> Also it keeps doing: >> >> > Yes, this is fine: by design ovirt-ha-agent periodically (about 30-40 > seconds) reconnects the hosted-engine storage domain. > > >> MainThread::INFO::2016-10-09 >> 17:06:01,025::hosted_engine::612::ovirt_hosted_engine_ha.age >> nt.hosted_engine.HostedEngine::(_initialize_vdsm) >> Initializing VDSM >> MainThread::INFO::2016-10-09 >> 17:06:05,118::hosted_engine::639::ovirt_hosted_engine_ha.age >> nt.hosted_engine.HostedEngine::(_initialize_storage_images) >> Connecting the storage >> MainThread::INFO::2016-10-09 >> 17:06:05,131::storage_server::218::ovirt_hosted_engine_ha.li >> b.storage_server.StorageServer::(connect_storage_server) >> Connecting storage server >> MainThread::INFO::2016-10-09 >> 17:06:13,459::storage_server::225::ovirt_hosted_engine_ha.li >> b.storage_server.StorageServer::(connect_storage_server) >> Connecting storage server >> MainThread::INFO::2016-10-09 >> 17:06:13,496::storage_server::232::ovirt_hosted_engine_ha.li >> b.storage_server.StorageServer::(connect_storage_server) >> Refreshing the storage domain >> MainThread::INFO::2016-10-09 >> 17:06:13,737::hosted_engine::666::ovirt_hosted_engine_ha.age >> nt.hosted_engine.HostedEngine::(_initialize_storage_images) >> Preparing images >> MainThread::INFO::2016-10-09 >> 17:06:13,737::image::126::ovirt_hosted_engine_ha.lib.image. >> Image::(prepare_images) >> Preparing images >> >> Does this make sense, Simone? >> >> Please check/share also /var/log/vdsm/* . Thanks. >> >> > Can you please set the agent in debug mode and share again its logs? >> > >> > You have to edit /etc/ovirt-hosted-engine-ha/agent-log.conf changing >> from >> > >> > [logger_root] >> > level=INFO >> > >> > to >> > [logger_root] >> > level=DEBUG >> > >> > and then restart ovirt-ha-agent. >> > >> > >> >> >> >> On Sun, Oct 9, 2016 at 3:19 PM, Doron Fediuck >> wrote: >> >>> >> >>> Can you please provide the HA agent logs? >> >>> >> >>> On Sun, Oct 9, 2016 at 1:57 PM, Susinthiran Sithamparanathan >> >>> wrote: >> >> Hi, >> i've been discussing this issue over at IRC (#ovirt on irc.oftc.net >> )) >> and was suggested to send an email with logs the ML. >> >> The problem i have is that the HE is a loop getting up and down ( ( i >> assume it's the HA trying to get it up) after upgrade to oVirt 4. I >> have a >> couple of VMs i use daily with SPICE and remoteviewer and i hope to >> get them >> up and running again. Due to this, i've put it in global maintenance >> mode. >> I'm attaching the server log in the hope of getting some help and can >> provide more details if needed to debug the issue further. >> >> The log is too big for fpaste so it's to be found here: >> https://my.owndrive.com/index.php/s/Gk4De1jy9fWmFa8 as well as >> attached >> here. >> >> >> Thanks! >> >> >> -- >> >> Susinthiran Sithamparanathan >> >> ___ >> Users mailing list >> Users@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/users >> >> >>> >> >> >> >> >> >> >> >> -- >> >> >> >> Susinthiran Sithamparanathan >> >> >> >> ___ >> >> Users mailing list >> >> Users@ovirt.org >> >> http://lists.ovirt.org/mailman/listinfo/users >> >> >> > >> > >> > ___ >> > Users mailing list >> > Users@ovirt.org >> > http://lists.ovirt.org/mailman/listinfo/users >> > >> >> >> >> -- >> Didi >> > > -- Susinthiran Sithamparanathan -- IMPORTANT! This message has been scanned for viruses and phishing links. However, it is your responsibility to evaluate the links and attachments you choose to click. If you are uncertain, we always try to help. Greetings helpd...@actnet.se -- IMPORTANT! This message has been scanned for viruses and phishing links. However, it is your responsibility to evaluate the links and attachments you choose to click. If you are uncertain, we always try to help. Greetings helpd...@actnet.se ___ Users
[ovirt-users] Re: Unable to get HE up after update
Can you please provide the HA agent logs? On Sun, Oct 9, 2016 at 1:57 PM, Susinthiran Sithamparanathan < chesu...@gmail.com> wrote: > Hi, > i've been discussing this issue over at IRC (#ovirt on irc.oftc.net)) and > was suggested to send an email with logs the ML. > > The problem i have is that the HE is a loop getting up and down ( ( i > assume it's the HA trying to get it up) after upgrade to oVirt 4. I have a > couple of VMs i use daily with SPICE and remoteviewer and i hope to get > them up and running again. Due to this, i've put it in global maintenance > mode. > I'm attaching the server log in the hope of getting some help and can > provide more details if needed to debug the issue further. > > The log is too big for fpaste so it's to be found here: > https://my.owndrive.com/index.php/s/Gk4De1jy9fWmFa8 as well as attached > here. > > > Thanks! > > > -- > > Susinthiran Sithamparanathan > > ___ > Users mailing list > Users@ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > > -- IMPORTANT! This message has been scanned for viruses and phishing links. However, it is your responsibility to evaluate the links and attachments you choose to click. If you are uncertain, we always try to help. Greetings helpd...@actnet.se -- IMPORTANT! This message has been scanned for viruses and phishing links. However, it is your responsibility to evaluate the links and attachments you choose to click. If you are uncertain, we always try to help. Greetings helpd...@actnet.se ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LLTZ3IG4VT6O3X5EZ2N4MNVOND5RLDL5/
