[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

I rebooted the server that was failing last week and haven't seen the error
since.

On 25 July 2018 at 11:48, Andrej Krejcir  wrote:

> I think the upgrade may not have affected the mom package, because no new
> mom rpm version was released for a few months.
>
> I'm not sure why the import fails even if the file is there.
>
> As a workaround, mom can be configured to use the older (deprecated) json
> rpc interface to communicate with vdsm. There is no difference in
> functionality.
> This can be done by editing /etc/vdsm/mom.conf. Set the
> 'hypervisor-interface' config value to 'vdsmjsonrpcbulk' insted of
> 'vdsmjsonrpcclient'.
>
> On Wed, 25 Jul 2018 at 08:56, Maton, Brett 
> wrote:
>
>> I upgraded my test cluster to 4.2.5.2-1 last night (hosts rebooted after
>> update) and I've started to see the error again.
>>
>> As before it seems to be limited to the one host,  I've checked that
>>
>> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/
>> vdsmjsonrpcclientInterface.py
>>
>> exists, which it does but there are no matching .pyc / .pyo files which
>> suggests that it's not been used?
>>
>> File permissions and SELinux contexts are the same on all files in the
>> HypervisorInterfaces directory.
>>
>> 0644 root root system_u:object_r:lib_t:s0
>>
>> vdsm / mom log from time period attached.
>>
>>
>> On 18 July 2018 at 16:10, Maton, Brett  wrote:
>>
>>> Thanks,
>>>
>>>   Cluster is all installed from pre-release, maybe I managed to get an
>>> iffy rpm
>>>
>>> On 18 July 2018 at 15:50, Andrej Krejcir  wrote:
>>>
 Yes, copying it from another host with mom version 0.5.12 is enough.

 I'm not sure why it was not installed. The file is in the rpm.

 On Wed, 18 Jul 2018 at 16:38, Maton, Brett 
 wrote:

> Bingo
>
> How could that file not be installed/deployed ?
>
>
> Should I simply copy it from one of the other hosts to make the
> message go away ?
>
> On 18 July 2018 at 15:30, Andrej Krejcir  wrote:
>
>> Maybe the python module is missing. Can you check if this file exists?
>>
>> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/
>> vdsmjsonrpcclientInterface.py
>>
>>
>> Andrej
>>
>> On Wed, 18 Jul 2018 at 16:04, Maton, Brett 
>> wrote:
>>
>>> I just checked the mom version, it's already at 0.5.12
>>>
>>> # rpm -qa mom
>>> mom-0.5.12-1.el7.centos.noarch
>>>
>>> All the physical hosts are fully updated /patched.
>>>
>>> Regards,
>>> Brett
>>>
>>> On 18 July 2018 at 14:08, Andrej Krejcir 
>>> wrote:
>>>
 Hi,

 Looking at MOM logs, the problem seems to be that MOM is configured
 to use 'vdsmjsonrpcclient' module to communicate with vdsm, but it
 cannot find this module, probably because it is an older version.

 Updating MOM to version 0.5.12 should fix it.


 Regards,
 Andrej

 On Wed, 18 Jul 2018 at 14:41, Maton, Brett <
 mat...@ltresources.co.uk> wrote:

> FWIW:
>
> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R)
> Xeon(R) CPU E3-1220 V2 @ 3.10GHz
>
> Network is bonded fail-over.
>
> Regards,
> Brett
>
> On 18 July 2018 at 13:15, Francesco Romani 
> wrote:
>
>> Thanks!
>>
>>
>> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>>
>>> Sure no problem, mom log attached.
>>>
>>
>> It seems to me that MOM is restarting *very* often:
>>
>> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
>> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>
>> And I totally not expect this. Martin, Andrej, is this the
>> expected behaviour?
>>
>>
>> Bests,
>>
>>
>>
>> --
>> Francesco Romani
>> Senior SW Eng., Virtualization R
>> Red Hat
>> IRC: fromani github: @fromanirh
>>
>>
>
>>>
>
>>>
>>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JVHD36FNHRKIXOMOJS7B5EQORTDGLPOR/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Andrej Krejcir]

I think the upgrade may not have affected the mom package, because no new
mom rpm version was released for a few months.

I'm not sure why the import fails even if the file is there.

As a workaround, mom can be configured to use the older (deprecated) json
rpc interface to communicate with vdsm. There is no difference in
functionality.
This can be done by editing /etc/vdsm/mom.conf. Set the
'hypervisor-interface' config value to 'vdsmjsonrpcbulk' insted of
'vdsmjsonrpcclient'.

On Wed, 25 Jul 2018 at 08:56, Maton, Brett  wrote:

> I upgraded my test cluster to 4.2.5.2-1 last night (hosts rebooted after
> update) and I've started to see the error again.
>
> As before it seems to be limited to the one host,  I've checked that
>
>
> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py
>
> exists, which it does but there are no matching .pyc / .pyo files which
> suggests that it's not been used?
>
> File permissions and SELinux contexts are the same on all files in the
> HypervisorInterfaces directory.
>
> 0644 root root system_u:object_r:lib_t:s0
>
> vdsm / mom log from time period attached.
>
>
> On 18 July 2018 at 16:10, Maton, Brett  wrote:
>
>> Thanks,
>>
>>   Cluster is all installed from pre-release, maybe I managed to get an
>> iffy rpm
>>
>> On 18 July 2018 at 15:50, Andrej Krejcir  wrote:
>>
>>> Yes, copying it from another host with mom version 0.5.12 is enough.
>>>
>>> I'm not sure why it was not installed. The file is in the rpm.
>>>
>>> On Wed, 18 Jul 2018 at 16:38, Maton, Brett 
>>> wrote:
>>>
 Bingo

 How could that file not be installed/deployed ?


 Should I simply copy it from one of the other hosts to make the message
 go away ?

 On 18 July 2018 at 15:30, Andrej Krejcir  wrote:

> Maybe the python module is missing. Can you check if this file exists?
>
>
> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py
>
>
> Andrej
>
> On Wed, 18 Jul 2018 at 16:04, Maton, Brett 
> wrote:
>
>> I just checked the mom version, it's already at 0.5.12
>>
>> # rpm -qa mom
>> mom-0.5.12-1.el7.centos.noarch
>>
>> All the physical hosts are fully updated /patched.
>>
>> Regards,
>> Brett
>>
>> On 18 July 2018 at 14:08, Andrej Krejcir  wrote:
>>
>>> Hi,
>>>
>>> Looking at MOM logs, the problem seems to be that MOM is configured
>>> to use 'vdsmjsonrpcclient' module to communicate with vdsm, but it
>>> cannot find this module, probably because it is an older version.
>>>
>>> Updating MOM to version 0.5.12 should fix it.
>>>
>>>
>>> Regards,
>>> Andrej
>>>
>>> On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
>>> wrote:
>>>
 FWIW:

 This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R)
 Xeon(R) CPU E3-1220 V2 @ 3.10GHz

 Network is bonded fail-over.

 Regards,
 Brett

 On 18 July 2018 at 13:15, Francesco Romani 
 wrote:

> Thanks!
>
>
> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>
>> Sure no problem, mom log attached.
>>
>
> It seems to me that MOM is restarting *very* often:
>
> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>
> And I totally not expect this. Martin, Andrej, is this the
> expected behaviour?
>
>
> Bests,
>
>
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R
> Red Hat
> IRC: fromani github: @fromanirh
>
>

>>

>>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/OJSGX7XKAKA3DVJ7VLFHS75A7XTXROYS/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

I upgraded my test cluster to 4.2.5.2-1 last night (hosts rebooted after
update) and I've started to see the error again.

As before it seems to be limited to the one host,  I've checked that

/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py

exists, which it does but there are no matching .pyc / .pyo files which
suggests that it's not been used?

File permissions and SELinux contexts are the same on all files in the
HypervisorInterfaces directory.

0644 root root system_u:object_r:lib_t:s0

vdsm / mom log from time period attached.


On 18 July 2018 at 16:10, Maton, Brett  wrote:

> Thanks,
>
>   Cluster is all installed from pre-release, maybe I managed to get an
> iffy rpm
>
> On 18 July 2018 at 15:50, Andrej Krejcir  wrote:
>
>> Yes, copying it from another host with mom version 0.5.12 is enough.
>>
>> I'm not sure why it was not installed. The file is in the rpm.
>>
>> On Wed, 18 Jul 2018 at 16:38, Maton, Brett 
>> wrote:
>>
>>> Bingo
>>>
>>> How could that file not be installed/deployed ?
>>>
>>>
>>> Should I simply copy it from one of the other hosts to make the message
>>> go away ?
>>>
>>> On 18 July 2018 at 15:30, Andrej Krejcir  wrote:
>>>
 Maybe the python module is missing. Can you check if this file exists?

 /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vd
 smjsonrpcclientInterface.py


 Andrej

 On Wed, 18 Jul 2018 at 16:04, Maton, Brett 
 wrote:

> I just checked the mom version, it's already at 0.5.12
>
> # rpm -qa mom
> mom-0.5.12-1.el7.centos.noarch
>
> All the physical hosts are fully updated /patched.
>
> Regards,
> Brett
>
> On 18 July 2018 at 14:08, Andrej Krejcir  wrote:
>
>> Hi,
>>
>> Looking at MOM logs, the problem seems to be that MOM is configured
>> to use 'vdsmjsonrpcclient' module to communicate with vdsm, but it
>> cannot find this module, probably because it is an older version.
>>
>> Updating MOM to version 0.5.12 should fix it.
>>
>>
>> Regards,
>> Andrej
>>
>> On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
>> wrote:
>>
>>> FWIW:
>>>
>>> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R)
>>> Xeon(R) CPU E3-1220 V2 @ 3.10GHz
>>>
>>> Network is bonded fail-over.
>>>
>>> Regards,
>>> Brett
>>>
>>> On 18 July 2018 at 13:15, Francesco Romani 
>>> wrote:
>>>
 Thanks!


 On 07/18/2018 02:11 PM, Maton, Brett wrote:

> Sure no problem, mom log attached.
>

 It seems to me that MOM is restarting *very* often:

 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
 2018-07-18 06:07:30,658 - mom - INFO - MOM starting

 And I totally not expect this. Martin, Andrej, is this the expected
 behaviour?


 Bests,



 --
 Francesco Romani
 Senior SW Eng., Virtualization R
 Red Hat
 IRC: fromani github: @fromanirh


>>>
>
>>>
>


logs.tgz
Description: GNU Zip compressed data
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/5FSBPGZQ6LW6GGLB7ZTCJF3E75B3OYG6/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

Thanks,

  Cluster is all installed from pre-release, maybe I managed to get an iffy
rpm

On 18 July 2018 at 15:50, Andrej Krejcir  wrote:

> Yes, copying it from another host with mom version 0.5.12 is enough.
>
> I'm not sure why it was not installed. The file is in the rpm.
>
> On Wed, 18 Jul 2018 at 16:38, Maton, Brett 
> wrote:
>
>> Bingo
>>
>> How could that file not be installed/deployed ?
>>
>>
>> Should I simply copy it from one of the other hosts to make the message
>> go away ?
>>
>> On 18 July 2018 at 15:30, Andrej Krejcir  wrote:
>>
>>> Maybe the python module is missing. Can you check if this file exists?
>>>
>>> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/
>>> vdsmjsonrpcclientInterface.py
>>>
>>>
>>> Andrej
>>>
>>> On Wed, 18 Jul 2018 at 16:04, Maton, Brett 
>>> wrote:
>>>
 I just checked the mom version, it's already at 0.5.12

 # rpm -qa mom
 mom-0.5.12-1.el7.centos.noarch

 All the physical hosts are fully updated /patched.

 Regards,
 Brett

 On 18 July 2018 at 14:08, Andrej Krejcir  wrote:

> Hi,
>
> Looking at MOM logs, the problem seems to be that MOM is configured to
> use 'vdsmjsonrpcclient' module to communicate with vdsm, but it
> cannot find this module, probably because it is an older version.
>
> Updating MOM to version 0.5.12 should fix it.
>
>
> Regards,
> Andrej
>
> On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
> wrote:
>
>> FWIW:
>>
>> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R)
>> Xeon(R) CPU E3-1220 V2 @ 3.10GHz
>>
>> Network is bonded fail-over.
>>
>> Regards,
>> Brett
>>
>> On 18 July 2018 at 13:15, Francesco Romani 
>> wrote:
>>
>>> Thanks!
>>>
>>>
>>> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>>>
 Sure no problem, mom log attached.

>>>
>>> It seems to me that MOM is restarting *very* often:
>>>
>>> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
>>> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
>>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>>
>>> And I totally not expect this. Martin, Andrej, is this the expected
>>> behaviour?
>>>
>>>
>>> Bests,
>>>
>>>
>>>
>>> --
>>> Francesco Romani
>>> Senior SW Eng., Virtualization R
>>> Red Hat
>>> IRC: fromani github: @fromanirh
>>>
>>>
>>

>>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/VYDBPUXSC5SONF7UFSCBAEJKD72B3XIZ/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Andrej Krejcir]

Yes, copying it from another host with mom version 0.5.12 is enough.

I'm not sure why it was not installed. The file is in the rpm.

On Wed, 18 Jul 2018 at 16:38, Maton, Brett  wrote:

> Bingo
>
> How could that file not be installed/deployed ?
>
>
> Should I simply copy it from one of the other hosts to make the message go
> away ?
>
> On 18 July 2018 at 15:30, Andrej Krejcir  wrote:
>
>> Maybe the python module is missing. Can you check if this file exists?
>>
>>
>> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py
>>
>>
>> Andrej
>>
>> On Wed, 18 Jul 2018 at 16:04, Maton, Brett 
>> wrote:
>>
>>> I just checked the mom version, it's already at 0.5.12
>>>
>>> # rpm -qa mom
>>> mom-0.5.12-1.el7.centos.noarch
>>>
>>> All the physical hosts are fully updated /patched.
>>>
>>> Regards,
>>> Brett
>>>
>>> On 18 July 2018 at 14:08, Andrej Krejcir  wrote:
>>>
 Hi,

 Looking at MOM logs, the problem seems to be that MOM is configured to
 use 'vdsmjsonrpcclient' module to communicate with vdsm, but it cannot
 find this module, probably because it is an older version.

 Updating MOM to version 0.5.12 should fix it.


 Regards,
 Andrej

 On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
 wrote:

> FWIW:
>
> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R)
> Xeon(R) CPU E3-1220 V2 @ 3.10GHz
>
> Network is bonded fail-over.
>
> Regards,
> Brett
>
> On 18 July 2018 at 13:15, Francesco Romani  wrote:
>
>> Thanks!
>>
>>
>> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>>
>>> Sure no problem, mom log attached.
>>>
>>
>> It seems to me that MOM is restarting *very* often:
>>
>> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
>> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>
>> And I totally not expect this. Martin, Andrej, is this the expected
>> behaviour?
>>
>>
>> Bests,
>>
>>
>>
>> --
>> Francesco Romani
>> Senior SW Eng., Virtualization R
>> Red Hat
>> IRC: fromani github: @fromanirh
>>
>>
>
>>>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/2CBNRFQTXEWN3YVFEFQYWVG4C4HAMHOV/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

Bingo

How could that file not be installed/deployed ?


Should I simply copy it from one of the other hosts to make the message go
away ?

On 18 July 2018 at 15:30, Andrej Krejcir  wrote:

> Maybe the python module is missing. Can you check if this file exists?
>
> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/
> vdsmjsonrpcclientInterface.py
>
>
> Andrej
>
> On Wed, 18 Jul 2018 at 16:04, Maton, Brett 
> wrote:
>
>> I just checked the mom version, it's already at 0.5.12
>>
>> # rpm -qa mom
>> mom-0.5.12-1.el7.centos.noarch
>>
>> All the physical hosts are fully updated /patched.
>>
>> Regards,
>> Brett
>>
>> On 18 July 2018 at 14:08, Andrej Krejcir  wrote:
>>
>>> Hi,
>>>
>>> Looking at MOM logs, the problem seems to be that MOM is configured to
>>> use 'vdsmjsonrpcclient' module to communicate with vdsm, but it cannot
>>> find this module, probably because it is an older version.
>>>
>>> Updating MOM to version 0.5.12 should fix it.
>>>
>>>
>>> Regards,
>>> Andrej
>>>
>>> On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
>>> wrote:
>>>
 FWIW:

 This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R)
 Xeon(R) CPU E3-1220 V2 @ 3.10GHz

 Network is bonded fail-over.

 Regards,
 Brett

 On 18 July 2018 at 13:15, Francesco Romani  wrote:

> Thanks!
>
>
> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>
>> Sure no problem, mom log attached.
>>
>
> It seems to me that MOM is restarting *very* often:
>
> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>
> And I totally not expect this. Martin, Andrej, is this the expected
> behaviour?
>
>
> Bests,
>
>
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R
> Red Hat
> IRC: fromani github: @fromanirh
>
>

>>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/TDIWVSH53LVIMHWA2VIQWAZZ47P6WYHV/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Andrej Krejcir]

Maybe the python module is missing. Can you check if this file exists?

/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py


Andrej

On Wed, 18 Jul 2018 at 16:04, Maton, Brett  wrote:

> I just checked the mom version, it's already at 0.5.12
>
> # rpm -qa mom
> mom-0.5.12-1.el7.centos.noarch
>
> All the physical hosts are fully updated /patched.
>
> Regards,
> Brett
>
> On 18 July 2018 at 14:08, Andrej Krejcir  wrote:
>
>> Hi,
>>
>> Looking at MOM logs, the problem seems to be that MOM is configured to
>> use 'vdsmjsonrpcclient' module to communicate with vdsm, but it cannot
>> find this module, probably because it is an older version.
>>
>> Updating MOM to version 0.5.12 should fix it.
>>
>>
>> Regards,
>> Andrej
>>
>> On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
>> wrote:
>>
>>> FWIW:
>>>
>>> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R) Xeon(R)
>>> CPU E3-1220 V2 @ 3.10GHz
>>>
>>> Network is bonded fail-over.
>>>
>>> Regards,
>>> Brett
>>>
>>> On 18 July 2018 at 13:15, Francesco Romani  wrote:
>>>
 Thanks!


 On 07/18/2018 02:11 PM, Maton, Brett wrote:

> Sure no problem, mom log attached.
>

 It seems to me that MOM is restarting *very* often:

 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
 2018-07-18 06:07:30,658 - mom - INFO - MOM starting

 And I totally not expect this. Martin, Andrej, is this the expected
 behaviour?


 Bests,



 --
 Francesco Romani
 Senior SW Eng., Virtualization R
 Red Hat
 IRC: fromani github: @fromanirh


>>>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/KTHJ66ZMEMTWS436GKU6TLM3WZE7HSXO/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

I just checked the mom version, it's already at 0.5.12

# rpm -qa mom
mom-0.5.12-1.el7.centos.noarch

All the physical hosts are fully updated /patched.

Regards,
Brett

On 18 July 2018 at 14:08, Andrej Krejcir  wrote:

> Hi,
>
> Looking at MOM logs, the problem seems to be that MOM is configured to use
> 'vdsmjsonrpcclient' module to communicate with vdsm, but it cannot find
> this module, probably because it is an older version.
>
> Updating MOM to version 0.5.12 should fix it.
>
>
> Regards,
> Andrej
>
> On Wed, 18 Jul 2018 at 14:41, Maton, Brett 
> wrote:
>
>> FWIW:
>>
>> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R) Xeon(R)
>> CPU E3-1220 V2 @ 3.10GHz
>>
>> Network is bonded fail-over.
>>
>> Regards,
>> Brett
>>
>> On 18 July 2018 at 13:15, Francesco Romani  wrote:
>>
>>> Thanks!
>>>
>>>
>>> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>>>
 Sure no problem, mom log attached.

>>>
>>> It seems to me that MOM is restarting *very* often:
>>>
>>> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
>>> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
>>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>>
>>> And I totally not expect this. Martin, Andrej, is this the expected
>>> behaviour?
>>>
>>>
>>> Bests,
>>>
>>>
>>>
>>> --
>>> Francesco Romani
>>> Senior SW Eng., Virtualization R
>>> Red Hat
>>> IRC: fromani github: @fromanirh
>>>
>>>
>>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/6MQMYJ7MKBYJJWNBXVAIEWZAGC4R4WKC/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Andrej Krejcir]

Hi,

Looking at MOM logs, the problem seems to be that MOM is configured to use
'vdsmjsonrpcclient' module to communicate with vdsm, but it cannot find
this module, probably because it is an older version.

Updating MOM to version 0.5.12 should fix it.


Regards,
Andrej

On Wed, 18 Jul 2018 at 14:41, Maton, Brett  wrote:

> FWIW:
>
> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R) Xeon(R)
> CPU E3-1220 V2 @ 3.10GHz
>
> Network is bonded fail-over.
>
> Regards,
> Brett
>
> On 18 July 2018 at 13:15, Francesco Romani  wrote:
>
>> Thanks!
>>
>>
>> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>>
>>> Sure no problem, mom log attached.
>>>
>>
>> It seems to me that MOM is restarting *very* often:
>>
>> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
>> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>>
>> And I totally not expect this. Martin, Andrej, is this the expected
>> behaviour?
>>
>>
>> Bests,
>>
>>
>>
>> --
>> Francesco Romani
>> Senior SW Eng., Virtualization R
>> Red Hat
>> IRC: fromani github: @fromanirh
>>
>>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/AJIX366ESGYMOXCNOCZHNM27HB4IQIH4/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

FWIW:

This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R) Xeon(R) CPU
E3-1220 V2 @ 3.10GHz

Network is bonded fail-over.

Regards,
Brett

On 18 July 2018 at 13:15, Francesco Romani  wrote:

> Thanks!
>
>
> On 07/18/2018 02:11 PM, Maton, Brett wrote:
>
>> Sure no problem, mom log attached.
>>
>
> It seems to me that MOM is restarting *very* often:
>
> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting
> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting
> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting
>
> And I totally not expect this. Martin, Andrej, is this the expected
> behaviour?
>
>
> Bests,
>
>
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R
> Red Hat
> IRC: fromani github: @fromanirh
>
>
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/446MWAKJYM2ZY5TCYUCCMVWXGDFM3SHC/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Francesco Romani]

Thanks!


On 07/18/2018 02:11 PM, Maton, Brett wrote:

Sure no problem, mom log attached.


It seems to me that MOM is restarting *very* often:

2018-07-18 06:07:04,909 - mom - INFO - MOM starting
2018-07-18 06:07:20,407 - mom - INFO - MOM starting
2018-07-18 06:07:30,658 - mom - INFO - MOM starting
2018-07-18 06:07:30,658 - mom - INFO - MOM starting

And I totally not expect this. Martin, Andrej, is this the expected 
behaviour?



Bests,


--
Francesco Romani
Senior SW Eng., Virtualization R
Red Hat
IRC: fromani github: @fromanirh
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/YOD45VBKT6BFGGABV4XUYEL6IQSJIMRL/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

Sure no problem, mom log attached.

On 18 July 2018 at 12:36, Francesco Romani  wrote:

>
> On 07/18/2018 07:24 AM, Maton, Brett wrote:
>
>> Thanks Francesco,
>>
>>   Log attached.
>>
>
> Interestings, it seems the fault comes from MOM:
>
> 2018-07-18 06:07:22,836+0100 INFO  (jsonrpc/3) [api.host] FINISH getStats
> error=:'NoneType' object has
> no attribute 'statistics'"> from=:::10.11.30.253,58840 (api:50)
> 2018-07-18 06:07:22,836+0100 ERROR (jsonrpc/3) [jsonrpc.JsonRpcServer]
> Internal server error (__init__:611)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line
> 606, in _handle_request
> res = method(**params)
>   File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201,
> in _dynamicMethod
> result = fn(*methodArgs)
>   File "", line 2, in getStats
>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in
> method
> ret = func(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1402, in
> getStats
> sampling.host_samples.stats())}
>   File "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 75, in
> get_stats
> ret.update(cif.mom.getKsmStats())
>   File "/usr/lib/python2.7/site-packages/vdsm/momIF.py", line 75, in
> getKsmStats
> stats = self._mom.getStatistics()['host']
>   File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
> return self.__send(self.__name, args)
>   File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
> verbose=self.__verbose
>   File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
> return self.single_request(host, handler, request_body, verbose)
>   File "/usr/lib64/python2.7/xmlrpclib.py", line 1306, in single_request
> return self.parse_response(response)
>   File "/usr/lib64/python2.7/xmlrpclib.py", line 1482, in parse_response
> return u.close()
>   File "/usr/lib64/python2.7/xmlrpclib.py", line 794, in close
> raise Fault(**self._stack[0])
> Fault: :'NoneType' object
> has no attribute 'statistics'">
> 2018-07-18 06:07:22,836+0100 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC
> call Host.getStats failed (error -32603) in 0.01 seconds (__init__:573)
> 2018-07-18 06:07:22,909+0100 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC
> call Host.ping2 succeeded in 0.00 seconds (__init__:573)
> 2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [api.virt] START getStats()
> from=::1,60058, vmId=a1af259b-7917-49ac-8bcc-eb4cc302e723 (api:46)
> 2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [api] FINISH getStats
> error=Virtual machine does not exist: {'vmId':
> u'a1af259b-7917-49ac-8bcc-eb4cc302e723'} (api:127)
> 2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [api.virt] FINISH getStats
> return={'status': {'message': "Virtual machine does not exist: {'vmId':
> u'a1af259b-7917-49ac-8bcc-eb4cc302e723'}", 'code': 1}} from=::1,60058,
> vmId=a1af259b-7917-49ac-8bcc-eb4cc302e723 (api:52)
> 2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC
> call VM.getStats failed (error 1) in 0.00 seconds (__init__:573)
> 2018-07-18 06:07:23,043+0100 INFO  (jsonrpc/6) [root]
> /usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err=
> (hooks:110)
>
> It could be a (benign) race on shutdown, but better continue the
> investigation. Could you please also chare the mom.log corresponding to the
> timeframe you already provided?
>
> Thanks,
>
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R
> Red Hat
> IRC: fromani github: @fromanirh
>
>


mom.log.gz
Description: GNU Zip compressed data
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/ABHDXFCEOXATO2VTTXFHF7QAFLHYKERP/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Francesco Romani]

On 07/18/2018 07:24 AM, Maton, Brett wrote:

Thanks Francesco,

  Log attached.


Interestings, it seems the fault comes from MOM:

2018-07-18 06:07:22,836+0100 INFO  (jsonrpc/3) [api.host] FINISH 
getStats error=:'NoneType' 
object has no attribute 'statistics'"> from=:::10.11.30.253,58840 
(api:50)
2018-07-18 06:07:22,836+0100 ERROR (jsonrpc/3) [jsonrpc.JsonRpcServer] 
Internal server error (__init__:611)

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 
606, in _handle_request

    res = method(**params)
  File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 201, 
in _dynamicMethod

    result = fn(*methodArgs)
  File "", line 2, in getStats
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, 
in method

    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1402, in 
getStats

    sampling.host_samples.stats())}
  File "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 75, in 
get_stats

    ret.update(cif.mom.getKsmStats())
  File "/usr/lib/python2.7/site-packages/vdsm/momIF.py", line 75, in 
getKsmStats

    stats = self._mom.getStatistics()['host']
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1306, in single_request
    return self.parse_response(response)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1482, in parse_response
    return u.close()
  File "/usr/lib64/python2.7/xmlrpclib.py", line 794, in close
    raise Fault(**self._stack[0])
Fault: :'NoneType' object 
has no attribute 'statistics'">
2018-07-18 06:07:22,836+0100 INFO  (jsonrpc/3) [jsonrpc.JsonRpcServer] 
RPC call Host.getStats failed (error -32603) in 0.01 seconds (__init__:573)
2018-07-18 06:07:22,909+0100 INFO  (jsonrpc/1) [jsonrpc.JsonRpcServer] 
RPC call Host.ping2 succeeded in 0.00 seconds (__init__:573)
2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [api.virt] START 
getStats() from=::1,60058, vmId=a1af259b-7917-49ac-8bcc-eb4cc302e723 
(api:46)
2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [api] FINISH getStats 
error=Virtual machine does not exist: {'vmId': 
u'a1af259b-7917-49ac-8bcc-eb4cc302e723'} (api:127)
2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [api.virt] FINISH 
getStats return={'status': {'message': "Virtual machine does not exist: 
{'vmId': u'a1af259b-7917-49ac-8bcc-eb4cc302e723'}", 'code': 1}} 
from=::1,60058, vmId=a1af259b-7917-49ac-8bcc-eb4cc302e723 (api:52)
2018-07-18 06:07:22,912+0100 INFO  (jsonrpc/4) [jsonrpc.JsonRpcServer] 
RPC call VM.getStats failed (error 1) in 0.00 seconds (__init__:573)
2018-07-18 06:07:23,043+0100 INFO  (jsonrpc/6) [root] 
/usr/libexec/vdsm/hooks/after_get_caps/50_openstacknet: rc=0 err= 
(hooks:110)


It could be a (benign) race on shutdown, but better continue the 
investigation. Could you please also chare the mom.log corresponding to 
the timeframe you already provided?


Thanks,

--
Francesco Romani
Senior SW Eng., Virtualization R
Red Hat
IRC: fromani github: @fromanirh
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZSEGPBX5D7CK2HSH3B5VM2Y35CTV4ZUN/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Maton, Brett]

Thanks Francesco,

  Log attached.

On 17 July 2018 at 13:12, Francesco Romani  wrote:

> On 07/17/2018 07:30 AM, Maton, Brett wrote:
>
> I've got one physical host in a 3 host CentOS7.5 cluster that reports the
> following error several times a day
>
> VDSM node3.example.com command Get Host Statistics failed: Internal
> JSON-RPC error: {'reason': ' \'exceptions.AttributeError\'>:\'NoneType\'
> object has no attribute \'statistics\'">'}
>
> Any ideas what the problem might be?
>
>
> Hi,
>
> I can make a wild guess[1], but let's try educated guesses first: could
> you please share the Vdsm logs around the time on which you see this error?
>
> Thanks,
>
> +++
>
> [1] It seems to me that a NIC card was removed from a VM during a
> statistic reporting cycle, or somehow failed to report stats. But really,
> is a wild guess.Let's look at the logs first.
>
> --
> Francesco Romani
> Senior SW Eng., Virtualization R
> Red Hat
> IRC: fromani github: @fromanirh
>
>
> ___
> Users mailing list -- users@ovirt.org
> To unsubscribe send an email to users-le...@ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-
> guidelines/
> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/
> message/JSRLKAZSSPXGUBDE7CXRQDXZTQJBY5YY/
>
>


vdsm.log.gz
Description: GNU Zip compressed data
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JQBKBLKCI7WJOERG6W6DOJ7P2DM4S34S/

[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

[Francesco Romani]

On 07/17/2018 07:30 AM, Maton, Brett wrote:
I've got one physical host in a 3 host CentOS7.5 cluster that reports 
the following error several times a day


VDSM node3.example.com  command Get Host 
Statistics failed: Internal JSON-RPC error: {'reason': '":\'NoneType\' object has no 
attribute \'statistics\'">'}


Any ideas what the problem might be?


Hi,

I can make a wild guess[1], but let's try educated guesses first: could 
you please share the Vdsm logs around the time on which you see this error?


Thanks,

+++

[1] It seems to me that a NIC card was removed from a VM during a 
statistic reporting cycle, or somehow failed to report stats. But 
really, is a wild guess.Let's look at the logs first.


--
Francesco Romani
Senior SW Eng., Virtualization R
Red Hat
IRC: fromani github: @fromanirh

___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/JSRLKAZSSPXGUBDE7CXRQDXZTQJBY5YY/