Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ Aug. 8th 2017 ] [ 001_upgrade_engine.test_initialize_engine ]

[Dafna Ron]

There is something going on withe the artifacts logs, they are only
partially downloaded (I sent a mail to lago team to have a look).

Here is the link to the setup logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/

The error would appear in the ovirt-engine-setup-20170808040357-4iiirj.log:

http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170808040357-4iiirj.log

Hope this helps.



On 08/08/2017 01:30 PM, Yedidyah Bar David wrote:
> On Tue, Aug 8, 2017 at 12:27 PM, Yaniv Kaul  wrote:
>>
>> On Tue, Aug 8, 2017 at 12:21 PM, Dafna Ron  wrote:
>>> Hi,
>>>
>>> We see a sporadic failure in the upgrade test. from what I can see from
>>> the log it is related to the firewalD package.
>>>
>>> Test failed: 001_upgrade_engine.test_initialize_engine
>>>
>>> Link to suspected patches:
>>>
>>> Link to Job:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/
>>>
>>> Link to all logs:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/
> Downloaded all exported artifacts, unzipped, and:
>
>>>
>>> Error snippet from the log:
>>>
>>> 
>>>
>>>
>>>
>>> 2017-08-08 04:09:11,006-0400 DEBUG otopi.plugins.otopi.network.firewalld
>>> plugin.execute:926 execute-output: ('/bin/firewall-cmd', '--zone',
>>> u'public', '--permanent', '--add-service', 'ovirt-postgres') stderr:
>>> ESC[91mError: Action org.fedoraproject.FirewallD1.all is not
>>> registeredESC[00m
>>>
>>> 2017-08-08 04:09:11,007-0400 DEBUG otopi.context
>>> context._executeMethod:142 method exception
>>> Traceback (most recent call last):
>>>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in
>>> _executeMethod
>>> method['method']()
>>>   File "/usr/share/otopi/plugins/otopi/network/firewalld.py", line 334, in
>>> _closeup
>>> '--add-service', service,
>>>   File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 931, in
>>> execute
>>> command=args[0],
>>> RuntimeError: Command '/bin/firewall-cmd' failed to execute
>>>
>>> 
> can't find this error.
>
> In the console log [1] I see:
>
> 08:09:14 [upgrade-from-prevrelease-suit] Error occured, aborting
> 08:09:14 [upgrade-from-prevrelease-suit] Traceback (most recent call last):
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 360, in
> do_run
> 08:09:14 [upgrade-from-prevrelease-suit]
> self.cli_plugins[args.ovirtverb].do_run(args)
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in
> do_run
> 08:09:14 [upgrade-from-prevrelease-suit] self._do_run(**vars(args))
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper
> 08:09:14 [upgrade-from-prevrelease-suit] return func(*args, **kwargs)
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper
> 08:09:14 [upgrade-from-prevrelease-suit] return func(*args,
> prefix=prefix, **kwargs)
> 08:09:14 [upgrade-from-prevrelease-suit]   File
> "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 99, in
> do_ovirt_runtest
> 08:09:14 [upgrade-from-prevrelease-suit] raise RuntimeError('Some
> tests failed')
> 08:09:14 [upgrade-from-prevrelease-suit] RuntimeError: Some tests failed
>
> But it's hard to understand what failed.
>
> If I look around in the web interface, I can find this in [2].
>
> So for some reason, downloading "all files in zip" misses many files.
> Not sure why. IIRC we already had similar cases in the past.
> When I press "(all files in zip)" in [3], I get a 134016 bytes
> archive.zip, which only has inside it "basic-suit-master-el7" (and
> "JenkinsTestedChangeList.dat"), no "upgrade-from-prevrelease-suit-master-el7"
> or "upgrade-from-release-suit-master-el7". Please check.
>
> [1] 
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/consoleFull
> [2] 
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170808040357-4iiirj.log
> [3] 
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/
>
>>>
>> Google shows we are not the only one suffering from it - see [1] and the
>> 

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ Aug. 8th 2017 ] [ 001_upgrade_engine.test_initialize_engine ]

[Yedidyah Bar David]

On Tue, Aug 8, 2017 at 12:27 PM, Yaniv Kaul  wrote:
>
>
> On Tue, Aug 8, 2017 at 12:21 PM, Dafna Ron  wrote:
>>
>> Hi,
>>
>> We see a sporadic failure in the upgrade test. from what I can see from
>> the log it is related to the firewalD package.
>>
>> Test failed: 001_upgrade_engine.test_initialize_engine
>>
>> Link to suspected patches:
>>
>> Link to Job:
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/
>>
>> Link to all logs:
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/

Downloaded all exported artifacts, unzipped, and:

>>
>>
>> Error snippet from the log:
>>
>> 
>>
>>
>>
>> 2017-08-08 04:09:11,006-0400 DEBUG otopi.plugins.otopi.network.firewalld
>> plugin.execute:926 execute-output: ('/bin/firewall-cmd', '--zone',
>> u'public', '--permanent', '--add-service', 'ovirt-postgres') stderr:
>> ESC[91mError: Action org.fedoraproject.FirewallD1.all is not
>> registeredESC[00m
>>
>> 2017-08-08 04:09:11,007-0400 DEBUG otopi.context
>> context._executeMethod:142 method exception
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in
>> _executeMethod
>> method['method']()
>>   File "/usr/share/otopi/plugins/otopi/network/firewalld.py", line 334, in
>> _closeup
>> '--add-service', service,
>>   File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 931, in
>> execute
>> command=args[0],
>> RuntimeError: Command '/bin/firewall-cmd' failed to execute
>>
>> 

can't find this error.

In the console log [1] I see:

08:09:14 [upgrade-from-prevrelease-suit] Error occured, aborting
08:09:14 [upgrade-from-prevrelease-suit] Traceback (most recent call last):
08:09:14 [upgrade-from-prevrelease-suit]   File
"/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 360, in
do_run
08:09:14 [upgrade-from-prevrelease-suit]
self.cli_plugins[args.ovirtverb].do_run(args)
08:09:14 [upgrade-from-prevrelease-suit]   File
"/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in
do_run
08:09:14 [upgrade-from-prevrelease-suit] self._do_run(**vars(args))
08:09:14 [upgrade-from-prevrelease-suit]   File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper
08:09:14 [upgrade-from-prevrelease-suit] return func(*args, **kwargs)
08:09:14 [upgrade-from-prevrelease-suit]   File
"/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper
08:09:14 [upgrade-from-prevrelease-suit] return func(*args,
prefix=prefix, **kwargs)
08:09:14 [upgrade-from-prevrelease-suit]   File
"/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 99, in
do_ovirt_runtest
08:09:14 [upgrade-from-prevrelease-suit] raise RuntimeError('Some
tests failed')
08:09:14 [upgrade-from-prevrelease-suit] RuntimeError: Some tests failed

But it's hard to understand what failed.

If I look around in the web interface, I can find this in [2].

So for some reason, downloading "all files in zip" misses many files.
Not sure why. IIRC we already had similar cases in the past.
When I press "(all files in zip)" in [3], I get a 134016 bytes
archive.zip, which only has inside it "basic-suit-master-el7" (and
"JenkinsTestedChangeList.dat"), no "upgrade-from-prevrelease-suit-master-el7"
or "upgrade-from-release-suit-master-el7". Please check.

[1] 
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/consoleFull
[2] 
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/exported-artifacts/upgrade-from-prevrelease-suit-master-el7/test_logs/upgrade-from-prevrelease-suite-master/post-001_upgrade_engine.py/lago-upgrade-from-prevrelease-suite-master-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170808040357-4iiirj.log
[3] http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/

>>
>>
>
> Google shows we are not the only one suffering from it - see [1] and the
> workarounds suggested.

The workaround [4] seems to have been to restart polkitd.
Do we want to do this?
Where? If in otopi, I'd consider this a somewhat ugly hack, and
would rather not, unless we have a clear and realistic reproducer.
If in CI, not sure where/when exactly.

Best,

[4] 
https://github.com/openshift/openshift-ansible/pull/3831/commits/457605bc7b8ba57b653eb14491f988a5565e3252

>
> Y.
> [1] https://github.com/openshift/openshift-ansible/issues/3213
>
>>
>>
>>
>>
>> ___
>> Devel mailing list
>> de...@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>
>
>
> ___
> Infra mailing list
> Infra@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>



-- 
Didi
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ Aug. 8th 2017 ] [ 001_upgrade_engine.test_initialize_engine ]

[Yaniv Kaul]

On Tue, Aug 8, 2017 at 12:21 PM, Dafna Ron  wrote:

> *Hi, *
>
>
>
> * We see a sporadic failure in the upgrade test. from what I can see from
> the log it is related to the firewalD package. Test failed:
> 001_upgrade_engine.test_initialize_engine Link to suspected patches: Link
> to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/
>  Link
> to all logs:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/
> 
> Error snippet from the log:  *
>
>
>
>
>
>
>
>
>
>
>
>
>
> *2017-08-08 04:09:11,006-0400 DEBUG otopi.plugins.otopi.network.firewalld
> plugin.execute:926 execute-output: ('/bin/firewall-cmd', '--zone',
> u'public', '--permanent', '--add-service', 'ovirt-postgres') stderr:
> ESC[91mError: Action org.fedoraproject.FirewallD1.all is not
> registeredESC[00m 2017-08-08 04:09:11,007-0400 DEBUG otopi.context
> context._executeMethod:142 method exception Traceback (most recent call
> last):   File "/usr/lib/python2.7/site-packages/otopi/context.py", line
> 132, in _executeMethod method['method']()   File
> "/usr/share/otopi/plugins/otopi/network/firewalld.py", line 334, in
> _closeup '--add-service', service,   File
> "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 931, in execute
> command=args[0], RuntimeError: Command '/bin/firewall-cmd' failed to
> execute  *
>

Google shows we are not the only one suffering from it - see [1] and the
workarounds suggested.

Y.
[1] https://github.com/openshift/openshift-ansible/issues/3213


>
>
>
> ___
> Devel mailing list
> de...@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

[ OST Failure Report ] [ oVirt Master ] [ Aug. 8th 2017 ] [ 001_upgrade_engine.test_initialize_engine ]

[Dafna Ron]

**

*Hi, *

*

**

We see a sporadic failure in the upgrade test. from what I can see from
the log it is related to the firewalD package.

Test failed: *001_upgrade_engine.test_initialize_engine*

Link to suspected patches:

Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/

Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/1684/artifact/


Error snippet from the log:




*
*2017-08-08 04:09:11,006-0400 DEBUG
otopi.plugins.otopi.network.firewalld plugin.execute:926 execute-output:
('/bin/firewall-cmd', '--zone', u'public', '--permanent',
'--add-service', 'ovirt-postgres') stderr:
ESC[91mError: Action org.fedoraproject.FirewallD1.all is not
registeredESC[00m

2017-08-08 04:09:11,007-0400 DEBUG otopi.context
context._executeMethod:142 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in
_executeMethod
method['method']()
  File "/usr/share/otopi/plugins/otopi/network/firewalld.py", line 334,
in _closeup
'--add-service', service,
  File "/usr/lib/python2.7/site-packages/otopi/plugin.py", line 931, in
execute
command=args[0],
RuntimeError: Command '/bin/firewall-cmd' failed to execute




*




___
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra