Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-24 Thread Julien Grall

Hi,

On 23/10/2017 16:57, Wei Liu wrote:

On Mon, Oct 23, 2017 at 03:38:33PM +0100, Andrew Cooper wrote:

On 23/10/17 15:34, Jan Beulich wrote:

On 23.10.17 at 15:58,  wrote:

On 23/10/17 09:40, Jan Beulich wrote:

On 23.10.17 at 01:49,  wrote:

flight 115037 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115037/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
  test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 114644
  test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 114644
  test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 114644

I'm puzzled by these recurring failures: Until flight 114525 all three
(plus the fourth sibling, which is in "guest-stop fail never pass" state)
were fail-never-pass on windows-install (the 64-bit host ones) or
guest-saverestore (the 32-bit host ones). Then flights 114540 and
114644 were successes, and since then guest-stop has been failing.
The guest console doesn't show any indication that the guest may
have received a shutdown signal.

Would it be possible of a platform specific bug? The last two flights
are failing on merlot1.

Not very likely here, I would say.


These tests have reliably never passed before, and there are no changes
recently (I'm aware of) which would cause them to start passing.


There is on osstest side -- we bumped the disk from 10G to 20G.

Previously the tests failed due to there was insufficient space to store
this iso and the guest image.


They seemed to have passed twice on two separate machine and then failed 
reliably again on merlot1/pinot0.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-23 Thread Wei Liu
On Mon, Oct 23, 2017 at 03:38:33PM +0100, Andrew Cooper wrote:
> On 23/10/17 15:34, Jan Beulich wrote:
>  On 23.10.17 at 15:58,  wrote:
> >> On 23/10/17 09:40, Jan Beulich wrote:
> >> On 23.10.17 at 01:49,  wrote:
>  flight 115037 xen-unstable real [real]
>  http://logs.test-lab.xenproject.org/osstest/logs/115037/ 
> 
>  Regressions :-(
> 
>  Tests which did not succeed and are blocking,
>  including tests which could not be run:
>    test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. 
>  vs. 114644
>    test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. 
>  vs. 114644
>    test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. 
>  vs. 114644
> >>> I'm puzzled by these recurring failures: Until flight 114525 all three
> >>> (plus the fourth sibling, which is in "guest-stop fail never pass" state)
> >>> were fail-never-pass on windows-install (the 64-bit host ones) or
> >>> guest-saverestore (the 32-bit host ones). Then flights 114540 and
> >>> 114644 were successes, and since then guest-stop has been failing.
> >>> The guest console doesn't show any indication that the guest may
> >>> have received a shutdown signal.
> >> Would it be possible of a platform specific bug? The last two flights 
> >> are failing on merlot1.
> > Not very likely here, I would say.
> 
> These tests have reliably never passed before, and there are no changes
> recently (I'm aware of) which would cause them to start passing.

There is on osstest side -- we bumped the disk from 10G to 20G.

Previously the tests failed due to there was insufficient space to store
this iso and the guest image.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-23 Thread Andrew Cooper
On 23/10/17 15:34, Jan Beulich wrote:
 On 23.10.17 at 15:58,  wrote:
>> On 23/10/17 09:40, Jan Beulich wrote:
>> On 23.10.17 at 01:49,  wrote:
 flight 115037 xen-unstable real [real]
 http://logs.test-lab.xenproject.org/osstest/logs/115037/ 

 Regressions :-(

 Tests which did not succeed and are blocking,
 including tests which could not be run:
   test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 
 114644
   test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 
 114644
   test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 
 114644
>>> I'm puzzled by these recurring failures: Until flight 114525 all three
>>> (plus the fourth sibling, which is in "guest-stop fail never pass" state)
>>> were fail-never-pass on windows-install (the 64-bit host ones) or
>>> guest-saverestore (the 32-bit host ones). Then flights 114540 and
>>> 114644 were successes, and since then guest-stop has been failing.
>>> The guest console doesn't show any indication that the guest may
>>> have received a shutdown signal.
>> Would it be possible of a platform specific bug? The last two flights 
>> are failing on merlot1.
> Not very likely here, I would say.

These tests have reliably never passed before, and there are no changes
recently (I'm aware of) which would cause them to start passing.

The windows VMs aren't running PV drivers, so have no clue about the
xenstore control key, or what a setting of shutdown is supposed to mean.

The bug is why there are two spurious passes.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-23 Thread Jan Beulich
>>> On 23.10.17 at 15:58,  wrote:
> On 23/10/17 09:40, Jan Beulich wrote:
> On 23.10.17 at 01:49,  wrote:
>>> flight 115037 xen-unstable real [real]
>>> http://logs.test-lab.xenproject.org/osstest/logs/115037/ 
>>>
>>> Regressions :-(
>>>
>>> Tests which did not succeed and are blocking,
>>> including tests which could not be run:
>>>   test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 
>>> 114644
>>>   test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 
>>> 114644
>>>   test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 
>>> 114644
>> 
>> I'm puzzled by these recurring failures: Until flight 114525 all three
>> (plus the fourth sibling, which is in "guest-stop fail never pass" state)
>> were fail-never-pass on windows-install (the 64-bit host ones) or
>> guest-saverestore (the 32-bit host ones). Then flights 114540 and
>> 114644 were successes, and since then guest-stop has been failing.
>> The guest console doesn't show any indication that the guest may
>> have received a shutdown signal.
> 
> Would it be possible of a platform specific bug? The last two flights 
> are failing on merlot1.

Not very likely here, I would say.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-23 Thread Ian Jackson
Julien Grall writes ("Re: [Xen-devel] [xen-unstable test] 115037: regressions - 
FAIL"):
> Would it be possible of a platform specific bug? The last two flights 
> are failing on merlot1.

The merlots are a highly unusual AMD machines which have NUMA nodes
with no memory and seem to sometimes have performance problems...

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-23 Thread Julien Grall

+ Andrew

Hi,

On 23/10/17 09:40, Jan Beulich wrote:

On 23.10.17 at 01:49,  wrote:

flight 115037 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115037/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
  test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 114644
  test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 114644
  test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 114644


I'm puzzled by these recurring failures: Until flight 114525 all three
(plus the fourth sibling, which is in "guest-stop fail never pass" state)
were fail-never-pass on windows-install (the 64-bit host ones) or
guest-saverestore (the 32-bit host ones). Then flights 114540 and
114644 were successes, and since then guest-stop has been failing.
The guest console doesn't show any indication that the guest may
have received a shutdown signal.


Would it be possible of a platform specific bug? The last two flights 
are failing on merlot1.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


Re: [Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-23 Thread Jan Beulich
>>> On 23.10.17 at 01:49,  wrote:
> flight 115037 xen-unstable real [real]
> http://logs.test-lab.xenproject.org/osstest/logs/115037/ 
> 
> Regressions :-(
> 
> Tests which did not succeed and are blocking,
> including tests which could not be run:
>  test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 
> 114644
>  test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 
> 114644
>  test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 
> 114644

I'm puzzled by these recurring failures: Until flight 114525 all three
(plus the fourth sibling, which is in "guest-stop fail never pass" state)
were fail-never-pass on windows-install (the 64-bit host ones) or
guest-saverestore (the 32-bit host ones). Then flights 114540 and
114644 were successes, and since then guest-stop has been failing.
The guest console doesn't show any indication that the guest may
have received a shutdown signal.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel


[Xen-devel] [xen-unstable test] 115037: regressions - FAIL

2017-10-22 Thread osstest service owner
flight 115037 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/115037/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stopfail REGR. vs. 114644
 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stop   fail REGR. vs. 114644
 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stop   fail REGR. vs. 114644

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt-xsm 14 saverestore-support-checkfail  like 114644
 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 114644
 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 114644
 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 114644
 test-armhf-armhf-libvirt 14 saverestore-support-checkfail  like 114644
 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail  like 114644
 test-amd64-amd64-xl-pvhv2-intel 12 guest-start fail never pass
 test-amd64-amd64-xl-pvhv2-amd 12 guest-start  fail  never pass
 test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  13 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  13 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qcow2 12 migrate-support-checkfail  never pass
 test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail   never pass
 test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl-credit2  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass
 test-armhf-armhf-xl-credit2  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass
 test-armhf-armhf-xl  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  13 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-xsm  14 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 13 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop  fail never pass
 test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail   never pass
 test-amd64-i386-xl-qemut-win10-i386 10 windows-install fail never pass
 test-amd64-i386-xl-qemuu-win10-i386 10 windows-install fail never pass
 test-amd64-amd64-xl-qemuu-win10-i386 10 windows-installfail never pass
 test-amd64-amd64-xl-qemut-win10-i386 10 windows-installfail never pass

version targeted for testing:
 xen  8e77dabc58c4b6c747dfb4b948551147905a7840
baseline version:
 xen  24fb44e971a62b345c7b6ca3c03b454a1e150abe

Last test of basis   114644  2017-10-17 10:49:11 Z5 days
Failing since114670  2017-10-18 05:03:38 Z4 days6 attempts
Testing same since   114808  2017-10-20 14:56:19 Z2 days4 attempts


People who touched revisions under test:
  Andrew Cooper 
  Anthony PERARD 
  David Esler 
  George Dunlap 
  Ian Jackson 
  Jan Beulich 
  Julien Grall 
  Roger Pau Monné 
  Stefano Stabellini 
  Tim Deegan 
  Wei Liu 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64-xtf  pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt