Actually I realuzed that the old test 0.14.0-1ubuntu5 is working and also had 
no regression-test.py nor the ppm files.
But the code in server/tests/test-display-base.c calls that python helper just 
the same:
  execve("regression-test.py", argv, envp);
So why isn't that failing, maybe it was never a valid test to begin with?.

I entered a test env of the test on Disco (with these issues present)
and checked how/why the test complete there.

It has an RC of 0, but that it has always if not crashing.
Still no files around:
ubuntu@autopkgtest:/tmp/autopkgtest.xBVfjt/build.efW/src$ find -name '*.ppm*'
ubuntu@autopkgtest:/tmp/autopkgtest.xBVfjt/build.efW/src$ find -name 
'*regression*'

Function regression_test is hit often.
Thread 1 "test-display-st" hit Breakpoint 1, regression_test () at 
test-display-base.c:101
101     {

There a child is supposed to start the test:
 114     pid = fork();                                                          
      
 115     if (pid == 0) {                                                        
      
 116         char buf[PATH_MAX];                                                
      
 117         char *argv[] = { NULL };                                           
      
 118         char *envp[] = {buf, NULL};                                        
      
 119                                                                            
      
 120         snprintf(buf, sizeof(buf), "PATH=%s", getenv("PATH"));             
      
 121         execve("regression-test.py", argv, envp);  

Setting gdb to follow that
(gdb) set follow-fork-mode child
(gdb) set detach-on-fork off
(gdb) run --automated-tests

And yeah it now is confirmed that the regression test always only "ran"
the test (which if breaking on execution would flag a bug) but it never
got to compare the actual output in the former modes.

So my upload will fix that to be on the same level as it was before -
but we need upstream and some understanding if we can make it actually
work (create valid ppm files with output that matches).

** Description changed:

  After the merge of 14.2 I realized that the autopkgtests failed.
  
  There are a few changes required that are due to the new version.
-     - d/t/automated-tests: spice-common moved into dir subprojects
-     - d/t/automated-tests: option --enable-automated-tests now is always on"
-     - d/t/automated-tests, d/t/control: make tests more debuggable by allowing
-       stderr
-     - d/t/control: install new test dependency python-pil
-     - d/t/base_test.ppm, d/t/regression-test.py: provide test resources from
-       upstream git not part of the released tarball anymore
-     - d/source/include-binaries: allow binary base_test.ppm in package
+     - d/t/automated-tests: spice-common moved into dir subprojects
+     - d/t/automated-tests: option --enable-automated-tests now is always on"
+     - d/t/automated-tests, d/t/control: make tests more debuggable by allowing
+       stderr
+     - d/t/control: install new test dependency python-pil
+     - d/t/base_test.ppm, d/t/regression-test.py: provide test resources from
+       upstream git not part of the released tarball anymore
+     - d/source/include-binaries: allow binary base_test.ppm in package
  
  Of these particularly interesting is that the python script and the
  image files are not even part of the release tarball anymore. So I
  wonder if this is really supposed to still be run that way.
  
  I got it resolved to "run" the test but it complained about an image mismatch.
  [...]
  req_cursor_notification
  wrote screen shot to output.ppm
  FAIL: Regressions were found!
-       Please, take a look in your code and go fix it!
+       Please, take a look in your code and go fix it!
  (process:21472): Spice-WARNING **: 08:00:30.990: Connection reset by peer
  (process:21472): Spice-WARNING **: 08:00:31.136: Connection reset by peer
  set_client_capabilities: present 0 caps 0
  client_monitors_config: NULL monitors_config
  
  There were plenty of changes to this test to make it work on windows, I
  wonder if one of them might have broken it - at least in the way we used
  to execute it.
  
  The images it generates are reproducible, but don't match.
  
  Since I had to tweak quite a lot to get it running in the first place
  I'm not entirely sure the test is valid at the moment.
  
- Therefore I'll upload the test fixes to run them (but it will ignore this 
error for now).
+ See comment #3 below for details on the old behavior which also was
+ incomplete running the test but not comparing output at all.
+ 
+ Therefore I'll upload the test fixes to run them (but it will ignore this 
error for now) - which is exactly the old state.
  => 
https://code.launchpad.net/~paelzer/ubuntu/+source/spice/+git/spice/+merge/369300
  
  To eventually resolve this I will report the problem upstream and
  depending on the outcome of that discussion we can fix our usage of the
  test OR drop it completely if it is no more meant to be run that way.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1834286

Title:
  autopkgtest fail in version 14.2 on test-display-streaming

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/spice/+bug/1834286/+subscriptions

-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to