Bug#832806: Build time test failure

[Andreas Tille]

Hi Jose,

On Fri, Feb 09, 2018 at 01:58:48PM -0800, Jose Luis Rivero wrote:
> 
> I've been in contact with upstream and seems like it is a flaky test so
> they recommend me to disable it since it highly depend on time. I did that:
> 
> https://salsa.debian.org/science-team/ignition-common/commit/6aa2abd28993a131a99b1004af8c7eccea82cb2e
> 
> Let me know if this finally works for you or I need to keep working on it.

I've uploaded after adding some nocheck option to the test override. (Please 
pull)
(lintian -i would have told you about this.)

Kind regards

Andreas.

-- 
http://fam-tille.de

Bug#832806: Build time test failure

[Jose Luis Rivero]

On 08/02/18 05:25, Andreas Tille wrote:
>
> I can confirm that the problem is somehow randomized.  I tried the
> following patch which sometimes helps for non-reproducable issues
> 
>  $ git diff
> diff --git a/debian/rules b/debian/rules
> index 6b7ed6a..e677071 100755
> --- a/debian/rules
> +++ b/debian/rules
> @@ -21,7 +21,7 @@ override_dh_clean:
>  
>  override_dh_auto_test:
> mkdir -p $(BUILDHOME)
> -   HOME=$(BUILDHOME) dh_auto_test
> +   HOME=$(BUILDHOME) dh_auto_test --no-parallel
>  
>  %:
> dh $@ --buildsystem=cmake
>

The no parallel is good idea. Thanks, I added it to the package.

> 
> but this just led to a different problem:
> 
> ...

> [ RUN  ] WorkerPool.ThingsRunInParallel
> /build/ignition-common-1.0.1/src/WorkerPool_TEST.cc:136: Failure
> Value of: pool.WaitForResults(time)
>   Actual: false
> Expected: true
> /build/ignition-common-1.0.1/src/WorkerPool_TEST.cc:137: Failure
>   Expected: 2
> To be equal to: sentinel
>   Which is: 1
> [  FAILED  ] WorkerPool.ThingsRunInParallel (12 ms)
> [--] 6 tests from WorkerPool (37 ms total)
> 
> [--] Global test environment tear-down
> [==] 6 tests from 1 test case ran. (37 ms total)
> [  PASSED  ] 5 tests.
> [  FAILED  ] 1 test, listed below:
> [  FAILED  ] WorkerPool.ThingsRunInParallel
>

> Unfortunately I have no idea how to deal with this.  I'd recommend
> you ask about this problem at debian-scie...@lists.debian.org and/or
> debian-ment...@lists.debian.org.
>

I've been in contact with upstream and seems like it is a flaky test so
they recommend me to disable it since it highly depend on time. I did that:

https://salsa.debian.org/science-team/ignition-common/commit/6aa2abd28993a131a99b1004af8c7eccea82cb2e

Let me know if this finally works for you or I need to keep working on it.

Thanks a lot.

-- 
Jose Luis Rivero 

Bug#832806: Build time test failure

[Andreas Tille]

Hi Jose,

On Wed, Feb 07, 2018 at 03:33:00PM -0800, Jose Luis Rivero wrote:
> I was not able to reproduce the problem with gbp using the --pbuilder
> option. Our buildfarm that only runs docker (not pbuilder) also did not
> catch it. I've modified the rules file to run tests on an writable HOME
> directory:
> https://salsa.debian.org/science-team/ignition-common/commit/671996525d8c09327bce10cab47aa1f7d0800d89

I have pulled but I'm not sure whether this is related.  I keep on getting

...
 6/61 Test  #6: check_UNIT_Battery_TEST .   Passed0.12 sec
 7/61 Test  #5: UNIT_Battery_TEST ...   Passed0.12 sec
  Start  9: UNIT_Dem_TEST
  Start 10: check_UNIT_Dem_TEST
  Start 11: UNIT_EnumIface_TEST
 8/61 Test #11: UNIT_EnumIface_TEST .   Passed0.09 sec
 9/61 Test  #7: UNIT_Console_TEST ...***Failed0.22 sec
[==] Running 17 tests from 1 test case.
[--] Global test environment set-up.
[--] 17 tests from Console_TEST
[ RUN  ] Console_TEST.NoInitAndLog
/build/ignition-common-1.0.1/src/Console_TEST.cc:40: Failure
Value of: ignition::common::removeAll(absPath)
  Actual: false
Expected: true
[  FAILED  ] Console_TEST.NoInitAndLog (0 ms)
[ RUN  ] Console_TEST.InitAndLog
...
60/61 Test #60: STATIC_ASSERT_plugin_not_global_ns ..   Passed0.72 sec
61/61 Test #61: STATIC_ASSERT_plugin_wrong_base_class ...   Passed0.66 sec

98% tests passed, 1 tests failed out of 61

Total Test time (real) =   2.63 sec

The following tests FAILED:
  7 - UNIT_Console_TEST (Failed)
Errors while running CTest


I can confirm that the problem is somehow randomized.  I tried the
following patch which sometimes helps for non-reproducable issues

 $ git diff
diff --git a/debian/rules b/debian/rules
index 6b7ed6a..e677071 100755
--- a/debian/rules
+++ b/debian/rules
@@ -21,7 +21,7 @@ override_dh_clean:
 
 override_dh_auto_test:
mkdir -p $(BUILDHOME)
-   HOME=$(BUILDHOME) dh_auto_test
+   HOME=$(BUILDHOME) dh_auto_test --no-parallel
 
 %:
dh $@ --buildsystem=cmake



but this just led to a different problem:

...
54/61 Test #54: check_UNIT_VideoEncoder_TEST    Passed0.01 sec
  Start 55: UNIT_WorkerPool_TEST
55/61 Test #55: UNIT_WorkerPool_TEST ***Failed0.15 sec
[==] Running 6 tests from 1 test case.
[--] Global test environment set-up.
[--] 6 tests from WorkerPool
[ RUN  ] WorkerPool.OneWorkNoCallback
[   OK ] WorkerPool.OneWorkNoCallback (1 ms)
[ RUN  ] WorkerPool.OneWorkWithCallback
[   OK ] WorkerPool.OneWorkWithCallback (5 ms)
[ RUN  ] WorkerPool.LotsOfWork
[   OK ] WorkerPool.LotsOfWork (7 ms)
[ RUN  ] WorkerPool.WaitWithTimeout
[   OK ] WorkerPool.WaitWithTimeout (7 ms)
[ RUN  ] WorkerPool.WaitWithTimeoutThatTimesOut
[   OK ] WorkerPool.WaitWithTimeoutThatTimesOut (5 ms)
[ RUN  ] WorkerPool.ThingsRunInParallel
/build/ignition-common-1.0.1/src/WorkerPool_TEST.cc:136: Failure
Value of: pool.WaitForResults(time)
  Actual: false
Expected: true
/build/ignition-common-1.0.1/src/WorkerPool_TEST.cc:137: Failure
  Expected: 2
To be equal to: sentinel
  Which is: 1
[  FAILED  ] WorkerPool.ThingsRunInParallel (12 ms)
[--] 6 tests from WorkerPool (37 ms total)

[--] Global test environment tear-down
[==] 6 tests from 1 test case ran. (37 ms total)
[  PASSED  ] 5 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] WorkerPool.ThingsRunInParallel

 1 FAILED TEST

  Start 56: check_UNIT_WorkerPool_TEST
56/61 Test #56: check_UNIT_WorkerPool_TEST ..   Passed0.03 sec
  Start 57: INTEGRATION_plugin
...
61/61 Test #61: STATIC_ASSERT_plugin_wrong_base_class ...   Passed0.91 sec

98% tests passed, 1 tests failed out of 61

Total Test time (real) =   7.49 sec

The following tests FAILED:
 55 - UNIT_WorkerPool_TEST (Failed)
Errors while running CTest
Makefile:143: recipe for target 'test' failed


 
> Our buildfarm is happy (but it was not able to catch the previous error):
> https://build.osrfoundation.org/view/All/job/ignition-common-pkg_builder-master-debian_sid-amd64/46/
> 
> Andreas could you verify please if this fix the error for you?

Unfortunately I have no idea how to deal with this.  I'd recommend
you ask about this problem at debian-scie...@lists.debian.org and/or
debian-ment...@lists.debian.org.

Kind regards

   Andreas.

> On Mon, Feb 5, 2018 at 10:22 AM, Andreas Tille  wrote:
> 
> > Hi Jose,
> >
> > I had a sponsor look at ignition-common but when building in a pbuilder
> > chroot I've got:
> >
> > ...
> >  #7: UNIT_Console_TEST ...***Timeout 240.00 sec
> > [==] Running 17 tests from 1 test case.
> > [--] Global test environment set-up.
> > [--] 17 tests from Console_TEST
> > [ RUN  ] Console_TEST.NoInitAndLog
> > Error opening log 

Bug#832806: Build time test failure

[Jose Luis Rivero]

I was not able to reproduce the problem with gbp using the --pbuilder
option. Our buildfarm that only runs docker (not pbuilder) also did not
catch it. I've modified the rules file to run tests on an writable HOME
directory:
https://salsa.debian.org/science-team/ignition-common/commit/671996525d8c09327bce10cab47aa1f7d0800d89

Our buildfarm is happy (but it was not able to catch the previous error):
https://build.osrfoundation.org/view/All/job/ignition-common-pkg_builder-master-debian_sid-amd64/46/

Andreas could you verify please if this fix the error for you?


On Mon, Feb 5, 2018 at 10:22 AM, Andreas Tille  wrote:

> Hi Jose,
>
> I had a sponsor look at ignition-common but when building in a pbuilder
> chroot I've got:
>
> ...
>  #7: UNIT_Console_TEST ...***Timeout 240.00 sec
> [==] Running 17 tests from 1 test case.
> [--] Global test environment set-up.
> [--] 17 tests from Console_TEST
> [ RUN  ] Console_TEST.NoInitAndLog
> Error opening log file: /nonexistent/.ignition/auto_default.log
>
>
> 98% tests passed, 1 tests failed out of 61
>
> Total Test time (real) = 240.07 sec
>
> The following tests FAILED:
>   7 - UNIT_Console_TEST (Timeout)
> Errors while running CTest
> Makefile:143: recipe for target 'test' failed
> make[1]: *** [test] Error 8
> make[1]: Leaving directory '/build/ignition-common-1.0.1/
> obj-x86_64-linux-gnu'
>
>
> Kind regards
>
>Andreas.
>
> --
> http://fam-tille.de
>

Bug#832806: Build time test failure

[Andreas Tille]

Hi Jose,

I had a sponsor look at ignition-common but when building in a pbuilder
chroot I've got:

...
 #7: UNIT_Console_TEST ...***Timeout 240.00 sec
[==] Running 17 tests from 1 test case.
[--] Global test environment set-up.
[--] 17 tests from Console_TEST
[ RUN  ] Console_TEST.NoInitAndLog
Error opening log file: /nonexistent/.ignition/auto_default.log

  
98% tests passed, 1 tests failed out of 61

Total Test time (real) = 240.07 sec

The following tests FAILED:
  7 - UNIT_Console_TEST (Timeout)
Errors while running CTest
Makefile:143: recipe for target 'test' failed
make[1]: *** [test] Error 8
make[1]: Leaving directory '/build/ignition-common-1.0.1/obj-x86_64-linux-gnu'


Kind regards

   Andreas.

-- 
http://fam-tille.de