Hi Bryan, Thank you very much for reaching us.
In general, the tests aren't expected to fail. I think that the problem is that the tests were written on Fedora, without considering other Linux distributions. If a test fails, it doesn't necessarily mean that the tested part of the OpenSCAP code doesn't work on Debian. It's more likely that there is something specific for Fedora in the tests. We welcome any patches that will fix the tests for Debian. Feel free to open a pull request. Fixing the tests will require deeper investigation. I suggest examining each failing test individually. To get detailed test results, you can run `ctest --verbose -R name_of_the_test` in the build directory. Also, the test cases are Bash scripts, so putting `set -x` at the beginning of each .sh file can help you to debug. If a test fails on "assert_exists" command, examine the results XML files generated during the tests. Adding `--verbose INFO` to `oscap` calls in the tests sometimes helps as well. There can be various different things that can cause the failures. For example, the tests can depend on something that doesn't exist on Debian, eg. they read from /etc/fedora-release or consult the RPM database. Things like that probably need to be replaced by something that exist on most of Linux distributions, or the test will have to contain 2 variants depending on the OS. Also consider disabling the RPM-related tests on Debian. Regards On Mon, Dec 9, 2019 at 5:48 PM Bryan Schneiders <pschneid...@trisept.com> wrote: > I’m trying to build or fix OpenSCAP content for Debian 10. I’m > encountering problems with existing content for Debian 8 and 9. To ensure > I’m not running into known or fixed issues, I’m attempting to build the > latest available openscap binaries since the Debian packages are old. > > > > When building the latest openscap on Debian 9 or 10, it fails several > tests in `make test`. I can run the failed tests individually and with > more verbose output but it’s not obvious why each test fails. Are these > tests expected to fail? Are there known solutions? Let me know what > details I can provide or point me in the right direction to troubleshoot > this deeper. I’m happy to contribute in any way toward solutions if there > are outstanding problems with some of the tests in this environment. > > > > I’m following the build instructions on > https://github.com/OpenSCAP/openscap/blob/maint-1.3/docs/developer/developer.adoc > > > > This is the output from the end of `make test` on Debian 9: > > > > ``` > > 94% tests passed, 10 tests failed out of 168 > > > > Total Test time (real) = 290.80 sec > > > > The following tests FAILED: > > 13 - API/OVAL/unittests/all.sh (Failed) > > 77 - API/XCCDF/unittests/test_xccdf_transformation.sh (Not > Run) > > 143 - probes/rpmverify/all.sh (Failed) > > 144 - probes/rpmverifyfile/all.sh (Failed) > > 145 - > probes/rpmverifypackage/test_probes_rpmverifypackage.sh (Not Run) > > 146 - probes/runlevel/test_probes_runlevel.sh (Failed) > > 150 - probes/symlink/test_offline_mode_symlink.sh (Not Run) > > 151 - probes/sysctl/all.sh (Failed) > > 157 - probes/uname/test_probes_uname.sh (Failed) > > 159 - sce/test_sce.sh (Failed) > > Errors while running CTest > > Makefile:127: recipe for target 'test' failed > > make: *** [test] Error 8 > > ``` > > > > This is the output from the end of `make test` on Debian 10: > > > > ``` > > 96% tests passed, 6 tests failed out of 167 > > > > Total Test time (real) = 317.42 sec > > > > The following tests did not run: > > 145 - > probes/rpmverifypackage/test_probes_rpmverifypackage.sh (Skipped) > > > > The following tests FAILED: > > 13 - API/OVAL/unittests/all.sh (Failed) > > 112 - DS/test_ds_misc.sh (Failed) > > 115 - DS/test_sds_eval.sh (Failed) > > 146 - probes/runlevel/test_probes_runlevel.sh (Failed) > > 150 - probes/sysctl/all.sh (Failed) > > 156 - probes/uname/test_probes_uname.sh (Failed) > > Errors while running CTest > > make: *** [Makefile:130: test] Error 8 > > ``` > > > > [image: TriSept Logo] > > > > *www.trisept.com <http://www.trisept.com/>* > > *Paul Bryan Schneiders* > > Principal Software Engineer > 302-858-0329 > > *pschneid...@trisept.com <pschneid...@trisept.com>* > > *CONFIDENTIALITY NOTICE:* This e-mail, including attachments, is intended > for the exclusive use of the person or entity to whom it is addressed and > may contain confidential or privileged information. If the reader of this > e-mail is not the intended recipient or his or her authorized agent, the > reader is hereby notified that any dissemination, distribution or copying > of this e-mail is prohibited. If you think that you have received this > e-mail in error, please advise the sender by reply e-mail of the error and > then delete this e-mail immediately. > > > > > _______________________________________________ > Open-scap-list mailing list > Open-scap-list@redhat.com > https://www.redhat.com/mailman/listinfo/open-scap-list -- Jan Černý Security Technologies | Red Hat, Inc.
_______________________________________________ Open-scap-list mailing list Open-scap-list@redhat.com https://www.redhat.com/mailman/listinfo/open-scap-list
