Hi, Just a note: running tests is optional part of the build process, you can skip the tests and proceed to install.
On Tue, Dec 10, 2019 at 1:00 PM Jan Cerny <jce...@redhat.com> wrote: > Hi Bryan, > > Thank you very much for reaching us. > > In general, the tests aren't expected to fail. I think that the problem is > that the tests were written on Fedora, without considering other Linux > distributions. If a test fails, it doesn't necessarily mean that the tested > part of the OpenSCAP code doesn't work on Debian. It's more likely that > there is something specific for Fedora in the tests. > > We welcome any patches that will fix the tests for Debian. Feel free to > open a pull request. > > Fixing the tests will require deeper investigation. I suggest examining > each failing test individually. > To get detailed test results, you can run `ctest --verbose -R > name_of_the_test` in the build directory. Also, the test cases are Bash > scripts, so putting `set -x` at the beginning of each .sh file can help you > to debug. > If a test fails on "assert_exists" command, examine the results XML files > generated during the tests. Adding `--verbose INFO` to `oscap` calls in the > tests sometimes helps as well. > > There can be various different things that can cause the failures. For > example, the tests can depend on something that doesn't exist on Debian, > eg. they read from /etc/fedora-release or consult the RPM database. Things > like that probably need to be replaced by something that exist on most of > Linux distributions, or the test will have to contain 2 variants depending > on the OS. > Also consider disabling the RPM-related tests on Debian. > > Regards > > On Mon, Dec 9, 2019 at 5:48 PM Bryan Schneiders <pschneid...@trisept.com> > wrote: > >> I’m trying to build or fix OpenSCAP content for Debian 10. I’m >> encountering problems with existing content for Debian 8 and 9. To ensure >> I’m not running into known or fixed issues, I’m attempting to build the >> latest available openscap binaries since the Debian packages are old. >> >> >> >> When building the latest openscap on Debian 9 or 10, it fails several >> tests in `make test`. I can run the failed tests individually and with >> more verbose output but it’s not obvious why each test fails. Are these >> tests expected to fail? Are there known solutions? Let me know what >> details I can provide or point me in the right direction to troubleshoot >> this deeper. I’m happy to contribute in any way toward solutions if there >> are outstanding problems with some of the tests in this environment. >> >> >> >> I’m following the build instructions on >> https://github.com/OpenSCAP/openscap/blob/maint-1.3/docs/developer/developer.adoc >> >> >> >> This is the output from the end of `make test` on Debian 9: >> >> >> >> ``` >> >> 94% tests passed, 10 tests failed out of 168 >> >> >> >> Total Test time (real) = 290.80 sec >> >> >> >> The following tests FAILED: >> >> 13 - API/OVAL/unittests/all.sh (Failed) >> >> 77 - API/XCCDF/unittests/test_xccdf_transformation.sh (Not >> Run) >> >> 143 - probes/rpmverify/all.sh (Failed) >> >> 144 - probes/rpmverifyfile/all.sh (Failed) >> >> 145 - >> probes/rpmverifypackage/test_probes_rpmverifypackage.sh (Not Run) >> >> 146 - probes/runlevel/test_probes_runlevel.sh (Failed) >> >> 150 - probes/symlink/test_offline_mode_symlink.sh (Not Run) >> >> 151 - probes/sysctl/all.sh (Failed) >> >> 157 - probes/uname/test_probes_uname.sh (Failed) >> >> 159 - sce/test_sce.sh (Failed) >> >> Errors while running CTest >> >> Makefile:127: recipe for target 'test' failed >> >> make: *** [test] Error 8 >> >> ``` >> >> >> >> This is the output from the end of `make test` on Debian 10: >> >> >> >> ``` >> >> 96% tests passed, 6 tests failed out of 167 >> >> >> >> Total Test time (real) = 317.42 sec >> >> >> >> The following tests did not run: >> >> 145 - >> probes/rpmverifypackage/test_probes_rpmverifypackage.sh (Skipped) >> >> >> >> The following tests FAILED: >> >> 13 - API/OVAL/unittests/all.sh (Failed) >> >> 112 - DS/test_ds_misc.sh (Failed) >> >> 115 - DS/test_sds_eval.sh (Failed) >> >> 146 - probes/runlevel/test_probes_runlevel.sh (Failed) >> >> 150 - probes/sysctl/all.sh (Failed) >> >> 156 - probes/uname/test_probes_uname.sh (Failed) >> >> Errors while running CTest >> >> make: *** [Makefile:130: test] Error 8 >> >> ``` >> >> >> >> [image: TriSept Logo] >> >> >> >> *www.trisept.com <http://www.trisept.com/>* >> >> *Paul Bryan Schneiders* >> >> Principal Software Engineer >> 302-858-0329 >> >> *pschneid...@trisept.com <pschneid...@trisept.com>* >> >> *CONFIDENTIALITY NOTICE:* This e-mail, including attachments, is >> intended for the exclusive use of the person or entity to whom it is >> addressed and may contain confidential or privileged information. If the >> reader of this e-mail is not the intended recipient or his or her >> authorized agent, the reader is hereby notified that any dissemination, >> distribution or copying of this e-mail is prohibited. If you think that you >> have received this e-mail in error, please advise the sender by reply >> e-mail of the error and then delete this e-mail immediately. >> >> >> >> >> _______________________________________________ >> Open-scap-list mailing list >> Open-scap-list@redhat.com >> https://www.redhat.com/mailman/listinfo/open-scap-list > > > > -- > Jan Černý > Security Technologies | Red Hat, Inc. > -- Jan Černý Security Technologies | Red Hat, Inc.
_______________________________________________ Open-scap-list mailing list Open-scap-list@redhat.com https://www.redhat.com/mailman/listinfo/open-scap-list
