Hello folks, in relation with recent SCAP content failures: [1] https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-November/004547.html (I will come with a final fix for this one yet) [2] https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-November/004557.html (confirmed as valid report) [3] https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-November/004559.html (confirmed as valid) [4] https://lists.fedorahosted.org/pipermail/scap-security-guide/2013-November/004571.html (pending review)
the more urgent requirement / need how to have a way for automated current RHEL6 (and Fedora) SSG SCAP content testing available has arisen. While these issues (when noticed) are mainly easy to be fixed, if went unnoticed they could cause serious harm (false sense of security for failing RHEL6's sshd rule claiming it passed, not working configuration for the login.defs / auditd cases). To summary looks there's is an urgent need to have the content automatically tested somehow. We were thinking about this yesterday a bit with Simon. Below are the (mainly in form of brainstorming) proposals / comments: * looks it would not be possible to have the test suite directly as part of SSG content performing the tests during scap-security-guide package builds (for example to check sshd rules it would require to modify underlying system configuration which would not be straightforward on the system to be scanned), * selected subset of the rules (mainly checks and remediations changing configuration) could be tested via chroot directory. There's openscap offline scan feature: [5] https://www.redhat.com/archives/open-scap-list/2013-May/msg00007.html which could be used for this purpose (set up desired test directory structure somewhere on the filesystem [possibly /tmp?] and instruct oscap via those environment variables to perform rule evaluation for that directory structure). Have expected evaluation result for each of such configurations, and compare the obtained results with the expected ones. * the previous proposal wouldn't still catch case when for example 'PermitRootLogin' would be allowed by default in the sshd config. Rules like this require the OVAL rule evaluation to be performed on one hand, on the other hand some kind of bash script to be performed on the other (for this concrete case try to login remotely for root via SSH to see if it's allowed), and compare result of the oval scan with actual result from the system. For this case we have been thinking to have set of predefined system configurations (hopefully via kickstart files), and scripts to perform actual system behaviour checks. The party responsible for the testing would then use the kickstart file to automatically create the guest image of particular system, and would run those scripts on it (there could be some unified interface to simplify the test runs yet). We think that this could: * prevent occurrence of issues like the above [1] up to [4] in the future, * allow us to notice when something in the test outputs changed very early, so we can possibly adjust the OVAL checks and / or remediations, * speedup the certification process - since many of the failures would be caught yet sooner than the content is provided to some independent party for certification * besides all the above it would be possible right in the moment of providing new OVAL check / remediation to provide testing script for them too. The proposal would still require us to perform the tests manually. But would at least provided unified environment, the tests are expected to be performed at, and would provide testing scripts (IOW requirements these concrete scripts must pass for the OVAL check / remediation rule definition to be acceptable). To be honest I see that this is just brainstorming proposal. Many things would need to be changed during the way. Hopefully at the end we would end up combining the above two approaches (running set of tests via oscap chroot's / offline scan mode, another set via live testing operating system images and scripts modifying their behaviour in desired way). This has been written mainly with the intention, if there would be willingness SSG upstream / members to participate in this effort. If you have proposals, how this could be improved yet, we would like to hear about them. Once we have agreed on the preliminary form of the testing suite, we could move on how it would be implemented, and where the content (the testing configuration files, kickstart files, testing scripts) would be hosted. Another possibility is us to start doing this for current Fedora content, and apply it back against RHEL-6 content only in moment it has shown to be proper approach, mature enough. Comments / feedback welcome. Thank you && Regards, Jan. -- Jan iankko Lieskovsky / Red Hat Security Technologies Team _______________________________________________ scap-security-guide mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide
