----- Original Message ----- > From: "Greg Elin" <grege...@gitmachines.com> > To: "SCAP Security Guide" <scap-security-guide@lists.fedorahosted.org> > Sent: Tuesday, September 2, 2014 5:21:23 PM > Subject: Re: New report and guide in openscap 1.1.0 > > On Mon, Sep 1, 2014 at 5:32 AM, Martin Preisler <mprei...@redhat.com> wrote: > > > ----- Original Message ----- > > > From: "Greg Elin" <grege...@gitmachines.com> > > > To: "SCAP Security Guide" <scap-security-guide@lists.fedorahosted.org> > > > Sent: Friday, August 29, 2014 11:18:37 PM > > > Subject: Re: New report and guide in openscap 1.1.0 > > > > > > I agree with Trey. The details would be more useful if they contained > > more > > > information about how something failed. I would expect that in a detail. > > > > As I said in my previous email, we show these details. We always have! > > > > They were not present in one of the previous ! draft ! versions of > > XCCDF report. When I released that draft I explicitly said that check > > details > > are not there but will be there in the final version. Are you sure you are > > looking at the new report? > > > > Thanks for the prodding. I looked more closely last night. At least two of > us had some confusion with this.
Many more people are confused by this AFAIK :-) > AFAIK, Oval results only show up if the `--oval-results` is added to `oscap > xccdf eval` command. It seems to me specific errors should be shared by > default and flag used to hide them. Yes, it is in the man page and of > course we all read the man page... (But I guess that is a comment for the > OpenSCAP List?) Yes, that is the case. We are planning to fix this with Result DataStreams at some point in the future but it's not in git yet. > Oval details show up on only some failed tests in the sample report ( > https://mpreisle.fedorapeople.org/openscap/1.1.0_xslt/report.html) > "package_aide_installed" for example does not show any oval details though > it fails while "service_atd_disabled" does show oval results. Our OVAL details implementation supports a subset of all possible OVAL objects. I think we support most of the use-cases where you really need to know which item is the culprit. We may support more at some point in the future. Patches are always welcome. > [snip] > > It seems OVAL details show up when there are specific "items violating." > That is _incredibly_ helpful for a test like "rpm_verify_permissions". It's > awesome. > > It seems OVAL details do not show up if the control just fails. That is > confusing. Didn't an OVAL criterion fail? Which one? Maybe there is more > than one criterion for a given test? The fact sometimes OVAL details show > up and sometimes they don't seems like yet another secret, undocumented > detail that is obvious to those in the know, but confusing to those that > aren't. At the very least, there should always be an explanation of the > OVAL detail even if it is "OVAL failed without any detail. Here is the > criterion that failed: " When we don't show anything in OVAL details it most likely means we don't support details for that particular OVAL object. > > > I'd be OK with default report ONLY showing details for the severity=high > > > items that I failed and all other details went either into a second > > report > > > > We show check system details for all failed rules, always. This works for > > both OVAL and SCE checks. > > > > What I was trying to get at is that I think fails that are high severity > should be treated differently in the report than those of medium or high. > For example, they could their own section, or -- as I was musing -- be the > only fails included in the report. I wanted to implement some sort of a "remediation priority list" that the report would generate for users on demand. You could print it and go through it in order. However I didn't have enough time to implement this in the end. If there is customer demand it may happen in the near future. > [snip] > > > Are you sure you want detailed reports of vulnerabilities on your > > infrastructure to end up on the "WEB"? One of our goals is to keep the > > reports and guides self sufficient. We do not want to rely on remote tools. > > That's why we bundle even all the JavaScript and CSS. > > > > > No, I do not want results of what controls *my* system passed or failed > posted on the web. > > I am speaking about the generic content that is a part of each control > being on the web: > rationale, links, related identifiers, remediation scripts, etc. > > The most important information in the results of my scan, are the details > about my system and my scan results. Generic information is _generic_. I > only want that generic information co-located with details when it > specifically helps me take an action. This is a general feature/issue with XCCDF and/or Source/Result DataStreams. Sorry but we can't fix this in report without making proprietary extensions. > Maybe what I am driving at as a general theme here is the difference > between an "Action/Working Report" optimized to help me get to work > resolving issues and "Complete Report" that represents an artifact > capturing a state that is less transitory. > > I'll try to mockup what I mean. Maybe consider patching the XSLT directly. I have restructured it and it's much easier to follow how it works now. > [snip] > > By static details I am referring to information in the SSG that does not > change from system to system or from scan result to scan result. > > I feel I should say a bit more about this because the above may seem > contradictory. > > It might seem I am one hand suggesting more information be added to the > report to make the report easier for beginners (e.g., include OVAL detail > on all failed tests). > > On the other hand, it might seem I am suggesting less information be > included in the report (e.g., removing generic static details, or putting > medium and low control fails details in a second report). > > I am arguing for clarity and less cognitively taxing presentation that does > way better than most other scan reports out there. > > Only some fails having an OVAL detail section seems ambiguous. Meanwhile > including all explanatory detail for every item often seems extraneous. > > If I knew the perfect algorithm, I would share it. I do know what I want to > read most of all in the report is : "The control ____ failed because > __________" and be able to follow a link to get more detail if I happen to > need it. OK, this clarification helped a lot. The only superfluous detail I know about is rule description. And we only show this in the modal dialog. Titles, IDs, references, identifiers, remediation fixes, severity, ... were all feature requests. > I think I really want a simple report that lists what failed and why, and > then a second report (or hidden second half of the report) that has way > more details. Having multiple reports adds maintenance costs. But maybe we can hide this additional information under a [+] button or something like that. -- Martin Preisler -- SCAP Security Guide mailing list scap-security-guide@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide https://github.com/OpenSCAP/scap-security-guide/
