----- Original Message ----- > From: "Martin Preisler" <mprei...@redhat.com> > To: "SCAP Security Guide" <scap-security-gu...@lists.fedorahosted.org>, "Jan > Lieskovsky" <jlies...@redhat.com> > Cc: "open-scap-list" <open-scap-list@redhat.com> > Sent: Wednesday, July 27, 2016 8:01:19 PM > Subject: Re: Latest OpenSCAP changes to speed up SSG builds > > ----- Original Message ----- > > From: "Jan Lieskovsky" <jlies...@redhat.com> > > To: "SCAP Security Guide" <scap-security-gu...@lists.fedorahosted.org> > > Cc: "open-scap-list" <open-scap-list@redhat.com> > > Sent: Wednesday, July 27, 2016 9:34:04 AM > > Subject: Re: Latest OpenSCAP changes to speed up SSG builds > > > > ----- Original Message ----- > > > From: "Gabe Alford" <redhatri...@gmail.com> > > > To: "SCAP Security Guide" <scap-security-gu...@lists.fedorahosted.org> > > > Cc: "open-scap-list" <open-scap-list@redhat.com> > > > Sent: Wednesday, July 27, 2016 3:00:34 PM > > > Subject: Re: Latest OpenSCAP changes to speed up SSG builds > > > > > > > > > On Tue, Jul 26, 2016 at 3:24 PM, Martin Preisler < mprei...@redhat.com > > > > wrote: > > > > > > > > > I found pretty bad inefficiencies in some of our XSLTs. > > > > > > Check out > > > https://github.com/OpenSCAP/openscap/commit/a65bf27dec4a93e2b87cec8cbcd80bec4fd2328a > > > or > > > https://github.com/OpenSCAP/openscap/commit/1dd5573b2c964b00af57215cadb7f13b1938bac6 > > > > > > Long story short I was able to cut SSG build time on my machine from > > > 2m21.061s to 1m08.771s. In other words my machine now needs roughly half > > > the time to build SSG. I was timing `make -j 4`. > > > > > > It will take a while for these changes to make it into releases > > > and into distributions but I am writing this email because the > > > savings are significant and perhaps you want to enjoy them sooner. > > > SCAP Security Guide contributors are doing a lot of content builds. > > > > > > This will only work if you !!! have OPENSCAP 1.2.x !!! > > > > > > If you want to speed up the build, replace > > > > > > /usr/share/openscap/xsl/xccdf_1.1_to_1.2.xsl with > > > https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf_1.1_to_1.2.xsl > > > > > > /usr/share/openscap/xsl/xccdf-share.xsl with > > > https://raw.githubusercontent.com/OpenSCAP/openscap/maint-1.2/xsl/xccdf-share.xsl > > > > > > and > > > > > > /usr/share/openscap/xsl/xccdf-guide-impl.xsl with > > > https://github.com/OpenSCAP/openscap/blob/maint-1.2/xsl/xccdf-guide-impl.xsl > > > > > > > > > Perhaps it would be worth it to deploy this on our Jenkins slaves > > > even before it hits releases? What do you think? > > > > > > +1 > > > > Not like I would want to be the blocker for speedups / SSG Jenkins > > optimizations, but how would you like to deploy these changes? > > > > Two scenarios come to mind: > > * just replace the affected XSLT transformations in recent RHEL openscap > > builds, > > * make a separate openscap RPM (applied on Jenkins slaves outside of RHEL > > release cycle process) > > > > While the advantage might be more quicker completion of Jenkins SSG build > > jobs, there are two disadvantages / concerns related with this: > > * if modified oscap returns different result than latest oscap available > > in RHEL - which report should be considered the right one? Which one is > > valid? > > * the concern with maybe even more importance is we wouldn't test on > > default > > RHEL / Fedora systems like we do now. The environment would be modified a > > bit > > (and it might or might not return same results). So to ensure we "will > > continue working" on default RHEL / Fedora - should we create another > > Jenkins > > CI jobs for default RHEL / Fedora? > > > > Maybe we could use "speed optimized" openscap on > > 'scap-security-guide-pull-requests' > > Jenkins CI job, and 'default one' on 'scap-security-guide' Jenkins CI job > > (the latter > > one being which actually tells if we don't regress across PRs). > > I think that would overcomplicate things. I can relate to your concerns. > The only reason why we are even considering this is because the speed-up > is so dramatic. > > What do you think about me generating RHEL 6 and 7 datastreams and guides > and diff-ing them? If the results are the same across the board that would > take care of my concerns. Do you agree?
That would be helpful, yes. Thanks, Martin. There will be some differences (for example timestamp) for sure. But the point is to ensure there won't be some other inevitable deviations (that could lead e.g. to the reduction of the file size at the end). IOW verification if those XSLT changes are that isolated enough, it won't hurt when they are used also together with old openscap code base (without other patches from upstream being applied). Thanks, Jan -- Jan iankko Lieskovsky / Red Hat Security Technologies Team > > -- > Martin Preisler > Identity Management and Platform Security | Red Hat, Inc. > _______________________________________________ Open-scap-list mailing list Open-scap-list@redhat.com https://www.redhat.com/mailman/listinfo/open-scap-list
