On 5/13/14, 8:52 AM, Jan Lieskovsky wrote:
Hello Simon,

   thanks for the suggestions.

----- Original Message -----
From: "Simon Lukasik" <[email protected]>
To: "SCAP Security Guide" <[email protected]>
Sent: Tuesday, May 13, 2014 10:37:22 AM
Subject: Re: [PATCH] [RHEL] Make new 0.1.17 release

On 05/12/2014 10:03 PM, Shawn Wells wrote:
On 5/9/14, 2:04 PM, Jan Lieskovsky wrote:
----- Original Message -----
From: "Shawn Wells" <[email protected]>
To: [email protected]
Sent: Friday, May 9, 2014 6:03:20 PM
Subject: Re: [PATCH] [RHEL] Make new 0.1.17 release


On 5/9/14, 10:24 AM, Jan Lieskovsky wrote:
----- Original Message -----
From: "David Smith"<[email protected]>
To: "SCAP Security Guide"<[email protected]>
Sent: Friday, May 9, 2014 4:44:14 PM
Subject: Re: [PATCH] [RHEL] Make new 0.1.17 release



Jan,

Totally agreed on the need for the update; ack + thanks!
Thank you, David. Pushed to master. New tarball & EPEL build to come
shortly.
Jan, would you be able to document (even informally) the various steps?
While only extremely limited people have access to cut RPMs, it'd be
good to document things.
Sure thing (should I forgot some item - doing them on per moment basis
when realized forgot to do some of them, will re-iterate later).

Upstream process:
* update main scap-security-guide.spec file
(scap-security-guide/scap-security-guide.spec)
    with new version (value of "redhatssgversion" version variable).
Ensure
    the Release: field contains 1%{?dist} (1 as release version). Add
particular
    changelog entry (possibly verify for & fix whitespace noise)
* build & test the content (i.e. run 'make', 'make srpm', 'make rpm')
to verify
    it builds successfully. Also try to scan some systems with selected
profiles to see
    it the content works
* if it works, "make clean" in the git repository to start with clean
table.
    Make the source tarball via "make tarball". Upload the tarball to
repos.ssgproject.org.

EPEL process:
* file-in new EPEL-6 bugzilla (Summary = "Upgrade scap-security-guide
package to scap-security-guide-X.Y.Z")
    Note: That bugzilla is required later when creating Bodhi update
request. See below.
    Note#2: It would be created automa{g,t}ically once the "latest
upstream source tarball checking Red Hat
            Bugzilla functionality" would realize there's is new source
tarball available. But since we wan't
            immediate upgrade, we create that bug manually.

* take that bugzilla (state change NEW => ASSIGNED)
* clone the scap-security-guide git repository via fedpkg tool (as
documented in:

https://fedoraproject.org/wiki/Join_the_package_collection_maintainers#Import.2C_Commit.2Cand_Build_Your_Package


    section "Check out the module" and later ones). Split into coins
for our case it means:

    $ fedpkg clone scap-security-guide
    $ cd scap-security-guide/
* ensure to change the git branch from master/ to origin/el6 via
'switch-branch' fedpkg's option
    (this ensures the changes will be actually committed into EPEL-6
branch, and not into the master,
    IOW F-21 branch, which we don't want). To see the list of available
branches, issue the following:

    $ fedpkg switch-branch -l
    Locals:
    * master
    Remotes:
      origin/el6
      origin/epel7
      origin/f18
      origin/f19
      origin/f20
      origin/master

    To switch to el6 branch, issue:

      $ fedpkg switch-branch el6
      Branch el6 set up to track remote branch el6 from origin.

    command. Now it's possible actually to see the actual content of
EPEL-6 branch:

      $ ls
      scap-security-guide.spec  sources

      scap-security-guide.spec is the SPEC file used for build of
EPEL-6's rpm, sources
      text file contains md5sum of scap-security-guide tarball, which
will be build during
      SRPM / RPM build.

* to refresh both of them (*.spec & content of source) at once, it's
possible to create source
    RPM package & import it into fedpkg.

    Two important notes to mention here:

    - The spec file needs to be the updated one => it's necessary to
update the actual epel-6 one
    with changes from upstream one or replace the epel-6 one with
upstream one (the latter is still
    possible because as of right now there aren't epel-6 downstream
specific patches that woulnd't
    be present in upstream already. But should there be such changes in
the future, the epel-6 spec
    should be updated to include changes from upstream spec but
simultaneously to keep epel-6 custom
    patches. IOW just replacing epel-6's spec with upstream's one
wouldn't work, but manual changes
    would be necessary).

    - The new source tarball needs to be the last one uploaded to
repos.ssgproject.org (so md5sum
    would match during package build).

    This means:
    * start with clean /rpmbuild directory structure
    * download latest tarball from repos.ssgproject.org into
/rpmbuild/SOURCES
    * place the modified epel-6 spec file into /rpmbuild/SPECS
    * build the source rpm (result will be in /rpmbuild/SRPMS)

* return back to fedpkg & import the SRPM created in previous steps:

    $ fedpkg import path_to_srpm

    This will change content of 'sources' file (include new md5sum) &
update scap-security-guide.spec.
    $ git status

    to see what will get committed.

    $ git commit

    to confirm the changes. The commit message should contain the string:

    "Resolves: rh bz# id_of_the_epel_6_bug_we_created_before"

    in point #1 of the EPEL process section.

    Make scratch build to see the uploaded content (spec + tarball)
would actually build in the Koji
    build system via:

    $ fedpkg scratch-build --srpm path_to_srpm_created_locally_before

    Note: scratch-build to work with actually committed git repository
content, it requires the
    new content to be already "git push-ed" to the repository. But
since we want to verify if the
    content would build yet before pushing changes into the EPEL-6
repository, we need to provide
    the --srpm option pointing fedpkg to the local source RPM package
we have created one step before.

    Once the scratch build passes (visible in Koji web interface, or
also on command line), we can push
    the changes to git repository via:

    $ git push origin el6

    After successful push, our / latest push should be visible at (in
el6 branch)
      http://pkgs.fedoraproject.org/cgit/scap-security-guide.git/

    Now it's safe (scratch build succeeded & we pushed the changes to
the Fedora's git) to build real
    new package via:

    $ fedpkg build

    This again generates clickable link, at which it's possible to see
the progress / result of the build.
    Once the new package build in Koji finishes successfully, we flip
the previously created
    EPEL-6 bug to MODIFIED (ASSIGNED => MODIFIED) and mention the new
package name-version-release
    in the "Fixed in Version:" field of that bug.

* Having new build available, it's necessary to schedule new Bodhi
update (something like
    advisory to be tied with new packages). I am using web UI:
      https://admin.fedoraproject.org/updates/new/

    but there's command-line interface too (see [1] for further details).

    Add New Update screen is shown (containing following fields / items):

    Add New Update
    --------------
        Package: name-version-release of Koji build goes here (e.g.
scap-security-guide-0.1-16.el6)
        Type:    select one of - bugfix
/* intended for updates fixing bugs */
                                 enhancement
/* intended for updates adding new features */
                                 security
/* intended for updates fixing security flaws */
                                 newpackage
/* intended for updates introducing new RPM packages */
                 options
        Request: select 'testing' option of - testing
/* intended for updates that should reach -testing repository first
(before reaching -stable) */
                                              stable
/* intended for updates that should go into -stable directly - maybe
for critical security updates,
                                                                         other
package updates should be pushed to -testing repository first, then
only after some time
                                                                         into
-stable yum repository */
                                              none
/* not sure what's this one intended for */

        Bugs:   provide previously created EPEL-6 RH BZ# id here
[v] Close bugs when update is stable

/* Keep this checkbox checked in. It will automagically close the
                                                                         created
EPEL-6 bug, when the particular build is pushed to -stable yum
repository */

        Notes: Describe the changes in this text field (i.e. which bugs
got fixed, which new functionality got introduced etc.)
               The content of this field appear in the advisory (sent
on fedora-package-announce mailing list), when the build is pushed to
-stable.

       Suggest Reboot: []                   /* should the package
update require reboot after install */
       Enable karma automatism: [v]         /* if to use the karma
threshold the updates push system to use to decide if the build
                                               should be pushed to
-stable channel or not */
       Threshold for pushing to -stable: [3]      /* minimum threshold
/ level of karma build needs to obtain from package testers to be
                                                     able to push it
into -stable channel */
       Threshold for unpushing [-3]:              /* lower bound for
negative karma, which is a sign for the push system to move the
                                                     package from the
-testing repository. IOW the build has received so much negative
                                                     karma /
experiences, it's not usable even for the -testing repository and
should be
                                                     rebuilt */

      Once all the information is filed, click "Save Update".
      This will generate automated email about that build being pushed
to -testing. After some time at the same day (depending on TZ)
      the build is pushed to -testing repository.

* The maintainer should check Bodhi pages for that update for positive
/ negative karma / comments. If the build has reached positive karma >= 3
     it can be pushed to -stable (if it hasn't reached positive karma
= 3 in 7 days, it will be pushed to -stable - 7 days is considered
sufficient
     period). If there are signs of negative karma, the build should be
either unpushed / deleted & new one made.

* After 7 days the build can be pushed to -stable (under assumption it
didn't reach positive karma >= 3 sooner), meaning in next day or
    two it's reachable via yum subscribed to epel-6 repository directly.

* And the whole process goes back to the first item.
Hopefully I have covered all the points. Should I realize there's
missing some yet, will clarify later.
Whoa. This is great.
Yes. Thank You, Jan!

I've blocked off ~10 hours between now and Fri to catchup on SSG
activities. I'll spend some of that converting this into DocBook and
merging into our Developers Guide. It'll give us a solid starting point,
and should make future edits easier. Will report back Fri on progress
(+hopefully patches).
I believe that the el6 branch from Fedora repository should be dropped
once/when/if the scap-security-guide is released through standard Red
Hat Enterprise channel.
Correct, is this process described somewhere? Or just after scap-security-guide
RPM is released within standard Red Hat Enterprise Linux software updates
channels it's enough to create FESCO ticket to drop the EPEL-6 branch?


In the meantime, I have updated epel7 branch with the newest content
Yeah, noticed new build. Thanks for that.

and
I have one observation to the ssg release process. Currently we have two
latest releases:

      scap-security-guide-0.1.5.tar.gz
        - this contains latest fedora stuff
      scap-security-guide-0.1.17.tar.gz
        - this contains latest rhel stuff

Note that Fedora and EPEL7 packages ship combined content from both
packages. The reasoning behind is that with scap-workbench on your
fedora box you may want to use RHEL6 content to assess servers in the
basement.
Right, that's correct.

However, I believe it is not sound project governance to release
tarballs of the same name with the different content.
Agree this is not fortunate solution, but it "somehow" evolved that
way during the time (first there was Fedora rpm, then EPEL-6 rpm etc.)
Before (while having the same name) scap-security-guide contained
different content depending if we were on Red Hat Enterprise Linux or
Fedora. But we started to ship Red Hat Enterprise Linux datastreams content
also on Fedora (from reasons you outlined above), so now the originally
disjoint is converging more and it would make sense to build one tarball
containing everything (and to have just particular *.spec to decide which
part of the content should be extracted and which filtered out).

I would love to
see either solution:

   (1) Release everything in one tarball named scap-security-guide.
Timely releases would be plus. Or
I like the first option better. Will have a look (if others won't have
objections to it - if so, please speak up) which changes would be necessary
to have just one scap-security-guide-N-V-R.tar.gz generated.

Having all RPM releases on the same version (EPEL-6, EPEL-7, Fedora) seems most ideal.

Content versioning (within the XCCDF itself) can be done on a per-sub-project basis if needed.

   (2) Rename Fedora tarball to scap-security-guide-extras

If you could tag (`git tag` on particular commit) your releases in git
it would be also helpful.
Yeah, another good recommendation. Was thinking recently about that when
making 0.1.17 release, that it would be great if, e.g.

   $ git describe

in cloned repo would be able to provide version information. Now it does:

   $ git describe
   v0.5.0-InitialDraft-1070-gfc50879

so there obviously was such effort in the past, but we hopefully stopped
to perform the tagging?

IIRC, I added that tag during the first code submission to DISA FSO for the RHEL6 STIG. Eventually we started doing RPMs, versus sending raw code, so I stopped tagging. There's no reason why we shouldn't start that process up again.
_______________________________________________
scap-security-guide mailing list
[email protected]
https://lists.fedorahosted.org/mailman/listinfo/scap-security-guide

Reply via email to