I've written up some notes with things to think about when testing
changes to nightly(1). I've attached my draft, which is in XWiki
markup. If you've got any comments, please let me know. Otherwise,
I'll post this on the Tools community page sometime next week.
thanks,
mike
== Introduction ==
The ##nightly## script is used by many developers, in many different
situations, with a variety of different options. It's hard to
remember all the edge cases, and it's not unusual for someone to break
##nightly## when they introduce a change. This page gives some guidance
on what to consider when you change ##nightly##.
Think of the various test scenarios and inputs as a set of variables:
hardware (SPARC vs x86), debug versus non-debug, clobber build versus
incremental build, etc.
The first thing to do is to identify the variables that are important
for the change that you're making. Then identify interesting
combinations of those variables. To keep the number of test cases
sane, try to arrange your test cases so that they test multiple
variables. For example, if you need to test debug and non-debug
builds, and you need to test on both SPARC and x86, you could test a
debug build on SPARC and a non-debug build on x86. (For additional
guidance on this approach, see __The
Craft of Software Testing__ by Brian Marick, Prentice Hall, 1995.)
== Variables ==
This section gives the variables for ##nightly##. Each subsection names
a (conceptual) variable, gives the values for the variable, and
discusses testing the variable.
=== Hardware ===
* sparc
* i386
For the purposes of testing ##nightly##, we only care about testing
SPARC and x86 builds. This is because an ON build always builds both
32-bit and 64-bit binaries.
Depending on what part of the build you're changing, it may be okay to
do most of your testing on one [ISA]. Some sanity testing on the
other ISA is usually a good idea, though.
Note that the SPARC build only builds a 64-bit kernel, whereas the x86
build builds both 32-bit and 64-bit kernels. So if your changes
affect the kernel build, be sure to test on both platforms.
=== Debug Builds ===
* non-debug only (default)
* non-debug and debug (-D)
* debug only (-DF)
There are 2 options in ##NIGHTLY_OPTIONS## that control whether debug
binaries are built. The ##-D## option enables the debug build, and the
##-F## option suppresses the non-debug build.
Some of ##nightly##'s options, such as ##-l## (lint) are only effective
for debug builds.
If ##nightly## is doing both a non-debug and debug build, the non-debug
build is always done first. Be careful not to introduce changes that
cause one build to depend on the other.
By default, the same proto area is used for both debug and non-debug
builds. In some circumstances, ##nightly## will give the non-debug
build its proto area, which can cause problems if the user has
specified only a non-debug build. See MULTI_PROTO, below, for details.
=== MULTI_PROTO ===
* yes (separate proto areas for debug and non-debug builds)
* no (shared proto area for both builds)
If the user sets ##MULTI_PROTO## to ##yes##, or if this is forced by
options such as ##-O##, the non-debug build is given its own proto area,
typically ##$CODEMGR_WS/proto/root_$MACH-nd##. This is implemented by
temporarily changing the value of ##ROOT##. As a result, it can cause
problems if the user is only doing a non-debug build. See CR NNNNNNN
for details.
=== Incremental Build ===
* clobber build (default)
* incremental build (-i)
By default, ##nightly## cleans out the workspace before pulling from its
parent or starting the "make install". This cleanup include "make
install", deleting all the ##.make.state## files in the source tree,
and deleting the proto area(s). It also includes removing library
binaries and .o files, just in case the "make clobber" missed them.
If -i is specified, either in ##NIGHTLY_OPTIONS## or on the command
line, the above steps are skipped. If -i is specified on the command
line, some options, such as ##-l## (lint) are disabled.
It is okay for your changes to introduce warnings for an incremental
build, but the build must still be able to complete. If you do
introduce warnings, you should send a heads-up email after your
putback to tell gatelings not to worry about the warnings. This
typically happens when you have deleted or moved installed files,
which causes the packaging step of the build of complain that it
doesn't know how to package the old files.
=== Old/new Nightly ===
* new nightly, new source
* new nightly, old source
* old nightly, new source
Many developers run ##nightly## from /ws/onnv-tools/onbld/bin, which is
updated daily. In general, your new version of ##nightly## should be
able to build workspaces that haven't yet merged in your changes.
Note that a workspace might contain old source for the clobber phase
and then contain your changes for the "make install" phase. (See also
the section on "updates" below.)
It's not necessary to make the build work if the user pulls your
changes and then tries to build it with an old version of ##nightly##.
Unless you know that the old version of ##nightly## will work, you
should send a flag-day email to tell people to build with the new
version of ##nightly##.
=== updates ===
* create repo from parent
* update existing repo
* use existing repo (no update)
Don't assume that the repo existed before the user started
##nightly##.
As mentioned above ("old/new nightly"), the user's repo might not
contain your changes prior to the update. Make sure you handle that
transition case okay.
=== closed tree ===
* closed tree is present
* closed tree is absent
If the repo already exists, ##nightly## auto-detects whether the closed
tree is present. The user can override this by setting
##CLOSED_IS_PRESENT## to "yes" or "no" in the environment file.
If ##nightly## is creating the repo from the parent (as per "updates"
above), the user can tell ##nightly## not to create the closed tree
by setting ##CLOSED_CLONE_WS## to an empty string in the environment
file.
Some ##nightly## options, such as ##-O##, require that the closed tree
be present.
=== Consolidation ===
* ON
* SFW
Although ##nightly## was originally written for ON and it lives in the
ON source tree, do not assume that you are building an ON tree. SFW,
for example, uses TeamWare rather than Mercurial. It does not have a
closed tree, nor does it have signed crypto. Unless you are sure your
changes are innocuous, test building the other consolidations
listed above.
Check the consolidation's public README for the
location of public build machines, who to contact if you have
problems, etc. Do not assume that an ON build machine can be used to
build non-ON consolidations.
=== Tonic Build (-O) ===
* yes
* no
This option should not be confused with the ##-S O## option (see "Source
Builds", below).
The ##-O## option tells ##nightly## to generate deliverables for developers
outside of Oracle/Sun. These deliverables are used by corporate
partners, so Tonic builds still need to work.
=== Source Builds ===
* not a "source build"
* ##-S O##
* ##-S E##
* ##-S D##
The ##-S O## option verifies that the open tree builds, i.e., that it
does not have unexpected compile-time dependencies on the closed tree.
It has different semantics from the ##-O## option (above) and uses
different code paths in ##nightly##, so it must be tested separately.
In the past. Sun sold a source product which included (most of) the ON
source, with variations depending on whether the sources were for
outside the US ("export") or inside ("domestic"). The ##-S E## option
generates the export version and then verifies that it builds, the ##-S
D## option does the same for the domestic version.
If your test of the ##-S E## or ##-S D## option fails to build, check
whether the ON gate builds with that option. If it doesn't and there
isn't already a CR for the issue, file a defect CR to report the
problem (solaris/consolidation/os-net-misc).
=== Other NIGHTLY_OPTIONS ===
At a minimum, do a test build using the standard developer options. For
ON, these are the default options in usr/src/tools/env/developer.sh.
Other consolidations may have a corresponding environment file, or the
standard options might be listed in the consolidation's public README.
Review the supported options for any that are related to the changes
that you are making. For example, if you are changing how "make
check" works, you may want to do a test run without ##-C## to ensure
that you have not accidentally introduced a dependency on your new
"make check" code.
=== signing build ===
* yes
* no
By default, cryptographic binaries are signed with a certificate and
key that enables their use on the Sun/Oracle internal network. A
limited number of people are able to do a "signing build", in which
the resulting crypto binaries can be run anywhere.
If you are making changes that affect the code path for signing
builds, you can do some limited testing by setting the ##CODESIGN_USER##
environment variable to a non-null dummy value. Then comment out the
section of ##nightly## that re-signs the binaries (grep for
##CODESIGN_USER## to find the appropriate chunk of code). You will
probably want to do a final test with a real signing build. Contact
the gatekeepers or find someone in the security group who can do a
signing build with your changes.
_______________________________________________
tools-discuss mailing list
tools-discuss@opensolaris.org
