Steve,
I'm primarily writing you because you're the only other serious scons
hacker on the dev team, but I'm including the dev list so everyone can
see what I'm thinking and perhaps comment. For those of you who don't
really care about the details of scons, you might still find the
attached patch handy as a nice set of utilities for helping you
compile and run m5.
1) First, I want to create a python package full of helper stuff for
SCons. The big question is, where should it go? I can put it in the
top level directory and call it buildlib or something like that. I'd
really rather not start the directory name with s, c, t, or u since
that will harm tab completion. I don't put my build directory in the
source directory, so I can imagine that others might even not really
like it to start with b. Any ideas? I could also put it in
src/python, but that might be too buried and since it really isn't
part of the source tree, it seems like the wrong place.
2) Do you think that build_opts stuff really helps? There are really
two things that the build_opts allows you to do. One is define the
ISA and emulation, and the other is define some other compile time
parameters such as SS_COMPATIBLE_FP or FAST_ALLOC_DEBUG. In practice,
I think users don't touch this, and developers only use the former to
provide new ISAs on a semi-permanent basis. If the latter is still
useful, I think we need to separate these two ideas because I think
that the new framework will require things to work a bit differently
so that options can import other options. e.g., there should be
something called ALPHA, something called SE, and ALPHA_SE should
include both of those. (This way I can construct the proper scons
environments and simplify the build.) For the ISA and emulation
parameters specifically, I expect to use something a bit more pythonic
remove them from the SCons options entirely. I'd probably just have a
subpackage in buildlib that describes the various ISA/Emulation
options, so there will be simple files that developers can modify as
we work with more ISAs. Once I do this, I wonder if build_opts is
really beneficial. If it is, it could be applied to a whole build
directory with a command line parameter instead of being part of the
build directory name. This would fit in with the stuff I've written
below.
3) right now, we parse the scons command line arguments quite a bit to
figure out what to build and where to build it. For example, if we're
not using the normal build directory, we parse the target to figure
out where the build directory is. This would allow us specify several
different build locations at once build1/build/ALPHA_FS,
build2/build/ALPHA_SE. In practice, I see no utility in this. I
suspect that we would continue to want to specify which binary to
build and thus parse that part of the command line. For the where to
build part, I just add a command line parameter "-D" perhaps that
specifies where the build should go? This would shorten command lines
quite a bit.
4) The previous question gets into a utility that I wrote for myself
(attached) that greatly simplifies building and running m5 by helping
you build the right command lines. I can contribute these as-is, but
I'd actually prefer to move much of the functionality directly into
scons. This would result in a scons command looking like:
scons -D <my build dir> --isas=alpha,sparc
--emulations=fullsys,syscall --regression=quick, instead of the crazy
command lines that we have right now (I will continue to support the
crazy command lines because that is inherently how scons works).
A big part (for me at least) of m5build and m5run is that they allow
you to label source tree, build directory pairs with labels and you
can build or run a labeled directory from anywhere. Much of this
can't really live in the SConstruct file because it's really used to
locate the sconstruct file itself.
I know this is a lot, but I'd love to hear your thoughts.
Nate
build: Add utilities for helping you build scons command lines and run m5.
The m5build script does two things, it helps you build m5 in any build
directory from any directory. The second thing it does is simplify the
construction of the scons command line. For example, you can say
% m5build --all work
This will construct a scons command line that will use your work tree
and build all variants of the code. (These things are configurable and
there is help information in the files.)
The m5run script simplifies building m5 command lines in that you can
say something like:
% m5run -g work <m5 options>
This will go and find the debug binary for the work tree and run it under
gdb passing the m5 options that you specified.
diff -r 5645632d594c util/build/dotm5.py
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/util/build/dotm5.py Wed Feb 11 12:51:40 2009 -0800
@@ -0,0 +1,48 @@
+# Copyright (c) 2006-2009 The Hewlett-Packard Development Company
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+# Authors: Nathan Binkert
+
+import os
+import sys
+
+from os.path import expanduser, isdir, isfile, join as joinpath
+
+homedir = os.environ['HOME']
+confdir = os.environ.get('M5_CONFIG', joinpath(homedir, '.m5'))
+confdir = expanduser(confdir)
+if not isdir(confdir):
+ raise ImportError, \
+ "could not find M5 configuration directory at %s" % confdir
+
+def find(filename):
+ path = joinpath(confdir, filename)
+ if not isfile(path):
+ raise AttributeError, \
+ "could not find %s in config directory: %s" % (filename, confdir)
+ return path
+
+__all__ = [ 'homedir', 'confdir', 'find' ]
diff -r 5645632d594c util/build/dotm5_build.py
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/util/build/dotm5_build.py Wed Feb 11 12:51:40 2009 -0800
@@ -0,0 +1,184 @@
+# Copyright (c) 2006-2009 The Hewlett-Packard Development Company
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+# Authors: Nathan Binkert
+
+import os
+import sys
+
+from os.path import expanduser, isdir, isfile, join as joinpath
+
+help = """\
+%prog assumes that the user has a ~/.m5/build.py (the directory
+~/.m5 is configurable with the M5_CONFIG environment variable) file
+describing the various versions of M5 that have been built.
+
+the config directory describes where in the filesystem source
+directories and potentially build directories live. Source
+directories must be specified, but build directories are optional.
+
+For example, one could keep sources in a shared filesystem (like nfs),
+say in ~/m5 and keep build output on local disks for each machine
+e.g. /home/<username>/build. This separation easily allows different
+os configurations to be used while at the same time significantly
+improving build time because output files don't go over the network.
+
+example .m5/build.py:
+
+# This file is actually -*- mode:python -*-
+
+default['binaries'] = [ 'opt' ]
+default['isas'] = [ 'alpha' ]
+default['emulations'] = [ 'fullsys' ]
+
+# arm build
+arm = M5Build('~/m5/work', '~/build/arm')
+arm.scons.append('EXTRAS=~/m5/arm_extras')
+
+# work build
+work = M5Build('~/m5/work', '~/build/work')
+work.scons.append('EXTRAS=~/m5/encumbered')
+work.isas = [ 'alpha', 'mips', 'sparc' ]
+work.binaries = [ 'debug', 'opt', 'fast', 'prof' ]
+work.emulations = [ 'fullsys', 'syscall' ]
+
+# x86 code (notice that it's working out of the work directory
+x86 = M5Build('~/m5/work', '~/build/work')
+x86.isas = [ 'x86' ]
+x86.emulations = [ 'fullsys', 'syscall' ]
+x86.experimental = True
+"""
+
+# default scons command line options
+scons_options = []
+
+# default options
+default = {
+ 'binaries' : [ 'debug', 'opt' ],
+ 'isas' : [ 'alpha' ],
+ 'emulations' : [ 'fullsys', 'syscall' ] }
+
+# options chosen when --all is selected
+all = {
+ 'binaries' : [ 'debug', 'opt', 'fast' ],
+ 'isas' : [ 'alpha', 'arm', 'mips', 'sparc', 'x86' ],
+ 'emulations' : [ 'fullsys', 'syscall' ] }
+
+# valid options
+valid = {
+ 'binaries' : [ 'debug', 'opt', 'fast', 'prof' ],
+ 'isas' : [ 'alpha', 'arm', 'mips', 'sparc', 'x86' ],
+ 'emulations' : [ 'fullsys', 'syscall' ] }
+
+# valid combinations of ISA and emulation mode
+valid_combinations = {
+ ('alpha', 'syscall') : 'ALPHA_SE',
+ ('alpha', 'fullsys') : 'ALPHA_FS',
+ ('mips', 'syscall') : 'MIPS_SE',
+ ('sparc', 'syscall') : 'SPARC_SE',
+ ('sparc', 'fullsys') : 'SPARC_FS',
+ ('x86', 'syscall') : 'X86_SE',
+ ('x86', 'fullsys') : 'X86_FS' }
+
+# experimental combinations of ISA and emulation mode
+experimental_combinations = {
+ ('arm', 'syscall') : 'ARM_SE' }
+
+class M5Build(object):
+ def __init__(self, source, build='', binaries=None, isas=None,
+ emulations=None, scons=None):
+ self.source = expanduser(source)
+ self.build = build and expanduser(build)
+ self.binaries = binaries
+ self.isas = isas
+ self.emulations = emulations
+ self.scons = scons or []
+ self.experimental = False
+
+ self.validated = False
+ self.run_mode = False
+
+ def target(self, *target):
+ if self.run_mode:
+ build = self.build
+ if not build:
+ build = self.source
+ binary = joinpath(build, 'build', *target)
+ if not isfile(binary):
+ raise AttributeError, \
+ 'cannot find file %s' % binary
+
+ return binary
+
+
+ if not self.validated:
+ if not isdir(self.source):
+ raise AttributeError, \
+ 'source directory %s not found' % self.source
+
+ if self.build and not isdir(self.build):
+ os.mkdir(self.build)
+
+ self.validated = True
+
+ return joinpath(self.build, 'build', *target)
+
+ def valid_combination(self, isa, emul):
+ if (isa, emul) in valid_combinations:
+ return valid_combinations[isa, emul]
+
+ if self.experimental and (isa, emul) in experimental_combinations:
+ return experimental_combinations[isa, emul]
+
+ return None
+
+ def _target(self, isa, emul, *args):
+ comb = self.valid_combination(isa, emul)
+ if not comb:
+ raise AttributeError, \
+ "Invalid isa/emulation combination: %s/%s" % (isa, emul)
+
+ return self.target(comb, *args)
+
+ def test(self, binary, isa, emul, test):
+ args = [isa, emul, 'tests', binary]
+ if test != 'all':
+ args.append(test)
+
+ return self._target(*args)
+
+ def unittest(self, binary, isa, emul, test):
+ return self._target(isa, emul, 'unittest', '%s.%s' % (test, binary))
+
+ def binary(self, binary, isa, emul):
+ return self._target(isa, emul, 'm5.%s' % binary)
+
+import dotm5
+conf_build = dotm5.find('build.py')
+
+distcc=False
+ccache=False
+execfile(conf_build)
diff -r 5645632d594c util/build/m5build
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/util/build/m5build Wed Feb 11 12:51:40 2009 -0800
@@ -0,0 +1,306 @@
+#!/usr/bin/env python
+
+# Copyright (c) 2006-2009 The Hewlett-Packard Development Company
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+# Authors: Nathan Binkert
+
+import os
+import sys
+
+import dotm5_build
+from dotm5_build import all, default, valid
+
+#
+# do argument parsing
+#
+progname = sys.argv[0]
+
+import optparse
+
+usage = '''%prog [compile options] <version> [SCons options]
+
+%prog assumes that the user has a .m5 directory with a build.py file
+describing the various versions of M5 that have been compiled. See
+--help-config for more information.
+'''
+
+version = '%prog 0.1'
+parser = optparse.OptionParser(usage=usage, version=version,
+ formatter=optparse.TitledHelpFormatter())
+parser.disable_interspersed_args()
+
+#
+# helper functions for the option parser stuff
+#
+
+# current option group
+group = None
+
+def set_group(*args, **kwargs):
+ '''set the current option group'''
+ global group
+ if not args and not kwargs:
+ group = None
+ else:
+ group = parser.add_option_group(*args, **kwargs)
+
+class splitter(object):
+ def __init__(self, split):
+ self.split = split
+ def __call__(self, option, opt_str, value, parser):
+ getattr(parser.values, option.dest).extend(value.split(self.split))
+
+def add_option(*args, **kwargs):
+ '''add an option to the current option group, or global none set'''
+
+ # if action=split, but allows the option arguments
+ # themselves to be lists separated by the split variable'''
+
+ if kwargs.get('action', None) == 'append' and 'split' in kwargs:
+ split = kwargs.pop('split')
+ kwargs['default'] = []
+ kwargs['type'] = 'string'
+ kwargs['action'] = 'callback'
+ kwargs['callback'] = splitter(split)
+
+ if group:
+ return group.add_option(*args, **kwargs)
+ else:
+ return parser.add_option(*args, **kwargs)
+
+def bool_option(name, default, help):
+ '''add a boolean option called --name and --no-name.
+ Display help depending on which is the default'''
+
+ tname = '--%s' % name
+ fname = '--no-%s' % name
+ dest = name.replace('-', '_')
+ if default:
+ thelp = optparse.SUPPRESS_HELP
+ fhelp = help
+ else:
+ thelp = help
+ fhelp = optparse.SUPPRESS_HELP
+
+ add_option(tname, action="store_true", default=default, help=thelp)
+ add_option(fname, action="store_false", dest=dest, help=fhelp)
+
+#
+# Compile options
+#
+
+parser.add_option('--help-config', default=False, action='store_true',
+ help="Print out help for config files")
+
+# Options that affect scons
+add_option('-C', "--ccache", default=dotm5_build.ccache, action='store_true',
+ help="use ccache to help compile")
+add_option('-D', "--distcc", default=dotm5_build.distcc, action='store_true',
+ help="use distcc to help compile")
+add_option('-I', "--interactive", action='store_true', default=False,
+ help="enable interactive builds")
+add_option('-j', '--jobs', type='int', default=1,
+ help='number of parallel jobs to use')
+add_option('-k', '--keep-going', action='store_true', default=False,
+ help="continue as much as possible after an error")
+add_option('-n', "--no-exec", default=False, action='store_true',
+ help="don't actually invoke scons, just echo SCons command line")
+
+# Options selecting what to build
+add_option('-A', "--all", default=False, action='store_true',
+ help="compile everything that can be compiled")
+add_option('-E', "--experimental", action='store_true', default=False,
+ help="enable experimental builds")
+add_option('-r', "--regress", metavar="TEST[,TEST]", action='append',
+ split=',', help="do specific tests")
+add_option('-u', "--unittest", metavar="TEST[,TEST]", action='append',
+ split=',', help="do specific unit tests")
+add_option('-v', "--verbose", default=False, action='store_true',
+ help="be verbose")
+add_option('-t', '--targets', metavar='TARGET,[TARGET]', action='append',
+ split=',', help="specify targets")
+
+
+set_group("Binary Type Options")
+add_option('-b', "--binaries", metavar="BIN[,BIN]", action='append', split=',',
+ default=[],
+ help="Output binary types")
+
+for binary in valid['binaries']:
+ bool_option(binary, None, "compile %s binaries" % binary)
+
+set_group("ISA Options")
+add_option('-i', "--isas", metavar="ISA[,ISA]", action='append', split=',',
+ default=[],
+ help="Compiled ISAs")
+
+for isa in valid['isas']:
+ bool_option(isa, None, "%s instruction set" % isa.upper())
+
+set_group("Emulation Options")
+add_option('-e', "--emulations", metavar="EMUL[,EMUL]", action='append',
+ split=',', default=[],
+ help="Emulation options")
+
+for emulation in valid['emulations']:
+ bool_option(emulation, None, "%s emulation" % emulation)
+
+#
+# Usage
+#
+def usage(exitcode=None):
+ parser.print_help()
+ if exitcode is not None:
+ sys.exit(exitcode)
+
+#
+# Option defaults and validation
+#
+(options, args) = parser.parse_args()
+
+if options.help_config:
+ print dotm5_build.help
+ sys.exit(0)
+
+if not args:
+ usage(2)
+
+builder_name = args.pop(0)
+builder = getattr(dotm5_build, builder_name)
+
+def get_info(builder, options, name):
+ values = getattr(options, name)
+ if not values:
+ if options.all:
+ values = all[name]
+ elif 'all' in values:
+ values = all[name]
+
+ for opt in valid[name]:
+ if not hasattr(options, opt):
+ continue
+
+ val = getattr(options, opt)
+ if val is None:
+ continue
+
+ if not val:
+ values.remove(opt)
+ else:
+ values.append(opt)
+
+ if not values:
+ values = getattr(builder, name)
+ if not values:
+ values = default[name]
+
+ for val in values:
+ if val not in valid[name]:
+ raise AttributeError, \
+ "'%s' is not a valid attribute for '%s'" % (val, name)
+ return values
+
+binaries = get_info(builder, options, 'binaries')
+isas = get_info(builder, options, 'isas')
+emulations = get_info(builder, options, 'emulations')
+
+scons_options = dotm5_build.scons_options[:]
+
+scons_options.extend(builder.scons)
+
+builds = []
+for isa in isas:
+ for emul in emulations:
+ if builder.valid_combination(isa, emul):
+ builds.append((isa, emul))
+
+if not builds:
+ sys.exit("must specify at least one valid combination of ISA and mode")
+
+#
+# Convert options into SCons command line arguments
+#
+if options.jobs != 1:
+ args.append('-j %d' % options.jobs)
+
+if options.interactive:
+ args.append('--interactive')
+
+if options.keep_going:
+ args.append('-k')
+
+if options.ccache or options.distcc:
+ scons_options.append("BATCH=True")
+if options.ccache:
+ if options.distcc:
+ os.environ['CCACHE_PREFIX'] = "distcc"
+ scons_options.append("BATCH_CMD=ccache")
+elif options.distcc:
+ scons_options.append("BATCH_CMD=distcc")
+
+targets = []
+if options.targets:
+ for target in options.targets:
+ targets.append(builder.target(target))
+else:
+ for bin in binaries:
+ for isa,emul in builds:
+ for test in options.regress:
+ targets.append(builder.test(bin, isa, emul, test))
+
+ for test in options.unittest:
+ targets.append(builder.unittest(bin, isa, emul, test))
+
+ if not options.regress and not options.unittest:
+ targets.append(builder.binary(bin, isa, emul))
+
+#
+# set up compile
+#
+os.chdir(builder.source)
+
+sys.argv = [ progname ]
+sys.argv.extend(scons_options)
+sys.argv.extend(args)
+sys.argv.extend(targets)
+
+if options.verbose or options.no_exec:
+ for arg in sys.argv[1:]:
+ print arg
+
+if not options.no_exec:
+ path = os.environ['PATH'].split(os.pathsep)
+ for dir in path:
+ filename = os.path.join(dir, "scons")
+ if os.path.isfile(filename):
+ break
+ else:
+ raise AttributeError, "could not find scons"
+
+ sys.path[0:0] = [ dir ]
+
+ execfile(filename)
diff -r 5645632d594c util/build/m5run
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/util/build/m5run Wed Feb 11 12:51:40 2009 -0800
@@ -0,0 +1,302 @@
+#!/usr/bin/env python
+
+# Copyright (c) 2006-2008 The Hewlett-Packard Development Company
+# Copyright (c) 2003 The Regents of The University of Michigan
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met: redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer;
+# redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution;
+# neither the name of the copyright holders nor the names of its
+# contributors may be used to endorse or promote products derived from
+# this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+#
+# Authors: Nathan Binkert
+# Steve Reinhardt
+
+import optparse
+import os
+import sys
+
+from os.path import basename, isfile
+
+import dotm5_build
+from dotm5_build import all, default, valid
+
+usage = """%prog [options] <version> [m5 args]
+ %prog [options] -D <version> <version> [m5 args]
+
+%prog assumes that the user has a .m5 directory with a build.py file
+describing the various versions of M5 that have been compiled. See
+--help-config for more information.
+
+version refers to either a version defined in .m5/build.py or an
+executable program
+
+Diff mode (-D) runs two copies of m5 expecting to diff their trace
+outputs, so you *must* enable some trace flags in the m5 arguments in
+order to do anything useful.
+
+In diff mode, arguments are handled uniformly as follows:
+- If the argument does not contain a '|' character, it is appended
+ to both command lines.
+- If the argument has a '|' character in it, the text on either side
+ of the '|' is appended to the respective command lines. Note that
+ you'll have to quote the arg or escape the '|' with a backslash
+ so that the shell doesn't think you're doing a pipe.
+- Arguments with '#' characters are split at those characters,
+ processed for alternatives ('|'s) as independent terms, then
+ pasted back into a single argument (without the '#'s). (Sort of
+ inspired by the C preprocessor '##' token pasting operator.)
+
+In other words, the arguments should look like the command line you
+want to run, with "|" used to list the alternatives for the parts
+that you want to differ between the two runs.
+
+For example:
+
+% %prog -D foo --opt1 '--opt2|--opt3' --opt4
+would compare these two runs:
+m5 --opt1 --opt2 --opt4
+m5 --opt1 --opt3 --opt4
+
+% %prog '--trace-flags=flag1,#flag2|flag3' --opt1 --opt2
+would compare these two runs:
+m5 --trace-flags=flag1,flag2 --opt1 --opt2
+m5 --trace-flags=flag1,flag3 --opt1 --opt2
+
+If you want to add arguments to one run only, just put a '|' in with
+text only on one side ('--onlyOn1|'). You can do this with multiple
+arguments together too ('|-a -b -c' adds three args to the second
+run only).
+"""
+
+version = '%prog 0.1'
+parser = optparse.OptionParser(usage=usage, version=version,
+ formatter=optparse.TitledHelpFormatter())
+parser.disable_interspersed_args()
+
+parser.add_option('--help-config', default=False, action='store_true',
+ help="Print out help for config files")
+
+parser.add_option('-D', '--diff', default=False, action='store_true',
+ help="diff two versions of M5")
+
+parser.add_option('-d', '--debug', default=False, action='store_true',
+ help="run the debug version of m5")
+parser.add_option('-o', '--opt', default=False, action='store_true',
+ help="run the opt version of m5")
+parser.add_option('-f', '--fast', default=False, action='store_true',
+ help="run the fast version of m5")
+parser.add_option('-p', '--prof', default=False, action='store_true',
+ help="run the profiled version of m5")
+parser.add_option('-g', '--gdb', default=False, action='store_true',
+ help="run gdb on m5")
+parser.add_option('-V', '--valgrind', default=False, action='store_true',
+ help="run under valgrind")
+
+parser.add_option('-u', '--unittest', default='', metavar='TEST',
+ help="run named unit test")
+
+parser.add_option('-e', '--emulation', default='fullsys', metavar='EMUL',
+ help="emulation type: fullsys (default) or syscall")
+parser.add_option('-i', '--isa', default='alpha', metavar='ISA',
+ help="isa: Alpha (default), MIPS, SPARC, or x86")
+
+parser.add_option('-n', '--dry-run', default=False, action='store_true',
+ help="Don't run, just print command line")
+
+#
+# Usage
+#
+def usage(exitcode=None):
+ parser.print_help()
+ if exitcode is not None:
+ sys.exit(exitcode)
+
+#
+# Option defaults and validation
+#
+(options, args) = parser.parse_args()
+
+if options.help_config:
+ print dotm5_build.help
+ sys.exit(0)
+
+if options.debug + options.opt + options.fast + options.prof > 1:
+ print "Only one option is allowed"
+ print
+ usage(1)
+
+if not args:
+ usage(1)
+
+if options.gdb and options.valgrind:
+ print >>sys.stderr, "It doesn't make sense to use both gdb and valgrind"
+ sys.exit(1)
+
+if options.diff and options.valgrind:
+ print >>sys.stderr, "It doesn't make sense to use valgrind in diff mode"
+ sys.exit(1)
+
+if options.diff and options.gdb:
+ print >>sys.stderr, "It doesn't make sense to use gdb in diff mode"
+ sys.exit(1)
+
+if options.diff and options.fast:
+ print >>sys.stderr, "diff mode needs tracing, so fast doesn't make sense"
+ sys.exit(1)
+
+build = None
+
+if build is None:
+ if options.gdb or options.valgrind:
+ build = 'debug'
+ else:
+ build = 'opt'
+
+if options.emulation not in valid['emulations']:
+ print "invalid emulation: %s" % options.emulation
+ usage(1)
+
+if options.isa not in valid['isas']:
+ print "invalid ISA: %s" % options.isa
+ usage(1)
+
+def get_binary(name, build, options):
+ builder = getattr(dotm5_build, name, None)
+ if builder is None:
+ if isfile(name):
+ return name, basename(name)
+
+ sys.exit('"%s" not defined in .m5/build.py and not a file' % name)
+
+ builder.run_mode = True
+
+ if options.unittest:
+ bin = builder.unittest(build, options.isa, options.emulation,
+ options.unittest)
+ name = options.unittest
+ else:
+ bin = builder.binary(build, options.isa, options.emulation)
+
+ return bin, name
+
+if options.diff:
+ binary1, name1 = get_binary(args.pop(0), build, options)
+ binary2, name2 = get_binary(args.pop(0), build, options)
+
+ args1 = [ binary1 ]
+ args2 = [ binary2 ]
+
+ # Run individual invocations in separate dirs so output and
+ # intermediate files (particularly config.py and config.ini) don't
+ # conflict.
+ pid = os.getpid()
+ dir1 = "tracediff.%d.%s" % (pid, name1)
+ dir2 = "tracediff.%d.%s" % (pid, name2)
+ args1.append("-d%s" % dir1)
+ args2.append("-d%s" % dir2)
+
+ for arg in args:
+ a1 = ''
+ a2 = ''
+ subargs = arg.split('#');
+ for subarg in subargs:
+ if not subarg:
+ continue
+
+ pair = subarg.split('|')
+ if len(pair) == 1:
+ a1 += subarg
+ a2 += subarg
+ elif len(pair) == 2:
+ a1 += pair[0]
+ a2 += pair[1]
+ else:
+ sys.exit('Parse error: too many |s in %s' % arg)
+
+
+ args1.append(a1)
+ args2.append(a2)
+
+ if options.dry_run:
+ print ' '.join(args1)
+ print ' '.join(args2)
+ sys.exit(0)
+
+ import rundiff
+ import signal
+ from subprocess import Popen, PIPE, STDOUT
+
+ os.mkdir(dir1)
+ os.mkdir(dir2)
+
+ args = [ 'rundiff', ' '.join(args1) + '|', ' '.join(args1) + '|' ]
+
+ outfile = file("tracediff.%d" % pid, "w")
+ os.dup2(outfile.fileno(), sys.stdout.fileno())
+ try:
+ os.execvp(args[0], args)
+ except:
+ sys.exit("Could not execute or find: %s" % exec_binary)
+
+ exec1 = Popen(args1, stdout=PIPE, stderr=STDOUT)
+ exec2 = Popen(args2, stdout=PIPE, stderr=STDOUT)
+
+ differ = rundiff.RunDiff(exec1.stdout, exec2.stdout,
+ name1=name1, name2=name2)
+
+ try:
+ for line in differ():
+ outfile.write(line)
+ outfile.flush()
+ except rundiff.SyncError, e:
+ sys.exit(e)
+ except KeyboardInterrupt, e:
+ sys.exit("diff interrupted by user")
+ finally:
+ os.kill(exec1.pid, signal.SIGTERM)
+ os.kill(exec2.pid, signal.SIGTERM)
+
+else:
+ binary, name = get_binary(args.pop(0), build, options)
+
+ if options.gdb:
+ exec_binary = 'gdb'
+ if args:
+ args = [ '--args', binary ] + args
+ else:
+ args = [ binary ]
+ elif options.valgrind:
+ exec_binary = 'valgrind'
+ args = [ binary ] + args
+ else:
+ exec_binary = binary
+
+ args = [ exec_binary ] + args
+ print args
+ if options.dry_run:
+ print ' '.join(args)
+ sys.exit(0)
+
+ try:
+ os.execvp(exec_binary, args)
+ except:
+ sys.exit("Could not execute or find: %s" % exec_binary)
_______________________________________________
m5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/m5-dev
