Re: Looking for good QA tests... Part 1..
A bit of cleanup and filing off of serial numbers, etc...
test script c program... very stupid test, except that in large doses
brings systems to their knees and very much exercises buffer ownership
issues. The test looks too stupid to catch things, but believe me, it
does.
#!/bin/sh
#
# Copyright (c) 1999 Matthew Jacob
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#notice, this list of conditions, and the following disclaimer,
#without modification, immediately at the beginning of the file.
# 2. The name of the author may not be used to endorse or promote products
#derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
# $Id: testfil,v 1.1 1999/08/27 18:11:19 mjacob Exp $
#
FILESIZ=${FILESIZ-1m}
FILECNT=${FILECNT-10}
tripcount=1
if [ $# -ne 1 ]; then
echo "usage: $0 testdirectory"
exit 1
fi
while :
do
echo "Begin Pass $tripcount"
i=1
while [ $i -lt ${FILECNT} ]
do
filbuf -p $i -s ${FILESIZ} -o $1/file.$i
i=`expr $i '+' 1`
done
wait
i=1
while [ $i -lt ${FILECNT} ]
do
filbuf -p $i -s ${FILESIZ} -i $1/file.$i
if [ $? -ne 0 ]
then
exit 1
fi
rm -f $1/file.$i
i=`expr $i '+' 1`
done
tripcount=`expr $tripcount '+' 1`
done
/*
* Copyright (c) 1999 Matthew Jacob
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
*notice, this list of conditions, and the following disclaimer,
*without modification, immediately at the beginning of the file.
* 2. The name of the author may not be used to endorse or promote products
*derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: filbuf.c,v 1.1 1999/08/27 18:11:09 mjacob Exp $
*/
#include unistd.h
#include stdlib.h
#include stdio.h
#include errno.h
#include fcntl.h
#include string.h
#include memory.h
static off_t szarg(char *);
static void miscompare(char *, char *, int, off_t);
#define DEFAULT_BUFLEN (63 10)
#define DEFAULT_FILESIZE(1 20)
extern int optind;
extern char *optarg;
int
main(int argc, char **argv)
{
static const char *usage =
"usage: %s -{o,i} filename [ -s size[{k|m|g}] ] [ -p idpattern ]";
int c, fd, io, amt;
off_t filesize, curoff;
size_t buflen;
char *idp, *filename, *buf, *ptr;
io = 0;
idp = argv[0];
filename = NULL;
buflen = DEFAULT_BUFLEN;
filesize = DEFAULT_FILESIZE;
while ((c = getopt(argc, argv, "p:s:o:i:")) != -1) {
switch (c) {
case 'p':
idp = optarg;
break;
case 'i':
if (filename != NULL) {
fprintf(stderr, usage, argv[0]);
return (1);
}
filename = optarg;
Re: Looking for good QA tests... Part 2..
Raw pattern checker...less well polished...runs on a lot of platforms...
Avoids trashing disk labels...
Usage is a little less than obvious...
1. patchk RAWDISK TransferSize c
Create/Validate/Test patterns.. creates patterns in TransferSize
chunks..validates that they were correctly written, then randoml reads or
updates same patterns.
2. patchk RAWDISK TransferSize u
Skips the create step
3. patchk RAWDISK TransferSize v
Skips the create and refresh test- just validates the pattern.
Gary Palmer managed to find some pretty serious problems in a disk array
by running multiple 'u' instantiations on the same partitition after an
initial 'c' test.
/*
* Copyright (c) 1999 Matthew Jacob
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
*notice, this list of conditions, and the following disclaimer,
*without modification, immediately at the beginning of the file.
* 2. The name of the author may not be used to endorse or promote products
*derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: patchk.c,v 1.3 1999/08/27 18:14:34 mjacob Exp $
*/
#ifdef convex
#include sys/types.h
extern int optind;
extern int getopt(int, char **, const char *);
#define SEEK_T off64_t
#define SEEKlseek64
#define FSTAT fstat64
#define STAT_T stat64_t
#else
#define SEEK_T off_t
#define SEEKlseek
#define FSTAT fstat
#define STAT_T struct stat
#endif
#include unistd.h
#include stdlib.h
#include stdio.h
#include errno.h
#include fcntl.h
#include string.h
#include sys/stat.h
#include sys/time.h
#include sys/param.h
#include sys/ioctl.h
#ifdef sun
#define randlrand48
#define srand srand48
#ifdef __SVR4
#include sys/dkio.h
#else
#include sun/dkio.h
extern int gettimeofday(struct timeval *, struct timezone *);
extern void bzero(char *, int);
extern int strtol(const char *, char **, int);
#endif
extern int optind;
#endif
#ifdef __linux__
#include sys/ioctl.h
#include linux/fs.h
#endif
#ifdef convex
#include sys/ioctl.h
#include interfaces/io_if/scsi/scsi.h
#endif
#if defined(__NetBSD__) || defined(__OpenBSD__)
#include sys/disklabel.h
#include sys/dkio.h
#endif
#ifdef __FreeBSD__
#include sys/disklabel.h
#endif
#ifdef __ultrix
extern int optind;
#endif
#ifndef O_LARGEFILE
#define O_LARGEFILE 0
#endif
static int szarg(char *);
int
main(int a, char **v)
{
SEEK_T seekbase, seeklim, *buffer, wloc;
int blksize;
long long sb, sl;
STAT_T st;
int fd, i, error, create, nowrite;
seekbase = (SEEK_T) 0;
nowrite = 0;
srand((int)(time((time_t *) 0)/getpid()));
if (a != 4) {
usage:
fprintf(stderr,
"Usage: %s raw-disk xfersize {c[reate]|u[se]|v[alidate]}\n", *v);
return (1);
}
blksize = szarg(v[2]);
buffer = (SEEK_T *) calloc((size_t) blksize, sizeof (SEEK_T));
if (buffer == NULL) {
perror("malloc");
return (1);
}
if (*v[3] == 'c') {
create = 1;
} else if (*v[3] == 'u') {
create = 0;
} else if (*v[3] == 'v') {
create = 0;
nowrite = 1;
} else {
goto usage;
}
fd = open(v[1], nowrite? O_RDONLY : O_RDWR, 0666);
if (fd 0) {
perror(v[2]);
exit(1);
}
if (FSTAT(fd, st) 0) {
perror("fstat");
exit(1);
}
if (S_ISCHR(st.st_mode)) {
#if defined(__NetBSD__) || defined(__FreeBSD__) || defined(__OpenBSD__)
int part;
struct disklabel x;
if (ioctl(fd, DIOCGDINFO, (caddr_t) x) 0) {
perror("DIOCGDINFO");
exit(1);
}
seekbase = 8192;
part = v[1][strlen(v[1]) - 1] - 'a';
seeklim = ((SEEK_T) x.d_partitions[part].p_size) * (SEEK_T) DEV_BSIZE;
#elif defined(sun)
struct dk_allmap x;
int part;
if (blksize DEV_BSIZE) {
fprintf(stderr, "%s: block size must be at least %d bytes on "
"raw device\n", *v, DEV_BSIZE);
exit(1);
}
#if defined(__svr4__)
Re: Looking for good QA tests... Part 1..
A bit of cleanup and filing off of serial numbers, etc...
test script c program... very stupid test, except that in large doses
brings systems to their knees and very much exercises buffer ownership
issues. The test looks too stupid to catch things, but believe me, it
does.
#!/bin/sh
#
# Copyright (c) 1999 Matthew Jacob
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
#notice, this list of conditions, and the following disclaimer,
#without modification, immediately at the beginning of the file.
# 2. The name of the author may not be used to endorse or promote products
#derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
# SUCH DAMAGE.
#
# $Id: testfil,v 1.1 1999/08/27 18:11:19 mjacob Exp $
#
FILESIZ=${FILESIZ-1m}
FILECNT=${FILECNT-10}
tripcount=1
if [ $# -ne 1 ]; then
echo usage: $0 testdirectory
exit 1
fi
while :
do
echo Begin Pass $tripcount
i=1
while [ $i -lt ${FILECNT} ]
do
filbuf -p $i -s ${FILESIZ} -o $1/file.$i
i=`expr $i '+' 1`
done
wait
i=1
while [ $i -lt ${FILECNT} ]
do
filbuf -p $i -s ${FILESIZ} -i $1/file.$i
if [ $? -ne 0 ]
then
exit 1
fi
rm -f $1/file.$i
i=`expr $i '+' 1`
done
tripcount=`expr $tripcount '+' 1`
done
/*
* Copyright (c) 1999 Matthew Jacob
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
*notice, this list of conditions, and the following disclaimer,
*without modification, immediately at the beginning of the file.
* 2. The name of the author may not be used to endorse or promote products
*derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: filbuf.c,v 1.1 1999/08/27 18:11:09 mjacob Exp $
*/
#include unistd.h
#include stdlib.h
#include stdio.h
#include errno.h
#include fcntl.h
#include string.h
#include memory.h
static off_t szarg(char *);
static void miscompare(char *, char *, int, off_t);
#define DEFAULT_BUFLEN (63 10)
#define DEFAULT_FILESIZE(1 20)
extern int optind;
extern char *optarg;
int
main(int argc, char **argv)
{
static const char *usage =
usage: %s -{o,i} filename [ -s size[{k|m|g}] ] [ -p idpattern ];
int c, fd, io, amt;
off_t filesize, curoff;
size_t buflen;
char *idp, *filename, *buf, *ptr;
io = 0;
idp = argv[0];
filename = NULL;
buflen = DEFAULT_BUFLEN;
filesize = DEFAULT_FILESIZE;
while ((c = getopt(argc, argv, p:s:o:i:)) != -1) {
switch (c) {
case 'p':
idp = optarg;
break;
case 'i':
if (filename != NULL) {
fprintf(stderr, usage, argv[0]);
return (1);
}
filename = optarg;
Re: Looking for good QA tests... Part 2..
Raw pattern checker...less well polished...runs on a lot of platforms...
Avoids trashing disk labels...
Usage is a little less than obvious...
1. patchk RAWDISK TransferSize c
Create/Validate/Test patterns.. creates patterns in TransferSize
chunks..validates that they were correctly written, then randoml reads or
updates same patterns.
2. patchk RAWDISK TransferSize u
Skips the create step
3. patchk RAWDISK TransferSize v
Skips the create and refresh test- just validates the pattern.
Gary Palmer managed to find some pretty serious problems in a disk array
by running multiple 'u' instantiations on the same partitition after an
initial 'c' test.
/*
* Copyright (c) 1999 Matthew Jacob
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
*notice, this list of conditions, and the following disclaimer,
*without modification, immediately at the beginning of the file.
* 2. The name of the author may not be used to endorse or promote products
*derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $Id: patchk.c,v 1.3 1999/08/27 18:14:34 mjacob Exp $
*/
#ifdef convex
#include sys/types.h
extern int optind;
extern int getopt(int, char **, const char *);
#define SEEK_T off64_t
#define SEEKlseek64
#define FSTAT fstat64
#define STAT_T stat64_t
#else
#define SEEK_T off_t
#define SEEKlseek
#define FSTAT fstat
#define STAT_T struct stat
#endif
#include unistd.h
#include stdlib.h
#include stdio.h
#include errno.h
#include fcntl.h
#include string.h
#include sys/stat.h
#include sys/time.h
#include sys/param.h
#include sys/ioctl.h
#ifdef sun
#define randlrand48
#define srand srand48
#ifdef __SVR4
#include sys/dkio.h
#else
#include sun/dkio.h
extern int gettimeofday(struct timeval *, struct timezone *);
extern void bzero(char *, int);
extern int strtol(const char *, char **, int);
#endif
extern int optind;
#endif
#ifdef __linux__
#include sys/ioctl.h
#include linux/fs.h
#endif
#ifdef convex
#include sys/ioctl.h
#include interfaces/io_if/scsi/scsi.h
#endif
#if defined(__NetBSD__) || defined(__OpenBSD__)
#include sys/disklabel.h
#include sys/dkio.h
#endif
#ifdef __FreeBSD__
#include sys/disklabel.h
#endif
#ifdef __ultrix
extern int optind;
#endif
#ifndef O_LARGEFILE
#define O_LARGEFILE 0
#endif
static int szarg(char *);
int
main(int a, char **v)
{
SEEK_T seekbase, seeklim, *buffer, wloc;
int blksize;
long long sb, sl;
STAT_T st;
int fd, i, error, create, nowrite;
seekbase = (SEEK_T) 0;
nowrite = 0;
srand((int)(time((time_t *) 0)/getpid()));
if (a != 4) {
usage:
fprintf(stderr,
Usage: %s raw-disk xfersize {c[reate]|u[se]|v[alidate]}\n, *v);
return (1);
}
blksize = szarg(v[2]);
buffer = (SEEK_T *) calloc((size_t) blksize, sizeof (SEEK_T));
if (buffer == NULL) {
perror(malloc);
return (1);
}
if (*v[3] == 'c') {
create = 1;
} else if (*v[3] == 'u') {
create = 0;
} else if (*v[3] == 'v') {
create = 0;
nowrite = 1;
} else {
goto usage;
}
fd = open(v[1], nowrite? O_RDONLY : O_RDWR, 0666);
if (fd 0) {
perror(v[2]);
exit(1);
}
if (FSTAT(fd, st) 0) {
perror(fstat);
exit(1);
}
if (S_ISCHR(st.st_mode)) {
#if defined(__NetBSD__) || defined(__FreeBSD__) || defined(__OpenBSD__)
int part;
struct disklabel x;
if (ioctl(fd, DIOCGDINFO, (caddr_t) x) 0) {
perror(DIOCGDINFO);
exit(1);
}
seekbase = 8192;
part = v[1][strlen(v[1]) - 1] - 'a';
seeklim = ((SEEK_T) x.d_partitions[part].p_size) * (SEEK_T) DEV_BSIZE;
#elif defined(sun)
struct dk_allmap x;
int part;
if (blksize DEV_BSIZE) {
fprintf(stderr, %s: block size must be at least %d bytes on
raw device\n, *v, DEV_BSIZE);
exit(1);
}
#if defined(__svr4__)
part =
Looking for good QA tests...
Hi everyone... Back in June, at Usenix, Jordan and I talked about doing some more formalized QA on FreeBSD releases. Unfortunately, this has yet to go really far, as I was trying to complete some commitments, and have some infrastructure in place before I started making this endeavor overly public, as I'm also trying to keep the amount of time I have to put in to it to a moderate pace. However, I'm now at the point where I'd like to start collecting materials to do this. By "materials", I mean both test scenarios and code for performing these tests. While I now have your mind spinning, I'd like to lay down some limits. Firstly, my current resources for testing is relatively small. I have four machines that I can use most of the time. Therefore, we can't do "everything, all the time", so tests should be limited to cover as broad a range of items as we have time for. My estimation is we will get approximately 48 hours per release candidate as we move towards Jordan cutting a release. In between, we might get a bit more slack, but not a lot. Secondly, these tests can not take a lot of person hours to perform. _I_ can give approximately 1-2 man hours per test cycle. Therefore, these tests need to be automated, have good reporting (especially if they fail), and be easy to set up and run. Lastly, these tests should be tests, and not benchmarks. I can't stress this enough. Knowing how fasts my disks are is useless compared to a bug in the network interface that causes the machine to panic. Now, as more people sign on to both write code/test plans, as well as do the testing, we might be able to expand the types of tests we can run. My plans are to wrap up whatever I'm given in the form of code in to a port/package set, and have it distributed so that anyone with a spare machine can come onboard and help out. Therefore, I'm asking people to put their minds to work, and talk about valid tests that we could run to catch things that might slip through the cracks. I'm hoping that some of the long-timers can point out areas that have been missed before, and others to point out areas where they have seen local problems. I am hoping that there are also enough interested people out there with coding skills (this may be an area where heavy-duty skills are not required) can take some of these ideas, and convert them in to test code that operates within the limits above. I look forward to hearing your comments. -Brian To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
Therefore, I'm asking people to put their minds to work, and talk about valid tests that we could run to catch things that might slip through the cracks. I'm hoping that some of the long-timers can point out areas that have been missed before, and others to point out areas where they have seen local problems. As you are a networking outfit, I would imagine that you would be interested in two things: 1) Does the core OS still work, and 2) Does the networking still work? 1) Is usually well checkable with a "make world", and 2) could (I imagine) be well tested with a make world with NFS mounted disks, possibly using another FreeBSD box as a router for extra stress. Is this what you are looking for? M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
I have at least one filesystem killer test. And it is?... Like I mentioned, we're trying to collect tests/code that will demonstrate bugs. I'll try and figure out a good place to hang it in the source tree, but I believe that it's usage is a mandatory "must use" for validating FFS and VM code. Tell you what... Ship the code to me, and I'll put it together with anything else I get :) I already devote several machines to freebsd, so I can offer help here. I am very overbooked, but I can certainly review and comment upon test plans and maybe add a few of my own. Good to have you on board. -Brian To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. -matt To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. - -matt Great. Bundle something up and send it along, and I'll take a look. -Brian To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
Therefore, I'm asking people to put their minds to work, and talk about valid tests that we could run to catch things that might slip through the cracks. I'm hoping that some of the long-timers can point out areas that have been missed before, and others to point out areas where they have seen local problems. As you are a networking outfit, I would imagine that you would be interested in two things: Hmm... Its good to see that there are people in the world who don't think we're a food company (for the non-locals, there is a Sysco in New England (USA) that does food distribution. They always confuse them with us (Cisco)). 1) Does the core OS still work, and 2) Does the networking still work? 1) Is usually well checkable with a "make world", and 2) could (I imagine) be well tested with a make world with NFS mounted disks, possibly using another FreeBSD box as a router for extra stress. Is this what you are looking for? Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a "lets help the project by coordinating testing" thing -Brian To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
On Thu, Aug 26, 1999 at 10:41:58AM -0700, Matthew Jacob wrote: I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. Does it check do things that should fail aswell as things that should work? One of the problems with using buildworld's and the like is that they don't try anything that should fail. (What I have in mind here are things like cross device links, symlink loops, operating files that you shouldn't be able to, filling the disk and checking you get an error...) It would be nice to go through the man (2) pages and look at the error code sections and write code that deliberately provokes as many errors as possible. David. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
I have at least one filesystem killer test. And it is?... Like I mentioned, we're trying to collect tests/code that will demonstrate bugs. I'll try and figure out a good place to hang it in the source tree, but I believe that it's usage is a mandatory "must use" for validating FFS and VM code. Tell you what... Ship the code to me, and I'll put it together with anything else I get :) I already devote several machines to freebsd, so I can offer help here. I am very overbooked, but I can certainly review and comment upon test plans and maybe add a few of my own. Good to have you on board. -Brian It needs a little cleanup (i.e., a man page and perhaps some other cleanups to try and autosize things correctly, but here it is). It is one of the dumber but most effective tests I've run across. Lemme get to this tonite and package up a couple of things and put them somewhere visible. Then people can see if they're worth packaging or starting from. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a "lets help the project by coordinating testing" thing A regression test to make sure that the OS is not broken before you inflict it on your colleagues/engineers? M Still too small a scope. How about "A regression test to make sure that the OS is not broken before Jordan inflicts it on the world" ? -Brian To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
: Still too small a scope. How about "A regression test to make sure : that the OS is not broken before Jordan inflicts it on the world" ? : :Athough it does not actively hunt down bugs, a NFS mounted, :FreeBSD-routed build should stress enough of the system to disprove many :serious problems. : :Inflicting it on "standard idiots" to check for install problems is a :human-engineering aproach you could also take? Impossible to automate, :though. : :M :-- :Mark Murray :Join the anti-SPAM movement: http://www.cauce.org This is what I usually do: * machine artificially limited to 32M of ram. * NFSv3 mounted /usr/src * NFS mounted /usr/obj or VN(swap-backed,softupdates-enabled) mounted /usr/obj or ccd'd /usr/obj depending on what I am testing * lots of swap space (either NFS-based or a real disk, depending) * make -j 12 buildworld Matthew Dillon [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
As Brian McGovern wrote ... Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a "lets help the project by coordinating testing" thing A regression test to make sure that the OS is not broken before you inflict it on your colleagues/engineers? M Still too small a scope. How about "A regression test to make sure that the OS is not broken before Jordan inflicts it on the world" ? A bunch of years ago we used the ATT SVVS and the X/Open VSX testsuite to check on our SysV (!) system. Both code sets run a *lot* of tests but are A. non-BSD and B. non open source. Wilko -- | / o / / _ Arnhem, The Netherlands- Powered by FreeBSD - |/|/ / / /( (_) BulteWWW : http://www.tcja.nl http://www.freebsd.org To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
On Thu, 26 Aug 1999, Brian McGovern wrote: However, I'm now at the point where I'd like to start collecting materials to do this. By "materials", I mean both test scenarios and code for performing these tests. I suggest going over all of the various stress-test scripts/code which have been submitted as PRs (and in the mailing lists - maybe a judicious grep of the archives) as regression tests. There are (unfortunately) still at least a few PRs which will kill a box dead (last time I tried w/ CURRENT), but no doubt many others which have shown up and been fixed. Maybe also include some of the known DoS attacks collected from rootshell which have worked in the past to guard against reintroduction of network bugs. Kris To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Looking for good QA tests...
However, I'm now at the point where I'd like to start collecting materials to
do this. By "materials", I mean both test scenarios and code for performing
these tests.
While I now have your mind spinning, I'd like to lay down some
limits. Firstly, my current resources for testing is relatively
small. I have four machines that I can use most of the time.
Therefore, we can't do "everything, all the time", so tests
should be limited to cover as broad a range of items as we
have time for. My estimation is we will get approximately 48
hours per release candidate as we move towards Jordan cutting
a release. In between, we might get a bit more slack, but not
a lot. Secondly, these tests can not take a lot of person hours
to perform. _I_ can give approximately 1-2 man hours per test
cycle. Therefore, these tests need to be automated, have good
reporting (especially if they fail), and be easy to set up and
run. Lastly, these tests should be tests, and not benchmarks.
I can't stress this enough. Knowing how fasts my disks are is
useless compared to a bug in the network interface that causes
the machine to panic.
Now, as more people sign on to both write code/test plans, as
well as do the testing, we might be able to expand the types
of tests we can run. My plans are to wrap up whatever I'm
given in the form of code in to a port/package set, and have
it distributed so that anyone with a spare machine can come
onboard and help out.
1) Download the NIST/PCTS. This should be one of the tests
that is run against FreeBSD before it is released. We
will preferrably pass, and include the test results on
the CDROM, even if we can't afford to pay a testing lab
to run the exact same code and get legal license to the
trademark (that can come later).
2) The test framework I put together for monitoring memory
pool leaks through code exercises that utilize the pools
being monitored is a Good Thing(tm). I believe that
Michael Hancock ported it to -current. My initial tests
that I provided with this framework were able to exercise
file operations on varios file systems, and detect namei
buffer leaks on any mounted FS that they were run on.
The test code did full branch path coverage on that
particular memory pool.
This would be a good framework for similar kernel memory
allocation and deallovation testing. I believe that Doug
Rabson also had a copy of this code when he was reviewing
my VFS megapatch (which included nameifree(), which leaked
on NFS under certain obscure circumstances, and I didn't
have NFS resources to test with).
3) Back when UNIX International closed shop, I saved all of
their TET and E/TET ("(Extended) Test Environment Toolkit").
This code is normally used for SVID III testing by USL
and other parties, but it is a very good validation
testing framework, capable of being scripted for test
cases for both positive and negative results.
This code should be available from many sources; originally,
DigiBoard (ftp.digibd.com at the time) put up a mirror of
my rescued copy of the U.I. archive (this is actually where
the draft copy of Spec. 1170 that went all over the place
came from: their copy of my copy of U.I.'s archive).
4) I suggest lmbench be incorporated, as well. Not because
it is any good as a comparative macro benchmark between
heterogeneous systems, but because comparison between
FreeBSD releases with prior releases results on the same
hardware could be useful to indicate when pessimizations
occur.
Any or all of these would be a nice start. You might also want to
contact Doug Ambrisko. He may be interested, assuming that these
tests achieve formal standing for acceptance testing.
Terry Lambert
[EMAIL PROTECTED]
---
Any opinions in this posting are my own and not those of my present
or previous employers.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Looking for good QA tests...
Hi everyone... Back in June, at Usenix, Jordan and I talked about doing some more formalized QA on FreeBSD releases. Unfortunately, this has yet to go really far, as I was trying to complete some commitments, and have some infrastructure in place before I started making this endeavor overly public, as I'm also trying to keep the amount of time I have to put in to it to a moderate pace. However, I'm now at the point where I'd like to start collecting materials to do this. By materials, I mean both test scenarios and code for performing these tests. While I now have your mind spinning, I'd like to lay down some limits. Firstly, my current resources for testing is relatively small. I have four machines that I can use most of the time. Therefore, we can't do everything, all the time, so tests should be limited to cover as broad a range of items as we have time for. My estimation is we will get approximately 48 hours per release candidate as we move towards Jordan cutting a release. In between, we might get a bit more slack, but not a lot. Secondly, these tests can not take a lot of person hours to perform. _I_ can give approximately 1-2 man hours per test cycle. Therefore, these tests need to be automated, have good reporting (especially if they fail), and be easy to set up and run. Lastly, these tests should be tests, and not benchmarks. I can't stress this enough. Knowing how fasts my disks are is useless compared to a bug in the network interface that causes the machine to panic. Now, as more people sign on to both write code/test plans, as well as do the testing, we might be able to expand the types of tests we can run. My plans are to wrap up whatever I'm given in the form of code in to a port/package set, and have it distributed so that anyone with a spare machine can come onboard and help out. Therefore, I'm asking people to put their minds to work, and talk about valid tests that we could run to catch things that might slip through the cracks. I'm hoping that some of the long-timers can point out areas that have been missed before, and others to point out areas where they have seen local problems. I am hoping that there are also enough interested people out there with coding skills (this may be an area where heavy-duty skills are not required) can take some of these ideas, and convert them in to test code that operates within the limits above. I look forward to hearing your comments. -Brian To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
Therefore, I'm asking people to put their minds to work, and talk about valid tests that we could run to catch things that might slip through the cracks. I'm hoping that some of the long-timers can point out areas that have been missed before, and others to point out areas where they have seen local problems. As you are a networking outfit, I would imagine that you would be interested in two things: 1) Does the core OS still work, and 2) Does the networking still work? 1) Is usually well checkable with a make world, and 2) could (I imagine) be well tested with a make world with NFS mounted disks, possibly using another FreeBSD box as a router for extra stress. Is this what you are looking for? M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
I have at least one filesystem killer test. And it is?... Like I mentioned, we're trying to collect tests/code that will demonstrate bugs. I'll try and figure out a good place to hang it in the source tree, but I believe that it's usage is a mandatory must use for validating FFS and VM code. Tell you what... Ship the code to me, and I'll put it together with anything else I get :) I already devote several machines to freebsd, so I can offer help here. I am very overbooked, but I can certainly review and comment upon test plans and maybe add a few of my own. Good to have you on board. -Brian To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. -matt To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. - -matt Great. Bundle something up and send it along, and I'll take a look. -Brian To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
Therefore, I'm asking people to put their minds to work, and talk about valid tests that we could run to catch things that might slip through the cracks. I'm hoping that some of the long-timers can point out areas that have been missed before, and others to point out areas where they have seen local problems. As you are a networking outfit, I would imagine that you would be interested in two things: Hmm... Its good to see that there are people in the world who don't think we're a food company (for the non-locals, there is a Sysco in New England (USA) that does food distribution. They always confuse them with us (Cisco)). 1) Does the core OS still work, and 2) Does the networking still work? 1) Is usually well checkable with a make world, and 2) could (I imagine) be well tested with a make world with NFS mounted disks, possibly using another FreeBSD box as a router for extra stress. Is this what you are looking for? Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a lets help the project by coordinating testing thing -Brian To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
On Thu, Aug 26, 1999 at 10:41:58AM -0700, Matthew Jacob wrote: I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. Does it check do things that should fail aswell as things that should work? One of the problems with using buildworld's and the like is that they don't try anything that should fail. (What I have in mind here are things like cross device links, symlink loops, operating files that you shouldn't be able to, filling the disk and checking you get an error...) It would be nice to go through the man (2) pages and look at the error code sections and write code that deliberately provokes as many errors as possible. David. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
On Thu, Aug 26, 1999 at 10:41:58AM -0700, Matthew Jacob wrote: I have a filesystem stress tests that are worth incorporating. I also have a raw disk pattern checker, but that's less of a test than analysis tool. Does it check do things that should fail aswell as things that should work? One of the problems with using buildworld's and the like is that they don't try anything that should fail. No, it's a stress tester. (What I have in mind here are things like cross device links, symlink loops, operating files that you shouldn't be able to, filling the disk and checking you get an error...) That's more than I've written up. There are a set of other tests I've collected over the years- some of them have this (like building directory trees 100 deep and removing them). Not much other than the SVVS tests has attempted to also fully test the positive and negative errors. I have had access to the SVVS source, but I suspect it's still encumbered so it oculdn't be used here. It would be nice to go through the man (2) pages and look at the error code sections and write code that deliberately provokes as many errors as possible. Yes. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
I have at least one filesystem killer test. And it is?... Like I mentioned, we're trying to collect tests/code that will demonstrate bugs. I'll try and figure out a good place to hang it in the source tree, but I believe that it's usage is a mandatory must use for validating FFS and VM code. Tell you what... Ship the code to me, and I'll put it together with anything else I get :) I already devote several machines to freebsd, so I can offer help here. I am very overbooked, but I can certainly review and comment upon test plans and maybe add a few of my own. Good to have you on board. -Brian It needs a little cleanup (i.e., a man page and perhaps some other cleanups to try and autosize things correctly, but here it is). It is one of the dumber but most effective tests I've run across. Lemme get to this tonite and package up a couple of things and put them somewhere visible. Then people can see if they're worth packaging or starting from. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a lets help the project by coordinating testing thing A regression test to make sure that the OS is not broken before you inflict it on your colleagues/engineers? M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a lets help the project by coordinating testing thing A regression test to make sure that the OS is not broken before you inflict it on your colleagues/engineers? M Still too small a scope. How about A regression test to make sure that the OS is not broken before Jordan inflicts it on the world ? -Brian To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
Still too small a scope. How about A regression test to make sure that the OS is not broken before Jordan inflicts it on the world ? Athough it does not actively hunt down bugs, a NFS mounted, FreeBSD-routed build should stress enough of the system to disprove many serious problems. Inflicting it on standard idiots to check for install problems is a human-engineering aproach you could also take? Impossible to automate, though. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
: Still too small a scope. How about A regression test to make sure : that the OS is not broken before Jordan inflicts it on the world ? : :Athough it does not actively hunt down bugs, a NFS mounted, :FreeBSD-routed build should stress enough of the system to disprove many :serious problems. : :Inflicting it on standard idiots to check for install problems is a :human-engineering aproach you could also take? Impossible to automate, :though. : :M :-- :Mark Murray :Join the anti-SPAM movement: http://www.cauce.org This is what I usually do: * machine artificially limited to 32M of ram. * NFSv3 mounted /usr/src * NFS mounted /usr/obj or VN(swap-backed,softupdates-enabled) mounted /usr/obj or ccd'd /usr/obj depending on what I am testing * lots of swap space (either NFS-based or a real disk, depending) * make -j 12 buildworld Matthew Dillon dil...@backplane.com To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
As Brian McGovern wrote ... Mmmm... No. _I_ (read: Not Cisco as a whole) am looking for tests that will help locate bugs pre-release copies of the OS so that there is still time for others to debug the code before Jordan cuts the release. Its more of a lets help the project by coordinating testing thing A regression test to make sure that the OS is not broken before you inflict it on your colleagues/engineers? M Still too small a scope. How about A regression test to make sure that the OS is not broken before Jordan inflicts it on the world ? A bunch of years ago we used the ATT SVVS and the X/Open VSX testsuite to check on our SysV (!) system. Both code sets run a *lot* of tests but are A. non-BSD and B. non open source. Wilko -- | / o / / _ Arnhem, The Netherlands- Powered by FreeBSD - |/|/ / / /( (_) BulteWWW : http://www.tcja.nl http://www.freebsd.org To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
On Thu, 26 Aug 1999, Brian McGovern wrote: However, I'm now at the point where I'd like to start collecting materials to do this. By materials, I mean both test scenarios and code for performing these tests. I suggest going over all of the various stress-test scripts/code which have been submitted as PRs (and in the mailing lists - maybe a judicious grep of the archives) as regression tests. There are (unfortunately) still at least a few PRs which will kill a box dead (last time I tried w/ CURRENT), but no doubt many others which have shown up and been fixed. Maybe also include some of the known DoS attacks collected from rootshell which have worked in the past to guard against reintroduction of network bugs. Kris To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
Re: Looking for good QA tests...
However, I'm now at the point where I'd like to start collecting materials to do this. By materials, I mean both test scenarios and code for performing these tests. While I now have your mind spinning, I'd like to lay down some limits. Firstly, my current resources for testing is relatively small. I have four machines that I can use most of the time. Therefore, we can't do everything, all the time, so tests should be limited to cover as broad a range of items as we have time for. My estimation is we will get approximately 48 hours per release candidate as we move towards Jordan cutting a release. In between, we might get a bit more slack, but not a lot. Secondly, these tests can not take a lot of person hours to perform. _I_ can give approximately 1-2 man hours per test cycle. Therefore, these tests need to be automated, have good reporting (especially if they fail), and be easy to set up and run. Lastly, these tests should be tests, and not benchmarks. I can't stress this enough. Knowing how fasts my disks are is useless compared to a bug in the network interface that causes the machine to panic. Now, as more people sign on to both write code/test plans, as well as do the testing, we might be able to expand the types of tests we can run. My plans are to wrap up whatever I'm given in the form of code in to a port/package set, and have it distributed so that anyone with a spare machine can come onboard and help out. 1) Download the NIST/PCTS. This should be one of the tests that is run against FreeBSD before it is released. We will preferrably pass, and include the test results on the CDROM, even if we can't afford to pay a testing lab to run the exact same code and get legal license to the trademark (that can come later). 2) The test framework I put together for monitoring memory pool leaks through code exercises that utilize the pools being monitored is a Good Thing(tm). I believe that Michael Hancock ported it to -current. My initial tests that I provided with this framework were able to exercise file operations on varios file systems, and detect namei buffer leaks on any mounted FS that they were run on. The test code did full branch path coverage on that particular memory pool. This would be a good framework for similar kernel memory allocation and deallovation testing. I believe that Doug Rabson also had a copy of this code when he was reviewing my VFS megapatch (which included nameifree(), which leaked on NFS under certain obscure circumstances, and I didn't have NFS resources to test with). 3) Back when UNIX International closed shop, I saved all of their TET and E/TET ((Extended) Test Environment Toolkit). This code is normally used for SVID III testing by USL and other parties, but it is a very good validation testing framework, capable of being scripted for test cases for both positive and negative results. This code should be available from many sources; originally, DigiBoard (ftp.digibd.com at the time) put up a mirror of my rescued copy of the U.I. archive (this is actually where the draft copy of Spec. 1170 that went all over the place came from: their copy of my copy of U.I.'s archive). 4) I suggest lmbench be incorporated, as well. Not because it is any good as a comparative macro benchmark between heterogeneous systems, but because comparison between FreeBSD releases with prior releases results on the same hardware could be useful to indicate when pessimizations occur. Any or all of these would be a nice start. You might also want to contact Doug Ambrisko. He may be interested, assuming that these tests achieve formal standing for acceptance testing. Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-hackers in the body of the message
