On 01/21/2013 10:25 PM, Curtis Dutton wrote:
I've been using racket now for about 4 years now. I use it for
everything that I can and I love it. It is really an awesome system, and
I just can't say "THANKS" enough to all of you for racket.
That being said, I'd like to become more active with the development
process. In a past life, I worked for Microsoft as a development tools
engineer. Most of what I did was develop and operate large scale
automated testing systems for very large teams. I once had a room full
of 500 or so machines at my disposal, which I tried hard to keep busy.
I've maintained rolling build systems, that included acceptance tests, a
performance testing system, a stress testing system and a security
fuzzing system.
I'm not sure how people feel about automated systems like this, part of
this email is just to see what people think. But used in the right way
they can be used to shape and control the directions that a project
evolves in.
We feel very good about them, and we can always use a lot more.
Every push, a machine named "DrDr" (see drdr.racket-lang.org) rebuilds
all of Racket, including GUI stuff and tools, from the updated codebase,
and runs almost every file. Many of them are in a directory called
"tests" and subdirectories of top-level collections like "plot/tests".
DrDr emails the pusher and any responsible parties if tests fail or a
file's output changes.
Most of these tests are standard, deterministic tests. Some are
randomized tests (Typed Racket and the math library do this, and
probably Redex). A small number are stress tests and performance tests.
Here's where I can't speak for everybody, just myself. It seems to me
that most of the testing is ad-hoc, depending on the whims of the
collection owners. It could be very nice to standardize most of it.
Now I'm willing to build it and I'm willing to host it with a number of
machines. I have pieces and parts of code lying around and I already
have a decent harness implementation that collects statistics about a
racket process as it runs.
Jay McCarthy owns DrDr, and runs it from our research lab. You'll want
to talk to him about hardware.
I'd like to collect a list of desired "tests" that this system would
monitor for us. If you already have code that you run on your own, even
better! Detailed examples would be welcome, as I need to gather some
ideas about what people would want to do with this thing.
I've wanted more automated testing for `math/special-functions' and
`math/distributions' exports, and performance testing for `math/array'
and `math/matrix', but I haven't gotten around to doing it. I'm not sure
what I'd use as a baseline for the latter two. For `math/distributions',
I've got no idea how to reliably test a sampler, since there's always a
nonzero probability that any computed statistic is wrong. Do you have
experience with that?
Neil ⊥
_________________________
Racket Developers list:
http://lists.racket-lang.org/dev
