On 08/26/2012 06:33 PM, Steve Jenson wrote:
Hi rustics,
I spent some time this weekend going over the rust tutorial and
documentation for 0.3.1. I thought a fun exercise would be writing an
xUnit testing framework[1] in a "classic" OO style. I ran into a few
problems:
I'm glad you've been considering this. I have always viewed it as a goal
to get an xUnit-style framework working in Rust, hopefully integrating
in some way with the basic test runner built into std. Unfortunately, a
lot of features necessary to do what JUnit does don't exist yet.
1) xUnit historically relies on inheritance but it's not clear to me
how to model an is-a relationship in Rust. For instance, define an
abstract base class (TestSuite) and an implementation that tests a set
of functions (say, a calculator).
TestSuite will probably be a trait, and implementing the trait is how
the test runner identifies test suites. But TestSuite itself is pretty
limited in what it can implement.
2) How to colocate state and behavior in impls. Embedding lets in my
impls results in errors.
You get a single type to represent the state of the test suite. impls
don't get their own state, just the state contained in their self types.
struct MyTestSuite {
mut state: ...
}
impl MyTestSuite: TestSuite {
}
3) Reflection. I have no documented way to iterate over the functions
in an impl and call them. (I'm sure this is on the way and I'm just
early to the party)
This is the biggest roadblock to making a bigger, more dynamic testing
framework. There is no reflection or dynamic function loading yet. Here
are two related issues:
https://github.com/mozilla/rust/issues/458
https://github.com/mozilla/rust/issues/2213
Just for the existing minimal test framework I want to be able to use
reflection to grab all the #[test] functions from library crates.
I also think I'm approaching this from the wrong direction and that
Rust's OO with typeclasses are different from how I'm using to
building software in another language with typeclasses (Scala). I'm
still looking for the zen of Rust OO.
Yes, aspects of xUnit's design doesn't fit all that well with Rust, but
hopefully we can borrow the parts the work and the parts that don't we
can do differently.
I ran into some old blog posts that discuss a class keyword but I
wasn't able to make those examples run in 0.3.1. Do we only have impls
now?
Yes. There is no longer a self-contained class type. Structs + impls +
constructor functions are the way to define objects now.
I realize that Rust is young and moving quickly but I'm struggling to
build something that's more than a few functions and an enum.
I don't blame you. There's a big mismatch here that I've struggled with
as well.
If we were going to outline something that looked like an xUnit test
suite it would look something like this
/// Marker that indicates a test suite
trait TestSuite {
// Something likely goes here
}
struct MyTestSuite {
// test state goes here
}
impl MyTestSuite: TestSuite {
fn startup() {
// optional function that runs before each test
}
fn teardown() {
// optional function that runs after each test
}
// Any number of test functions
fn test1() {
}
fn test2() {
}
}
It looks really simple, but then when you start thinking about how the
test runner actually runs this type of test it gets ugly. This is what a
typical xUnit runner does:
1) Find all the implementations of TestSuite
2) For each TestSuite:
a) Instantiate the suite with a default constructor
b) Find all the test methods dynamically
c) For each test method, run startup(), the test, then teardown()
Rust basically can't do any of these things yet.
Let's first consider the reflection aspect. The test runner needs
reflection to find the suite, the tests, possibly the startup and
teardown methods. Since reflection doesn't much exist yet, we can
speculate on several ways to do this. One may might be more declarative,
using attributes, like NUnit (and I think later JUnit). The following
would be enough to locate tests:
// Don't actually need a trait just to locate a test object. An
// attribute will also work.
#[test_suite]
struct MyTestSuite {
// state
}
impl MyTestSuite {
#[startup] // This can probably be optional, since we could just
// reflect on the method name
fn startup() {
}
fn shutdown() { }
fn test() {
// methods that start with 'test' are tests
}
#[test]
fn descriptive_test_case() {
// methods with the test attribute are tests
}
}
Again, most of the machinery to make this possible does not exist.
The next set of problems is around the statefulness of an xUnit test
suite. xUnit wants to instantiate a mutable test suite object, then
reuse it for each test, with the startup and shutdown methods typically
resetting the mutable state of the test suite to initial test conditions.
Without default values and null pointers Rust can't just instantiate an
empty test case for you, so we'll need to tell the test runner where the
constructor is. So maybe our test suite looks like the following:
// I have no idea how much of this syntax is available or even under
// consideration
trait<T> TestSuite<T> {
static fn new() -> T; // No default implementation
fn startup(&mut self) {
self <- new() // By default we completely reset our state
}
fn shutdown(&mut self) {
}
}
struct MyTestSuite {
mut state1: bool
}
impl MyTestSuite: TestSuite<MyTestSuite> {
static fn new() -> MyTestSuite {
MyTestSuite { state1: false }
}
fn test() { ... }
}
Given a lot of assumptions about what might ultimately be available in
the language, this looks like the beginnings of a workable design.
Unfortunately, given the stateful nature of a test suite, you aren't
going to be able to run the tests in parallel (just entire suites). Also
unfortunately, it's difficult to even begin approaching this in the
existing language.
In the end I think the ideal test framework for Rust will look somewhat
different than xUnit, be designed by default to isolate and run tests in
parallel, and will treat stateful test suits with interdependent tests
as a special case.
Ok, maybe some of that was useful.
Regards,
Brian
_______________________________________________
Rust-dev mailing list
[email protected]
https://mail.mozilla.org/listinfo/rust-dev
