Hi, I've added some new machinery to the testing systems in rust and wanted to draw a little attention to them. At least this will all be true shortly when https://github.com/mozilla/rust/pull/7829 lands.
They're (lightly) documented at the bottom of: https://github.com/mozilla/rust/wiki/Doc-unit-testing and https://github.com/mozilla/rust/wiki/Note-testsuite The point-form version is: - #[bench] tests can now save their results as json. run the testrunner with --save-metrics=foo.json - If you want to avoid regressing on something, use --ratchet-metrics=foo.json instead; it will save and reload with each run, and consider any regression beyond a given noise threshold to be a test-fail. (Removing, renaming, omitting or adding metrics does not cause failure. it emits a warning and assumes you meant to do it.) - If you configure with --ratchet-bench then all the crate #[bench] benchmarks will be ratcheted in your workspace. We will turn this on on the buildbot soon-ish. At that point, adding a #[bench] function will make it impervious to perf regressions, so add them regularly but also with some care and thought. - If you're on a particularly noisy machine, you can override the inferred noise thresholds with --ratchet-noise-percent=N - There are new compiletests in src/test/codegen. These consist of pairs of files, foo.rs and foo.cc, each containing a non-mangled extern C function 'test'. The runner will compile the rust one with rustc and the C++ one with clang, extract their LLVM bitcode, disassemble it and compare the sizes. The result is recorded in a metrics file and _ratcheted by default_. - The codegen tests also leave behind foo-extract.ll and foo-clang-extract.ll files, which are helpful for eyeballing problems in our codegen. Please populate the codegen directory liberally. All you need to do is write a function in rust and a function that "does the same thing" in C++. If you have any questions, or find none of it works for you and want help making it work, please let me know. I'll hopefully turn on --ratchet-bench and export the metrics files from the buildbot soon. -Graydon _______________________________________________ Rust-dev mailing list [email protected] https://mail.mozilla.org/listinfo/rust-dev
