Hi everyone! I am giving a talk about randomized testing in a month or so and I thought it'd be cool to share some real-life "ah-crap", "gotcha", "wtf" moments related to the bugs discovered by randomized testing.
My personal favorite is LUCENE-5168 (g1gc/ impossible code path) which, by the way, is still unresolved and we have no idea how to reproduce it reliably. Another is one of Robert's bugs with Unicode where he debugged what seemed like a "megabutt" of crazy-hairy random unicode that triggered the problem. These sorts of things. You can reply to the list or to my prv. e-mail if you wish. Dawid P.S. The talk's abstract is below here. Randomized Testing: When a Monkey Can do Better than Human. The talk will take a look at the concept of writing (unit) tests with predictable randomness. We will try to generalize: describe what such tests are, how to write them and how randomized testing can help in finding bugs in otherwise deterministic routines. We will also try to demonstrate on real-life examples that human understanding and predictions with regard to software are typically, ehm, incorrect (the same applies to mice and towels, see: THGTTG for full explanation). Randomized testing has been used extensively in Apache Lucene, Solr and ElasticSearch. Developers generally love it because it's effective at finding bugs and hate it when the bugs it finds are theirs to debug. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
