Shad Storhaug created LUCENENET-627:
---------------------------------------
Summary: Port RandomizedTesting Test Runner
Key: LUCENENET-627
URL: https://issues.apache.org/jira/browse/LUCENENET-627
Project: Lucene.Net
Issue Type: Task
Components: Lucene.Net.TestFramework
Affects Versions: Lucene.Net 4.8.0
Reporter: Shad Storhaug
While the test framework can run with the standard NUnit test runner and we
have managed to make it work, it is missing the ability to reproduce a random
test failure, which is crucial for debugging some of the more complicated
random tests. Lucene uses a custom JUnit test runner called
[RandomizedTesting|https://github.com/randomizedtesting/randomizedtesting] to
accomplish this.
It also includes some other nice features
# Ensure when a test fails, the random seeds are included in error messages and
logs
# Code analysis to ensure the tests are setup properly
# Run tests in a random order to ensure they are not dependent upon each other
Preliminary analysis shows that the API for NUnit allows building custom test
runners and it is a close enough match to implement the functionality. Most
likely, this will require a custom adapter as well, so the test runner can
integrate into Visual Studio and dotnet test/mstest.
Do note that without doing some pretty heavy refactoring on the Codec,
Directory, and LuceneTestCase classes, it is not possible to run tests in
parallel within the same AppDomain because the codecs use a static variable to
turn codec impersonation on/off. For now, it would probably be best to run
tests serially.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
