Ariel Weisberg created CASSANDRA-9007:
-----------------------------------------
Summary: Run stress nightly against trunk in a way that validates
Key: CASSANDRA-9007
URL: https://issues.apache.org/jira/browse/CASSANDRA-9007
Project: Cassandra
Issue Type: Task
Reporter: Ariel Weisberg
Assignee: Philip Thompson
Stress has some very basic validation functionality when used without workload
profiles. It found a bug on trunk when I first ran it so it has value even
though the validation is basic.
As a beachhead for the kind of blackbox validation that we are missing we can
start by running stress nightly or 24/7 in some rotation.
There should be two jobs. One job has inverted success criteria (C* should lose
some data) and the job should only "pass" if the failure is detected. This is
just to prove that the harness reports failure if failure occurs.
Another would be the real job that runs stress, parses and parses the output
for reports of missing data.
This job is the first pass and basis of what we can point to when a developer
makes a change, implements a feature, or fixes a bug, and say "go add
validation to this job."
Follow on tickets to link to this
* Test multiple configurations
* Get stress to validate more query functionality and APIs (counters, LWT,
batches)
* Parse logs and fail tests on error level logs (great way to improve log
messages over time)
* ?
I am going to hold off on creating a ton of issues until we have a basic
version of the job running.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
