[jira] [Commented] (CASSANDRA-7436) Dtest for tracking metrics regressions
[ https://issues.apache.org/jira/browse/CASSANDRA-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702013#comment-14702013 ] Steve Wang commented on CASSANDRA-7436: --- Created jmxmetrics_test, which takes JMX metrics and tests whether they work as expected. Unfortunately, I'm not too familiar as to how each value should change, so I only judged their expected change by tracking the change in values a couple times myself. Dtest for tracking metrics regressions -- Key: CASSANDRA-7436 URL: https://issues.apache.org/jira/browse/CASSANDRA-7436 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire Assignee: Steve Wang We should create a dtest that runs a small load test and collects a bunch of the JMX metrics and introduce assertions to ensure they are roughly the same every time. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7436) Dtest for tracking metrics regressions
[ https://issues.apache.org/jira/browse/CASSANDRA-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041153#comment-14041153 ] Benedict commented on CASSANDRA-7436: - I think I'd prefer to work this into a performance regression suite off the back of the performance test framework, where we collect JMX stats along with our other data for a standardized set of stress runs. We can then post process the results to check it's within tolerance of prior runs. I suspect this would be too noisy for a dtest, and not actually that useful without a long enough test to create a continuum of values. Dtest for tracking metrics regressions -- Key: CASSANDRA-7436 URL: https://issues.apache.org/jira/browse/CASSANDRA-7436 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire We should create a dtest that runs a small load test and collects a bunch of the JMX metrics and introduce assertions to ensure they are roughly the same every time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7436) Dtest for tracking metrics regressions
[ https://issues.apache.org/jira/browse/CASSANDRA-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041164#comment-14041164 ] Ryan McGuire commented on CASSANDRA-7436: - I think we need both. A simple dtest would at least exercise the code path for coverage purposes. I totally understand what you're talking about though, and I agree we need to build that into the cstar_perf stuff too. Dtest for tracking metrics regressions -- Key: CASSANDRA-7436 URL: https://issues.apache.org/jira/browse/CASSANDRA-7436 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire We should create a dtest that runs a small load test and collects a bunch of the JMX metrics and introduce assertions to ensure they are roughly the same every time. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7436) Dtest for tracking metrics regressions
[ https://issues.apache.org/jira/browse/CASSANDRA-7436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041173#comment-14041173 ] Benedict commented on CASSANDRA-7436: - I just think you'll struggle to get a viable baseline that doesn't break regularly, which might cause unnecessary overhead in trying to get it fixed - but the principle is sound Dtest for tracking metrics regressions -- Key: CASSANDRA-7436 URL: https://issues.apache.org/jira/browse/CASSANDRA-7436 Project: Cassandra Issue Type: Test Reporter: Ryan McGuire Assignee: Ryan McGuire We should create a dtest that runs a small load test and collects a bunch of the JMX metrics and introduce assertions to ensure they are roughly the same every time. -- This message was sent by Atlassian JIRA (v6.2#6252)
