[
https://issues.apache.org/jira/browse/OLIO-29?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12680721#action_12680721
]
Akara Sucharitakul commented on OLIO-29:
----------------------------------------
Faban just does the Little's law verification as a guiding measure, not a
pass/fail. At the end of UIDriverMetrics.getResults(), we've added this check:
el[9] = new Element();
el[9].description = "Concurrent user to ops/sec ratio";
el[9].target = "<= 5.25";
double ratio = r.getScale() / r.getMetric();
el[9].result = String.format("%.2f", ratio);
if (ratio <= 5.25d)
el[9].passed = true;
else
el[9].passed = false;
return el;
> Driver does not check for accuracy in throughput to determine pass/fail
> -----------------------------------------------------------------------
>
> Key: OLIO-29
> URL: https://issues.apache.org/jira/browse/OLIO-29
> Project: Olio
> Issue Type: Bug
> Components: rails-driver
> Reporter: Akara Sucharitakul
> Assignee: Akara Sucharitakul
>
> The driver only checks whether the various metrics reported in the summary
> report have passed the necessary criteria but does not check whether the
> resulting throughput (ops/sec) is correct for the scale of the run (number of
> concurrent users). Thus for example, it is possible to get 150 ops/sec for a
> run done with 1000 users and be reported as PASSED.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
