[
https://issues.apache.org/jira/browse/TINKERPOP-1233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stephen mallette closed TINKERPOP-1233.
---------------------------------------
Resolution: Won't Do
I'm going to close this. It doesn't feel actionable to me. If we want to
implement something here we should create a specific issue to do that
particular thing.
> Gremlin-Benchmark wish list.
> ----------------------------
>
> Key: TINKERPOP-1233
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1233
> Project: TinkerPop
> Issue Type: Improvement
> Components: benchmark
> Affects Versions: 3.2.0-incubating
> Reporter: Marko A. Rodriguez
> Priority: Major
>
> [~twilmes] has developed {{gremlin-benchmark}} which is slated for 3.2.0
> (TINKERPOP-1016). This is really good as now we can ensure the Gremlin
> traversal machine only speeds up with each release. Here is a collection of
> things I would like to be able to do with {{gremlin-benchmark}}.
> ----
> *Benchmarks in the Strategy Tests*
> {code}
> // ensure that traversalA is at least 1.5 times faster than traversalB
> assertTrue(Benchmark.compare(traversalA,traversalB) > 1.50d)
> {code}
> With this, I can have an {{OptimizationStrategy}} applied to {{traversalA}}
> and not to {{traversalB}} and prove via "mvn clean install" that the strategy
> is in fact "worth it." I bet there are other good static methods we could
> create. Hell, why not just have a {{BenchmarkAsserts}} that we can statically
> import like JUnit {{Asserts}}. Then its just:
> {code}
> assertFaster(traversalA,traversalB,1.50d)
> assertSmaller(traversalA,traversalB) // memory usage or object creation?
> assertTime(traversal, 1000, TimeUnits.MILLISECONDS) // has to complete in 1
> second?
> ... ?
> {code}
> Its a little scary as not all computers are the same, but it would be nice to
> know that we have tests for space and time costs.
> ----
> *Benchmarks saved locally over the course of a release*
> This is tricky, but it would be cool if local folders (not to GitHub) were
> created like this:
> {code}
> tinkerpop3/gremlin-benchmark/benchmarks/g_V_out_out_12:23:66UT23.txt
> {code}
> Then a test case could ensure that all newer runs of that benchmark are
> faster than older ones. If its, lets say, 10%+ slower, {{Exception}} and test
> fails. ??
> What else can we do? Can we know whether a certain area of code is faster?
> For instance, strategy application or requirements aggregation? If we can
> introspect like that, that would be stellar.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
