[
https://issues.apache.org/jira/browse/TINKERPOP-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15076050#comment-15076050
]
Ted Wilmes commented on TINKERPOP-1016:
---------------------------------------
Yeah, that's a good point. I think we could run into issues. I looked around
some more and discovered that Netty has a nice JMH microbenchmark setup:
[http://netty.io/wiki/microbenchmarks.html]. They forgo the uberjar and
instead kick JMH off via junit. I'm going to do some comparisons between the
uberjar results and this approach but I can't see why it would make any
difference. Regardless, JMH forks off new JVMs so if the classes are there,
seems like the results should be the same. This simplifies things greatly and
would make it easy to either do the separate {{gremlin-benchmark}} approach or
embed the tests within existing modules.
I have a basic working version based off Netty at
[https://github.com/apache/incubator-tinkerpop/tree/TINKERPOP-1016] if you want
to try it out. You can navigate down into {{gremlin-benchmark}} and run it
from the command line using {{mvn clean test}} or right click on the
GraphTraversalBenchmark in the IDE and run it from there. A few things to
note, you can't run individual benchmarks within the class right now, it's all
or nothing. Also, I'll add the {{skipPerformanceTests}} flag back but they'll
just run by default for now. Finally, the benchmarks themselves still need to
be tweaked for the addV/addVertex comparison so this should be treated as more
of a JMH example now than sacrosanct performance numbers.
> Replace junit-benchmarks with JMH
> ---------------------------------
>
> Key: TINKERPOP-1016
> URL: https://issues.apache.org/jira/browse/TINKERPOP-1016
> Project: TinkerPop
> Issue Type: Improvement
> Components: test-suite
> Affects Versions: 3.1.0-incubating
> Reporter: Ted Wilmes
> Assignee: Ted Wilmes
> Priority: Minor
>
> Replace junit-benchmark with JMH. This includes the following tasks:
> * Evaluate which existing benchmarks should be kept and port them to JMH.
> * Add an initial JMH benchmark for TINKERPOP-957
> * See if we can trigger perf tests with the same scheme that is currently
> used {noformat}mvn verify -DskipPerformanceTests=true/false{noformat}
> * Write developer docs that outline our initial performance testing approach.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
