[
https://issues.apache.org/jira/browse/TINKERPOP3-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526835#comment-14526835
]
Daniel Kuppitz commented on TINKERPOP3-231:
-------------------------------------------
I created a separate project that...
a) spawns a Docker new container
b) checks out the current master and
c) records the performance of a bunch of queries
What I found was that the results are pretty worthless. Differences in
execution times are mostly (or always?) just noise. In my opinion TinkerGraph
is too fast - automated benchmarks between releases / commits can lead to a lot
of wrong assumptions. You can even get totally different results if a commit
contains only changes in the docs, which wouldn't make any sense at all.
> Expand the PerformanceSuite
> ---------------------------
>
> Key: TINKERPOP3-231
> URL: https://issues.apache.org/jira/browse/TINKERPOP3-231
> Project: TinkerPop 3
> Issue Type: Improvement
> Components: test-suite
> Reporter: stephen mallette
> Assignee: Daniel Kuppitz
> Fix For: 3.0.0.GA
>
>
> We current have:
> https://github.com/tinkerpop/tinkerpop3/blob/master/gremlin-test/src/main/java/com/tinkerpop/gremlin/structure/StructurePerformanceSuite.java
> It doesn't do much - basic/simple tests for simple reads/writes. We should
> build this out for "traversal" oriented tests. Not sure if we want to have a
> {{ProcessPerformanceSuite}} for that or not. The model would be the same.
> This would be nice because it's then easy to check for performance
> regressions between branches, releases, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
