[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15267232#comment-15267232
]
Marcus Eriksson commented on CASSANDRA-11514:
-
What are we measuring here? Write performance or compaction performance? The
graph looks like it is write perf?
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15262168#comment-15262168
]
Sylvain Lebresne commented on CASSANDRA-11514:
--
What is the test run by {{daily_regressions_trunk-compaction_lcs}} so someone
else can try to reproduce/have a look at it?
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260406#comment-15260406
]
Michael Shuler commented on CASSANDRA-11514:
I was unable to find a concrete method to bisect this - I attempted a good
number of variations to find a way to call a commit "good" or "bad", but was
unsuccessful. Those are on a private jira
[CSTAR-478|https://datastax.jira.com/browse/CSTAR-478], which I'm going to
close, since I'm currently unsure of how to proceed.
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230918#comment-15230918
]
Michael Shuler commented on CASSANDRA-11514:
Larger run on DTCS
[trunk_3.5_3.0-compaction_DTCS|http://cstar.datastax.com/tests/artifacts/c9dd18fc-fcd7-11e5-8f8b-0256e416528f/graph]
seems to support that trunk and 3.5 both have this commit. The differences
seem pretty small, so it could be difficult to determine a difference between 2
commits.
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230795#comment-15230795
]
Michael Shuler commented on CASSANDRA-11514:
Bumped up from 10M to 25M writes/reads and this round on
[trunk_3.5_3.0-compaction_LCS|http://cstar.datastax.com/tests/artifacts/aa6dee6a-fcd7-11e5-8f8b-0256e416528f/graph]
looks like 3.5 is pretty evenly split performance-wise between 3.0 and trunk.
(Waiting on DTCS to finish)
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230421#comment-15230421
]
Michael Shuler commented on CASSANDRA-11514:
[trunk_3.5_3.0-compaction_LCS|http://cstar.datastax.com/tests/artifacts/7b642270-fccc-11e5-bf65-0256e416528f/graph]
shows that both trunk and 3.5 seem to have the same issue. 'op rate' text and
'Total operations' graph seem to agree.
[trunk_3.5_3.0-compaction_DTCS|http://cstar.datastax.com/tests/artifacts/2aace7d6-fccc-11e5-bf65-0256e416528f/graph]
is less conclusive.
I'm going to run both for a longer period and write more data to see if I can
get a little more definitive answer.
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15230323#comment-15230323
]
Michael Shuler commented on CASSANDRA-11514:
I'm starting some perf runs to see if a) is this commit in the 3.5 release
HEAD, comparing with trunk and 3.0 HEAD, then b) see if I can determine the
commit culprit.
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228948#comment-15228948
]
Michael Shuler commented on CASSANDRA-11514:
Automatically on cstar_perf, unfortunately not. It may be possible feeding jobs
with git sha's.
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11514) trunk compaction performance regression
[
https://issues.apache.org/jira/browse/CASSANDRA-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228935#comment-15228935
]
Sylvain Lebresne commented on CASSANDRA-11514:
--
Would it be possible to bisect the culprit?
> trunk compaction performance regression
> ---
>
> Key: CASSANDRA-11514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11514
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: cstar_perf
>Reporter: Michael Shuler
> Labels: performance
> Fix For: 3.x
>
> Attachments: trunk-compaction_dtcs-op_rate.png,
> trunk-compaction_lcs-op_rate.png
>
>
> It appears that a commit between Mar 29-30 has resulted in a drop in
> compaction performance. I attempted to get a log list of commits to post
> here, but
> {noformat}
> git log trunk@{2016-03-29}..trunk@{2016-03-31}
> {noformat}
> appears to be incomplete, since reading through {{git log}} I see netty and
> och were upgraded during this time period.
> !trunk-compaction_dtcs-op_rate.png!
> !trunk-compaction_lcs-op_rate.png!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
