[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-6572: --- Fix Version/s: (was: 2.1.3) 3.0 Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Carl Yeksigian Fix For: 3.0 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-6572: -- Fix Version/s: (was: 2.1.2) 2.1.3 Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Carl Yeksigian Fix For: 2.1.3 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6572: -- Reviewer: (was: Benedict) Have you had a chance to look at this, Carl? Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Carl Yeksigian Fix For: 2.1.1 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6572: -- Assignee: Carl Yeksigian Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Carl Yeksigian Fix For: 2.1.1 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-6572: Assignee: (was: Lyuben Todorov) Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Fix For: 2.1.1 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6572: -- Fix Version/s: (was: 2.1.0) 2.1.1 Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Lyuben Todorov Fix For: 2.1.1 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tyler Hobbs updated CASSANDRA-6572: --- Reviewer: Benedict (was: Tyler Hobbs) Alright, I've set you as the reviewer. I do have plenty of other things on my plate :) Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Lyuben Todorov Fix For: 2.1.1 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6572: -- Fix Version/s: (was: 2.0.8) 2.1 bq. It's a pretty safe patch, but as a non-essential feature I think it should be reserved for 2.1 or 3.0. Let's do 2.1.1 then. Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Lyuben Todorov Fix For: 2.1 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6572: -- Reviewer: Tyler Hobbs WDYT [~thobbs], is this uninvasive enough to make it into 2.0? Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Assignee: Lyuben Todorov Fix For: 2.0.8 Attachments: 6572-trunk.diff Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lyuben Todorov updated CASSANDRA-6572:
--
Attachment: 6572-trunk.diff
The process goes like this:
# We enable recording via JMX SS#enableQueryRecording
# Insert n number of queries
# Replay queries to a new cluster using tools/bin/workloadreplayer
Running through an example:
# JMX Call to SS#enableQueryRecording where we supply parameters of 5 for 5MB
log limit, 4 for record ever 1/4 queries and finally
{{/var/lib/cassadra/querylog}} as the directory for the logs
# Insert 100k rows
This should result in 2 query logs, one of which is 5mb and has been renamed to
store a timestamp in its name, the other will be named QueryLog.log. Between
the two logs there should be 25k queries.
# Replaying the logs is done via the replay tool (workloadreplayer) where we
first supply the directory of the query logs and then various flags ([see git
branch
here|https://github.com/lyubent/cassandra/commit/526672982870bec49e5b234e8d11ef5e1f17cd28#diff-91cd490dd94b74e10ade733f61dc6ab7R207])
e.g:
{{./tools/bin/workloadreplayer /Users/lyubentodorov/Desktop/Log/ -t 100}}
Concerns:
Two synchronize blocks (one in
[QueryProcessor#maybeLogQuery|https://github.com/lyubent/cassandra/commit/526672982870bec49e5b234e8d11ef5e1f17cd28#diff-9c19942eca6c858baad84e942b3c7e21R402]
and the other in
[QueryRecorder#append|https://github.com/lyubent/cassandra/commit/526672982870bec49e5b234e8d11ef5e1f17cd28#diff-7d2a64c8ee2a2b78b3f1921e673b423eR73])
have been added on the read path, but since these blocks will only be hit when
query logging is enabled it shouldn't hinder performance where it matters most.
I've used the thrift client so I'm not sure if queries routing will be optimal.
Feature branch [here|https://github.com/lyubent/cassandra/tree/6572], also
attaching a patch for trunk. I'll patch this for cassandra-2.0 tomorrow :)
Workload recording / playback
-
Key: CASSANDRA-6572
URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
Project: Cassandra
Issue Type: New Feature
Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
Fix For: 2.0.8
Attachments: 6572-trunk.diff
Write sample mode gets us part way to testing new versions against a real
world workload, but we need an easy way to test the query side as well.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Ellis updated CASSANDRA-6572: -- Fix Version/s: 2.0.5 As for the UI, I think a JMX method to enable recording every Nth query is enough to start with, and then a separate tool to run those queries against another cluster. Suggest saving a timestamp as well as the query itself so that we can playback with the same timing as the original as well as as fast as you can run them. Tagging this for 2.0 but we can revise that if it turns out to be more invasive than I think. Workload recording / playback - Key: CASSANDRA-6572 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 Project: Cassandra Issue Type: New Feature Components: Core, Tools Reporter: Jonathan Ellis Fix For: 2.0.5 Write sample mode gets us part way to testing new versions against a real world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
