Re: Using Cassandra for session tokens
Hi Phil, just my 2 cents, just watch out for these issues like counter type (replicate on write), compaction (when node load goes huge) and cassandra instance gc. This issues exists in 1.x perhaps it has been resolved in 2.x. hth jason On Mon, Dec 1, 2014 at 10:44 PM, Matt Brown m...@mattnworb.com wrote: This sounds like a good use case for http://www.datastax.com/dev/blog/datetieredcompactionstrategy On Dec 1, 2014, at 3:07 AM, Phil Wise p...@advancedtelematic.com wrote: We're considering switching from using Redis to Cassandra to store short lived (~1 hour) session tokens, in order to reduce the number of data storage engines we have to manage. Can anyone foresee any problems with the following approach: 1) Use the TTL functionality in Cassandra to remove old tokens. 2) Store the tokens in a table like: CREATE TABLE tokens ( id uuid, username text, // (other session information) PRIMARY KEY (id) ); 3) Perform ~100 writes/sec like: INSERT INTO tokens (id, username ) VALUES (468e0d69-1ebe-4477-8565-00a4cb6fa9f2, 'bob') USING TTL 3600; 4) Perform ~1000 reads/sec like: SELECT * FROM tokens WHERE ID=468e0d69-1ebe-4477-8565-00a4cb6fa9f2 ; The tokens will be about 100 bytes each, and we will grant 100 per second on a small 3 node cluster. Therefore there will be about 360k tokens alive at any time, with a total size of 36MB before database overhead. My biggest worry at the moment is that this kind of workload will stress compaction in an unusual way. Are there any metrics I should keep an eye on to make sure it is working fine? I read over the following links, but they mostly talk about DELETE-ing and tombstones. Am I right in thinking that as soon as a node performs a compaction then the rows with an expired TTL will be thrown away, regardless of gc_grace_seconds? https://issues.apache.org/jira/browse/CASSANDRA-7534 http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets https://issues.apache.org/jira/browse/CASSANDRA-6654 Thank you Phil
Re: Error when dropping keyspaces; One row required, 0 found
I don’t know what it is but I also saw “empty” keyspaces via CQL while migrating an existing test cluster from 2.0.9 to 2.1.0 (final release bits prior to labelling). Since I was doing this manually (and had cqlsh problems due to python change) I figured it might have been me. My observation was the same - this wasn’t actual data corruption but some nodes’ in memory state was incorrect - I didn’t investigate the problem too much as it soon went away, however it seemed to happen when there was schema disagreement in the cluster. Fixing that and restarting the nodes solved it. On Dec 1, 2014, at 7:24 PM, Mark Greene green...@gmail.com wrote: I'm running Cassandra 2.1.0. I was attempting to drop two keyspaces via cqlsh and encountered an error in the CLI as well as the appearance of losing all my keyspaces. Below is the output from my cqlsh session: $ cqlsh Connected to Production Cluster at 127.0.0.1:9042 http://127.0.0.1:9042/. [cqlsh 5.0.1 | Cassandra 2.1.0 | CQL spec 3.2.0 | Native protocol v3] Use HELP for help. cqlsh desc keyspaces; contacts_index contacts_testing contacts system OpsCenter system_traces cqlsh drop keyspace contacts_index; cqlsh drop keyspace contacts; ErrorMessage code= [Server error] message=java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException cqlsh drop keyspace contacts; ErrorMessage code= [Server error] message=java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: One row required, 0 found cqlsh desc keyspaces; empty -- OH SHIT -- After it appeared that I had lost all my keyspaces, I looked at the system.log and found this: (full log attached) ERROR [MigrationStage:1] 2014-12-01 23:54:05,622 CassandraDaemon.java:166 - Exception in thread Thread[MigrationStage:1,5,main] java.lang.IllegalStateException: One row required, 0 found at org.apache.cassandra.cql3.UntypedResultSet$FromResultSet.one(UntypedResultSet.java:78) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.config.KSMetaData.fromSchema(KSMetaData.java:275) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.db.DefsTables.mergeKeyspaces(DefsTables.java:230) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:186) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:164) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49) ~[apache-cassandra-2.1.0.jar:2.1.0] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.1.0.jar:2.1.0] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_65] at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_65] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_65] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65] At this point I wasn't sure quite what to do about this and did a rolling restart of the entire ring. After which, the keyspaces that were not attempted to be deleted returned when running 'desc keyspaces' and my intended keyspaces to be deleted had been removed as expected. Strangely enough, because we run OpsCenter, we lost the dashboards we had configured. Not a total deal breaker, but concerning that data loss occurred here assuming it's related. Anyone run into something like this before? system.log smime.p7s Description: S/MIME cryptographic signature
Recommissioned node is much smaller
As a a test, I took down a node, deleted /var/lib/cassandra and restarted it. After it joined the cluster, it’s about 75% the size of its neighbors (both in terms of bytes and numbers of keys). Prior to my test it was approximately the same size. I have no explanation for why that node would shrink so much, other than data loss. I have no deleted data, and no TTL’s. Only a small percentage of my data has had any updates (and some of my tables have had only inserts, and those have shrunk by 25% as well). I don’t really know how to check if I have records that have fewer than three replicas (RF=3). Any thoughts would be greatly appreciated. Thanks Robert
Re: Recommissioned node is much smaller
On Tue, Dec 2, 2014 at 12:21 PM, Robert Wille rwi...@fold3.com wrote: As a a test, I took down a node, deleted /var/lib/cassandra and restarted it. After it joined the cluster, it’s about 75% the size of its neighbors (both in terms of bytes and numbers of keys). Prior to my test it was approximately the same size. I have no explanation for why that node would shrink so much, other than data loss. I have no deleted data, and no TTL’s. Only a small percentage of my data has had any updates (and some of my tables have had only inserts, and those have shrunk by 25% as well). I don’t really know how to check if I have records that have fewer than three replicas (RF=3). Sounds suspicious, actually. I would suspect partial-bootstrap. To determine if you have under-replicated data, run repair. That's what it's for. =Rob
Re: Nodes get stuck in crazy GC loop after some time, leading to timeouts
On Mon, Dec 1, 2014 at 11:07 PM, Jason Wee peich...@gmail.com wrote: Hi Rob, any recommended documentation on describing explanation/configuration of the JVM heap and permanent generation ? We stucked in this same situation too. :( The archives of this list are chock full of explorations of various cases. Your best bet is to look for a good Aaron Morton reference where he breaks down the math between generations. I swear there was a blog post of his on this subject, but the best I can find is this slidedeck : http://www.slideshare.net/aaronmorton/cassandra-tk-2014-large-nodes =Rob
Re: Cassandra add a node and remove a node
On Mon, Dec 1, 2014 at 7:10 PM, Neha Trivedi nehajtriv...@gmail.com wrote: No the old node is not defective. We Just want to separate out that Server for testing. And add a new node. (Present cluster has two Nodes and RF=2) If you currently have two nodes and RF=2, you must add the new node before removing the old node, so that you always have at least two nodes in the cluster. =Rob
Re: Recommissioned node is much smaller
I meant to mention that I had run repair, but neglected to do so. Sorry about that. Repair runs pretty quick (a fraction of the time that compaction takes) and doesn’t seem to do anything. On Dec 2, 2014, at 1:44 PM, Robert Coli rc...@eventbrite.commailto:rc...@eventbrite.com wrote: On Tue, Dec 2, 2014 at 12:21 PM, Robert Wille rwi...@fold3.commailto:rwi...@fold3.com wrote: As a a test, I took down a node, deleted /var/lib/cassandra and restarted it. After it joined the cluster, it’s about 75% the size of its neighbors (both in terms of bytes and numbers of keys). Prior to my test it was approximately the same size. I have no explanation for why that node would shrink so much, other than data loss. I have no deleted data, and no TTL’s. Only a small percentage of my data has had any updates (and some of my tables have had only inserts, and those have shrunk by 25% as well). I don’t really know how to check if I have records that have fewer than three replicas (RF=3). Sounds suspicious, actually. I would suspect partial-bootstrap. To determine if you have under-replicated data, run repair. That's what it's for. =Rob
opscenter: 0 of 0 agents connected, but /nodes/all gives 3 results
Hi all -
Just getting started setting up OpsCenter today. I have a 3 node cassandra
cluster and (afaict) the agent installed and running happily on all 3
nodes. I also have OpsCenter up and running on a 4th node. I do not have
SSL enabled between these nodes.
In the OpsCenter interface, I see 0 of 0 agents connected and restarting
OpsCenter does not change this. However, in my cluster settings I have the
following entered for the least one host / IP in the cluster portion, and
these hostnames are all resolvable/pingable from the OpsCenter machine.
cassandra-db-1
cassandra-db-2
cassandra-db-3
Also, from this post
http://www.datastax.com/support-forums/topic/agents-running-without-error-but-not-connecting-to-opscenter
I
found a suggestion to curl cluster/nodes/all and when I do that I (think
I) get results showing 3 nodes.
Any suggestions? Thanks!
- Ian
curl http://localhost:/FullStory/nodes/all
[{load: 0.0, has_jna: false, vnodes: true, devices:
{saved_caches: sdb, commitlog: sdb, other: [sda], data:
[sdb]}, task_progress: {}, node_ip: 10.240.167.187,
network_interfaces: [eth0, lo], ec2: {instance-type: null,
placement: null, ami-id: null, instance-id: null}, node_version:
{search: null, jobtracker: null, tasktracker: null, spark:
{master: null, version: null, worker: null}, dse: null,
cassandra: 2.0.9}, dc: us-central1, node_name:
cassandra-db-3.c.fs-staging.internal, num_procs: 1, streaming: {},
token: -3220529141950148793, data_held: 54064133.0, mode: normal,
rpc_ip: 10.240.167.187, partitions: {saved_caches: /dev/sdb,
commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/121fd70f_e625_4394_89ce_5e5eff3d25e0, rootfs],
data: [/dev/sdb]}, os: linux, rack: f, last_seen: 0},
{load: 0.0, has_jna: false, vnodes: true, devices: {saved_caches:
sdb, commitlog: sdb, other: [sda], data: [sdb]},
task_progress: {}, node_ip: 10.240.111.79, network_interfaces:
[eth0, lo], ec2: {instance-type: null, placement: null, ami-id:
null, instance-id: null}, node_version: {search: null, jobtracker:
null, tasktracker: null, spark: {master: null, version: null,
worker: null}, dse: null, cassandra: 2.0.9}, dc: us-central1,
node_name: cassandra-db-1.c.fs-staging.internal, num_procs: 1,
streaming: {}, token: 6048217978730786970, data_held: 54319361.0,
mode: normal, rpc_ip: 10.240.111.79, partitions: {saved_caches:
/dev/sdb, commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/2795e34f_4e9b_46a9_9e39_e6cc0f1f1478, rootfs],
data: [/dev/sdb]}, os: linux, rack: a, last_seen: 0},
{load: 0.04, has_jna: false, vnodes: true, devices:
{saved_caches: sdb, commitlog: sdb, other: [sda], data:
[sdb]}, task_progress: {}, node_ip: 10.240.129.185,
network_interfaces: [eth0, lo], ec2: {instance-type: null,
placement: null, ami-id: null, instance-id: null}, node_version:
{search: null, jobtracker: null, tasktracker: null, spark:
{master: null, version: null, worker: null}, dse: null,
cassandra: 2.0.9}, dc: us-central1, node_name:
cassandra-db-2.c.fs-staging.internal, num_procs: 1, streaming: {},
token: 4393271482730462700, data_held: 51388793.0, mode: normal,
rpc_ip: 10.240.129.185, partitions: {saved_caches: /dev/sdb,
commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/2795e34f_4e9b_46a9_9e39_e6cc0f1f1478, rootfs],
data: [/dev/sdb]}, os: linux, rack: b, last_seen: 0}]
Re: Recommissioned node is much smaller
On Tue, Dec 2, 2014 at 2:21 PM, Robert Wille rwi...@fold3.com wrote: As a a test, I took down a node, deleted /var/lib/cassandra and restarted it. Did you decommission or removenode it when you took it down? If you didn't, the old node is still in the ring, and affects the replication. -- Tyler Hobbs DataStax http://datastax.com/
Re: opscenter: 0 of 0 agents connected, but /nodes/all gives 3 results
I have observed this kind of situation with 0 agents connected. Restarting
the agents always helped so far. By the way, check the agent's logs and
opscenterd logs, there may be some clues there.
On Tue, Dec 2, 2014 at 4:59 PM, Ian Rose ianr...@fullstory.com wrote:
Hi all -
Just getting started setting up OpsCenter today. I have a 3 node
cassandra cluster and (afaict) the agent installed and running happily on
all 3 nodes. I also have OpsCenter up and running on a 4th node. I do not
have SSL enabled between these nodes.
In the OpsCenter interface, I see 0 of 0 agents connected and restarting
OpsCenter does not change this. However, in my cluster settings I have the
following entered for the least one host / IP in the cluster portion, and
these hostnames are all resolvable/pingable from the OpsCenter machine.
cassandra-db-1
cassandra-db-2
cassandra-db-3
Also, from this post
http://www.datastax.com/support-forums/topic/agents-running-without-error-but-not-connecting-to-opscenter
I
found a suggestion to curl cluster/nodes/all and when I do that I (think
I) get results showing 3 nodes.
Any suggestions? Thanks!
- Ian
curl http://localhost:/FullStory/nodes/all
[{load: 0.0, has_jna: false, vnodes: true, devices:
{saved_caches: sdb, commitlog: sdb, other: [sda], data:
[sdb]}, task_progress: {}, node_ip: 10.240.167.187,
network_interfaces: [eth0, lo], ec2: {instance-type: null,
placement: null, ami-id: null, instance-id: null}, node_version:
{search: null, jobtracker: null, tasktracker: null, spark:
{master: null, version: null, worker: null}, dse: null,
cassandra: 2.0.9}, dc: us-central1, node_name:
cassandra-db-3.c.fs-staging.internal, num_procs: 1, streaming: {},
token: -3220529141950148793, data_held: 54064133.0, mode: normal,
rpc_ip: 10.240.167.187, partitions: {saved_caches: /dev/sdb,
commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/121fd70f_e625_4394_89ce_5e5eff3d25e0, rootfs],
data: [/dev/sdb]}, os: linux, rack: f, last_seen: 0},
{load: 0.0, has_jna: false, vnodes: true, devices: {saved_caches:
sdb, commitlog: sdb, other: [sda], data: [sdb]},
task_progress: {}, node_ip: 10.240.111.79, network_interfaces:
[eth0, lo], ec2: {instance-type: null, placement: null, ami-id:
null, instance-id: null}, node_version: {search: null, jobtracker:
null, tasktracker: null, spark: {master: null, version: null,
worker: null}, dse: null, cassandra: 2.0.9}, dc: us-central1,
node_name: cassandra-db-1.c.fs-staging.internal, num_procs: 1,
streaming: {}, token: 6048217978730786970, data_held: 54319361.0,
mode: normal, rpc_ip: 10.240.111.79, partitions: {saved_caches:
/dev/sdb, commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/2795e34f_4e9b_46a9_9e39_e6cc0f1f1478, rootfs],
data: [/dev/sdb]}, os: linux, rack: a, last_seen: 0},
{load: 0.04, has_jna: false, vnodes: true, devices:
{saved_caches: sdb, commitlog: sdb, other: [sda], data:
[sdb]}, task_progress: {}, node_ip: 10.240.129.185,
network_interfaces: [eth0, lo], ec2: {instance-type: null,
placement: null, ami-id: null, instance-id: null}, node_version:
{search: null, jobtracker: null, tasktracker: null, spark:
{master: null, version: null, worker: null}, dse: null,
cassandra: 2.0.9}, dc: us-central1, node_name:
cassandra-db-2.c.fs-staging.internal, num_procs: 1, streaming: {},
token: 4393271482730462700, data_held: 51388793.0, mode: normal,
rpc_ip: 10.240.129.185, partitions: {saved_caches: /dev/sdb,
commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/2795e34f_4e9b_46a9_9e39_e6cc0f1f1478, rootfs],
data: [/dev/sdb]}, os: linux, rack: b, last_seen: 0}]
--
Nikolai Grigoriev
(514) 772-5178
Re: Recommissioned node is much smaller
I didn’t do anything except kill the server process, delete /var/lib/cassandra, and start it back up again. nodetool status shows all nodes as UN, and doesn’t display any unexpected nodes. I don’t know if this sheds any light on the issue, but I’ve added a considerable amount of data to the cluster since I did the aforementioned test. The difference in size between the nodes is shrinking. The other nodes are growing more slowly than the one I recommissioned. That was definitely not something that I expected, and I don’t have any explanation for that either. Robert On Dec 2, 2014, at 3:38 PM, Tyler Hobbs ty...@datastax.commailto:ty...@datastax.com wrote: On Tue, Dec 2, 2014 at 2:21 PM, Robert Wille rwi...@fold3.commailto:rwi...@fold3.com wrote: As a a test, I took down a node, deleted /var/lib/cassandra and restarted it. Did you decommission or removenode it when you took it down? If you didn't, the old node is still in the ring, and affects the replication. -- Tyler Hobbs DataStaxhttp://datastax.com/
Apache Spark connector and saving collection typed columns
Hi, Is it possible to save columns of collection types (Map in particular) with the Spark connector? I see that you can easily read a collection column, but i can’t find any documentation on writing such columns. mn
Re: opscenter: 0 of 0 agents connected, but /nodes/all gives 3 results
Often this is caused by the agent version not matching the OpsCenter
version. Unfortunately the UI currently reports a version mismatch as 'not
installed'. Can you check the versions of your OpsCenter and agent installs?
On Tue, Dec 2, 2014 at 5:06 PM, Nikolai Grigoriev ngrigor...@gmail.com
wrote:
I have observed this kind of situation with 0 agents connected. Restarting
the agents always helped so far. By the way, check the agent's logs and
opscenterd logs, there may be some clues there.
On Tue, Dec 2, 2014 at 4:59 PM, Ian Rose ianr...@fullstory.com wrote:
Hi all -
Just getting started setting up OpsCenter today. I have a 3 node
cassandra cluster and (afaict) the agent installed and running happily on
all 3 nodes. I also have OpsCenter up and running on a 4th node. I do not
have SSL enabled between these nodes.
In the OpsCenter interface, I see 0 of 0 agents connected and
restarting OpsCenter does not change this. However, in my cluster settings
I have the following entered for the least one host / IP in the cluster
portion, and these hostnames are all resolvable/pingable from the OpsCenter
machine.
cassandra-db-1
cassandra-db-2
cassandra-db-3
Also, from this post
http://www.datastax.com/support-forums/topic/agents-running-without-error-but-not-connecting-to-opscenter
I
found a suggestion to curl cluster/nodes/all and when I do that I (think
I) get results showing 3 nodes.
Any suggestions? Thanks!
- Ian
curl http://localhost:/FullStory/nodes/all
[{load: 0.0, has_jna: false, vnodes: true, devices:
{saved_caches: sdb, commitlog: sdb, other: [sda], data:
[sdb]}, task_progress: {}, node_ip: 10.240.167.187,
network_interfaces: [eth0, lo], ec2: {instance-type: null,
placement: null, ami-id: null, instance-id: null}, node_version:
{search: null, jobtracker: null, tasktracker: null, spark:
{master: null, version: null, worker: null}, dse: null,
cassandra: 2.0.9}, dc: us-central1, node_name:
cassandra-db-3.c.fs-staging.internal, num_procs: 1, streaming: {},
token: -3220529141950148793, data_held: 54064133.0, mode: normal,
rpc_ip: 10.240.167.187, partitions: {saved_caches: /dev/sdb,
commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/121fd70f_e625_4394_89ce_5e5eff3d25e0, rootfs],
data: [/dev/sdb]}, os: linux, rack: f, last_seen: 0},
{load: 0.0, has_jna: false, vnodes: true, devices: {saved_caches:
sdb, commitlog: sdb, other: [sda], data: [sdb]},
task_progress: {}, node_ip: 10.240.111.79, network_interfaces:
[eth0, lo], ec2: {instance-type: null, placement: null, ami-id:
null, instance-id: null}, node_version: {search: null, jobtracker:
null, tasktracker: null, spark: {master: null, version: null,
worker: null}, dse: null, cassandra: 2.0.9}, dc: us-central1,
node_name: cassandra-db-1.c.fs-staging.internal, num_procs: 1,
streaming: {}, token: 6048217978730786970, data_held: 54319361.0,
mode: normal, rpc_ip: 10.240.111.79, partitions: {saved_caches:
/dev/sdb, commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/2795e34f_4e9b_46a9_9e39_e6cc0f1f1478, rootfs],
data: [/dev/sdb]}, os: linux, rack: a, last_seen: 0},
{load: 0.04, has_jna: false, vnodes: true, devices:
{saved_caches: sdb, commitlog: sdb, other: [sda], data:
[sdb]}, task_progress: {}, node_ip: 10.240.129.185,
network_interfaces: [eth0, lo], ec2: {instance-type: null,
placement: null, ami-id: null, instance-id: null}, node_version:
{search: null, jobtracker: null, tasktracker: null, spark:
{master: null, version: null, worker: null}, dse: null,
cassandra: 2.0.9}, dc: us-central1, node_name:
cassandra-db-2.c.fs-staging.internal, num_procs: 1, streaming: {},
token: 4393271482730462700, data_held: 51388793.0, mode: normal,
rpc_ip: 10.240.129.185, partitions: {saved_caches: /dev/sdb,
commitlog: /dev/sdb, other:
[/dev/disk/by_uuid/2795e34f_4e9b_46a9_9e39_e6cc0f1f1478, rootfs],
data: [/dev/sdb]}, os: linux, rack: b, last_seen: 0}]
--
Nikolai Grigoriev
(514) 772-5178
Wide rows best practices and GC impact
Hi, We have a cluster (2.0.11) of 6 nodes (RF=3), c3.4xlarge instances, about 50 column families. Cassandra heap takes 8GB out of the 30GB of every instance. We mainly store time series-like data, where each data point is a binary blob of 5-20KB. We use wide rows, and try to put in the same row all the data that we usually need in a single query (but not more than that). As a result, our application logic is very simple (since we have to do just one query to read the data on average) and read/write response times are very satisfactory. This is a cfhistograms and a cfstats of our heaviest CF: SSTables per Read 1 sstables: 3198856 2 sstables: 45 Write Latency (microseconds) 4 us: 37 5 us: 1247 6 us: 9987 7 us: 31442 8 us: 66121 10 us: 400503 12 us: 1158329 14 us: 2873934 17 us: 11843616 20 us: 24464275 24 us: 30574717 29 us: 24351624 35 us: 16788801 42 us: 3935374 50 us: 797781 60 us: 272160 72 us: 121819 86 us: 64641 103 us: 41085 124 us: 33618 149 us: 199463 179 us: 255445 215 us: 38238 258 us: 12300 310 us: 5307 372 us: 3180 446 us: 2443 535 us: 1773 642 us: 1314 770 us: 991 924 us: 748 1109 us: 606 1331 us: 465 1597 us: 433 1916 us: 453 2299 us: 484 2759 us: 983 3311 us: 976 3973 us: 338 4768 us: 312 5722 us: 237 6866 us: 198 8239 us: 163 9887 us: 138 11864 us: 115 14237 us: 231 17084 us: 550 20501 us: 603 24601 us: 635 29521 us: 875 35425 us: 731 42510 us: 497 51012 us: 476 61214 us: 347 73457 us: 331 88148 us: 273 105778 us: 143 126934 us: 92 152321 us: 47 182785 us: 16 219342 us: 5 263210 us: 2 315852 us: 2 379022 us: 1 454826 us: 1 545791 us: 1 654949 us: 0 785939 us: 0 943127 us: 1 1131752 us: 1 Read Latency (microseconds) 20 us: 1 24 us: 9 29 us: 18 35 us: 96 42 us: 6989 50 us: 113305 60 us: 552348 72 us: 772329 86 us: 654019 103 us: 578404 124 us: 300364 149 us: 111522 179 us: 37385 215 us: 18353 258 us: 10733 310 us: 7915 372 us: 9406 446 us: 7645 535 us: 2773 642 us: 1323 770 us: 1351 924 us: 953 1109 us: 857 1331 us: 1122 1597 us: 800 1916 us: 806 2299 us: 686 2759 us: 581 3311 us: 671 3973 us: 318 4768 us: 318 5722 us: 226 6866 us: 164 8239 us: 161 9887 us: 134 11864 us: 125 14237 us: 184 17084 us: 285 20501 us: 315 24601 us: 378 29521 us: 431 35425 us: 468 42510 us: 469 51012 us: 466 61214 us: 407 73457 us: 337 88148 us: 297 105778 us: 242 126934 us: 135 152321 us: 109 182785 us: 57 219342 us: 41 263210 us: 28 315852 us: 16 379022 us: 12 454826 us: 6 545791 us: 6 654949 us: 0 785939 us: 0 943127 us: 0 1131752 us: 2 Partition Size (bytes) 3311 bytes: 1 3973 bytes: 2 4768 bytes: 0 5722 bytes: 2 6866 bytes: 0 8239 bytes: 0 9887 bytes: 2 11864 bytes: 1 14237 bytes: 0 17084 bytes: 0 20501 bytes: 0 24601 bytes: 0 29521 bytes: 3 35425 bytes: 0 42510 bytes: 1 51012 bytes: 1 61214 bytes: 1 73457 bytes: 3 88148 bytes: 1 105778 bytes: 5 126934 bytes: 2 152321 bytes: 4 182785 bytes: 65 219342 bytes: 165 263210 bytes: 268 315852 bytes: 201 379022 bytes: 30 454826 bytes: 248 545791 bytes: 16 654949 bytes: 41 785939 bytes: 259 943127 bytes: 547 1131752 bytes: 243 1358102 bytes: 176 1629722 bytes: 59 1955666 bytes: 37 2346799 bytes: 41 2816159 bytes: 78 3379391 bytes: 243 4055269 bytes: 122 4866323 bytes: 209 5839588 bytes: 220 7007506 bytes: 266 8409007 bytes: 77 10090808 bytes: 103 12108970 bytes: 1 14530764 bytes: 2 17436917 bytes: 7 20924300 bytes: 410 25109160 bytes: 76 Cell Count per Partition 3 cells: 5 4 cells: 0 5 cells: 0 6 cells: 2 7 cells: 0 8 cells: 0 10 cells: 2 12 cells: 1 14 cells: 0 17 cells: 0 20 cells: 1 24 cells: 3 29 cells: 1 35 cells: 1 42 cells: 0 50 cells: 0 60 cells: 3 72 cells: 0 86 cells: 1 103 cells: 0 124 cells: 11 149 cells: 3 179 cells: 4 215 cells: 10 258 cells: 13 310 cells: 2181 372 cells: 2 446 cells: 2 535 cells: 2 642 cells: 4 770 cells: 7 924 cells: 488 1109 cells: 3 1331 cells: 24 1597 cells: 143 1916 cells: 332 2299 cells: 2 2759 cells: 5 3311 cells: 483 3973 cells: 0 4768 cells: 2 5722 cells: 1 6866 cells: 1 8239 cells: 0 9887 cells: 2 11864 cells: 244 14237 cells: 1 17084 cells: 248 20501 cells: 1 24601 cells: 1 29521 cells: 1 35425 cells: 2 42510 cells: 1 51012 cells: 2 61214 cells: 237 Read Count: 3202919 Read Latency: 0.16807454013042478 ms. Write Count: 118568574 Write Latency: 0.026566498615391967 ms. Pending Tasks: 0 Table: protobuf_by_agent1 SSTable count: 49 SSTables in each level: [1, 11/10, 37, 0, 0, 0, 0, 0, 0] Space used (live), bytes: 6934395462
Re: Cassandra add a node and remove a node
Thanks Jens and Robert !!! On Wed, Dec 3, 2014 at 2:20 AM, Robert Coli rc...@eventbrite.com wrote: On Mon, Dec 1, 2014 at 7:10 PM, Neha Trivedi nehajtriv...@gmail.com wrote: No the old node is not defective. We Just want to separate out that Server for testing. And add a new node. (Present cluster has two Nodes and RF=2) If you currently have two nodes and RF=2, you must add the new node before removing the old node, so that you always have at least two nodes in the cluster. =Rob
UPDATE statement is failed
Hi ALL,
There is a grogram to consume messages from queue; according to the message,
the program will READ a row and then UPDATE the row;
BUT, sometimes, the UPDATE statement is fail, the result of READ statement is
also the old content before UDPATE.
Any suggestions from you are appreciated.
The following are my program and the result.
---READ
from cassandra.query import SimpleStatement
from cassandra import ConsistencyLevel
self.interval_data_get_simple = SimpleStatement(SELECT TRADETIME,
OPENPRICE, HIGHPRICE, \
LOWPRICE, CLOSEPRICE, CHANGE, CHANGERATIO,
VOLUME, AMOUNT,SECURITYNAME, \
SECURITYID from {} WHERE SYMBOL = '{}' AND
TRADETIME = '{}';\
.format(self.cassandra_table, symbol, \
interval_trade_time.strftime(u'%Y-%m-%d
%H:%M:%S')), \
consistency_level=ConsistencyLevel.ALL)
cur_interval_future =
self.cassandra_session.execute_async(self.interval_data_get_simple)
-UPDATE
from cassandra.query import SimpleStatement
from cassandra import ConsistencyLevel
data_set_simple = SimpleStatement(UPDATE {} SET OPENPRICE = {},
HIGHPRICE = {}, LOWPRICE = {},\
CLOSEPRICE = {}, VOLUME = {}, AMOUNT = {}, MARKET = {}, SECURITYID
= {} WHERE \
SYMBOL = '{}' AND TRADETIME =
'{}';.format(self.cassandra_table, insert_data_list[0], \
insert_data_list[1], insert_data_list[2],
insert_data_list[3], \
insert_data_list[4], insert_data_list[5],
insert_data_list[6], \
insert_data_list[7], insert_data_list[8],
insert_data_list[9]), \
consistency_level=ConsistencyLevel.ALL)
update_future = self.cassandra_session.execute(data_set_simple)
test result--
#CQL UPDATE statement
UPDATE GTA_HFDCS_SSEL2.SSEL2_TRDMIN01_20141127 SET OPENPRICE = 8.460, HIGHPRICE
= 8.460, LOWPRICE = 8.460, CLOSEPRICE = 8.460, VOLUME = 1500,
AMOUNT = 12240.000, MARKET = 1, SECURITYID = 20103592 WHERE
SYMBOL = '600256' AND TRADETIME = '2014-11-27 10:00:00';
#the result of READ
[Row(tradetime=datetime.datetime(2014, 11, 27, 2, 0),
openprice=Decimal('8.460'), highprice=Decimal('8.460'),
lowprice=Decimal('8.460'), closeprice=Decimal('8.460'), change=None,
changeratio=None, volume=1500, amount=Decimal('12240.000'), securityname=None,
securityid=20103592)]
#CQL UPDATE statement
UPDATE GTA_HFDCS_SSEL2.SSEL2_TRDMIN01_20141127 SET OPENPRICE = 8.460, HIGHPRICE
= 8.460, LOWPRICE = 8.160, CLOSEPRICE = 8.160, VOLUME = 3500,
AMOUNT = 28560.000, MARKET = 1, SECURITYID = 20103592 WHERE
SYMBOL = '600256' AND TRADETIME = '2014-11-27 10:00:00';
#the result of READ
[Row(tradetime=datetime.datetime(2014, 11, 27, 2, 0),
openprice=Decimal('8.460'), highprice=Decimal('8.460'),
lowprice=Decimal('8.460'), closeprice=Decimal('8.460'), change=None,
changeratio=None, volume=1500, amount=Decimal('12240.000'), securityname=None,
securityid=20103592)]
Thanks Regards,
鄢来琼 / Peter YAN, Staff Software Engineer,
A3 Dept., GTA Information Technology Co., Ltd.
=
Mobile: 18620306659
E-Mail: laiqiong@gtafe.com
Website: http://www.gtafe.com/
=
Re: Nodes get stuck in crazy GC loop after some time, leading to timeouts
ack and many thanks for the tips and help.. jason On Wed, Dec 3, 2014 at 4:49 AM, Robert Coli rc...@eventbrite.com wrote: On Mon, Dec 1, 2014 at 11:07 PM, Jason Wee peich...@gmail.com wrote: Hi Rob, any recommended documentation on describing explanation/configuration of the JVM heap and permanent generation ? We stucked in this same situation too. :( The archives of this list are chock full of explorations of various cases. Your best bet is to look for a good Aaron Morton reference where he breaks down the math between generations. I swear there was a blog post of his on this subject, but the best I can find is this slidedeck : http://www.slideshare.net/aaronmorton/cassandra-tk-2014-large-nodes =Rob
re: UPDATE statement is failed
The system setting is as the following:
Cluster replication:
replication = {'class': 'NetworkTopologyStrategy', 'GTA_SZ_DC1':2}
Totally, 5 Nodes,
OS of Nodes are windows.
Thanks Regards,
鄢来琼 / Peter YAN, Staff Software Engineer,
A3 Dept., GTA Information Technology Co., Ltd.
=
Mobile: 18620306659
E-Mail: laiqiong@gtafe.com
Website: http://www.gtafe.com/
=
发件人: 鄢来琼 [mailto:laiqiong@gtafe.com]
发送时间: 2014年12月3日 11:49
收件人: user@cassandra.apache.org
主题: UPDATE statement is failed
Hi ALL,
There is a grogram to consume messages from queue; according to the message,
the program will READ a row and then UPDATE the row;
BUT, sometimes, the UPDATE statement is fail, the result of READ statement is
also the old content before UDPATE.
Any suggestions from you are appreciated.
The following are my program and the result.
---READ
from cassandra.query import SimpleStatement
from cassandra import ConsistencyLevel
self.interval_data_get_simple = SimpleStatement(SELECT TRADETIME,
OPENPRICE, HIGHPRICE, \
LOWPRICE, CLOSEPRICE, CHANGE, CHANGERATIO,
VOLUME, AMOUNT,SECURITYNAME, \
SECURITYID from {} WHERE SYMBOL = '{}' AND
TRADETIME = '{}';\
.format(self.cassandra_table, symbol, \
interval_trade_time.strftime(u'%Y-%m-%d
%H:%M:%S')), \
consistency_level=ConsistencyLevel.ALL)
cur_interval_future =
self.cassandra_session.execute_async(self.interval_data_get_simple)
-UPDATE
from cassandra.query import SimpleStatement
from cassandra import ConsistencyLevel
data_set_simple = SimpleStatement(UPDATE {} SET OPENPRICE = {},
HIGHPRICE = {}, LOWPRICE = {},\
CLOSEPRICE = {}, VOLUME = {}, AMOUNT = {}, MARKET = {}, SECURITYID
= {} WHERE \
SYMBOL = '{}' AND TRADETIME =
'{}';.format(self.cassandra_table, insert_data_list[0], \
insert_data_list[1], insert_data_list[2],
insert_data_list[3], \
insert_data_list[4], insert_data_list[5],
insert_data_list[6], \
insert_data_list[7], insert_data_list[8],
insert_data_list[9]), \
consistency_level=ConsistencyLevel.ALL)
update_future = self.cassandra_session.execute(data_set_simple)
test result--
#CQL UPDATE statement
UPDATE GTA_HFDCS_SSEL2.SSEL2_TRDMIN01_20141127 SET OPENPRICE = 8.460, HIGHPRICE
= 8.460, LOWPRICE = 8.460, CLOSEPRICE = 8.460, VOLUME = 1500,
AMOUNT = 12240.000, MARKET = 1, SECURITYID = 20103592 WHERE
SYMBOL = '600256' AND TRADETIME = '2014-11-27 10:00:00';
#the result of READ
[Row(tradetime=datetime.datetime(2014, 11, 27, 2, 0),
openprice=Decimal('8.460'), highprice=Decimal('8.460'),
lowprice=Decimal('8.460'), closeprice=Decimal('8.460'), change=None,
changeratio=None, volume=1500, amount=Decimal('12240.000'), securityname=None,
securityid=20103592)]
#CQL UPDATE statement
UPDATE GTA_HFDCS_SSEL2.SSEL2_TRDMIN01_20141127 SET OPENPRICE = 8.460, HIGHPRICE
= 8.460, LOWPRICE = 8.160, CLOSEPRICE = 8.160, VOLUME = 3500,
AMOUNT = 28560.000, MARKET = 1, SECURITYID = 20103592 WHERE
SYMBOL = '600256' AND TRADETIME = '2014-11-27 10:00:00';
#the result of READ
[Row(tradetime=datetime.datetime(2014, 11, 27, 2, 0),
openprice=Decimal('8.460'), highprice=Decimal('8.460'),
lowprice=Decimal('8.460'), closeprice=Decimal('8.460'), change=None,
changeratio=None, volume=1500, amount=Decimal('12240.000'), securityname=None,
securityid=20103592)]
Thanks Regards,
鄢来琼 / Peter YAN, Staff Software Engineer,
A3 Dept., GTA Information Technology Co., Ltd.
=
Mobile: 18620306659
E-Mail: laiqiong@gtafe.commailto:laiqiong@gtafe.com
Website: http://www.gtafe.com/
=
