Re: Cassandra schema migrator
I've added these as answers to a question I posted on Stack Overflow: http://stackoverflow.com/questions/26460932/how-to-deploy-changes-to-a-cassandra-cql-schema/27013426 Thank you Phil On 05.12.2014 15:23, Brian Sam-Bodden wrote: There is also https://github.com/hsgubert/cassandra_migrations On Fri, Dec 5, 2014 at 7:49 AM, Ben Hood 0x6e6...@gmail.com wrote: On Tue, Nov 25, 2014 at 12:49 PM, Phil Wise p...@advancedtelematic.com wrote: https://github.com/advancedtelematic/cql-migrate Great to see these tools out there! Just to add to the list https://github.com/mattes/migrate Might not be as C* specific as the other tools mentioned earlier in this thread, but it does integrate with Cassandra.
Using Cassandra for session tokens
We're considering switching from using Redis to Cassandra to store short lived (~1 hour) session tokens, in order to reduce the number of data storage engines we have to manage. Can anyone foresee any problems with the following approach: 1) Use the TTL functionality in Cassandra to remove old tokens. 2) Store the tokens in a table like: CREATE TABLE tokens ( id uuid, username text, // (other session information) PRIMARY KEY (id) ); 3) Perform ~100 writes/sec like: INSERT INTO tokens (id, username ) VALUES (468e0d69-1ebe-4477-8565-00a4cb6fa9f2, 'bob') USING TTL 3600; 4) Perform ~1000 reads/sec like: SELECT * FROM tokens WHERE ID=468e0d69-1ebe-4477-8565-00a4cb6fa9f2 ; The tokens will be about 100 bytes each, and we will grant 100 per second on a small 3 node cluster. Therefore there will be about 360k tokens alive at any time, with a total size of 36MB before database overhead. My biggest worry at the moment is that this kind of workload will stress compaction in an unusual way. Are there any metrics I should keep an eye on to make sure it is working fine? I read over the following links, but they mostly talk about DELETE-ing and tombstones. Am I right in thinking that as soon as a node performs a compaction then the rows with an expired TTL will be thrown away, regardless of gc_grace_seconds? https://issues.apache.org/jira/browse/CASSANDRA-7534 http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets https://issues.apache.org/jira/browse/CASSANDRA-6654 Thank you Phil
Re: Using Cassandra for session tokens
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The session will be written once at create time, and never modified after that. Will that affect things? Thank you - -Phil On 01.12.2014 15:58, Jonathan Haddad wrote: I don't think DateTiered will help here, since there's no clustering key defined. This is a pretty straightforward workload, I've done something similar. Are you overwriting the session on every request? Or just writing it once? On Mon Dec 01 2014 at 6:45:14 AM Matt Brown m...@mattnworb.com wrote: This sounds like a good use case for http://www.datastax.com/dev/blog/datetieredcompactionstrategy On Dec 1, 2014, at 3:07 AM, Phil Wise p...@advancedtelematic.com wrote: We're considering switching from using Redis to Cassandra to store short lived (~1 hour) session tokens, in order to reduce the number of data storage engines we have to manage. Can anyone foresee any problems with the following approach: 1) Use the TTL functionality in Cassandra to remove old tokens. 2) Store the tokens in a table like: CREATE TABLE tokens ( id uuid, username text, // (other session information) PRIMARY KEY (id) ); 3) Perform ~100 writes/sec like: INSERT INTO tokens (id, username ) VALUES (468e0d69-1ebe-4477-8565-00a4cb6fa9f2, 'bob') USING TTL 3600; 4) Perform ~1000 reads/sec like: SELECT * FROM tokens WHERE ID=468e0d69-1ebe-4477-8565-00a4cb6fa9f2 ; The tokens will be about 100 bytes each, and we will grant 100 per second on a small 3 node cluster. Therefore there will be about 360k tokens alive at any time, with a total size of 36MB before database overhead. My biggest worry at the moment is that this kind of workload will stress compaction in an unusual way. Are there any metrics I should keep an eye on to make sure it is working fine? I read over the following links, but they mostly talk about DELETE-ing and tombstones. Am I right in thinking that as soon as a node performs a compaction then the rows with an expired TTL will be thrown away, regardless of gc_grace_seconds? https://issues.apache.org/jira/browse/CASSANDRA-7534 http://www.datastax.com/dev/blog/cassandra-anti-patterns-queues-and-queue-like-datasets https://issues.apache.org/jira/browse/CASSANDRA-6654 Thank you Phil -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBAgAGBQJUfIR1AAoJEAvGtrO88FBAnpAP/0RCdwCy4Wi0ogz24SRKpCu0 c/i6O2HBTinl2RXLoH9xMOT8kXJ82P9tVDeKjLQAZYnBgRwF7Fcbvd40GPf+5aaj aU1TkU4jLnDCeFTwG/vx+TIfZEE27nppsECLtfmnzJEl/4yZwAG3Dy+VkuqBurMu J6If9bMnseEgvF1onmA7ZLygJq44tlgOGyHT0WdYRX7CwAE6HeyxMC38ArarRU37 dfGhsttBMqdxHreKE0CqRZZ67iT+KixGoUeCvZUnTvOLTsrEWO17yTezQDamAee0 jIsVfgKqqhoiKeAj99J75rcsIT3WAbS23MV1s92AQXYkpR1KmHTB6KvUjH2AQBew 9xwdDSg/eVsdQNkGbtSJ2cNPnFuBBZv2kzW5PVyQ625bMHNAF2GE9rLIKddMUbNQ LiwOPAJDWBJeZnJYj3cncdfC2Jw1H4rlV0k6BHwdzZUrEdbvUKlHtyl8/ZsZnJHs SrPsiYQa0NI6C+faAFqzBEyLhsWdJL3ygNZTo4CW3I8z+yYEyzZtmKPDmHdVzK/M M8GlaRYw1t7OY81VBXKcmPyr5Omti7wtkffC6bhopsPCm7ATSq2r46z8OFlkUdJl wcTMJM0E6gZtiMIr3D+WbOTzI5kPX6x4UB3ec3xq6+GIObPwioVAJf3ADmIK4iHT G106NwdUnag5XlnbwgMX =6zXb -END PGP SIGNATURE-
Re: Cassandra schema migrator
On 25.11.2014 10:22, Jens Rantil wrote: Anyone who is using, or could recommend, a tool for versioning schemas/migrating in Cassandra? I've recently written a tool to solve schema migration at our company which may be useful: https://github.com/advancedtelematic/cql-migrate My list of requirements is: * Support for adding tables. Yep, this works. * Support for versioning of table properties. All our tables are to be defaulted to LeveledCompactionStrategy. My parser/statement splitter doesn't currently recognise the WITH ... syntax on CREATE tables and ALTER TABLE, but it could easily be added. Do you need to modify the properties after the table has been created? If so the changes would need to be expressed as ALTER TABLE statements. * Support for adding non-existing columns. Supported. cql-migrate works by getting the user to express the upgrade as an ALTER TABLE ... ADD column statement. * Optional: Support for removing columns. * Optional: Support for removing tables. Not supported, although it wouldn't be hard to add. We are preferably a Java shop, but could potentially integrate something non-Java. I understand I could write a tool that would make these decisions using system.schema_columnfamilies and system.schema_columns, but as always reusing a proven tool would be preferable. It is implemented in python, and can be built into a Debian package for deployment. Rather than try and fully parse the CQL, cql-migrate works by splitting a .CQL script into individual statements, executing them in turn and ignoring errors that relate to tables etc already exiting. Doing it that way means there is very little chance that the schema created will differ from what running the same script through cqlsh would have produced. Let me know if you need a hand getting going with it. Cheers, Phil
