Re: How do you run integration tests for your cassandra code?
At SMX we use https://github.com/SMX-LTD/ccm-clj for our Cassandra integration tests. ccm-clj is a Clojure wrapper around CCM. -- Daniel On 14/10/2014, at 6:54 pm, Ranjib Dey dey.ran...@gmail.com wrote: you can use tools like chef along side vagrant to bring a cassandra. I personally prefer LXC containers, as they mimic full blown vms, along side chef-lxc which provides chef's awesome DSL for container customization (similar dockerfile, and you wont install chef inside the container), for our scenarios, i use 5 node clusters. and those whole spawn+ token set + rebalance is done as part of the test setup, this is assuming you want to run full blown integration test within a single host. if you can afford multiple hosts, you can set up either ephemeral hosts (chef-metal) or dedicated hosts (normal chef nodes but with a CI agent , like team city or go-cd etc). So, depending upon how big you want integration environment (from a single host developer environment) till a production clone (say for capacity planning load testing), you can automate cassandra cluster provisioning before you actually kick in your tests, which might be jmeter/gatling based testing to actual unit/functional/UAT style testing. I use chef, but im sure alternate exists in other frameworks hope this help ranjib On Mon, Oct 13, 2014 at 10:36 PM, Paco Trujillo f.truji...@genetwister.nl wrote: Hi Kevin We are using a similar solution than horschi. In the past we use CassandraUnit (https://github.com/jsevellec/cassandra-unit) but truncate the tables after and before each test works better for us. We also set gc_grace_seconds to zero. From: horschi [mailto:hors...@gmail.com] Sent: maandag 13 oktober 2014 22:17 To: user@cassandra.apache.org Subject: Re: How do you run integration tests for your cassandra code? Hi Kevin, I run my tests against my locally running Cassandra instance. I am not using any framework, but simply truncate all my tables after/before each test. With which I am quite happy. You have to enable the unsafeSystem property, disable durable writes on the CFs and disable auto-snapshot in the yaml for it to be fast. kind regards, Christian On Mon, Oct 13, 2014 at 9:50 PM, Kevin Burton bur...@spinn3r.com wrote: Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.com Location: San Francisco, CA blog: http://burtonator.wordpress.com … or check out my Google+ profile
How do you run integration tests for your cassandra code?
Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts http://spinn3r.com
Re: How do you run integration tests for your cassandra code?
Kevin, Have you looked at the Cassandra integration tests used by the Cassandra development team: https://github.com/riptano/cassandra-dtest ? They make use of CCM for integration testing: https://github.com/pcmanus/ccm Philip Thompson On Mon, Oct 13, 2014 at 2:50 PM, Kevin Burton bur...@spinn3r.com wrote: Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts http://spinn3r.com
Re: How do you run integration tests for your cassandra code?
Hi Kevin, I run my tests against my locally running Cassandra instance. I am not using any framework, but simply truncate all my tables after/before each test. With which I am quite happy. You have to enable the unsafeSystem property, disable durable writes on the CFs and disable auto-snapshot in the yaml for it to be fast. kind regards, Christian On Mon, Oct 13, 2014 at 9:50 PM, Kevin Burton bur...@spinn3r.com wrote: Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts http://spinn3r.com
Re: How do you run integration tests for your cassandra code?
FWIW we run a 3 node cluster with ccm on Travis to regression test the gocql driver - here's the descriptor: https://github.com/gocql/gocql/blob/master/.travis.yml On Mon, Oct 13, 2014 at 9:04 PM, Philip Thompson philip.thomp...@datastax.com wrote: Kevin, Have you looked at the Cassandra integration tests used by the Cassandra development team: https://github.com/riptano/cassandra-dtest ? They make use of CCM for integration testing: https://github.com/pcmanus/ccm Philip Thompson On Mon, Oct 13, 2014 at 2:50 PM, Kevin Burton bur...@spinn3r.com wrote: Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts http://spinn3r.com
RE: How do you run integration tests for your cassandra code?
Hi Kevin We are using a similar solution than horschi. In the past we use CassandraUnit (https://github.com/jsevellec/cassandra-unit) but truncate the tables after and before each test works better for us. We also set gc_grace_seconds to zero. From: horschi [mailto:hors...@gmail.com] Sent: maandag 13 oktober 2014 22:17 To: user@cassandra.apache.org Subject: Re: How do you run integration tests for your cassandra code? Hi Kevin, I run my tests against my locally running Cassandra instance. I am not using any framework, but simply truncate all my tables after/before each test. With which I am quite happy. You have to enable the unsafeSystem property, disable durable writes on the CFs and disable auto-snapshot in the yaml for it to be fast. kind regards, Christian On Mon, Oct 13, 2014 at 9:50 PM, Kevin Burton bur...@spinn3r.commailto:bur...@spinn3r.com wrote: Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.comhttp://Spinn3r.com Location: San Francisco, CA blog: http://burtonator.wordpress.com … or check out my Google+ profilehttps://plus.google.com/102718274791889610666/posts http://spinn3r.com http://spinn3r.com
Re: How do you run integration tests for your cassandra code?
you can use tools like chef along side vagrant to bring a cassandra. I personally prefer LXC containers, as they mimic full blown vms, along side chef-lxc which provides chef's awesome DSL for container customization (similar dockerfile, and you wont install chef inside the container), for our scenarios, i use 5 node clusters. and those whole spawn+ token set + rebalance is done as part of the test setup, this is assuming you want to run full blown integration test within a single host. if you can afford multiple hosts, you can set up either ephemeral hosts (chef-metal) or dedicated hosts (normal chef nodes but with a CI agent , like team city or go-cd etc). So, depending upon how big you want integration environment (from a single host developer environment) till a production clone (say for capacity planning load testing), you can automate cassandra cluster provisioning before you actually kick in your tests, which might be jmeter/gatling based testing to actual unit/functional/UAT style testing. I use chef, but im sure alternate exists in other frameworks hope this help ranjib On Mon, Oct 13, 2014 at 10:36 PM, Paco Trujillo f.truji...@genetwister.nl wrote: Hi Kevin We are using a similar solution than horschi. In the past we use CassandraUnit (https://github.com/jsevellec/cassandra-unit) but truncate the tables after and before each test works better for us. We also set gc_grace_seconds to zero. *From:* horschi [mailto:hors...@gmail.com] *Sent:* maandag 13 oktober 2014 22:17 *To:* user@cassandra.apache.org *Subject:* Re: How do you run integration tests for your cassandra code? Hi Kevin, I run my tests against my locally running Cassandra instance. I am not using any framework, but simply truncate all my tables after/before each test. With which I am quite happy. You have to enable the unsafeSystem property, disable durable writes on the CFs and disable auto-snapshot in the yaml for it to be fast. kind regards, Christian On Mon, Oct 13, 2014 at 9:50 PM, Kevin Burton bur...@spinn3r.com wrote: Curious to see if any of you have an elegant solution here. Right now I”m using cassandra unit; https://github.com/jsevellec/cassandra-unit for my integration tests. The biggest problem is that it doesn’t support shutdown. so I can’t stop or cleanup after cassandra between tests. I have other Java daemons that have the same problem. For example, ActiveMQ doesn’t clean up after itself. I was *thinking* of using docker or vagrant to startup a daemon in a container, then shut it down between tests. But this seems difficult to setup and configure … as well as being not amazingly portable. Another solution is to use a test suite, and a setUp/tearDown that drops all tables created by a test. This way you’re still on the same cassandra instance, but the tables are removed for each pass. Anyone have an elegant solution to this? -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile https://plus.google.com/102718274791889610666/posts http://spinn3r.com http://spinn3r.com
