Re: cross DC data sync starts without rebuilding nodes on new DC
Thanks Alain and Sean, your detailed explanation answers my question. Yes, nodetool status reflects new DC and nodetool netstats says not "No Streams". My all writes going to old DC with local_quorum. Yes this new data might be getting synced into new DC (repair was not running anywhere). I will proceed with rebuilding nodes on new DC. Thanks, Vijay On 5 April 2016 at 18:56, Alain RODRIGUEZ <arodr...@gmail.com> wrote: > Hi Vijay. > > After completing step 7.a (which is altering keyspace with desired >> replication factor for new DC) data automatically starts syncing from >> existing DC to new DC >> > > My guess: what you are seeing is not data syncing. Well it is, but not old > data being streamed but new writes being replicated. As soon as you set the > RF for the new DC, it starts accepting writes. > > Some background: > Using a Local_X consistency level means the operation to copy data to all > the DC won't happen, it means coordinator won't wait for ack from other DC > nodes, but write should reach all the DC set in the keyspace configuration. > So as soon as you say I want X copies of the data on the new Datacenter, > new data start to be replicated there. > > To check: > > Are you writing in your original DC? > Is the output of 'nodetool netstats' saying 'No streams' as I expect? > > When rebuilding run this command again and you should see streams. > > Any idea why it's happening? >> If this is the desired behaviour then what's the purpose of rebuilding >> each node on new DC (step 7.b)? >> > > So basically, the rebuild allows the new cluster to have the *old* / > *existing* data streamed from an other DC. We use rebuild instead of > auto_bootstrap to avoid nodes trying to stream data as soon as they are > added to the new DC because we want to add *all* the nodes, to have ranges > distributed evenly before starting streaming to stream just the correct > amount of data from the DC of our choice. > > C*heers, > --- > Alain Rodriguez - al...@thelastpickle.com > France > > The Last Pickle - Apache Cassandra Consulting > http://www.thelastpickle.com > > > 2016-04-05 8:26 GMT+02:00 Vijay Patil <vijay2110.t...@gmail.com>: > >> Hi, >> >> I have configured new DC as per instructions at below link. >> >> https://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_add_dc_to_cluster_t.html >> >> After completing step 7.a (which is altering keyspace with desired >> replication factor for new DC) data automatically starts syncing from >> existing DC to new DC, which is not expected because auto_bootstrap is >> false on all nodes (existing as well as new DC). >> >> Any idea why it's happening? >> If this is the desired behaviour then what's the purpose of rebuilding >> each node on new DC (step 7.b)? >> >> Cassandra version is 2.0.17 on all nodes in both DC's and I am using >> GossipingPropertyFileSnitch. >> >> Regards, >> Vijay >> > >
cross DC data sync starts without rebuilding nodes on new DC
Hi, I have configured new DC as per instructions at below link. https://docs.datastax.com/en/cassandra/2.0/cassandra/operations/ops_add_dc_to_cluster_t.html After completing step 7.a (which is altering keyspace with desired replication factor for new DC) data automatically starts syncing from existing DC to new DC, which is not expected because auto_bootstrap is false on all nodes (existing as well as new DC). Any idea why it's happening? If this is the desired behaviour then what's the purpose of rebuilding each node on new DC (step 7.b)? Cassandra version is 2.0.17 on all nodes in both DC's and I am using GossipingPropertyFileSnitch. Regards, Vijay
Bulk Load Hadoop to Cassandra
Hi, I intend to bulk load data from HDFS to Cassandra using a map-only program which uses the BulkOutputFormat class. Please advise me which versions of Cassandra and Hadoop would support such a use-case. I am using Hadoop 2.2.0 and Cassandra 2.0.6 and I am getting following error: Error: Found interface org.apache.hadoop.mapreduce.TaskAttemptContext, but class was expected Thanks, Vijay
Re: Cassandra Consistency problem with NTP
Just FYI, For one of the projects, i got around the NTP Drift problem by always reading more than i need, For example i want to read all the messages before x seconds then i would query cassandra for (x seconds + 500ms) then filter the duplicates in the client. Yes it does more network and yes client needs more logic to handle it. Regards, /VJ On Thu, Jan 17, 2013 at 10:47 AM, Edward Capriolo edlinuxg...@gmail.comwrote: If you have 40ms NTP drift something is VERY VERY wrong. You should have a local NTP server on the same subnet, do not try to use one on the moon. On Thu, Jan 17, 2013 at 4:42 AM, Sylvain Lebresne sylv...@datastax.comwrote: So what I want is, Cassandra provide some information for client, to indicate A is stored before B, e.g. global unique timestamp, or row order. The row order is determined by 1) the comparator you use for the column family and 2) the column names you, the client, choose for A and B. So what are the column names you use for A and B? Now what you could do is use a TimeUUID comparator for that column family and use a time uuid for A and B column names. In that case, provided A and B are sent from the same client node and B is sent after A on that client (which you said is the case), then any non buggy time uuid generator will guarantee that the uuid generated for A will be smaller than the one for B and thus that in Cassandra, A will be sorted before B. In any case, the point I want to make is that Cassandra itself cannot do anything for you problem, because by design the row ordering is something entirely controlled client side (and just so there is no misunderstanding, I want to make that point not because I'm not trying to suggest you were wrong asking this mailing list, but because we can't suggest a proper solution unless we clearly understand what the problem is). -- Sylvain 2013/1/17 Sylvain Lebresne sylv...@datastax.com I'm not sure I fully understand your problem. You seem to be talking of ordering the requests, in the order they are generated. But in that case, you will rely on the ordering of columns within whatever row you store request A and B in, and that order depends on the column names, which in turns is client provided and doesn't depend at all of the time synchronization of the cluster nodes. And since you are able to say that request A comes before B, I suppose this means said requests are generated from the same source. In which case you just need to make sure that the column names storing each request respect the correct ordering. The column timestamps Cassandra uses are here to which update *to the same column* is the more recent one. So it only comes into play if you requests A and B update the same column and you're interested in knowing which one of the update will win when you read. But even if that's your case (which doesn't sound like it at all from your description), the column timestamp is only generated server side if you use CQL. And even in that latter case, it's a convenience and you can force a timestamp client side if you really wish. In other words, Cassandra dependency on time synchronization is not a strong one even in that case. But again, that doesn't seem at all to be the problem you are trying to solve. -- Sylvain On Thu, Jan 17, 2013 at 2:56 AM, Jason Tang ares.t...@gmail.comwrote: Hi I am using Cassandra in a message bus solution, the major responsibility of cassandra is recording the incoming requests for later consumming. One strategy is First in First out (FIFO), so I need to get the stored request in reversed order. I use NTP to synchronize the system time for the nodes in the cluster. (4 nodes). But the local time of each node are still have some inaccuracy, around 40 ms. The consistency level is write all and read one, and replicate factor is 3. But here is the problem: A request come to node One at local time PM 10:00:01.000 B request come to node Two at local time PM 10:00:00.980 The correct order is A -- B But the timestamp is B -- A So is there any way for Cassandra to keep the correct order for read operation? (e.g. logical timestamp ?) Or Cassandra strong depence on time synchronization solution? BRs //Tang
Re: Flume-cassandra
FYI, Not sure about this particular problem, https://github.com/Vijay2win/flume-cassandra-sink supports latest flume version, if you want to try it out. Regards, /VJ On Sun, Jan 27, 2013 at 8:05 PM, Sri Ramya ramya.1...@gmail.com wrote: hi, I am storing flume events in to cassandra by using simpleCassandraSink. when i am storing a single event to cassandra from flume collector source in Cassandra i am getting nearly 100 rows , in which 1 row is containg my data and other rows are blank. So how do i solve this problem . Please provide me some help. Thank you
Re:
Hi Manu, Glad that you have the issue resolved. If i understand the issue correctly Your cassandra installation had RandomParitioner but the bulk loader configuration (cassandra.yaml) had Murmur3Partitioner? By fixing the cassandra.yaml for the bulk loader the issue got resolved? If not then we might have a bug and your feedback might help the community. Regards, /VJ On Wed, Sep 19, 2012 at 10:41 PM, Manu Zhang owenzhang1...@gmail.comwrote: the problem seems to have gone away with changing Murmur3Partitioner back to RandomPartitioner On Thu, Sep 20, 2012 at 11:14 AM, Manu Zhang owenzhang1...@gmail.comwrote: Yeah, BulkLoader. You did help me to elaborate my question. Thanks! On Thu, Sep 20, 2012 at 10:58 AM, Michael Kjellman mkjell...@barracuda.com wrote: I assumed you were talking about BulkLoader. I haven't played with trunk yet so I'm afraid I won't be much help here... On Sep 19, 2012, at 7:56 PM, Manu Zhang owenzhang1...@gmail.com mailto:owenzhang1...@gmail.com wrote: cassandra-trunk (so it's 1.2); no Hadoop, bulk load example here http://www.datastax.com/dev/blog/bulk-loading#comment-127019; buffer size is 64 MB as in the example; I'm dealing with about 1GB data. job config, you mean? On Thu, Sep 20, 2012 at 10:32 AM, Michael Kjellman mkjell...@barracuda.commailto:mkjell...@barracuda.com wrote: A few questions: what version of 1.1 are you running. What version of Hadoop? What is your job config? What is the buffer size you've chosen? How much data are you dealing with? On Sep 19, 2012, at 7:23 PM, Manu Zhang owenzhang1...@gmail.com mailto:owenzhang1...@gmail.com wrote: I've been bulk loading data into Cassandra and seen the following exception: ERROR 10:10:31,032 Exception in thread Thread[CompactionExecutor:5,1,main] java.lang.RuntimeException: Last written key DecoratedKey(-442063125946754, 313130303136373a31) = current key DecoratedKey(-465541023623745, 313036393331333a33) writing into /home/manuzhang/cassandra/data/tpch/lineitem/tpch-lineitem-tmp-ia-56-Data.db at org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:131) at org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:152) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:169) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:69) at org.apache.cassandra.db.compaction.CompactionManager$1.run(CompactionManager.java:152) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) The running Cassandra and that I load data into are the same one. What's the cause? 'Like' us on Facebook for exclusive content and other resources on all Barracuda Networks solutions. Visit http://barracudanetworks.com/facebook 'Like' us on Facebook for exclusive content and other resources on all Barracuda Networks solutions. Visit http://barracudanetworks.com/facebook
Re: Question on Read Repair
Yes, If you are using 1.1 take a look at: dclocal_read_repair_chance and read_repair_chance CF settings. Regards, /VJ On Sun, Sep 16, 2012 at 5:03 PM, Raj N raj.cassan...@gmail.com wrote: Hi, I have a 2 DC setup(DC1:3, DC2:3). All reads and writes are at LOCAL_QUORUM. The question is if I do reads at LOCAL_QUORUM in DC1, will read repair happen on the replicas in DC2? Thanks -Raj
Re: key_cache_size_in_mb
You might want to show us the GC log's... if there are promotion failures, count on the objects which you can get by inspecting the heapdumps etc, I am kind of assuming stuff :) Regards, /VJ On Thu, Jun 28, 2012 at 11:01 PM, Gurpreet Singh gurpreet.si...@gmail.comwrote: Vijay, Thanks for the suggestion. Row Key size is 8 bytes (its a long). Is the issue here incorrect calculation really? The server already had the key cache in memory when i stopped it, and it was using abt 6 gigs of heap total. How is it tht it cannot startup again with the same keycache? /G On Thu, Jun 28, 2012 at 4:25 PM, Vijay vijay2...@gmail.com wrote: It has 13.7million keys How big is your Row Keys? Current calculation (in 1.1) assumes that the average size of the EntryRowKey, Long in the keycache is 16 bytes. One thing which you can do to get out this issue is to reduce the Keycache to a low enough to get the server started or increase the Heap to hold all the objects. (If you are using the visualVM try to see the object counts). Regards, /VJ On Thu, Jun 28, 2012 at 3:40 PM, Gurpreet Singh gurpreet.si...@gmail.com wrote: Vijay, I am running into an issue right now, not sure if its related to this one. My cassandra 1.1 was running fine for a week, and the key cache size (as checked by jconsole) increased to almost 650 mb. It has 13.7million keys. I have set key_cache_size_in_mb to 800 mb. I checked the heap size, and it was at 6 gigs. All good! However, when i try to restart cassandra, it is unable to start now. Before i shut it down, it seemed fine at 6 gigs, which means its able to handle the memory requirements just fine. However it is not able to start up anymore with the same key cache. jconsole says that the heap is full. Thanks /G On Thu, Jun 28, 2012 at 12:22 PM, Vijay vijay2...@gmail.com wrote: in 1.1 we dont calculate the Key size accurately, hence we have the fix in https://issues.apache.org/jira/browse/CASSANDRA-4315 Regards, /VJ On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh gurpreet.si...@gmail.com wrote: anyone have an explanation for this? This kinda screws up memory calculations. /G On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh gurpreet.si...@gmail.com wrote: Hi, I have a question about cassandra 1.1 Just wanted to confirm if key_cache_size_in_mb is the maximum amount of memory that key cache will use in memory? If not, what is it? My observations: With key cache disabled, I started cassandra. I invoked Full GC through jconsole a couple of times just to make sure that memory is freed up. Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so this looked reasonable as the bloom filter itself is around 1 gig. I enabled key cache again by making key_cache_size_in_mb to a value 800. This would lead cassandra to load a previously saved key cache file of size 98 mb on disk. I restarted cassandra again. This time the heap size increased to 4.2 gigs. I checked the cache sizing in jconsole. KeyCacheCapacityInMb: 799 KeyCacheSize: 410,316,048 This says that the key cache size is 410 mb. However the increase in the heap is 2 gigs. Doesnt add up!! Can anyone explain this? Thanks Gurpreet
Re: key_cache_size_in_mb
in 1.1 we dont calculate the Key size accurately, hence we have the fix in https://issues.apache.org/jira/browse/CASSANDRA-4315 Regards, /VJ On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh gurpreet.si...@gmail.comwrote: anyone have an explanation for this? This kinda screws up memory calculations. /G On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh gurpreet.si...@gmail.comwrote: Hi, I have a question about cassandra 1.1 Just wanted to confirm if key_cache_size_in_mb is the maximum amount of memory that key cache will use in memory? If not, what is it? My observations: With key cache disabled, I started cassandra. I invoked Full GC through jconsole a couple of times just to make sure that memory is freed up. Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so this looked reasonable as the bloom filter itself is around 1 gig. I enabled key cache again by making key_cache_size_in_mb to a value 800. This would lead cassandra to load a previously saved key cache file of size 98 mb on disk. I restarted cassandra again. This time the heap size increased to 4.2 gigs. I checked the cache sizing in jconsole. KeyCacheCapacityInMb: 799 KeyCacheSize: 410,316,048 This says that the key cache size is 410 mb. However the increase in the heap is 2 gigs. Doesnt add up!! Can anyone explain this? Thanks Gurpreet
Re: key_cache_size_in_mb
It has 13.7million keys How big is your Row Keys? Current calculation (in 1.1) assumes that the average size of the EntryRowKey, Long in the keycache is 16 bytes. One thing which you can do to get out this issue is to reduce the Keycache to a low enough to get the server started or increase the Heap to hold all the objects. (If you are using the visualVM try to see the object counts). Regards, /VJ On Thu, Jun 28, 2012 at 3:40 PM, Gurpreet Singh gurpreet.si...@gmail.comwrote: Vijay, I am running into an issue right now, not sure if its related to this one. My cassandra 1.1 was running fine for a week, and the key cache size (as checked by jconsole) increased to almost 650 mb. It has 13.7million keys. I have set key_cache_size_in_mb to 800 mb. I checked the heap size, and it was at 6 gigs. All good! However, when i try to restart cassandra, it is unable to start now. Before i shut it down, it seemed fine at 6 gigs, which means its able to handle the memory requirements just fine. However it is not able to start up anymore with the same key cache. jconsole says that the heap is full. Thanks /G On Thu, Jun 28, 2012 at 12:22 PM, Vijay vijay2...@gmail.com wrote: in 1.1 we dont calculate the Key size accurately, hence we have the fix in https://issues.apache.org/jira/browse/CASSANDRA-4315 Regards, /VJ On Thu, Jun 28, 2012 at 11:25 AM, Gurpreet Singh gurpreet.si...@gmail.com wrote: anyone have an explanation for this? This kinda screws up memory calculations. /G On Mon, Jun 25, 2012 at 5:50 PM, Gurpreet Singh gurpreet.si...@gmail.com wrote: Hi, I have a question about cassandra 1.1 Just wanted to confirm if key_cache_size_in_mb is the maximum amount of memory that key cache will use in memory? If not, what is it? My observations: With key cache disabled, I started cassandra. I invoked Full GC through jconsole a couple of times just to make sure that memory is freed up. Total heap memory in use is 2.2 gigs. I have almost 550 million keys, so this looked reasonable as the bloom filter itself is around 1 gig. I enabled key cache again by making key_cache_size_in_mb to a value 800. This would lead cassandra to load a previously saved key cache file of size 98 mb on disk. I restarted cassandra again. This time the heap size increased to 4.2 gigs. I checked the cache sizing in jconsole. KeyCacheCapacityInMb: 799 KeyCacheSize: 410,316,048 This says that the key cache size is 410 mb. However the increase in the heap is 2 gigs. Doesnt add up!! Can anyone explain this? Thanks Gurpreet
Re: inconsistent snapshots
Can you attach the cassandra logs? How was it restored? Priam chunks and compresses before uploading it to S3 and when you restore you have to go thought Priam to restore... Regards, /VJ On Thu, May 24, 2012 at 3:58 AM, Jose Flexa jose.fl...@gmail.com wrote: Hi all, Having issues with cassandra 1.0.8 on AWS EC2 and Netflix Priam with a XFS FS: seems snapshots are broken. When I restore then and run nodetool repair cassandra cant load them complaning . Anybody had a similiar experience or any suggestions? Thanks in advance José
Re: Safely Disabling Compaction
I would rather set the Keyspace setting min_compaction_threshold and max_compaction_threshold to be a higher number and once i am ready i will put the value back... This way i dont need to restart. Having said that why not set the compaction throughput to 1 (low enough to not have contention) and complete the stream? Regards, /VJ On Wed, May 16, 2012 at 2:43 PM, sj.climber sj.clim...@gmail.com wrote: Hi, In an effort to minimize IO contention, I'd like to disable compactions while I'm streaming SSTables to the cluster. When done streaming, I intend on forcing a major compaction through nodetool. Elsewhere in the forums, various folks suggest setting max_compaction_threshold = 0 to disable compaction. While this works sometimes (via 'update column family family with max_compaction_threshold=0'), I've observed a number of serious issues with this approach: 1) You can't create a column family with max_compaction_threshold = 0. The CLI reports that min_compaction_threshold must have a value = 2, and max_compaction_threshold can't be lower than it. Worse yet, trying to create a column family with max_compaction_threshold = 0 gets the cluster into a Schema Disagreement Exception (since the node on which you issue the migration command fails with a fatal error). 2) Cassandra will allow me to update an existing column family with max_compaction_threshold = 0. But if I restart the node, it will crash on startup. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160) Caused by: java.lang.RuntimeException: java.lang.reflect.InvocationTargetException ... org.apache.cassandra.config.CFMetaData.createCompactionStrategyInstance(CFMetaData.java:839) ... 14 more Caused by: java.lang.RuntimeException: The max_compaction_threshold cannot be smaller than the min. at org.apache.cassandra.db.ColumnFamilyStore.setMaximumCompactionThreshold(ColumnFamilyStore.java:1740) at org.apache. Is there another solution for more safely enabling/disabling compaction? Thanks! -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Safely-Disabling-Compaction-tp7562777.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: Cassandra backup question regarding commitlogs
The incremental backups are generated when the flush is complete (Only during the flush), If the node crash before the flush completes then the commit logs in the local node's backup for the data in memory. It wouldn't help to copy the Commit log across because they are not immutable (They are recycled). There is commit log backup in 1.1.1 (Yet to be released) https://issues.apache.org/jira/browse/CASSANDRA-3690 Thanks, /VJ On Sun, Apr 29, 2012 at 3:29 PM, Roshan codeva...@gmail.com wrote: Hi Currently I am taking daily snapshot on my keyspace in production and already enable the incremental backups as well. According to the documentation, the incremental backup option will create an hard-link to the backup folder when new sstable is flushed. Snapshot will copy all the data/index/etc. files to a new folder. Question: What will happen (with enabling the incremental backup) when crash (due to any reason) the Cassandra before flushing the data as a SSTable (inserted data still in commitlog). In this case how can I backup/restore data? Do I need to backup the commitlogs as well and and replay during the server start to restore the data in commitlog files? Thanks. -- View this message in context: http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-question-regarding-commitlogs-tp7511918.html Sent from the cassandra-u...@incubator.apache.org mailing list archive at Nabble.com.
Re: getting status of long running repair
Are you by using Broadcast Address? if yes then you might be affected by https://issues.apache.org/jira/browse/CASSANDRA-3503 Nodes are all up while repairing is running. I should have been clear are you seeing the following messages in logs (UP/DOWN) during the period of the repair... INFO [GossipStage:1] 2012-05-01 19:52:00,515 Gossiper.java (line 804) InetAddress /xx.xx.xx.xx is now UP Regards, /VJ On Wed, May 9, 2012 at 5:49 AM, Bill Au bill.w...@gmail.com wrote: I am running 1.0.8. Two data center with 8 machines in each dc. Nodes are all up while repairing is running. No dropped Mutations/Messages. I do see HintedHandoff messages. Bill On Tue, May 8, 2012 at 11:15 PM, Vijay vijay2...@gmail.com wrote: What is the version you are using? is it Multi DC setup? Are you seeing a lot of dropped Mutations/Messages? Are the nodes going up and down all the time while the repair is running? Regards, /VJ On Tue, May 8, 2012 at 2:05 PM, Bill Au bill.w...@gmail.com wrote: There are no error message in my log. I ended up restarting all the nodes in my cluster. After that I was able to run repair successfully on one of the node. It took about 40 minutes. Feeling lucky I ran repair on another node and it is stuck again. tpstats show 1 active and 1 pending AntiEntropySessions. netstats and compactionstats show no activity. I took a close look at the log file, it shows that the node requested merkle tree from 4 nodes (including itself). It actually received 3 of those merkle trees. It looks like it is stuck waiting for that last one. I checked the node where the request was sent to, there isn't anything in the log on repair. So it looks like the merkle tree request has gotten lost some how. It has been 8 hours since the repair was issue and it is still stuck. I am going to let it run a bit longer to see if it will eventually finish. I have observed that if I restart all the nodes, I would be able to run repair successfully on a single node. I have done that twice already. But after that all repairs will hang. Since we are supposed to run repair periodically, having to restart all nodes before running repair on each node isn't really viable for us. Bill On Tue, May 8, 2012 at 6:04 AM, aaron morton aa...@thelastpickle.comwrote: When you look in the logs please let me know if you see this error… https://issues.apache.org/jira/browse/CASSANDRA-4223 I look at nodetool compactionstats (for the Merkle tree phase), nodetool netstats for the streaming, and this to check for streaming progress: while true; do date; diff (nodetool -h localhost netstats) (sleep 5 nodetool -h localhost netstats); done Or use Data Stax Ops Centre where possible http://www.datastax.com/products/opscenter Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 8/05/2012, at 2:15 PM, Ben Coverston wrote: Check the log files for warnings or errors. They may indicate why your repair failed. On Mon, May 7, 2012 at 10:09 AM, Bill Au bill.w...@gmail.com wrote: I restarted the nodes and then restarted the repair. It is still hanging like before. Do I keep repeating until the repair actually finish? Bill On Fri, May 4, 2012 at 2:18 PM, Rob Coli rc...@palominodb.com wrote: On Fri, May 4, 2012 at 10:30 AM, Bill Au bill.w...@gmail.com wrote: I know repair may take a long time to run. I am running repair on a node with about 15 GB of data and it is taking more than 24 hours. Is that normal? Is there any way to get status of the repair? tpstats does show 2 active and 2 pending AntiEntropySessions. But netstats and compactionstats show no activity. As indicated by various recent threads to this effect, many versions of cassandra (including current 1.0.x release) contain bugs which sometimes prevent repair from completing. The other threads suggest that some of these bugs result in the state you are in now, where you do not see anything that looks like appropriate activity. Unfortunately the only solution offered on these other threads is the one I will now offer, which is to restart the participating nodes and re-start the repair. I am unaware of any JIRA tickets tracking these bugs (which doesn't mean they don't exist, of course) so you might want to file one. :) =Rob -- =Robert Coli AIMGTALK - rc...@palominodb.com YAHOO - rcoli.palominob SKYPE - rcoli_palominodb -- Ben Coverston DataStax -- The Apache Cassandra Company
Re: Cassandra - crash with “free() invalid pointer”
Can you send us the stack trace which you can find in the hs_err_pid*.log? is the system memory all used up (free)? any errors in the logs just before the crash? Regards, /VJ On Mon, Mar 26, 2012 at 12:35 AM, Maciej Miklas mac.mik...@googlemail.comwrote: I have row cache - it's about 20GB big in this case. The problem can be reproduced with our load test - we are using 20 reader threads on single Cassandra node. I will retest it with Java 6 - still it looks to me like JNA problem and JDK in this case should not matter, but we will see. On Thu, Mar 22, 2012 at 8:27 PM, Benoit Perroud ben...@noisette.chwrote: Sounds like a race condition in the off heap caching while calling Unsafe.free(). Do you use cache ? What is your use case when you encounter this error ? Are you able to reproduce it ? 2012/3/22 Maciej Miklas mac.mik...@googlemail.com: Hi *, My Cassandra installation runs on flowing system: Linux with Kernel 2.6.32.22 jna-3.3.0 Java 1.7.0-b147 Sometimes we are getting following error: *** glibc detected *** /var/opt/java1.7/bin/java: free(): invalid pointer: 0x7f66088a6000 *** === Backtrace: = /lib/libc.so.6[0x7f661d7099a8] /lib/libc.so.6(cfree+0x76)[0x7f661d70bab6] /lib64/ld-linux-x86-64.so.2(_dl_deallocate_tls+0x59)[0x7f661e02f349] /lib/libpthread.so.0[0x7f661de09237] /lib/libpthread.so.0[0x7f661de0931a] /lib/libpthread.so.0[0x7f661de0a0bd] /lib/libc.so.6(clone+0x6d)[0x7f661d76564d] === Memory map: 0040-00401000 r-xp 68:07 537448203 /var/opt/jdk1.7.0/bin/java 0060-00601000 rw-p 68:07 537448203 /var/opt/jdk1.7.0/bin/java 01bae000-01fd rw-p 00:00 0 [heap] 01fd-15798000 rw-p 00:00 0 [heap] 40002000-40005000 ---p 00:00 0 40005000-40023000 rw-p 00:00 0 4003-40033000 ---p 00:00 0 40033000-40051000 rw-p 00:00 0 Does anyone have similar problems? or maybe some hints? Thanks, Maciej -- sent from my Nokia 3210
Re: Multic DC on EC2 with no VPC
Can you check if the ports are open and you are able to communicate? UNKNOWN-DC is printed when it is not gossiping with the other node. I would check the security group settings and verify if the telnet to the targe port is working as expected. Regards, /VJ On Thu, Mar 8, 2012 at 11:41 AM, Todd Nine todd.n...@gmail.com wrote: Hi all, I've recently upgraded a test cluster from 0.8.x to 1.0.8 for testing multi data center communications. I have the following configuration file on 3 nodes in a single data center. https://gist.github.com/4671e4ae562a47f96ed2 However, when I run node tool on any of these nodes, they recognize the others are up, but no data is available. nodetool -h localhost ring Address DC RackStatus State Load OwnsToken 113427455640312821154458202477256070485 10.172.106.192 UNKNOWN-DC UNKNOWN-RACKDown Normal ? 33.33% 0 50.18.3.222 us-west 1c Up Normal 3.95 GB 33.33% 56713727820156410577229101238628035242 10.172.134.239 UNKNOWN-DC UNKNOWN-RACKDown Normal ? 33.33% 113427455640312821154458202477256070485 If I remove the broadcast_address from the yaml, everything works fine, but I can't communicate across data centers. Any ideas why I'm getting this error? The listen_address is the private ip, and the broadcast_address is the public ip. Thanks, Todd
Astyanax: A New Cassandra Client.
http://techblog.netflix.com/2012/01/announcing-astyanax.html * * *What is Astyanax?* Astyanax is a Java Cassandra client. It borrows many concepts from Hector but diverges in the connection pool implementation as well as the client API. One of the main design considerations was to provide a clean abstraction between the connection pool and Cassandra API so that each may be customized and improved separately. Astyanax provides a fluent style API which guides the caller to narrow the query from key to column as well as providing queries for more complex use cases that we have encountered. The operational benefits of Astyanax over Hector include lower latency, reduced latency variance, and better error handling. PS: Author CC'ed Regards, /VJ
Re: Astyanax: A New Cassandra Client.
Fixing the CC list Regards, /VJ On Tue, Jan 31, 2012 at 5:40 PM, Vijay vijay2...@gmail.com wrote: http://techblog.netflix.com/2012/01/announcing-astyanax.html * * *What is Astyanax?* Astyanax is a Java Cassandra client. It borrows many concepts from Hector but diverges in the connection pool implementation as well as the client API. One of the main design considerations was to provide a clean abstraction between the connection pool and Cassandra API so that each may be customized and improved separately. Astyanax provides a fluent style API which guides the caller to narrow the query from key to column as well as providing queries for more complex use cases that we have encountered. The operational benefits of Astyanax over Hector include lower latency, reduced latency variance, and better error handling. PS: Author CC'ed Regards, /VJ
Re: MUTATION messages dropped
+ I would also check the GC settings :) and full gc events in the logs. Regards, /VJ On Wed, Jan 25, 2012 at 9:52 AM, aaron morton aa...@thelastpickle.comwrote: Am I missing data here? Yes, but you can repair it with nodetool. Is this means that my cluster is too loaded? yes. http://spyced.blogspot.com/2010/01/linux-performance-basics.html) Use nodetool tpstats to see when tasks are backing up, check io throughput with iostat (see Cheers - Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 26/01/2012, at 1:13 AM, Michael Vaknine wrote: Hi, I have a 1.0.3 cluster with 4 nodes and replication factor 3. I have lots of messages like the following in my logs: INFO [ScheduledTasks:1] 2012-01-25 07:07:18,410 MessagingService.java (line 613) 510 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:07:23,420 MessagingService.java (line 613) 129 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:07:28,429 MessagingService.java (line 613) 2 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:07:33,788 MessagingService.java (line 613) 1068 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:07:33,271 MessagingService.java (line 613) 391 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 12:07:52,981 MessagingService.java (line 613) 312 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:07:53,823 MessagingService.java (line 613) 1004 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:07:53,304 MessagingService.java (line 613) 443 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:08:08,854 MessagingService.java (line 613) 515 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:08:13,863 MessagingService.java (line 613) 225 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 12:08:28,047 MessagingService.java (line 613) 790 MUTATION messages dropped in last 5000ms INFO [ScheduledTasks:1] 2012-01-25 07:08:28,374 MessagingService.java (line 613) 833 MUTATION messages dropped in last 5000ms ** ** Am I missing data here? Is this means that my cluster is too loaded? ** ** Thanks Michael
Re: Encryption related question
I had the following writeup when i did the KS and TS creation... Hope this helps *Step 1:* Download your Organisation Cert/Cert Chain/Generate one. *Step 2:* Login to any of one machine do the following to create p12 # openssl pkcs12 -export -in cassandra-app.cert -inkey cassandra-app.key -certfile cassandra-app.cert -name cassandra-app -out cassandra-app.p12 *Step 3:* now you can create the Keystore # keytool -importkeystore -srckeystore cassandra-app.p12 -srcstoretype pkcs12 -destkeystore cassandra-app.jks -deststoretype JKS - You might need the password at this stage. *Step 4:* List to make sure you have the right one. # keytool -list -v -keystore cassandra-app.jks -storepass Password * * *TrustStore:* *Step 1:* Download the certificate chain from perforce. Do all the steps as above and you have a trust store (Name it sensibly to differentiate in the future) keytool -import -keystore cassandra-app.truststore -file ca.pem -alias cassandra-app -storepass diffrent pass *Finally:* Checkin the files into conf dir in Perforce. *Open Yaml File:* And Add: encryption_options: internode_encryption: *dc* keystore: conf/.keystore keystore_password: cassandra truststore: conf/.truststore truststore_password: cassandra Regards, /VJ On Fri, Jan 20, 2012 at 11:16 AM, A J s5a...@gmail.com wrote: Hello, I am trying to use internode encryption in Cassandra (1.0.6) for the first time. 1. Followed the steps 1 to 5 at http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#CreateKeystore Q. In cassandra.yaml , what value goes for keystore ? I exported the certificate per step #3 above in duke.cer. Do I put the location and name of that file for this parameter ? Siminarly, what value goes for truststore ? The steps 1-5 don't indicate any other file to be exported that would possibly go here. Also do I need to follow these steps on each of the node ? Thanks AJ
Re: Installing C* on EC2
http://www.datastax.com/docs/0.8/install/install_ami Image is the best way to go find a public AMI for 1.0 Are there any known issues when running C* on EC2? - Not that i know of but we prefer Ephimeral drives rather than EBS when using local storage on EC2 since local storage is gone at that point? - Tradeoff is in the replication factor you can also take regular snapshot and copy to some other location. You might want to place replicas in diffèrent ASG's to be safe. Launch new instance and run repair? - Nope you dont need it when you are bootstrapping the new nodes. Regards, /VJ On Thu, Jan 12, 2012 at 4:36 PM, Mohit Anchlia mohitanch...@gmail.comwrote: What's the best way to install C*? Any good links? Is it better to just create instances and install rpms on it first, just like regular cluster and then create image from it? I am assuming it's possible. Are there any known issues when running C* on EC2? How do other C* users deal with instance failures when using local storage on EC2 since local storage is gone at that point? Launch new instance and run repair?
Re: EC2 Maintenance Reboot
In addition to the below i would also think of rebooting one at a time, just to be safe... folks in my corp are seeing the reboot fail because of EBS or S3 dependencies while booting the image. Regards, /VJ On Thu, Dec 8, 2011 at 3:04 AM, Peter Sanford psanf...@nearbuysystems.comwrote: Just rebooting a machine with ephemeral drives is ok (it does an os level reboot). You will also keep the same IP address. If you stop and then start a machine with ephemeral drives you will lose data. See: http://alestic.com/2011/09/ec2-reboot-stop-start On Wed, Dec 7, 2011 at 6:43 PM, Stephen McKamey step...@mckamey.com wrote: I just received an email from AWS about rebooting my EC2 instance for maintenance. Thankfully this is just my webhead, but I'm curious have others experienced this with a Cassandra instance? http://aws.amazon.com/maintenance-help/ The details seem to say that if you allow them to do the reboot for you then your instance data will remain intact (as opposed to if you do it yourself). Has anyone experienced this to verify? Thanks, Stephen
Re: Single node
You can add a DNS entry with multiple IP's or something like a elastic ip which will keep switching between the active machines. or you can also write your custom seed provider class. Not sure if you will get a quorum when there dev's are on vacation :) Regards, /VJ On Thu, Dec 8, 2011 at 11:05 AM, Stephen Pope stephen.p...@quest.comwrote: Is there a way to set up a single node cluster without specifying anything about the specific machine in cassandra.yaml? I’ve cleared the values from listen_address and rpc_address, but it complains upon startup that no other nodes can be seen (presumably because the ip in the seeds doesn’t match). ** ** The reason I’m trying to do this is because we deploy cassandra on each developer’s machine, and we’d like to be able to use our client across machines using the hostname. Ideally, none of the developers would have to change the base config that gets deployed. ** ** The default config file works as a single node cluster, but won’t let you talk to it across machines (we’re using windows, in case it’s relevant).** ** ** ** Cheers, Steve
Re: Second Cassandra users survey
My wish list: 1) Conditional updates: if a column has a value then put column in the column family atomically else fail. 2) getAndSet: on counters: a separate API 3) Revert the count when client disconnects or receives a exception (so they can safely retry). 4) Something like a freeze API for updates to a row/CF (this can be used as a lock). Regards, /VJ On Tue, Nov 1, 2011 at 3:59 PM, Jonathan Ellis jbel...@gmail.com wrote: Hi all, Two years ago I asked for Cassandra use cases and feature requests. [1] The results [2] have been extremely useful in setting and prioritizing goals for Cassandra development. But with the release of 1.0 we've accomplished basically everything from our original wish list. [3] I'd love to hear from modern Cassandra users again, especially if you're usually a quiet lurker. What does Cassandra do well? What are your pain points? What's your feature wish list? As before, if you're in stealth mode or don't want to say anything in public, feel free to reply to me privately and I will keep it off the record. [1] http://www.mail-archive.com/cassandra-dev@incubator.apache.org/msg01148.html [2] http://www.mail-archive.com/cassandra-user@incubator.apache.org/msg01446.html [3] http://www.mail-archive.com/dev@cassandra.apache.org/msg01524.html -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: Client traffic encryption best practices....
https://issues.apache.org/jira/browse/THRIFT-106 seems to be the right way to go but the cassandra server needs to support too which we might want to add Regards, /VJ On Thu, Aug 11, 2011 at 2:54 PM, Chris Marino ch...@vcider.com wrote: Hello, is there any consensus on how to secure client/cluster communications??? I'm running an 8 node cluster across EC2 regions. I'm running inter-node encryption and I want to encrypt the traffic from the clients as well. My options seem to be: Have the client connect to only one node and encrypt that one connection with OpenVPN/stunnel (or something similar). Or, set up an encrypted tunnel from the client to each node. Is there a client library that could take care of this for me?? Setting up tunnels to each node is a major pain, but pointing the client to only one node is going to kill my performance. I'm running 4 nodes in each EC2 region with one client in each. Maybe I could connect the client only to the local nodes, which should simplify things a bit, but I was wondering if anyone had any experience with this or could suggest something that might be better. Please let me know. Thanks. CM
Re: Running across multiple EC2 regions...
So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. If you use 2452, communication within a region is via private ip and communication between the regions are public (Handshaking or intial communication will still be via public ip). In EC2 they dont have 2 interface but they nat the public IP even then this patch will do the right thing for you. There is comments in the patch + * 1) Snitch will automatically set the public IP by querying the AWS API + * + * 2) Snitch will set the private IP as a Gossip application state. + * + * 3) Snitch implements IESCS and will reset the connection if it is within the + * same region to communicate via private IP. + * + * Implements Ec2Snitch to inherit its functionality and extend it for + * Multi-Region. + * + * Operational: All the nodes in this cluster needs to be able to (modify the + * Security group settings in AWS) communicate via Public IP's. Regards, /VJ On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino ch...@vsider.com wrote: Hello, Me and some of my colleagues are about to start some experiments running Cassandra across EC2 regions using virtual networks and have some questions about how this is going to work. I've read these threads about patching the .yaml file to bind to the Listen address to the public IP... http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=g...@mail.gmail.com%3E com%3Ehttp://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=g...@mail.gmail.com%3E http://mail-archives.apache.org/mod_mbox/cassandra-user/201103.mbox/%3CAANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hg33...@mail.gmail.com%3E And this EC2 Snitch patch that lets it work across regions... https://issues.apache.org/jira/browse/CASSANDRA-2452 I'm pretty familiar with EC2 networking, but not very familiar how Cassandra will use the RPC and Listen ports with this Snitch. So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. What I'm trying to do is set up a virtual network that lets all the nodes use private IPs, but can still communicate across regions. We're going to try this with a virtual network as well as with the EC2 Snitch to see how things compare. Being able to use the EC2 private interface is going to make big difference. Thanks in advance. CM
Re: Running across multiple EC2 regions...
Chris, It is internal to cassandra.(IEndpointStateChangeSubscriber)... the snitch basically starts gossiping via public ip and once it sees the Location of the node to be local (it gets notified via IESCS) and then switches to private IP (safely - flush and reconnect). hope this helps... I didnt intend to make it complicated i thought the operations part is what you have to be worried about... (Change the Security group in both the regions to allow the Public IP's) Regards, /VJ On Fri, Jul 29, 2011 at 4:37 PM, Chris Marino ch...@vcider.com wrote: Thanks Vijay, that helps a lot. FYI, I did read the comments but didn't understand what I was reading since I don't know what IECSC is?? Googled it but still came up empty. What is this?? Sorry, if this is obvious, but I'm pretty new to all this Thanks CM On Fri, Jul 29, 2011 at 3:18 PM, Vijay vijay2...@gmail.com wrote: So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. If you use 2452, communication within a region is via private ip and communication between the regions are public (Handshaking or intial communication will still be via public ip). In EC2 they dont have 2 interface but they nat the public IP even then this patch will do the right thing for you. There is comments in the patch + * 1) Snitch will automatically set the public IP by querying the AWS API + * + * 2) Snitch will set the private IP as a Gossip application state. + * + * 3) Snitch implements IESCS and will reset the connection if it is within the + * same region to communicate via private IP. + * + * Implements Ec2Snitch to inherit its functionality and extend it for + * Multi-Region. + * + * Operational: All the nodes in this cluster needs to be able to (modify the + * Security group settings in AWS) communicate via Public IP's. Regards, /VJ On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino ch...@vsider.com wrote: Hello, Me and some of my colleagues are about to start some experiments running Cassandra across EC2 regions using virtual networks and have some questions about how this is going to work. I've read these threads about patching the .yaml file to bind to the Listen address to the public IP... http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=gQQ@mail.gmail.http://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=g...@mail.gmail.com%3E com%3Ehttp://mail-archives.apache.org/mod_mbox/cassandra-user/201104.mbox/%3CBANLkTikWGOtWkBOBAs+ibq5voSmjLm=g...@mail.gmail.com%3E http://mail-archives.apache.org/mod_mbox/cassandra-user/201103.mbox/%3CAANLkTikWsuUOjEU228niWi0iDTAO5J=5wO=i=hg33...@mail.gmail.com%3E And this EC2 Snitch patch that lets it work across regions... https://issues.apache.org/jira/browse/CASSANDRA-2452 I'm pretty familiar with EC2 networking, but not very familiar how Cassandra will use the RPC and Listen ports with this Snitch. So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. What I'm trying to do is set up a virtual network that lets all the nodes use private IPs, but can still communicate across regions. We're going to try this with a virtual network as well as with the EC2 Snitch to see how things compare. Being able to use the EC2 private interface is going to make big difference. Thanks in advance. CM
Re: Off-heap Cache
Use describe keyspace and see the settings are right. Check the logs on all the servers, Make sure you dont see errors... Check JNA jar in all the servers. Regards, /VJ On Wed, Jul 13, 2011 at 1:29 PM, Raj N raj.cassan...@gmail.com wrote: How do I ensure it is indeed using the SerializingCacheProvider. Thanks -Rajesh On Tue, Jul 12, 2011 at 1:46 PM, Jonathan Ellis jbel...@gmail.com wrote: You need to set row_cache_provider=SerializingCacheProvider on the columnfamily definition (via the cli) On Tue, Jul 12, 2011 at 9:57 AM, Raj N raj.cassan...@gmail.com wrote: Do we need to do anything special to turn off-heap cache on? https://issues.apache.org/jira/browse/CASSANDRA-1969 -Raj -- Jonathan Ellis Project Chair, Apache Cassandra co-founder of DataStax, the source for professional Cassandra support http://www.datastax.com
Re: gossiper problem
How about GC logs, what are your pause times? JVM settings might help If you are not sure how to enable GC logs check cassandra.yaml look for application pause times. it is highly recommended not to swap -- include JNA jar. Regards, /VJ On Thu, Jul 14, 2011 at 1:42 AM, Donna Li donna...@utstar.com wrote: All: I have four cassandra servers in cluster. I do not restart any one of the servers, why the following print show the four servers restart many times? What is the possible reason? The connection between the four server’s is good. Swap may be used, because there are other applications run with cassandra server. ** ** 10.63.61.71 log INFO [Timer-0] 2011-07-13 10:44:55,732 Gossiper.java (line 181) InetAddress /10.63.61.74 is now dead. INFO [GMFD:1] 2011-07-13 10:44:57,748 Gossiper.java (line 579) InetAddress /10.63.61.74 is now UP INFO [Timer-0] 2011-07-13 15:56:44,630 Gossiper.java (line 181) InetAddress /10.63.61.74 is now dead. INFO [GMFD:1] 2011-07-13 15:56:44,653 Gossiper.java (line 579) InetAddress /10.63.61.74 is now UP INFO [Timer-0] 2011-07-13 16:03:24,391 Gossiper.java (line 181) InetAddress /10.63.61.72 is now dead. INFO [GMFD:1] 2011-07-13 16:03:24,405 Gossiper.java (line 579) InetAddress /10.63.61.72 is now UP INFO [Timer-0] 2011-07-13 22:21:41,246 Gossiper.java (line 181) InetAddress /10.63.61.74 is now dead. INFO [Timer-0] 2011-07-13 22:22:45,602 Gossiper.java (line 181) InetAddress /10.63.61.73 is now dead. INFO [Timer-0] 2011-07-13 22:22:45,602 Gossiper.java (line 181) InetAddress /10.63.61.72 is now dead. INFO [GMFD:1] 2011-07-13 22:22:45,993 Gossiper.java (line 579) InetAddress /10.63.61.73 is now UP INFO [GMFD:1] 2011-07-13 22:22:46,107 Gossiper.java (line 579) InetAddress /10.63.61.72 is now UP INFO [GMFD:1] 2011-07-13 22:22:46,107 Gossiper.java (line 579) InetAddress /10.63.61.74 is now UP INFO [Timer-0] 2011-07-13 22:24:08,812 Gossiper.java (line 181) InetAddress /10.63.61.74 is now dead. INFO [GMFD:1] 2011-07-13 22:24:08,920 Gossiper.java (line 579) InetAddress /10.63.61.74 is now UP ** ** 10.63.61.72 log INFO [Timer-0] 2011-07-13 02:06:03,941 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 02:06:05,109 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 03:39:41,918 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 03:39:45,536 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 10:10:17,449 Gossiper.java (line 181) InetAddress /10.63.61.74 is now dead. INFO [Timer-0] 2011-07-13 10:10:17,471 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 10:10:18,451 Gossiper.java (line 579) InetAddress /10.63.61.74 is now UP INFO [GMFD:1] 2011-07-13 10:10:18,451 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 10:44:36,140 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 10:44:57,417 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 10:45:10,141 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 10:45:14,478 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 15:14:44,044 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 15:14:47,610 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 15:56:36,857 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 15:56:44,417 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 16:02:37,260 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 16:02:52,651 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 16:03:05,289 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 16:03:11,260 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 16:08:47,666 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 16:08:48,668 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 17:38:32,569 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 17:38:34,572 Gossiper.java (line 579) InetAddress /10.63.61.71 is now UP INFO [Timer-0] 2011-07-13 22:20:45,706 Gossiper.java (line 181) InetAddress /10.63.61.71 is now dead. INFO [GMFD:1] 2011-07-13 22:22:46,143 Gossiper.java (line 579)
Re: Compressing data types
You might want to watch https://issues.apache.org/jira/browse/CASSANDRA-47 Regards, /VJ On Tue, Jun 21, 2011 at 5:14 AM, Timo Nentwig timo.nent...@toptarif.dewrote: Hi! Just wondering why this doesn't already exist: wouldn't it make sense to have decorating data types that compress (gzip, snappy) other data types (esp. UTF8Type, AsciiType) transparently? -tcn
Re: Docs: Token Selection
The problem in the above approach is you have 2 nodes between 12 to 4 in DC1 but from 4 to 12 you just have 1 (Which will cause uneven distribution of data the node) It is easier to think of the DCs as ring and split equally and interleave them together DC1 Node 1 : token 0 DC1 Node 2 : token 8.. DC2 Node 1 : token 4.. DC2 Node 1 : token 12.. Regards, /VJ On Tue, Jun 14, 2011 at 7:31 PM, AJ a...@dude.podzone.net wrote: Yes, which means that the ranges overlap each other. Is this just a convention, or is it technically required when using NetworkTopologyStrategy? Would it be acceptable to split the ranges into quarters by ignoring the data centers, such as: DC1 node 1 = 0 Range: (12, 16], (0, 0] node 2 = 4 Range: (0, 4] DC2 node 3 = 8 Range: (4, 8] node 4 = 12 Range: (8, 12] If this is OK, are there any drawbacks to this? On 6/14/2011 6:10 PM, Vijay wrote: Yes... Thats right... If you are trying to say the below... DC1 Node1 Owns 50% (Ranges 8..4 - 8..5 8..5 - 0) Node2 Owns 50% (Ranges 0 - 1 1 - 8..4) DC2 Node1 Owns 50% (Ranges 8..5 - 0 0 - 1) Node2 Owns 50% (Ranges 1 - 8..4 8..4 - 8..5) Regards, /VJ On Tue, Jun 14, 2011 at 3:47 PM, AJ a...@dude.podzone.net wrote: This http://wiki.apache.org/cassandra/Operations#Token_selection says: With NetworkTopologyStrategy, you should calculate the tokens the nodes in each DC independantly. and gives the example: DC1 node 1 = 0 node 2 = 85070591730234615865843651857942052864 DC2 node 3 = 1 node 4 = 85070591730234615865843651857942052865 So, according to the above, the token ranges would be (abbreviated nums): DC1 node 1 = 0 Range: (8..4, 16], (0, 0] node 2 = 8..4 Range: (0, 8..4] DC2 node 3 = 1 Range: (8..5, 16], (0, 1] node 4 = 8..5 Range: (1, 8..5] If the above is correct, then I would be surprised as this paragraph is the only place were one would discover this and may be easy to miss... unless there's a doc buried somewhere in plain view that I missed. So, have I interpreted this paragraph correctly? Was this design to help keep data somewhat localized if that was important, such as a geographically dispersed DC? Thanks!
Re: Docs: Token Selection
Correction The problem in the above approach is you have 2 nodes between 12 to 4 in DC1 but from 4 to 12 you just have 1 should be The problem in the above approach is you have 1 node between 0-4 (25%) and and one node covering the rest which is 4-16, 0-0 (75%) Regards, /VJ On Wed, Jun 15, 2011 at 11:10 AM, Vijay vijay2...@gmail.com wrote: The problem in the above approach is you have 2 nodes between 12 to 4 in DC1 but from 4 to 12 you just have 1 (Which will cause uneven distribution of data the node) It is easier to think of the DCs as ring and split equally and interleave them together DC1 Node 1 : token 0 DC1 Node 2 : token 8.. DC2 Node 1 : token 4.. DC2 Node 1 : token 12.. Regards, /VJ On Tue, Jun 14, 2011 at 7:31 PM, AJ a...@dude.podzone.net wrote: Yes, which means that the ranges overlap each other. Is this just a convention, or is it technically required when using NetworkTopologyStrategy? Would it be acceptable to split the ranges into quarters by ignoring the data centers, such as: DC1 node 1 = 0 Range: (12, 16], (0, 0] node 2 = 4 Range: (0, 4] DC2 node 3 = 8 Range: (4, 8] node 4 = 12 Range: (8, 12] If this is OK, are there any drawbacks to this? On 6/14/2011 6:10 PM, Vijay wrote: Yes... Thats right... If you are trying to say the below... DC1 Node1 Owns 50% (Ranges 8..4 - 8..5 8..5 - 0) Node2 Owns 50% (Ranges 0 - 1 1 - 8..4) DC2 Node1 Owns 50% (Ranges 8..5 - 0 0 - 1) Node2 Owns 50% (Ranges 1 - 8..4 8..4 - 8..5) Regards, /VJ On Tue, Jun 14, 2011 at 3:47 PM, AJ a...@dude.podzone.net wrote: This http://wiki.apache.org/cassandra/Operations#Token_selection says: With NetworkTopologyStrategy, you should calculate the tokens the nodes in each DC independantly. and gives the example: DC1 node 1 = 0 node 2 = 85070591730234615865843651857942052864 DC2 node 3 = 1 node 4 = 85070591730234615865843651857942052865 So, according to the above, the token ranges would be (abbreviated nums): DC1 node 1 = 0 Range: (8..4, 16], (0, 0] node 2 = 8..4 Range: (0, 8..4] DC2 node 3 = 1 Range: (8..5, 16], (0, 1] node 4 = 8..5 Range: (1, 8..5] If the above is correct, then I would be surprised as this paragraph is the only place were one would discover this and may be easy to miss... unless there's a doc buried somewhere in plain view that I missed. So, have I interpreted this paragraph correctly? Was this design to help keep data somewhat localized if that was important, such as a geographically dispersed DC? Thanks!
Re: Docs: Token Selection
All you heard is right... You are not overriding Cassandra's token assignment by saying here is your token... Logic is: Calculate a token for the given key... find the node in each region independently (If you use NTS and if you set the strategy options which says you want to replicate to the other region)... Search for the ranges in each region independntly Replicate the data to that node. For multi DC cassandra needs nodes to be equally partitioned within each dc (If you care that the load equally distributed) as well as there shouldn't be any collusion of tokens within a cluster The documentation tried to explain the same and the example in the documentation. Hope this clarifies... More examples if it helps DC1 Node 1 : token 0 DC1 Node 2 : token 8.. DC2 Node 1 : token 4.. DC2 Node 1 : token 12.. or DC1 Node 1 : token 0 DC1 Node 2 : token 1.. DC2 Node 1 : token 8.. DC2 Node 1 : token 7.. Regards, /VJ On Wed, Jun 15, 2011 at 12:28 PM, AJ a...@dude.podzone.net wrote: On 6/15/2011 12:14 PM, Vijay wrote: Correction The problem in the above approach is you have 2 nodes between 12 to 4 in DC1 but from 4 to 12 you just have 1 should be The problem in the above approach is you have 1 node between 0-4 (25%) and and one node covering the rest which is 4-16, 0-0 (75%) Regards, /VJ Ok, I think you are saying that the computed token range intervals are incorrect and that they would be: DC1 *node 1 = 0 Range: (4, 16], (0, 0] node 2 = 4 Range: (0, 4] DC2 *node 3 = 8 Range: (12, 16], (0, 8] node 4 = 12 Range: (8, 12] If so, then yes, this is what I am seeking to confirm since I haven't found any documentation stating this directly and that reference that I gave only implies this; that is, that the token ranges are calculated per data center rather than per cluster. I just need someone to confirm that 100% because it doesn't sound right to me based on everything else I've read. SO, the question is: Does Cass calculate the consecutive node token ranges A.) per cluster, or B.) for the whole data center? From all I understand, the answer is B. But, that documentation (reprinted below) implies A... or something that doesn't make sense to me because of the token placement in the example: With NetworkTopologyStrategy, you should calculate the tokens the nodes in each DC independantly... DC1 node 1 = 0 node 2 = 85070591730234615865843651857942052864 DC2 node 3 = 1 node 4 = 850705917302346158658436518579 42052865 However, I do see why this would be helpful, but first I'm just asking if this token assignment is absolutely mandatory or if it's just a technique to achieve some end.
Re: Docs: Token Selection
No it wont it will assume you are doing the right thing... Regards, /VJ On Wed, Jun 15, 2011 at 2:34 PM, AJ a...@dude.podzone.net wrote: Vijay, thank you for your thoughtful reply. Will Cass complain if I don't setup my tokens like in the examples? On 6/15/2011 2:41 PM, Vijay wrote: All you heard is right... You are not overriding Cassandra's token assignment by saying here is your token... Logic is: Calculate a token for the given key... find the node in each region independently (If you use NTS and if you set the strategy options which says you want to replicate to the other region)... Search for the ranges in each region independntly Replicate the data to that node. For multi DC cassandra needs nodes to be equally partitioned within each dc (If you care that the load equally distributed) as well as there shouldn't be any collusion of tokens within a cluster The documentation tried to explain the same and the example in the documentation. Hope this clarifies... More examples if it helps DC1 Node 1 : token 0 DC1 Node 2 : token 8.. DC2 Node 1 : token 4.. DC2 Node 1 : token 12.. or DC1 Node 1 : token 0 DC1 Node 2 : token 1.. DC2 Node 1 : token 8.. DC2 Node 1 : token 7.. Regards, /VJ On Wed, Jun 15, 2011 at 12:28 PM, AJ a...@dude.podzone.net wrote: On 6/15/2011 12:14 PM, Vijay wrote: Correction The problem in the above approach is you have 2 nodes between 12 to 4 in DC1 but from 4 to 12 you just have 1 should be The problem in the above approach is you have 1 node between 0-4 (25%) and and one node covering the rest which is 4-16, 0-0 (75%) Regards, /VJ Ok, I think you are saying that the computed token range intervals are incorrect and that they would be: DC1 *node 1 = 0 Range: (4, 16], (0, 0] node 2 = 4 Range: (0, 4] DC2 *node 3 = 8 Range: (12, 16], (0, 8] node 4 = 12 Range: (8, 12] If so, then yes, this is what I am seeking to confirm since I haven't found any documentation stating this directly and that reference that I gave only implies this; that is, that the token ranges are calculated per data center rather than per cluster. I just need someone to confirm that 100% because it doesn't sound right to me based on everything else I've read. SO, the question is: Does Cass calculate the consecutive node token ranges A.) per cluster, or B.) for the whole data center? From all I understand, the answer is B. But, that documentation (reprinted below) implies A... or something that doesn't make sense to me because of the token placement in the example: With NetworkTopologyStrategy, you should calculate the tokens the nodes in each DC independantly... DC1 node 1 = 0 node 2 = 85070591730234615865843651857942052864 DC2 node 3 = 1 node 4 = 850705917302346158658436518579 42052865 However, I do see why this would be helpful, but first I'm just asking if this token assignment is absolutely mandatory or if it's just a technique to achieve some end.
Re: Docs: Token Selection
+1 for more documentation (I guess contributions are always welcomed) I will try to write it down sometime when we have a bit more time... 0.8 nodetool ring command adds the DC and RAC information http://www.datastax.com/dev/blog/deploying-cassandra-across-multiple-data-centers http://www.datastax.com/products/opscenter Hope this helps... Regards, /VJ On Wed, Jun 15, 2011 at 7:24 PM, AJ a...@dude.podzone.net wrote: Ok. I understand the reasoning you laid out. But, I think it should be documented more thoroughly. I was trying to get an idea as to how flexible Cass lets you be with the various combinations of strategies, snitches, token ranges, etc.. It would be instructional to see what a graphical representation of a cluster ring with multiple data centers looks like. Google turned-up nothing. I imagine it's a multilayer ring; one layer per data center with the nodes of one layer slightly offset from the ones in the other (based on the example in the wiki). I would also like to know which node is next in the ring such so as to understand replica placement in, for example, the OldNetworkTopologyStrategy when it's doc states, ...It places one replica in a different data center from the first (if there is any such data center), the third replica in a different rack in the first datacenter, and any remaining replicas on the first unused nodes on the ring. I can only assume for now that the ring referred to is the local ring of the first data center. On 6/15/2011 5:51 PM, Vijay wrote: No it wont it will assume you are doing the right thing... Regards, /VJ On Wed, Jun 15, 2011 at 2:34 PM, AJ a...@dude.podzone.net wrote: Vijay, thank you for your thoughtful reply. Will Cass complain if I don't setup my tokens like in the examples? On 6/15/2011 2:41 PM, Vijay wrote: All you heard is right... You are not overriding Cassandra's token assignment by saying here is your token... Logic is: Calculate a token for the given key... find the node in each region independently (If you use NTS and if you set the strategy options which says you want to replicate to the other region)... Search for the ranges in each region independntly Replicate the data to that node. For multi DC cassandra needs nodes to be equally partitioned within each dc (If you care that the load equally distributed) as well as there shouldn't be any collusion of tokens within a cluster The documentation tried to explain the same and the example in the documentation. Hope this clarifies... More examples if it helps DC1 Node 1 : token 0 DC1 Node 2 : token 8.. DC2 Node 1 : token 4.. DC2 Node 1 : token 12.. or DC1 Node 1 : token 0 DC1 Node 2 : token 1.. DC2 Node 1 : token 8.. DC2 Node 1 : token 7.. Regards, /VJ On Wed, Jun 15, 2011 at 12:28 PM, AJ a...@dude.podzone.net wrote: On 6/15/2011 12:14 PM, Vijay wrote: Correction The problem in the above approach is you have 2 nodes between 12 to 4 in DC1 but from 4 to 12 you just have 1 should be The problem in the above approach is you have 1 node between 0-4 (25%) and and one node covering the rest which is 4-16, 0-0 (75%) Regards, /VJ Ok, I think you are saying that the computed token range intervals are incorrect and that they would be: DC1 *node 1 = 0 Range: (4, 16], (0, 0] node 2 = 4 Range: (0, 4] DC2 *node 3 = 8 Range: (12, 16], (0, 8] node 4 = 12 Range: (8, 12] If so, then yes, this is what I am seeking to confirm since I haven't found any documentation stating this directly and that reference that I gave only implies this; that is, that the token ranges are calculated per data center rather than per cluster. I just need someone to confirm that 100% because it doesn't sound right to me based on everything else I've read. SO, the question is: Does Cass calculate the consecutive node token ranges A.) per cluster, or B.) for the whole data center? From all I understand, the answer is B. But, that documentation (reprinted below) implies A... or something that doesn't make sense to me because of the token placement in the example: With NetworkTopologyStrategy, you should calculate the tokens the nodes in each DC independantly... DC1 node 1 = 0 node 2 = 85070591730234615865843651857942052864 DC2 node 3 = 1 node 4 = 850705917302346158658436518579 42052865 However, I do see why this would be helpful, but first I'm just asking if this token assignment is absolutely mandatory or if it's just a technique to achieve some end.
Re: Docs: Token Selection
Yes... Thats right... If you are trying to say the below... DC1 Node1 Owns 50% (Ranges 8..4 - 8..5 8..5 - 0) Node2 Owns 50% (Ranges 0 - 1 1 - 8..4) DC2 Node1 Owns 50% (Ranges 8..5 - 0 0 - 1) Node2 Owns 50% (Ranges 1 - 8..4 8..4 - 8..5) Regards, /VJ On Tue, Jun 14, 2011 at 3:47 PM, AJ a...@dude.podzone.net wrote: This http://wiki.apache.org/cassandra/Operations#Token_selection says: With NetworkTopologyStrategy, you should calculate the tokens the nodes in each DC independantly. and gives the example: DC1 node 1 = 0 node 2 = 85070591730234615865843651857942052864 DC2 node 3 = 1 node 4 = 85070591730234615865843651857942052865 So, according to the above, the token ranges would be (abbreviated nums): DC1 node 1 = 0 Range: (8..4, 16], (0, 0] node 2 = 8..4 Range: (0, 8..4] DC2 node 3 = 1 Range: (8..5, 16], (0, 1] node 4 = 8..5 Range: (1, 8..5] If the above is correct, then I would be surprised as this paragraph is the only place were one would discover this and may be easy to miss... unless there's a doc buried somewhere in plain view that I missed. So, have I interpreted this paragraph correctly? Was this design to help keep data somewhat localized if that was important, such as a geographically dispersed DC? Thanks!
Re: EC2 Snitch
We are using this patch in our multi-region testing... yes this approach is going to be integrated into https://issues.apache.org/jira/browse/CASSANDRA-2491 once it is committed (you might want to wait for that). Yes this fix the Amazon infrastructure problems and it will automatically detect the DC and RAC only thing is that we have to allow IP's in the security groups to talk to each other's region. Regards, /VJ On Tue, May 10, 2011 at 6:19 PM, Sameer Farooqui cassandral...@gmail.comwrote: Has anybody successfully used EC2 Snitch for cross-region deployments on EC2? Brandon Williams has not recommended using this just yet, but I was curious if anybody is using it with 0.8.0. Also, the snitch just let's the cluster automatically discover what the different regions (aka data centers) and racks (aka availability zones) are, right? So, does this fix the Amazon NATed infrastructure problem where we can't use the Amazon external IP as the Cassandra listen address? Cassandra can only bind to addresses that are attached to the server and the Amazon external IP is NATed so Cassandra can't see it. The EC2-Snitch patch was kinda unclear on this: https://issues.apache.org/jira/browse/CASSANDRA-2452 Also, the nightly builds link to Cassandra seems to be down, has it been relocated? (click on Latest Builds/Hudson link on right): http://cassandra.apache.org/download/ This link is dead: http://hudson.zones.apache.org/hudson/job/Cassandra/lastSuccessfulBuild/artifact/cassandra/build/
Re: Does anybody have experience with running Cassandra in Amazon's Virtual Private Cloud (VPC)?
We are planning to use EIP's + https://issues.apache.org/jira/browse/CASSANDRA-2452 for AWS multi region deployment. Regards, /VJ On Thu, May 5, 2011 at 5:01 PM, aaron morton aa...@thelastpickle.comwrote: There have been some recent discussions about different EC2 deployments, may be be exactly what you are looking for but try start here http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Re-IP-address-resolution-in-MultiDC-setup-EC2-VIP-td6306635.html http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Re-IP-address-resolution-in-MultiDC-setup-EC2-VIP-td6306635.html - Aaron Morton Freelance Cassandra Developer @aaronmorton http://www.thelastpickle.com On 6 May 2011, at 09:21, Sameer Farooqui wrote: Here is an image that shows what the Amazon VPC we're thinking about using looks like: http://i.imgur.com/OUe1i.png We would like to configure a 2 node Cassandra cluster in the private subnet and a read/write web application service in the public subnet. However, we also want to span the Cassandra cluster across from the Virginia VPC to a California VPC and have 2 Cassandra nodes in the private subnet in Virginia and 1 node in the private subnet in California. Amazon states that EC2 instances in Virginia’s VPC can communicate with EC2 instances in California’s VPC as long as all communication takes place over the internet gateway of each VPC and uses Elastic IP addresses as the specified source or destination address. But Amazon also says that “EIP addresses should only be used on instances in subnets configured to route their traffic directly to the Internet Gateway”. So, by this they must mean the public subnet in the diagram above. Amazon goes on to say “EIPs cannot be used on instances in subnets configured to use a NAT instance to access the Internet.” It looks like we cannot have a Cassandra cluster spanning two Private Subnets in two different regions. Can somebody with VPC experience confirm this? Also has anybody successfully deployed a Cassandra cluster in private subnets spanning multiple regions? Any tips from your experience? How did you find the latency over VPCs and gateways across regions? - Sameer Accenture Technology Labs
Re: Optimizing a few nodes to handle all client connections?
Are you saying you dont like the idea of the co-ordinator node being in the same ring? if yes have you looked at the cassandra fat client in contrib? Regards, /VJ On Fri, Mar 18, 2011 at 6:55 PM, Jason Harvey alie...@gmail.com wrote: Hola everyone, I have been considering making a few nodes only manage 1 token and entirely dedicating them to talking to clients. My reasoning behind this is I don't like the idea of a node having a dual-duty of handling data, and talking to all of the client stuff. Is there any merit to this thought? Cheers, Jason
Re: Is SuperColumn necessary?
I would rather be interested in Tree type structure where supercolumns have supercolumns in it. you dont need to compare all the columns to find a set of columns and will also reduce the bytes transfered for separator, at least string concatenation (Or something like that) for read and write column name generation. it is more logically stored and structured by this way and also we can make caching work better by selectively caching the tree (User defined if you will) But nothing wrong in supporting both :) Regards, /VJ On Wed, May 5, 2010 at 11:31 AM, Ed Anuff e...@anuff.com wrote: Follow-up from last weeks discussion, I've been playing around with a simple column comparator for composite column names that I put up on github. I'd be interested to hear what people think of this approach. http://github.com/edanuff/CassandraCompositeType Ed On Wed, Apr 28, 2010 at 12:52 PM, Ed Anuff e...@anuff.com wrote: It might make sense to create a CompositeType subclass of AbstractType for the purpose of constructing and comparing these types of composite column names so that if you could more easily do that sort of thing rather than having to concatenate into one big string. On Wed, Apr 28, 2010 at 10:25 AM, Mike Malone m...@simplegeo.com wrote: The only thing SuperColumns appear to buy you (as someone pointed out to me at the Cassandra meetup - I think it was Eric Florenzano) is that you can use different comparator types for the Super/SubColumns, I guess..? But you should be able to do the same thing by creating your own Column comparator. I guess my point is that SuperColumns are mostly a convenience mechanism, as far as I can tell. Mike
Re: SuperColumns
Yes a super column can only have columns in it. Regards, /VJ On Wed, Apr 14, 2010 at 10:28 PM, Christian Torres chtor...@gmail.comwrote: I'm defining a ColumnFamily (Table) type Super, It's posible to have a SuperColumn inside another SuperColumn or SuperColumns can only have normal columns? -- Christian Torres * Desarrollador Web * Guegue.com * Celular: +505 84 65 92 62 * Loving of the Programming
Re: Bug in Cassandra that occurs when removing a supercolumn.
What version do you use? i think that bug was fixed in .6 https://issues.apache.org/jira/browse/CASSANDRA-703 Regards, /VJ On Sat, Apr 3, 2010 at 5:27 AM, Arash Bazrafshan ara...@gmail.com wrote: ello. A bug occurs for me when working with Cassandra. With this e-mail I intend to show what I do to recreate it, and then perhaps you can try it out too. SUMMARY OF THE BUG: (1): insert a row with a supercolumn that contains a subcolumn. (2) remove the supercolumn. (3) reinsert the same row with the same supercolumn and subcolumn. (RESULT): You won't be able to retrieve the entire supercolumn. However you will be able to retrieve the specific subcolumn within the supercolumn. Removing cassandra's datalogs will make the problem to go away. PREREQUISITES: * Use the column families that are defined by storage-conf.xml in its default out-of-the-box configuration. Specifically I will use the keyspace Keyspace1 with the supercolumn Super1. * I use Cassandra 0.5.0-1 on Ubuntu Karmic 9.10. * I use Thrift 0.2.0 to generate a php api for cassandra. It is when I use this api that the bug occurs. * I run Cassandra on a single node. So I query against 127.0.0.1. STEP-BY-STEP INSTRUCTIONS FOR TRIGGERING THE BUG: I will now step by step show the PHP scripts that I execute in order to generate the bug. STEP 1: EXECUTE THIS SCRIPT. //We will first insert a row into the supercolumn family Super1. //=== $socket = new TSocket(127.0.0.1, 30003); $transport = new TBufferedTransport($socket, 1024, 1024); $protocol = new TBinaryProtocolAccelerated($transport); $client = new CassandraClient ($protocol); $transport-open(); $writeColumnPath = new cassandra_ColumnPath(); $writeColumnPath-column_family = 'Super1'; $writeColumnPath-super_column = 'info'; $writeColumnPath-column = 'phonenumber'; $client-insert ( 'Keyspace1', 'adam', $writeColumnPath, '02012312345', time(), cassandra_ConsistencyLevel::ZERO ); $transport-close(); //=== RESULT OF STEP 1: The row that contains a single supercolumn with a single column has been inserted. STEP 2: EXECUTE THIS SCRIPT. //Next we will fetch the supercolumn of the row that we just inserted, just to make sure that the subcolumn is really there. //=== $socket = new TSocket(127.0.0.1, 30003); $transport = new TBufferedTransport($socket, 1024, 1024); $protocol = new TBinaryProtocolAccelerated($transport); $client = new CassandraClient ($protocol); $transport-open(); $readColumnPath = new cassandra_ColumnPath(); $readColumnPath-column_family = 'Super1'; $readColumnPath-super_column = 'info'; $readColumnPath-column = null; //NOTE: We want to fetch the entire supercolumn. $res = $client-get ( 'Keyspace1', 'adam', $readColumnPath, cassandra_ConsistencyLevel::ONE ); echo $res-super_column-columns[0]-value; $transport-close(); //=== RESULT OF STEP 2: You receive the following output: 02012312345 STEP 3: EXECUTE THIS SCRIPT. //Now we will remove the supercolumn of the row, but we will keep the row itself. //=== $socket = new TSocket(127.0.0.1, 30003); $transport = new TBufferedTransport($socket, 1024, 1024); $protocol = new TBinaryProtocolAccelerated($transport); $client = new CassandraClient ($protocol); $transport-open(); $deleteColumnPath = new cassandra_ColumnPath(); $deleteColumnPath-column_family = 'Super1'; $deleteColumnPath-super_column = 'info'; $deleteColumnPath-column = null; //NOTE: We want to remove the entire supercolumn 'info'. $client-remove ( 'Keyspace1', 'adam', $deleteColumnPath, time(), cassandra_ConsistencyLevel::ZERO ); $transport-close(); //=== RESULT OF STEP 3: The row is removed from the column family. STEP 4: EXECUTE THIS SCRIPT. //Now let's try to fetch the column within the supercolumn again, just to make sure it is really gone. //=== $socket = new TSocket(127.0.0.1, 30003); $transport = new TBufferedTransport($socket, 1024, 1024); $protocol = new TBinaryProtocolAccelerated($transport); $client = new CassandraClient ($protocol); $transport-open(); $readColumnPath = new cassandra_ColumnPath(); $readColumnPath-column_family = 'Super1'; $readColumnPath-super_column = 'info'; $readColumnPath-column = null; //NOTE: Fetching the entire supercolumn. $res = $client-get ( 'Keyspace1', 'adam', $readColumnPath, cassandra_ConsistencyLevel::ONE ); echo $res-super_column-columns[0]-value; $transport-close(); //=== RESULT OF STEP 4: A NotFoundException is thrown. STEP 5: EXECUTE THIS
Re: renaming a SuperColumn
+1 for renaming the S Column/Column as a atomic operation :) Regards, /VJ On Thu, Mar 18, 2010 at 9:50 AM, Jonathan Ellis jbel...@gmail.com wrote: -1 on adding a special case for this. 2010/3/18 Ted Zlatanov t...@lifelogs.com: I find it useful in one particular schema to have SuperColumns with specific names and rename them sometimes. Rather than client-side (read, write, delete) it would be nice if there was a SuperColumnRename Mutation than encapsulated that sequence on the server side and perhaps implemented it more efficiently. Right now I pay a pretty high cost for the client roundtrips and this would make my writers much more efficient. I don't need it to be atomic. Is this feasible and has it been discussed before? Thanks Ted
