can I have a mix of 32 and 64 bit machines in a cluster?

2012-10-09 Thread Brian Tarbox
I can't imagine why this would be a problem but I wonder if anyone has experience with running a mix of 32 and 64 bit nodes in a cluster. (I'm not going to do this in production, just trying to make use of the gear I have for my local system). Thanks.

read performance plumetted

2012-10-12 Thread Brian Tarbox
I have a two node cluster hosting a 45 gig dataset. I periodically have to read a high fraction (20% or so) of my 'rows', grabbing a few thousand at a time and then processing them. This used to result in about 300-500 reads a second which seemed quite good. Recently that number has plummeted

why does my Effective-Ownership and Load from ring give such different answers?

2012-10-19 Thread Brian Tarbox
I had a two node cluster that I expanded to four nodes. I ran the token generation script and moved all the nodes so that when I run nodetool ring each node reports 25% Effective-Ownership. However, my load numbers map out to 39%, 30%, 15%, 17%. How can that be? Thanks.

my ubunutu system has fuseblk filesystem, would it go faster if I changed it to EXT4?

2012-10-30 Thread Brian Tarbox
I got some new ubuntu servers to add to my cluster and found that the file system is fuseblk which really means NTFS. All else being equal would I expect to get any performance boost if I converted the file system to EXT4? Edward Capriolo's Cookbook book seems to suggest so. Thanks. Brian

problem encrypting keys and data

2012-11-07 Thread Brian Tarbox
be appreciated, thanks. Brian Tarbox

Re: Memory Manager

2012-11-12 Thread Brian Tarbox
Can you supply your java parameters? On Mon, Nov 12, 2012 at 7:29 AM, Everton Lima peitin.inu...@gmail.comwrote: Hi people, I was using cassandra on distributed project. I am using java 6 and cassandra 1.1.6. My problem is in Memory manager (I think). My system was throwing heap limit

Re: State of Cassandra and Java 7

2012-12-15 Thread Brian Tarbox
We've reverted all machines back to Java 6 after running into numerous Java 7 issues...some running Cassandra, some running Zookeeper, others just general problems. I don't recall any other major language release being such a mess. On Fri, Dec 14, 2012 at 5:07 PM, Bill de hÓra b...@dehora.net

Re: State of Cassandra and Java 7

2012-12-17 Thread Brian Tarbox
and have no issues whatsoever. A related question is which which version of java 7 did you try? The first releases of java 7 were apparently famous for having many issues but it seems the more recent updates are much more stable. -- Sylvain On Dec 15, 2012, at 7:51 AM, Brian Tarbox tar

Re: State of Cassandra and Java 7

2012-12-22 Thread Brian Tarbox
, timestamp=1355836425784329, ttl=648000) A long is exactly 8 bytes: 10 -Bryan On Mon, Dec 17, 2012 at 7:33 AM, Brian Tarbox tar...@cabotresearch.comwrote: I was using jre-7u9-linux-x64 which was the latest at the time. I'll confess that I did not file any bugs...at the time

Re: Cassadra API for Java

2012-12-30 Thread Brian Tarbox
Anyone still use Pelops? On Sun, Dec 30, 2012 at 12:19 PM, Shahryar Sedghi shsed...@gmail.comwrote: I use JDBC with Cassandra 1.1 with CQL 3. I tried both Hector and Thrift and JDBC is much easier to code, I never tried Astyanax. Application servers have built-in connection pooling support

puzzled why my cluster is slowing down

2013-01-07 Thread Brian Tarbox
? Thanks! Brian Tarbox

help turning compaction..hours of run to get 0% compaction....

2013-01-07 Thread Brian Tarbox
I have a column family where I'm doing 500 inserts/sec for 12 hours or so at time. At some point my performance falls off a cliff due to time spent doing compactions. I'm seeing row after row of logs saying that after 1 or 2 hours of compactiing it reduced to 100% of 99% of the original. I'm

Re: help turning compaction..hours of run to get 0% compaction....

2013-01-07 Thread Brian Tarbox
? From: Brian Tarbox tar...@cabotresearch.com Reply-To: user@cassandra.apache.org user@cassandra.apache.org Date: Monday, January 7, 2013 12:03 PM To: user@cassandra.apache.org user@cassandra.apache.org Subject: help turning compaction..hours of run to get 0% compaction I have a column

Re: help turning compaction..hours of run to get 0% compaction....

2013-01-07 Thread Brian Tarbox
. This is especially true if your reads are fairly random and don’t focus on a single, hot dataset. From: Brian Tarbox tar...@cabotresearch.com Reply-To: user@cassandra.apache.org user@cassandra.apache.org Date: Monday, January 7, 2013 12:56 PM To: user@cassandra.apache.org user@cassandra.apache.org Subject

is there a way to list who is connected to my cluster?

2013-01-11 Thread Brian Tarbox
this I've turned off all of the programs I know of. But, DataStax still saying there are clients running against us...how can I find them? Thanks. Brian Tarbox

How can OpsCenter show me Read Request Latency where there are no read requests??

2013-01-15 Thread Brian Tarbox
but it stays up even after all the reads are done. At last glance I've had next to zero reads for 10 minutes but still have a read request latency thats basically unchanged from when there were actual reads. How am I to interpret this? Thanks. Brian Tarbox

Re: How can OpsCenter show me Read Request Latency where there are no read requests??

2013-01-16 Thread Brian Tarbox
. Normally the number of reads required to view metrics are small enough that they only make a minor difference in your overall read latency average, but when you have no other reads occurring, they're the only reads that are included in the average. On Tue, Jan 15, 2013 at 9:28 PM, Brian

trying to use row_cache (b/c we have hot rows) but nodetool info says zero requests

2013-01-16 Thread Brian Tarbox
how to proceed are appreciated. Thanks. Brian Tarbox

Is this how to read the output of nodetool cfhistograms?

2013-01-22 Thread Brian Tarbox
The output of this command seems to make no sense unless I think of it as 5 completely separate histograms that just happen to be displayed together. Using this example output should I read it as: my reads all took either 1 or 2 sstable. And separately, I had write latencies of 3,7,19. And

Re: Is this how to read the output of nodetool cfhistograms?

2013-01-22 Thread Brian Tarbox
Thank you! Since this is a very non-standard way to display data it might be worth a better explanation in the various online documentation sets. Thank you again. Brian On Tue, Jan 22, 2013 at 9:19 AM, Mina Naguib mina.nag...@adgear.com wrote: On 2013-01-22, at 8:59 AM, Brian Tarbox tar

Re: Is this how to read the output of nodetool cfhistograms?

2013-01-22 Thread Brian Tarbox
: This was described in good detail here: http://thelastpickle.com/2011/04/28/Forces-of-Write-and-Read/ On Tue, Jan 22, 2013 at 9:41 AM, Brian Tarbox tar...@cabotresearch.comwrote: Thank you! Since this is a very non-standard way to display data it might be worth a better explanation in the various online

Re: Is this how to read the output of nodetool cfhistograms?

2013-01-23 Thread Brian Tarbox
:41 AM, Brian Tarbox tar...@cabotresearch.comwrote: Thank you! Since this is a very non-standard way to display data it might be worth a better explanation in the various online documentation sets. Thank you again. Brian On Tue, Jan 22, 2013 at 9:19 AM, Mina Naguib mina.nag

trying to create encrypted ephemeral drive on Amazon

2013-01-29 Thread Brian Tarbox
I've heard that on Amazon EC2 I should be using ephemeral drives...but I want/need to be using encrypted volumes. On my local machine I use cryptsetup to encrypt a device and then mount it and so on...but on Amazon I get the error: Cannot open device /dev/xvdb for read-only access. Reading

Re: Upcoming conferences

2013-01-30 Thread Brian Tarbox
At what level will the NY talks be? I had been planning on attending Datastax's big summer conference and I might not be able to get approval for bothso I'd like to hear more about this one. On Wed, Jan 30, 2013 at 12:40 PM, Jonathan Ellis jbel...@gmail.com wrote: ApacheCon North America

what addresses to use in EC2 cluster (whenever an instance restarts it gets a new private ip)?

2013-02-11 Thread Brian Tarbox
addresses can change out from under you? Thanks. Brian Tarbox

Re: what addresses to use in EC2 cluster (whenever an instance restarts it gets a new private ip)?

2013-02-12 Thread Brian Tarbox
: You have to use private IPs, but if an instance dies you have to bootstrap it with replace token flag. If you use EC2 I'd recommend Netflix's Priam tool. It manages all that stuff, plus you have S3 backup. Andrey On Mon, Feb 11, 2013 at 11:35 AM, Brian Tarbox tar...@cabotresearch.comwrote

any other NYC* attendees find your usb stick of the proceedings empty?

2013-03-25 Thread Brian Tarbox
Last week I attended DataStax's NYC* conference and one of the give-aways was a wooden USB stick. Finally getting around to loading it I find it empty. Anyone else have this problem? Are the conference presentations available somewhere else? Brian Tarbox

Re: cfhistograms

2013-03-25 Thread Brian Tarbox
I think we all go through this learning curve. Here is the answer I gave last time this question was asked: The output of this command seems to make no sense unless I think of it as 5 completely separate histograms that just happen to be displayed together. Using this example output should I

Re: Cassandra Summit 2013

2013-04-12 Thread Brian Tarbox
on it and so now can't attend SFSummit25. NYC* was great but I feel that I got misled...or did I misunderstand somehow??? Brian Tarbox On Fri, Apr 12, 2013 at 11:50 AM, Jonathan Ellis jbel...@gmail.com wrote: Hi all, Last year's Summit saw fantastic talks [1] and over 800 attendees

how to monitor nodetool cleanup?

2013-05-07 Thread Brian Tarbox
total data). Anyway, any suggestions for monitoring / speeding up cleanup would be appreciated. Brian Tarbox

best practices on EC2 question

2013-05-16 Thread Brian Tarbox
From this list and the NYC* conference it seems that the consensus configuration of C* on EC2 is to put the data on an ephemeral drive and then periodically back it the drive to S3...relying on C*'s inherent fault tolerance to deal with any data loss. Fine, and we're doing this...but we find that

Re: Unreachable Nodes

2013-05-22 Thread Brian Tarbox
Have to disagree with the does no harm comment just a tiny bit. I had a similar situation recently and coincidentally needed to do a CF truncate. The system rejected the request saying that not all nodes were up. Nodetool ring said everyone was up but nodetool gossipinfo said there were

Re: Alternate major compaction

2013-07-11 Thread Brian Tarbox
Perhaps I should already know this but why is running a major compaction considered so bad? We're running 1.1.6. Thanks. On Thu, Jul 11, 2013 at 7:51 AM, Takenori Sato ts...@cloudian.com wrote: Hi, I think it is a common headache for users running a large Cassandra cluster in production.

Re: too many open files

2013-07-15 Thread Brian Tarbox
Odd that this discussion happens now as I'm also getting this error. I get a burst of error messages and then the system continues...with no apparent ill effect. I can't tell what the system was doing at the timehere is the stack. BTW Opscenter says I only have 4 or 5 SSTables in each of my

Re: Which of these VPS configurations would perform better for Cassandra ?

2013-08-05 Thread Brian Tarbox
We run a cluster in EC2 and it's working very well for us. The standard seems to be M2.2XLarge instances with data living on the ephemeral drives (which means its local and fast) and backups either to EBS, S3 or just relying on cluster size and replication (we avoid that last idea). Brian On

Re: Cassandra JVM heap sizes on EC2

2013-08-23 Thread Brian Tarbox
The advice I heard at the New York C* conference...which we follow is to use the m2.2xlarge and give it about 8 GB. The m2.4xlarge seems overkill (or at least over price). Brian On Fri, Aug 23, 2013 at 6:12 PM, David Laube d...@stormpath.com wrote: Hi All, We are evaluating our JVM heap

looking for advice before upgrading from 1.1.6 to 2.0.1

2013-10-10 Thread Brian Tarbox
consider? would be most appreciated. Thank you, Brian Tarbox

changing several things (almost) at once; is this the right order to make the changes?

2013-12-02 Thread Brian Tarbox
but figured I'd ask the experts first in case I'm doing something foolish. Thanks, Brian Tarbox

Re: bad interaction between CompositeTypes and Secondary index

2014-01-21 Thread Brian Tarbox
then updated. Not sure if that issue was fixed in 2.0.4, I'm avoiding altering tables completely for now. __ Sent from iPhone On 22 Jan 2014, at 7:50 am, Brian Tarbox tar...@cabotresearch.com wrote: We're trying to use CompositeTypes and Secondary indexes

Re: Opscenter tabs

2014-01-23 Thread Brian Tarbox
A vaguely related question...my OpsCenter now has two separate tabs for the same cluster...one tab shows all six nodes and has their agents...the other tab has the same six nodes but no agents. I see no way to get rid of the spurious tab. On Thu, Jan 23, 2014 at 12:47 PM, Ken Hancock

in AWS is it worth trying to talk to a server in the same zone as your client?

2014-02-12 Thread Brian Tarbox
zone would help? Thank you, Brian Tarbox

Re: in AWS is it worth trying to talk to a server in the same zone as your client?

2014-02-12 Thread Brian Tarbox
...@gmail.comwrote: yes, sure. Taking data from the same zone will reduce latency and save you some money. On Wed, Feb 12, 2014 at 10:13 AM, Brian Tarbox tar...@cabotresearch.com wrote: We're running a C* cluster with 6 servers spread across the four us-east1 zones. We also spread our clients

getting lots of dropped messages/requests/mutations but only on 2 of 6 servers

2014-03-20 Thread Brian Tarbox
I have a six node cluster (running m2-2xlarge instances in AWS) with RF=3 and I'm seeing two of the six nodes reporting lots of dropped messages. The six machines are identical (created from same AWS AMI) so this local behavior has me puzzled. BTW this is mostly happening when I'm reading via

Re: this seems like a flaw in Node Selection

2014-03-20 Thread Brian Tarbox
Does this still apply since we're using 1.2.13? (should have said that in the original message) Thank you. On Thu, Mar 20, 2014 at 3:57 PM, Robert Coli rc...@eventbrite.com wrote: On Thu, Mar 20, 2014 at 12:31 PM, Brian Tarbox tar...@cabotresearch.comwrote: I've seen this problem

Re: this seems like a flaw in Node Selection

2014-03-20 Thread Brian Tarbox
:07 PM, Robert Coli rc...@eventbrite.com wrote: On Thu, Mar 20, 2014 at 1:03 PM, Brian Tarbox tar...@cabotresearch.comwrote: Does this still apply since we're using 1.2.13? (should have said that in the original message) I checked the cassandra-1.2 branch to verify that the dynamic_snitch

getting dropped messages in log even with no one running

2014-03-24 Thread Brian Tarbox
I'm getting messages dropped messages in my cluster even when (like right now) there are no clients running against the cluster. 1) who could be generating the traffic if there are no clients? 2) is there a way to list active clients...on the off chance that there is a client I don't know about?

Re: getting dropped messages in log even with no one running

2014-03-24 Thread Brian Tarbox
the other nodes went right back into the dropping messages state. Help please. Brian On Mon, Mar 24, 2014 at 10:01 AM, Brian Tarbox tar...@cabotresearch.comwrote: I'm getting messages dropped messages in my cluster even when (like right now) there are no clients running against the cluster

Specifying startBefore with iterators with compositeKeys

2014-04-01 Thread Brian Tarbox
I have a composite key consisting of: (integer, bytes) and I have rows like: (1,abc), (1,def), (2,abc), (2,def) and I want to find all rows with the integer part = 2. I need to create a startBeyondName using CompositeType.Builder class and am wondering if specifying (2, Bytes.Empty) will sort

running out of diskspace during maintenance tasks

2014-06-18 Thread Brian Tarbox
require double the space of one's data. Since I can't change the size of attached storage for my instance type my question is can I somehow get these maintenance operations to use other volumes? Failing that, what are my options? Thanks. Brian Tarbox

Re: running out of diskspace during maintenance tasks

2014-06-18 Thread Brian Tarbox
on a separate volume. Thanks. On Wed, Jun 18, 2014 at 12:03 PM, Jeremy Jongsma jer...@barchart.com wrote: One option is to add new nodes, and do a node repair/cleanup on everything. That will at least reduce your per-node data size. On Wed, Jun 18, 2014 at 11:01 AM, Brian Tarbox tar

can I kill very old data files in my data folder (I know that sounds crazy but....)

2014-06-18 Thread Brian Tarbox
have been deleted but were not. The files are tens or hundreds of gigs so deleting would be good, unless its really bad! Thanks, Brian Tarbox

Re: can I kill very old data files in my data folder (I know that sounds crazy but....)

2014-06-18 Thread Brian Tarbox
on a rolling basis if I understand you. Thanks, Brian On Wed, Jun 18, 2014 at 2:37 PM, Robert Coli rc...@eventbrite.com wrote: On Wed, Jun 18, 2014 at 10:56 AM, Brian Tarbox tar...@cabotresearch.com wrote: I have a column family that only stores the last 5 days worth of some data...and yet I

Re: can I kill very old data files in my data folder (I know that sounds crazy but....)

2014-06-18 Thread Brian Tarbox
I don't think I have the space to run a major compaction right now (I'm above 50% disk space used already) and compaction can take extra space I think? On Wed, Jun 18, 2014 at 3:24 PM, Robert Coli rc...@eventbrite.com wrote: On Wed, Jun 18, 2014 at 12:05 PM, Brian Tarbox tar

nodetool repair saying starting and then nothing, and nothing in any of the server logs either

2014-07-01 Thread Brian Tarbox
I have a six node cluster in AWS (repl:3) and recently noticed that repair was hanging. I've run with the -pr switch. I see this output in the nodetool command line (and also in that node's system.log): Starting repair command #9, repairing 256 ranges for keyspace dev_a but then no other

Re: nodetool repair saying starting and then nothing, and nothing in any of the server logs either

2014-07-01 Thread Brian Tarbox
We're running 1.2.13. Any chance that doing a rolling-restart would help? Would running without the -pr improve the odds? Thanks. On Tue, Jul 1, 2014 at 1:40 PM, Robert Coli rc...@eventbrite.com wrote: On Tue, Jul 1, 2014 at 9:24 AM, Brian Tarbox tar...@cabotresearch.com wrote: I have

Re: nodetool repair saying starting and then nothing, and nothing in any of the server logs either

2014-07-01 Thread Brian Tarbox
) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) On Tue, Jul 1, 2014 at 2:09 PM, Brian Tarbox tar...@cabotresearch.com wrote

Re: nodetool repair saying starting and then nothing, and nothing in any of the server logs either

2014-07-01 Thread Brian Tarbox
1, 2014 at 11:09 AM, Brian Tarbox tar...@cabotresearch.com wrote: We're running 1.2.13. 1.2.17 contains a few streaming fixes which might help. Any chance that doing a rolling-restart would help? Probably not. Would running without the -pr improve the odds? No, that'd make it less

Re: nodetool repair saying starting and then nothing, and nothing in any of the server logs either

2014-07-01 Thread Brian Tarbox
, Jul 1, 2014 at 2:57 PM, Robert Coli rc...@eventbrite.com wrote: On Tue, Jul 1, 2014 at 11:54 AM, Brian Tarbox tar...@cabotresearch.com wrote: Given that an upgrade is (for various internal reasons) not an option at this point...is there anything I can do to get repair working again? I'll

do all nodes actually send the data to the coordinator when doing a read?

2014-07-25 Thread Brian Tarbox
We're considering a C* setup with very large columns and I have a question about the details of read. I understand that a read request gets handled by the coordinator which sends read requests to quorum of the nodes holding replicas of the data, and once quorum nodes have replied with consistent

Re: Options for expanding Cassandra cluster on AWS

2014-08-19 Thread Brian Tarbox
The last guidance I heard from DataStax was to use m2.2xlarge's on AWS and put data on the ephemeral drivehave they changed this guidance? Brian On Tue, Aug 19, 2014 at 9:41 AM, Oleg Dulin oleg.du...@gmail.com wrote: Distinguished Colleagues: Our current Cassandra cluster on AWS looks

read after write inconsistent even on a one node cluster

2014-11-06 Thread Brian Tarbox
to ALL with no effect. I've read CASSANDRA-876 which seems spot-on but it was closed as won't-fix...and I don't see what the solution is. Thanks in advance for any help. Brian Tarbox -- http://about.me/BrianTarbox

Re: read after write inconsistent even on a one node cluster

2014-11-06 Thread Brian Tarbox
that it's a Bad Idea™, but your email reads like you already know that =) On Thu Nov 06 2014 at 7:16:25 AM Brian Tarbox briantar...@gmail.com wrote: We're doing development on a single node cluster (and yes of course we're not really deploying that way), and we're getting inconsistent behavior

multiple threads updating result in TransportException

2014-11-26 Thread Brian Tarbox
We're running into a problem where things are fine if our client runs single threaded but gets TransportException if we use multiple threads. The datastax driver gets an NIO checkBounds error. Here is a link to a stack overflow question we found that describes the problem we're seeing. This

Re: High read latency after data volume increased

2015-01-09 Thread Brian Tarbox
C* seems to have more than its share of version x doesn't work, use version y type issues On Thu, Jan 8, 2015 at 2:23 PM, Robert Coli rc...@eventbrite.com wrote: On Thu, Jan 8, 2015 at 11:14 AM, Roni Balthazar ronibaltha...@gmail.com wrote: We are using C* 2.1.2 with 2 DCs. 30 nodes DC1

Re: write timeout

2015-03-23 Thread Brian Tarbox
My group is seeing the same thing and also can not figure out why its happening. On Mon, Mar 23, 2015 at 8:36 AM, Anishek Agarwal anis...@gmail.com wrote: Forgot to mention I am using Cassandra 2.0.13 On Mon, Mar 23, 2015 at 5:59 PM, Anishek Agarwal anis...@gmail.com wrote: Hello, I am

unsubscribe

2015-10-27 Thread Brian Tarbox
unsubscribe -- http://about.me/BrianTarbox

unsubscribe

2015-10-26 Thread Brian Tarbox
-- http://about.me/BrianTarbox