Re: Stream fails during repair, two nodes out-of-memory

2013-03-22 Thread Wei Zhu
compaction needs some disk I/O. Slowing down our compaction will improve overall system performance. Of course, you don't want to go too slow and fall behind too much. -Wei - Original Message - From: "Dane Miller" To: user@cassandra.apache.org Cc: "Wei Zhu" Sent: Friday, March 22, 201

Re: Stream fails during repair, two nodes out-of-memory

2013-03-22 Thread Dane Miller
On Thu, Mar 21, 2013 at 10:28 AM, aaron morton wrote: > heap of 1867M is kind of small. According to the discussion on this list, > it's advisable to have m1.xlarge. > > +1 > > In cassadrea-env.sh set the MAX_HEAP_SIZE to 4GB, and the NEW_HEAP_SIZE to > 400M > > In the yaml file set > > in_memory_

RE: High disk I/O during reads

2013-03-22 Thread Kanwar Sangha
Just as a test - can you disable/reduce compaction throughput and see if that makes a difference ? Compaction eats a lot of I/O. From: zod...@fifth-aeon.net [mailto:zod...@fifth-aeon.net] On Behalf Of Jon Scarborough Sent: 22 March 2013 15:01 To: user@cassandra.apache.org; Wei Zhu Subject: Re:

Re: Secondary Indexes

2013-03-22 Thread Mayank
Ok. I always know the row key before I start the Cassandra read operation. A full system could have 300-500k columns so secondary indexes don't seem a good idea here. I think the best option will be to query a range of columns for the given row key. Thanks a bunch guys. On Mar 20, 2013, at 11:2

Re: High disk I/O during reads

2013-03-22 Thread Jon Scarborough
Checked tpstats, there are very few dropped messages. Checked histograms. Mostly nothing surprising. The vast majority of rows are small, and most reads only access one or two SSTables. What I did discover is that of our 5 nodes, one is performing well, with disk I/O in the ballprk that seems rea

Re: cannot start Cassandra on Windows7

2013-03-22 Thread Wei Zhu
It's there: http://www.datastax.com/docs/1.2/cluster_architecture/cluster_planning#node-init-config It's a long document You need to look at the cassandra.yaml and cassandra-env.sh and make sure you understand the settings there. By the way, did datastax just face lift their document web s

Re: High disk I/O during reads

2013-03-22 Thread Wei Zhu
According to your cfstats, read latency is over 100 ms which is really really slow. I am seeing less than 3ms reads for my cluster which is on SSD. Can you also check the nodetool cfhistorgram, it tells you more about the number of SSTable involved and read/write latency. Somtimes average doesn'

Re: High disk I/O during reads

2013-03-22 Thread Jon Scarborough
Key distribution across probably varies a lot from row to row in our case. Most reads would probably only need to look at a few SSTables, a few might need to look at more. I don't yet have a deep understanding of C* internals, but I would imagine even the more expensive use cases would involve som

Re: index_interval file size is the same after modifying 128 to 512?

2013-03-22 Thread Yuki Morishita
Index.db file always contains *all* position of the keys in data file. index_interval is the rate that the position of the key in index file is store in memory. So that C* can begin scanning index file from closest position. On Friday, March 22, 2013 at 11:17 AM, Hiller, Dean wrote: > I was ju

Re: cannot start Cassandra on Windows7

2013-03-22 Thread Marina
Jabbar Azam gmail.com> writes: > > > > > > > Oops, I also had opscenter installed on my PC. > My  changes > log4j-server.properties file log4j.appender.R.File=c:/var/log/cassandra/system.log > cassandra.yaml file# directories w

index_interval file size is the same after modifying 128 to 512?

2013-03-22 Thread Hiller, Dean
I was just curious. Our RAM has significantly reduced but the *Index.db files are the same size size as before. Any ideas why this would be the case? Basically, Why is our disk size not reduced since RAM is way lower? We are running strong now with 512 index_interval for past 2-3 days and RAM

Re: cannot start Cassandra on Windows7

2013-03-22 Thread Jabbar Azam
Oops, I also had opscenter installed on my PC. My changes log4j-server.properties file log4j.appender.R.File=c:/var/log/cassandra/system.log cassandra.yaml file # directories where Cassandra should store data on disk. data_file_

Re: cannot start Cassandra on Windows7

2013-03-22 Thread Jabbar Azam
Viktor, you're right. I didn't get any errors on my windows console but cassandra.yaml and log4j-server.properties need modifying. On 22 March 2013 15:44, Viktor Jevdokimov wrote: > You NEED to edit cassandra.yaml and log4j-server.properties paths before > starting on Windows. > > There're a LOT

RE: cannot start Cassandra on Windows7

2013-03-22 Thread Viktor Jevdokimov
You NEED to edit cassandra.yaml and log4j-server.properties paths before starting on Windows. There're a LOT of things to learn for starters. Google for Cassandra on Windows. Best regards / Pagarbiai Viktor Jevdokimov Senior Developer Email: viktor.jevdoki...@adform.com Phone: +370 5 212 306

Re: cannot start Cassandra on Windows7

2013-03-22 Thread Jabbar Azam
Hello Marina, I've downloaded a fresh copy of v1.2.3 and it's running fine on my Windows 7 64 bit PC. I am using jdk 1.6.0 u29 64 bit. I have local admin permissions to my PC. On 22 March 2013 15:36, Marina wrote: > > > > > Hi, > > I have downloaded apache-cassandra-1.2.3-bin.tar.gz and un-zip

Backup strategies in a multi DC cluster

2013-03-22 Thread Jabbar Azam
Hello, I've been experimenting with cassandra for quite a while now. It's time for me to look at backups but I'm not sure what the best practice is. I want to be able to recover the data to a point in time before any user or software errors. We will have two datacentres with 4 servers and RF=3.

Re: cannot start Cassandra on Windows7

2013-03-22 Thread Marina
> > Hi, > I have downloaded apache-cassandra-1.2.3-bin.tar.gz and un-zipped it on my > Windows7 machine (I did not find a Windows-specific distributable...). Then, > I > tried to start Cassandra as following and got an error: > > C:\Marina\Tools\apache-cassandra-1.2.3\bin>cassandra.bat -f >

cannot start Cassandra on Windows7

2013-03-22 Thread Marina
Hi, I have downloaded apache-cassandra-1.2.3-bin.tar.gz and un-zipped it on my Windows7 machine (I did not find a Windows-specific distributable...). Then, I tried to start Cassandra as following and got an error: C:\Marina\Tools\apache-cassandra-1.2.3\bin>cassandra.bat -f Starting Cassandra Se

RE: High disk I/O during reads

2013-03-22 Thread Kanwar Sangha
Sorry...I meant to ask "is your key" spread across multiple sstables ? But with LCS, your reads should ideally be served from one sstable most of the times.. -Original Message- From: Hiller, Dean [mailto:dean.hil...@nrel.gov] Sent: 22 March 2013 10:16 To: user@cassandra.apache.org Sub

Re: High disk I/O during reads

2013-03-22 Thread Hiller, Dean
Did you mean to ask "are 'all' your keys spread across all SSTables"? I am guessing at your intention. I mean I would very well hope my keys are spread across all sstables or otherwise that sstable should not be there as he has no keys in it ;). And I know we had HUGE disk size from the duplic

RE: High disk I/O during reads

2013-03-22 Thread Kanwar Sangha
Are your Keys spread across all SSTables ? That will cause every sstable read which will increase the I/O. What compaction are you using ? From: zod...@fifth-aeon.net [mailto:zod...@fifth-aeon.net] On Behalf Of Jon Scarborough Sent: 21 March 2013 23:00 To: user@cassandra.apache.org Subject: Hig

Thrift key

2013-03-22 Thread Sávio Teles
I'm running a Cassandra 1.1.10 cluster with a ByteOrderedPartitioner. I'm generating a key to force an object to be stored in a specific machine. When I used org.apache.cassandra.thrift.CassandraServer to store the object, this object was stored on correct machine. When I used Thrift, the key is ch

Re: Cassandra freezes

2013-03-22 Thread Joel Samuelsson
Thanks for the GC suggestion. It seems we didn't have enough CPU power to handle both the data and GC. Increasing the number of CPU cores made everything run smoothly at the same load. 2013/3/21 Andras Szerdahelyi > Neat! > Thanks. > > From: Sylvain Lebresne > Reply-To: "user@cassandra.apac

Re: create secondary index on column family

2013-03-22 Thread Xu Renjie
Sorry, continued: I have created a column family User with no parameters specified, just create column family User. Then I checked that the default comparator is BytesType. Then I want to create secondary index on one column like below: update column family User with column_metadata=[

create secondary index on column family

2013-03-22 Thread Xu Renjie
Hello, guys: I am new to Cassandra. I am currently using cassandra-cli(version 1.1.6).

RE: java.lang.IllegalStateException: No sources found for (0,0]

2013-03-22 Thread Sloot, Hans-Peter
Hi, Yes I did. I was a bit confused when reading your previous update on my cell phone. I have 4 nodes and 2 seeds. Regards Hans-Peter From: Tyler Hobbs [mailto:ty...@datastax.com] Sent: donderdag 21 maart 2013 17:17 To: user@cassandra.apache.org Subject: Re: java.lang.IllegalStateException: N