unable to find sufficient sources for streaming range

2014-07-02 Thread Daning Wang
) at org.apache.cassandra.service.StorageService.shutdownClientServers(StorageService.java:362) at org.apache.cassandra.service.StorageService.access$000(StorageService.java:88) at org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:513) Daning

Bulk writes and key cache

2014-02-03 Thread Daning Wang
? Thanks, Daning

Move token to another node on 1.2.x

2013-11-07 Thread Daning Wang
168755834953206242653616795390304335560 This node has more than one token and cannot be moved thusly We don't want to use cassandra-shuffle, because it put too much load on the server. we just want to move some tokens. Thanks, Daning

Re: ReadCount change rate is different across nodes

2013-10-30 Thread Daning Wang
Thanks. actually I forgot to mention it is multi-center environment and we have dynamic snitch disabled. because we saw some performance impact on the multi-center environment. On Wed, Oct 30, 2013 at 11:12 AM, Piavlo lolitus...@gmail.com wrote: On 10/30/2013 02:06 AM, Daning Wang wrote

ReadCount change rate is different across nodes

2013-10-29 Thread Daning Wang
hector on client side, and clients are connecting to all the servers, we checked open connections on each server, the numbers are about same. What could cause this problem, how to debug this? Thanks in advance, Daning

Key cache size

2013-09-04 Thread Daning Wang
We noticed that key cache could not be fully populated, we have set the key cache size to 1024M. key_cache_size_in_mb: 1024 But none of nodes showed the cache capacity is 1G, we have recently upgraded to 1.2.5, could be an issue in that version? Token: (invoke with -T/--tokens to

Dynamic Snitch and EC2MultiRegionSnitch

2013-07-01 Thread Daning Wang
How does dynamic snitch work with EC2MultiRegionSnitch? Can dynamic routing only happen in one data center? We don't wan to have the requests routed to another center even nodes are idle in other side since the network could be slow. Thanks in advance, Daning

Re: Multiple data center performance

2013-06-12 Thread Daning Wang
other DC is shutdown. Thanks, Daning On Wed, Jun 12, 2013 at 7:48 AM, Sylvain Lebresne sylv...@datastax.comwrote: Is there something special of this kind regarding counters over multiDC ? No. Counters behave exactly as other writes as far the consistency level is concerned. Technically

Re: Multiple data center performance

2013-06-11 Thread Daning Wang
replicate_on_write on table. If you turn that off for counter might have better performance. but you are on high risk to lose data and create inconsistency. I did not try this option. Daning On Sat, Jun 8, 2013 at 6:53 AM, srmore comom...@gmail.com wrote: I am seeing the similar behavior, in my case I

replication factor is zero

2013-06-06 Thread Daning Wang
We have multi-center deployment. data from some tables we don't want to sync to other center. could we set replication factor to 0 on other data center? what is the best to way for not syncing some data in a cluster? Thanks in advance, Daning

How to change existing cluster to multi-center

2013-04-25 Thread Daning Wang
this before, please share some light, Thanks in advance, Daning

Cassandra remote backup solution

2013-04-25 Thread Daning Wang
Hi Guys, What is the cassandra solution for remote backup besides multi-center? I hope I can do incremental backup to remote database center. Thanks, Daning

Re: Upgrade to Cassandra 1.2

2013-02-14 Thread Daning Wang
Thanks Aaron and Manu. Since we are using 1.1, there is no num_taken parameter. when I upgrade to 1.2, should I set num_token=1 to start up, or I can set to other numbers? Daning On Tue, Feb 12, 2013 at 3:45 PM, Manu Zhang owenzhang1...@gmail.com wrote: num_tokens is only used

Re: Upgrade to Cassandra 1.2

2013-02-14 Thread Daning Wang
#num_tokens commented in the cassandra.yaml and would set the initial_token at the same value than in the pre-C*1.2.x-uprage configuration. Alain 2013/2/14 Daning Wang dan...@netseer.com Thanks Aaron and Manu. Since we are using 1.1, there is no num_taken parameter. when I upgrade to 1.2

Re: Upgrade to Cassandra 1.2

2013-02-12 Thread Daning Wang
No, I did not run shuffle since the upgrade was not successful. what do you mean reverting the changes to num_tokens and inital_token? set num_tokens=1? initial_token should be ignored since it is not bootstrap. right? Thanks, Daning On Tue, Feb 12, 2013 at 10:52 AM, aaron morton aa

Re: Upgrade to Cassandra 1.2

2013-02-11 Thread Daning Wang
10.210.101.12045.43 GB 256 0.4% 0fd912fb-3187-462b-8c8a-7d223751b649 rack1 UN 10.210.101.11127.08 GB 256 0.4% bd4c37bc-07dd-488b-bfab-e74e32c26f6e rack1 What was wrong? please help. I could provide more information if you need. Thanks, Daning On Mon, Feb 4, 2013 at 9

Cassandra jmx stats ReadCount

2013-02-07 Thread Daning Wang
We have 8 nodes cluster in Casandra 1.1.0, with replication factor is 3. We found that when you just insert data, not only WriteCount increases, the ReadCount also increases. How could this happen? I am under the impression that readCount only counts the reads from client. Thanks, Daning

Upgrade to Cassandra 1.2

2013-02-02 Thread Daning Wang
the num_token http://www.datastax.com/docs/1.2/install/upgrading Is there other doc about this upgrade path? Thanks, Daning

Problem on node join the ring

2013-01-28 Thread Daning Wang
I add a new node to ring(version 1.1.6), after more than 30 hours, it is still in the 'Joining' state Address DC RackStatus State Load Effective-Ownership Token 141784319550391026443072753096570088105 10.28.78.123datacenter1 rack1 Up

1.2 Authentication

2013-01-28 Thread Daning Wang
exist - create it with CREATE USER query first) What does create it with CREATE USER query first mean? I put debug information in SimpleAuthenticator class, that showed authentication is passed in the authenticate() method. Thanks, Daning

Re: Replication factor

2012-05-23 Thread Daning Wang
Thanks guys. Aaron, I am confused about this. from wiki http://wiki.apache.org/cassandra/ReadRepair, looks for any consistency level. Read Repair will be done either before or after responding data. Read Repair does not run at CL ONE Daning On Wed, May 23, 2012 at 3:51 AM, Viktor Jevdokimov

Re: Couldn't find cfId

2012-05-16 Thread Daning Wang
Thanks Aaron! We will upgrade to 1.0.9. Just curious, you said removing the HintedHandoff files from data/system, what do the HintedHandoff files look like? Thanks, Daning On Wed, May 16, 2012 at 2:32 AM, aaron morton aa...@thelastpickle.comwrote: Looks like this https://issues.apache.org

Couldn't find cfId

2012-05-15 Thread Daning Wang
) at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:248) at org.apache.cassandra.db.HintedHandOffManager.access$200(HintedHandOffManager.java:84) at org.apache.cassandra.db.HintedHandOffManager$3.runMayThrow(HintedHandOffManager.java:418) Thanks, Daning

Re: Request timeout and host marked down

2012-04-10 Thread Daning Wang
Developer @aaronmorton http://www.thelastpickle.com On 10/04/2012, at 8:08 AM, Daning Wang wrote: Thanks Aaron! Here is the exception, is that the timeout between nodes? any parameter I can change to reduce timeout? me.prettyprint.hector.api.exceptions.HectorTransportException

Re: Request timeout and host marked down

2012-04-09 Thread Daning Wang
://www.thelastpickle.com On 6/04/2012, at 5:30 AM, Daning Wang wrote: Hi all, We are using Hector and ofter we see lots of timeout exception in the log, I know that the hector can failover to other node, but I want to reduce the number of timeouts. any hector parameter I should change to reduce

Request timeout and host marked down

2012-04-05 Thread Daning Wang
Hi all, We are using Hector and ofter we see lots of timeout exception in the log, I know that the hector can failover to other node, but I want to reduce the number of timeouts. any hector parameter I should change to reduce this error? also, on the server side, any kind of tunning need to do

Re: Cassandra Exception

2012-03-28 Thread Daning Wang
We upgraded to 1.0.8, and looks the problem is gone. Thanks for your help, Daning On Sun, Mar 25, 2012 at 9:54 AM, aaron morton aa...@thelastpickle.comwrote: Can you go to those nodes and run describe cluster ? Also check the logs on the machines that are marked as UNREACHABLE . A node

How to find CF from cfId

2012-03-22 Thread Daning Wang
:113) at org.apache.cassandra.db.ReadResponseSerializer.deserialize(ReadResponse.java:82) at org.apache.cassandra.service.AbstractRowResolver.preprocess(AbstractRowResolver.java:64) Daning

Re: Cassandra Exception

2012-03-22 Thread Daning Wang
16.67% 85070591730234615865843651857942052863 10.123.83.114 datacenter1 rack1 Up Normal 5.49 GB 16.67% 113427455640312821154458202477256070484 10.240.113.171 datacenter1 rack1 Up Normal 5.43 GB 16.67% 141784319550391026443072753096570088105 Daning On Thu, Mar 22, 2012

Cassandra Exception

2012-03-21 Thread Daning Wang
) at org.apache.cassandra.service.RowRepairResolver.resolve(RowRepairResolver.java:85) at org.apache.cassandra.service.AsyncRepairCallback$1.runMayThrow(AsyncRepairCallback.java:54) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30) ... 3 more Thank you in advance, Daning

Re: Cassandra Exception

2012-03-21 Thread Daning Wang
and we are on 0.8.6. On Wed, Mar 21, 2012 at 10:24 AM, Daning Wang dan...@netseer.com wrote: Hi All, We got lots of Exception in the log, and later the server crashed. any idea what is happening and how to fix it? ERROR [RequestResponseStage:4] 2012-03-21 04:16:30,482

hector connection pool

2012-03-05 Thread Daning Wang
. But I think client should recover after server load was down. Any bug reported about this? I did search but could not find one. Thanks, Daning

Re: Rebalance cluster

2012-01-12 Thread Daning Wang
Thank you guys. very appreciated. How about just pulling the slow machines out of cluster? I think the most of reads should already from fast machine right now because of dynamic snitch. so removing two machines should not add much loads on the remaining nodes. How do you think? Thanks, Daning

Rebalance cluster

2012-01-11 Thread Daning Wang
in advance, Daning

Pending on ReadStage

2012-01-06 Thread Daning Wang
0 REQUEST_RESPONSE 0 Thanks you in advance. Daning

Re: Pending on ReadStage

2012-01-06 Thread Daning Wang
Thanks for your reply. Nodes are equally balanced. and it is RandomPartitioner. I think that machine is slower, Are you saying it is IO issue? Daning On Fri, Jan 6, 2012 at 10:25 AM, Mohit Anchlia mohitanch...@gmail.comwrote: Are all your nodes equally balanced in terms of read requests

Cassandra memory usage

2012-01-03 Thread Daning Wang
11g *S 66.9 65.5 338:02.72 java Thank you in advance, Daning

TimedOutException()

2012-01-03 Thread Daning Wang
this? Your help is much appreciated. Daning

Re: Key count mismatch in cluster for a column family

2011-11-07 Thread Daning
Sylvain - We have similar problem but the discrepancy is not that big. Do we have to do major compaction to fix it? We did not do 'nodetool compact', just did repair regularly, which triggers minor compaction. Thanks, Daning On 10/26/2011 03:23 AM, Sylvain Lebresne wrote: The estimate

Restore snapshots suggestion

2011-10-13 Thread Daning
DoConsistencyChecksBoolean(http://www.datastax.com/dev/blog/whats-new-cassandra-066) to disable read repair, what is the proper way to do it? I am on 0.8.6. Thank you in advance, Daning

Re: Weird problem with empty CF

2011-10-04 Thread Daning
deleted data will be synced back. In this case my data will be processed twice and it will not be a big deal to me. Thanks, Daning On 10/04/2011 01:27 AM, aaron morton wrote: Yes that's the slice query skipping past the tombstone columns. Cheers - Aaron Morton Freelance

Re: Weird problem with empty CF

2011-10-04 Thread Daning
Thanks. Do you have plan to improve this? I think tombstone should be separated with live data since it serves different purpose, built in separate SSTable or indexed differently. It is pretty costly to do filtering while reading. Daning On 10/04/2011 01:34 PM, aaron morton wrote: I would

Re: Weird problem with empty CF

2011-10-03 Thread Daning Wang
@aaronmorton http://www.thelastpickle.com On 30/09/2011, at 3:27 AM, Daning Wang wrote: Jonathan/Aaron, Thank you guy's reply, I will change GCGracePeriod to 1 day to see what will happen. Is there a way to purge tombstones at anytime? because if tombstones affect performance, we want them

Weird problem with empty CF

2011-09-28 Thread Daning
, Daning

Re: Unable to create compaction marker

2011-09-22 Thread Daning
Thank you guys. I don't think have permission issue or out of space. might be ulimit set to low(1024), we will change ulimit -Hn and unlimit -Sn to see if that could solve the problem. Daning On 09/22/2011 01:12 AM, aaron morton wrote: It's in the yaml file… # directories where Cassandra

Lots of GC in log

2011-09-22 Thread Daning
GCInspector.java (line 122) GC for ParNew: 534 ms for 1 collections, 3308763856 used; max is 8015314944 Thanks in advance. Daning

Re: Lots of GC in log

2011-09-22 Thread Daning
Thanks Peter. I saw cpu was shooting much higher. I am not sure if frequent GCs are caused by improperly sized generations. I'd like to get some tunning tips, or good document about Cassandra tuning. Daning On 09/22/2011 12:23 PM, Peter Schuller wrote: We are testing Cassandra with pretty

Unable to create compaction marker

2011-09-21 Thread Daning
Thank you in advance. Daning

Re: Unable to create compaction marker

2011-09-21 Thread Daning
Thanks Jonathan. where is the data directory configured? since I did not any permission problem. Daning On 09/21/2011 11:01 AM, Jonathan Ellis wrote: Means Cassandra couldn't create an empty file in the data directory designating a sstable as compacted. I'd look for permissions problems

Queue suggestion in Cassandra

2011-09-16 Thread Daning Wang
for other designs. I want to know if anybody has implemented a large ordered queue successfully. Let me know if you have suggestion, Thank you in advance. Daning

ByteOrderedPartitioner

2011-09-16 Thread Daning Wang
data to all 4 nodes, that could avoid hot spot, right? Thank you in advance, Daning

Disable hector stats

2011-09-09 Thread Daning
Avg(ms) Min Max Std Dev 95th Count (PeriodicalLog.java:221) INFO Thread-53040 2011-09-09 13:24:03,291 ( We have tried to set log4j like this but that does not work, log4j.logger.com.ecyrd.speed4j.log.PeriodicalLog=ERROR Thanks, Daning

Query indexed column with key filter‏

2011-06-28 Thread Daning
work around for it? At this moment my workaound is to create a column which is exactly same as key. Thanks, Daning