Re: Manual Indexing With Buckets

2015-07-24 Thread Anuj Wadehra
Can anyone take this one? Thanks Anuj Sent from Yahoo Mail on Android From:Anuj Wadehra anujw_2...@yahoo.co.in Date:Thu, 23 Jul, 2015 at 10:57 pm Subject:Manual Indexing With Buckets We have a primary table and we need search capability by batchid column. So we are creating a manual index

Re: Schema questions for data structures with recently-modified access patterns

2015-07-24 Thread Robert Wille
When performing an update, the following needs to happen: 1. Read document.last_modified 2. Get the current timestamp 3. Update document with last_modified=current timestamp 4. Insert into doc_by_last_modified with last_modified=current timestamp 5. Delete from doc_by_last_modified with

Re: Reduced write performance when reading

2015-07-24 Thread Soerian Lieve
I was on CFQ so I changed it to noop. The problem still persisted however. Do you have any other ideas? On Thu, Jul 23, 2015 at 5:00 PM, Jeff Ferland j...@tubularlabs.com wrote: Imbalanced disk use is ok in itself. It’s only saturated throughput that’s harmful. RAID 0 does give more consistent

Compaction no longer working properly

2015-07-24 Thread Robert Wille
I have a database which has a fair amount of churn. When I need to update a data structure, I create a new one, and when it is complete, I delete the old one. I have gc_grace_seconds=0, so the space for the old data structures should be reclaimed on the next compaction. This has been working

Re: Nodetool cleanup takes long time and no progress

2015-07-24 Thread Jeff Jirsa
You can check for progress using `nodetool compactionstats` (which will show Cleanup tasks), or check for ‘Cleaned up’ messages in the log (/var/log/cassandra/system.log). However, `nodetool cleanup` has a very specific and limited task - it deletes data no longer owned by the node, typically

Nodetool cleanup takes long time and no progress

2015-07-24 Thread rock zhang
Hi All, After I added node, I run node tool cleanup on the old notes , but it takes forever, no error message, and I don't see space are freed. What should I do ? Repair first ? Thanks Rock

Re: Nodetool cleanup takes long time and no progress

2015-07-24 Thread rock zhang
Hi Jeff, It already 2 hours, only progress is 6%, seems it is very slow. Is there any way to speedup ? If I interrupted the process, what gonna happen ? Next time it just compact again, right ? I think by default setting is the compaction occurs every day . Thanks Rock On Jul 24, 2015,

Re: Nodetool cleanup takes long time and no progress

2015-07-24 Thread Robert Coli
On Fri, Jul 24, 2015 at 5:03 PM, rock zhang r...@alohar.com wrote: It already 2 hours, only progress is 6%, seems it is very slow. Is there any way to speedup ? Cleanup is a type of compaction; it obeys the compaction throttle. If I interrupted the process, what gonna happen ? Next time

Re: Nodetool cleanup takes long time and no progress

2015-07-24 Thread rock zhang
Thank you Jeff. I just added one more node, so i want to delete moved tokens. ubuntu@ip-172-31-30-145:~$ nodetool compactionstats pending tasks: 1413 compaction type keyspace table completed totalunit progress Cleanuprawdata raw_data 25817918778

RE: Manual Indexing With Buckets

2015-07-24 Thread SEAN_R_DURITY
It is a bit hard to follow. Perhaps you could include your proposed schema (annotated with your size predictions) to spur more discussion. To me, it sounds a bit convoluted. Why is a batch so big (up to 100 million rows)? Is a row in the primary only associated with one batch? Sean Durity -