Re: Need help with simple schema for time-series

Use a timestamp instead of 2 separate fields and you can query on the range. CREATE TABLE mytable ( sensorname text, reading_time timestamp, data MAP, PRIMARY KEY (sensorname, reading_time) ); On Fri, Aug 26, 2016 at 8:17 PM Peter Figliozzi

Re: Need help with simple schema for time-series

Ah, i see what you're looking for. No, my schema wouldn't work for that. I had read through your question a little quickly. In cassandra 3.5 support was added for more flexible ALLOW FILTERING statements. Here's an example: CREATE TABLE mytable ( sensorname text, date date, time

Re: Need help with simple schema for time-series

Thanks, guys, for your help. I tried the filtering method and it works great. Sincerely, Pete On Sat, Aug 27, 2016 at 12:36 AM, Jonathan Haddad wrote: > Ah, i see what you're looking for. No, my schema wouldn't work for that. > I had read through your question a little

Need help with simple schema for time-series

I have data from many sensors as time-series: - Sensor name - Date - Time - value I want to query windows of both date and time. For example, 8am - 9am from Aug. 1st to Aug 10th. Here's what I did: CREATE TABLE mykeyspace.mytable ( sensorname text, date date, time

Re: Need help with simple schema for time-series

To do 8-9am on Aug-1 through Aug-10, you’d likely need to do either multiple queries in parallel (fire off async), or use some clever IN logic. Or, you’d need to break your table up so the first clustering key is the hour of the day, and then you could do this: CREATE TABLE mytable (

Re: Need help with simple schema for time-series

I don't believe that would let me query a time of day range, over a date range, would it? For example, between 8am and 9am, August 1st through August 10th. On Fri, Aug 26, 2016 at 11:52 PM, Jonathan Haddad wrote: > Use a timestamp instead of 2 separate fields and you can

Re: How to start using incremental repairs?

After running some tests I can confirm that using -pr leaves unrepaired SSTables, while removing it shows repaired SSTables only once repair is completed. The purpose of -pr was to lighten the repair process by not repairing ranges RF times, but just once. With incremental repair though, repaired

Re: How to start using incremental repairs?

> What is the underlying reason? Basically to minimize the amount of anti-compaction needed, since with RF=3 you'd need to perform anti-compaction 3 times in a particular node to get it fully repaired, while without it you can just repair the full node's range in one run. Assuming you run repair

Re: How to start using incremental repairs?

> I must admit that I fail to understand currently how running repair with -pr could leave unrepaired data though, even when ran on all nodes in all DCs, and how that could be specific to incremental repair (and would appreciate if someone shared the explanation). Anti-compaction, which marks

Re: Guidelines for configuring Thresholds for Cassandra metrics

Forgot the most important thing. LogsERROR you should investigateWARN you should have a list of known ones. Use case dependent. Ideally you change configuration accordingly.*PoolCleaner (slab or native) - good indication node is tuned badly if you see a ton of this. Set

Re: How to start using incremental repairs?

I see. Didn't think about it that way. Thanks for clarifying! On Fri, Aug 26, 2016 at 2:14 PM, Paulo Motta wrote: > > What is the underlying reason? > > Basically to minimize the amount of anti-compaction needed, since with > RF=3 you'd need to perform anti-compaction

Re: Guidelines for configuring Thresholds for Cassandra metrics

Thomas, Not all metrics are KPIs and are only useful when researching a specific issue or after a use case specific threshold has been set. The main "canaries" I monitor are:* Pending compactions (dependent on the compaction strategy chosen but 1000 is a sign of severe issues in all cases)*

Re: How to start using incremental repairs?

Hi Paulo, could you elaborate on 2? I didn't know incremental repairs were not compatible with -pr What is the underlying reason? Regards, Stefano On Fri, Aug 26, 2016 at 1:25 AM, Paulo Motta wrote: > 1. Migration procedure is no longer necessary after CASSANDRA-8004,

Re: How to start using incremental repairs?

An extract of this conversation should definitely be posted somewhere. Read a lot but never learnt all these bits... On Fri, Aug 26, 2016 at 2:53 PM, Paulo Motta wrote: > > I must admit that I fail to understand currently how running repair with > -pr could leave

Re: Guidelines for configuring Thresholds for Cassandra metrics

The default when I wrote it was 0.4 but it was found this did not saturate flush writers in JBOD configurations. Iirc it now defaults to 1/(1+#disks) which is not a terrible default, but obviously comes out much lower if you have many disks. This smaller value behaves better for peak performance,

Re: Flush activity and dropped messages

Hi Benedict, This makes sense now. Thank you very much for your input. Regards, Vasilis On 25 Aug 2016 10:30 am, "Benedict Elliott Smith" wrote: > You should update from 2.0 to avoid this behaviour, is the simple answer. > You are correct that when the commit log gets

Re: Flush activity and dropped messages

Hi Patrick and thanks for your reply, We are monitoring disk usage and more and we don't seem to be running out of space at the moment. We have separate partitions/disks for commitlog/data. Which one do you suspect and why? Regards, Vasilis On 25 Aug 2016 4:01 pm, "Patrick McFadin"

Re: Flush activity and dropped messages

It's not that your disks are getting full. I suspect you don't have enough throughput to handle the type of stress compaction and memtable flushing produce. Blocked flush writers is almost always a disk problem. Any storage with the words SAN, NAS, NFS or SATA in them, is going to make your life

Re: How to start using incremental repairs?

Thanks for confirmation Paulo. Then my understanding of proccess was correct. I'm curious why I still see unrepaired sstables after performing repair -pr on all nodes in all datacenters... пт, 26 Авг 2016 г., 3:25 Paulo Motta : > 1. Migration procedure is no longer

Re: How to start using incremental repairs?

There are 2 main reasons I see for still having unrepaired sstables after running nodetool repair -pr : 1- new data is still flowing in your database after the repair sessions were launched, and thus hasn't been repaired 2- some repair sessions failed and left unrepaired data on your nodes.

Re: How to start using incremental repairs?

Thanks Alexander. 1. No reads and writes were enabled during repair on keyspace. 2. All repairs were started sequentially on all nodes one by one (new repair started after repair completeon on previous node) I'll dig deeper into the logs, maybe there are some records about reason why some

Re: Stale value appears after consecutive TRUNCATE

Hi Christian, C* 2.2.7 doesn't cause this problem. I can always reproduce it on some servers and my laptop by using 2.2.6. I reviewed the source code of 2.2.7. The above ReplayPosition updating was fixed. Thank you for your cooperation. yuji On Thu, Aug 25, 2016 at 11:40 PM, horschi