Re: Cassandra seems slow when having many read operations

2017-07-22 Thread benjamin roth
creased heap to 24GB. do you use cms or g1gc? > did you change any other parameters? > As for the chunk size, we found change 64kb to 16kb didn't make a > difference in low key cache rate environment > > > > On Fri, Jul 21, 2017 at 9:27 PM, benjamin roth <brs...@gmail.com>

Re: Cassandra seems slow when having many read operations

2017-07-21 Thread benjamin roth
Apart from all that you can try to reduce the compression chunk size from the default 64kb to 16kb or even down to 4kb. This can help a lot if your read io on disk is very high and the page cache is not efficient. Am 21.07.2017 23:03 schrieb "Petrus Gomes" : > Thanks a lot to

Re: Corrupted commit log prevents Cassandra start

2017-07-07 Thread benjamin roth
Hi Hannu, I remember there have been discussions about this in the past. Most probably there is already a JIRA for this. I roughly remember a consense like that: - Default behaviour should remain - It should be configurable to the needs and preferences of the DBA - It should at least spit out

Re: Is it possible to repair a single partition.

2017-06-27 Thread benjamin roth
Then the partition is too big or has too many sstables that contain data for that partition so that the query times out. You can run a manual compaction on that table. That helped me several times. + I hope you are not trying to read that parition at once. Please use paging to query large

Re: Cassandra Cluster issues

2017-05-08 Thread benjamin roth
Hm that question is like "My car does not start - whats the problem?". You have to monitor, monitor, monitor, monitor. I'd strongly advice to graph as many metrics as you can. Read them from the JMX interface and write them to a TSDB, visualize them e.g. with Grafana. Then read logs, trace your

Re: TRUNCATE on a disk almost full - possible?

2017-04-21 Thread benjamin roth
Truncate needs no space. It just creates a hard link of all affected SSTables under the corresponding -SNAPSHOT dir (at least with default settings) and then removes the SSTables. Also this operation should be rather fast as it is mostly a file-deletion process with some metadata updates.

Re: WriteTimeoutException with LWT after few milliseconds

2017-04-19 Thread benjamin roth
Thanks, Jeff! As soon as I have some spare time I will try to reproduce and open a Jira for it. 2017-04-19 16:27 GMT+02:00 Jeff Jirsa <jji...@apache.org>: > > > On 2017-04-13 05:13 (-0700), benjamin roth <brs...@gmail.com> wrote: > > I found out that if the WTEs occur

Re: Counter performance

2017-04-17 Thread benjamin roth
Sending REQUEST_RESPONSE message to / > cassandra-01 [MessagingService-Outgoing-/ cassandra-01 ] | 2017-04-17 > 18:31:49.627000 | cassandra-05 | 3453 | cassandra-01 > > REQUEST_RESPONSE message received from / > cassandra-05 [Mes

RE: Counter performance

2017-04-17 Thread benjamin roth
r ways to log slow > queries in this version? Or, what do we expect with this log output? > > > > *From:* benjamin roth [mailto:brs...@gmail.com] > *Sent:* Monday, April 17, 2017 5:44 PM > *To:* user@cassandra.apache.org > *Subject:* RE: Counter performance > > > >

RE: Counter performance

2017-04-17 Thread benjamin roth
e little control over Usergrid-created data. *From:* benjamin roth [mailto:brs...@gmail.com] *Sent:* Monday, April 17, 2017 4:12 PM *To:* user@cassandra.apache.org *Subject:* Re: Counter performance Do you see difference when tracing the selects? 2017-04-17 13:36 GMT+02:00 Eren

Re: Counter performance

2017-04-17 Thread benjamin roth
changed them > to LeveledCompactionStrategy then. > > > > compaction = { 'class' : > 'org.apache.cassandra.db.compaction.LeveledCompactionStrategy', > 'sstable_size_in_mb' : 512 } > > > > *From:* benjamin roth [mailto:brs...@gmail.com] > *Sent:* Monday, April 17, 2017 12:12 PM > *To:*

Re: Counter performance

2017-04-17 Thread benjamin roth
Do you have a different compaction strategy on the counter tables? 2017-04-17 10:07 GMT+02:00 Eren Yilmaz : > We are using Cassandra (3.7) counter tables in our application, and there > are about 10 counter tables. The counter tables are in a separate keyspace > with

Re: hanging validation compaction

2017-04-13 Thread benjamin roth
sstables for that > particular column_family > > is it possible to track down which sstable of that cf is affected or > should i upload all of them? > > > br, > roland > > > On Thu, 2017-04-13 at 13:57 +0200, benjamin roth wrote: > > I think thats a good r

Re: WriteTimeoutException with LWT after few milliseconds

2017-04-13 Thread benjamin roth
gt; whats your consistency level for the insert? >> what if one ore more nodes are marked down and proper consistency cant be >> achieved? >> of course the error message does not indicate that problem (as it says >> its a timeout)... but in that case you would get an instant e

Re: hanging validation compaction

2017-04-13 Thread benjamin roth
.run(Unknown Source) > java.lang.Thread.run(Thread.java:745) > > > br, > roland > > > On Thu, 2017-04-13 at 10:04 +0000, Roland Otta wrote: > > i did 2 restarts before which did not help > > after that i have set for testing purposes file_cache_size_in_mb: 0 and > buf

Re: hanging validation compaction

2017-04-13 Thread benjamin roth
hausted: false and restarted again > > after that it worked ... but it also could be that it just worked by > accident after the last restart and is not related to my config changes > > On Thu, 2017-04-13 at 11:58 +0200, benjamin roth wrote: > > If you restart the server the same

Re: hanging validation compaction

2017-04-13 Thread benjamin roth
se/CASSANDRA > > On Thu, Apr 13, 2017 at 7:49 PM, benjamin roth <brs...@gmail.com> wrote: > > What I can tell you from that trace - given that this is the correct > thread and it really hangs there: > > The validation is stuck when reading from an SSTable. > Unfortunately

Re: force processing of pending hinted handoffs

2017-04-13 Thread benjamin roth
handoff handling in case it > hase been paused with the pausehandoff before. > i have tested it (resuming .. pausing & resuming) but it has no effect on > those old hints > > On Thu, 2017-04-13 at 10:27 +0200, benjamin roth wrote: > > There is a nodetool command to resume hints.

Re: force processing of pending hinted handoffs

2017-04-13 Thread benjamin roth
There is a nodetool command to resume hints. Maybe that helps? Am 13.04.2017 09:42 schrieb "Roland Otta" : > oh ... the operation is deprecated according to the docs ... > > > On Thu, 2017-04-13 at 07:40 +, Roland Otta wrote: > > i figured out that there is an mbean

Re: hanging validation compaction

2017-04-13 Thread benjamin roth
java.util.concurrent.FutureTask.run(FutureTask.java:266) > java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > org.apache.cassandra.concurrent.NamedThreadFactory. > lambda$threadLocalDeallocator$0(Na

Re: hanging validation compaction

2017-04-13 Thread benjamin roth
You should connect to the node with JConsole and see where the compaction thread is stuck 2017-04-13 8:34 GMT+02:00 Roland Otta : > hi, > > we have the following issue on our 3.10 development cluster. > > we are doing regular repairs with thelastpickle's fork of

Re: WriteTimeoutException with LWT after few milliseconds

2017-04-12 Thread benjamin roth
s not indicate that problem (as it says its > a timeout)... but in that case you would get an instant error for inserts. > wouldn't you? > > br, > roland > > > > On Wed, 2017-04-12 at 15:09 +0200, benjamin roth wrote: > > Hi folks, > > Can someone explain why that occur

WriteTimeoutException with LWT after few milliseconds

2017-04-12 Thread benjamin roth
Hi folks, Can someone explain why that occurs? Write timeout after 0.006s Query: 'INSERT INTO log_moment_import ("source", "reference", "user_id", "moment_id", "date", "finished") VALUES (3, '1305821272790495', 65675537, 0, '2017-04-12 13:00:51', NULL) IF NOT EXISTS Primary key and parition key

Re: Multiple nodes decommission

2017-04-11 Thread benjamin roth
I did not test it but I'd bet that parallel decommision will lead to inconsistencies. Each decommission results in range movements and range reassignments which becomes effective after a successful decommission. If you start several decommissions at once, I guess the calculated reassignments are

Re: Node always dieing

2017-04-06 Thread benjamin roth
> trickle_fsync_interval_in_kb=10240; truncate_request_timeout_in_ms=600; > unlogged_batch_across_partitions_warn_threshold=10; > user_defined_function_fail_timeout=1500; > user_defined_function_warn_timeout=500; > user_function_timeout_policy=die; windows_timer_interval=1; > wri

Re: Node always dieing

2017-04-06 Thread benjamin roth
= 1048575 net.ipv4.tcp_keepalive_time=600 On /etc/pam.d/su ... sessionrequired pam_limits.so ... Distro is the currently Ubuntu LTS. Thanks On 04/06/2017 10:39 AM, benjamin roth wrote: Cassandra cannot write an SSTable to disk. Are you sure the disk/volume where SSTables reside (normally /var

Re: Node always dieing

2017-04-06 Thread benjamin roth
Cassandra cannot write an SSTable to disk. Are you sure the disk/volume where SSTables reside (normally /var/lib/cassandra/data) is writeable for the CS user and has enough free space? The CDC warning also implies that. The other warnings indicate you are probably not running CS as root and you

Re: nodes are always out of sync

2017-04-02 Thread benjamin roth
Btw.: I created an issue for that some months ago https://issues.apache.org/jira/browse/CASSANDRA-12991 2017-04-01 22:25 GMT+02:00 Roland Otta <roland.o...@willhaben.at>: > thank you both chris and benjamin for taking time to clarify that. > > > On Sat, 2017-04-01 at 21:17 +0

Re: nodes are always out of sync

2017-04-01 Thread benjamin roth
Tl;Dr: there are race conditions in a repair and it is not trivial to fix them. So we rather stay with these race conditions. Actually they don't really hurt. The worst case is that ranges are repaired that don't really need a repair. Am 01.04.2017 21:14 schrieb "Chris Lohfink"

Re: nodes are always out of sync

2017-04-01 Thread benjamin roth
I think your way to communicate needs work. No one forces you to answer on questions. Am 01.04.2017 21:09 schrieb "daemeon reiydelle" : > What you are doing is correctly going to result in this, IF there is > substantial backlog/network/disk or whatever pressure. > > What do

Re: spikes in blocked native transport requests

2017-03-20 Thread benjamin roth
Did you check STW GCs? You can do that with 'nodetool gcstats', by looking at the gc.log or observing GC related JMX metrics. 2017-03-20 8:52 GMT+01:00 Roland Otta : > we have a datacenter which is currently used exlusively for spark batch > jobs. > > in case batch jobs

Re: Running cassandra

2017-03-19 Thread benjamin roth
You're welcome! 2017-03-19 18:41 GMT+01:00 Long Quanzheng <prc...@gmail.com>: > You are RIGHT! > It's working after I remove the env variable GREP_OPTIONS. > > Thanks! > > 2017-03-19 10:08 GMT-07:00 benjamin roth <brs...@gmail.com>: > >>

Re: Running cassandra

2017-03-19 Thread benjamin roth
I once had the same problem. In my case it was the coloured output of grep that injected ansi codes into the CS startup command. Am 19.03.2017 18:07 schrieb "Long Quanzheng" : > Hi > It still doesn't work. > > The real problem is this error: > > Error: Could not find or load

Re: repair performance

2017-03-17 Thread benjamin roth
b > threads: 1, ColumnFamilies: [], dataCenters: [ProdDC2], hosts: [], # of > ranges: 1758) > > 3.0.7 is also the reason why we are not using reaper ... as far as i could > figure out it's not compatible with 3.0+ > > > > On Fri, 2017-03-17 at 22:13 +0100, benjamin roth wrote: >

Re: repair performance

2017-03-17 Thread benjamin roth
It depends a lot ... - Repairs can be very slow, yes! (And unreliable, due to timeouts, outages, whatever) - You can use incremental repairs to speed things up for regular repairs - You can use "reaper" to schedule repairs and run them sliced, automated, failsafe The time repairs actually may

Re: scylladb

2017-03-13 Thread benjamin roth
@Dor,Jeff: I think Jeff pointed out an important fact: You cannot stop CS, swap binaries and start Scylla. To be honest that was AFAIR the only "Oooh :(" I had when reading the Scylla "marketing material". If that worked it would be very valuable from both Scylla's and a users' point of view. As

Re: scylladb

2017-03-11 Thread benjamin roth
There is no reason to be angry. This is progress. This is the circle of live. It happens anywhere at any time. Am 12.03.2017 07:34 schrieb "Dor Laor" : > On Sat, Mar 11, 2017 at 10:02 PM, Jeff Jirsa wrote: > >> >> >> On 2017-03-10 09:57 (-0800), Rakesh

Re: scylladb

2017-03-11 Thread benjamin roth
Why? Am 12.03.2017 07:02 schrieb "Jeff Jirsa" : > > > On 2017-03-10 09:57 (-0800), Rakesh Kumar wrote: > > Cassanda vs Scylla is a valid comparison because they both are > compatible. Scylla is a drop-in replacement for Cassandra. > > No, they aren't, and no, it isn't > > > > >

Re: scylladb

2017-03-11 Thread benjamin roth
Thanks a lot for your detailed explanation! I am very curious about the future development of Scylladb! Especially about mvs and lwt! Am 11.03.2017 02:05 schrieb "Dor Laor" : > On Fri, Mar 10, 2017 at 4:45 PM, Kant Kodali wrote: > >>

Re: Can I do point in time recover using nodetool

2017-03-08 Thread benjamin roth
I remember a very similar question on the list some months ago. The short answer is that there is no short answer. I'd recommend you search the mailing list archive for "backup" or "recover". 2017-03-08 10:17 GMT+01:00 Bhardwaj, Rahul : > Hi All, > > > > Is there any

Re: Limit on number of keyspaces/tables

2017-03-05 Thread benjamin roth
Why do you think 1 table consumes 1m?? Am 05.03.2017 20:36 schrieb "Vladimir Yudovin" : > Hi, > > there is no such hard limit, but each table consume at least 1M memory, so > 1000 tables takes at least 1G. > > Best regards, Vladimir Yudovin, > *Winguzone

Re: Limit on number of keyspaces/tables

2017-03-05 Thread benjamin roth
No seriously. Am 05.03.2017 2:54 nachm. schrieb "Rakesh Kumar" : > > I ask back: what's your intention > > May be documenting the limitations of Cassandra to show Oracle is better > :-) > > Am 05.03.2017 11:58 schrieb "Lata Kannan"

Re: Limit on number of keyspaces/tables

2017-03-05 Thread benjamin roth
I ask back: what's your intention Am 05.03.2017 11:58 schrieb "Lata Kannan" : > Hi > > I just wanted to check if there is any known limit to the number of > keyspaces one can create in a Cassandra cluster? Alternatively is there a > max on the number of tables that can be

Rebuild / removenode with MV is inconsistent

2017-03-01 Thread benjamin roth
Hi there, Today I come up with the following thesis: A rebuild / removenode may break the base-table <> MV contract. I'd even claim that a rebuild / removenode requires rebuilding all MVs to guarantee MV consistency. Reason: A node can have base tables with MVs. This is no problem. If these are

Re: Non-zero nodes are marked as down after restarting cassandra process

2017-03-01 Thread benjamin roth
You should always drain nodes before stopping the daemon whenever possible. This avoids commitlog replay on startup. This can take a while. But according to your description commit log replay seems not to be the cause. I once had a similar effect. Some nodes appeared down for some other nodes and

Re: Resources for fire drills

2017-03-01 Thread benjamin roth
@Doc: http://cassandra.apache.org/doc/latest/ is built from the git repo. So you can add documentation in doc/source and submit a patch. I personally think that is not the very best place or way to build a knowledge DB but thats what we have. 2017-03-01 13:39 GMT+01:00 Malte Pickhan

Re: Resources for fire drills

2017-03-01 Thread benjamin roth
the point. > > What I mean are some overview for basic scenarios for firedrills, so that > you can exercise them with your team. > > Best > > > On 1 Mar 2017, at 11:01, benjamin roth <brs...@gmail.com> wrote: > > Could you specify it a little bit? There are really a l

Re: Resources for fire drills

2017-03-01 Thread benjamin roth
can exercise them with your team. > > Best > > > On 1 Mar 2017, at 11:01, benjamin roth <brs...@gmail.com> wrote: > > Could you specify it a little bit? There are really a lot of things that > can go wrong. > > 2017-03-01 10:59 GMT+01:00 Malte Pickhan <malte

Re: Resources for fire drills

2017-03-01 Thread benjamin roth
Could you specify it a little bit? There are really a lot of things that can go wrong. 2017-03-01 10:59 GMT+01:00 Malte Pickhan : > Hi Cassandra users, > > I am looking for some resources/guides for firedrill scenarios with apache > cassandra. > > Do you know anything

Re: Is periodic manual repair necessary?

2017-02-28 Thread benjamin roth
for it. > > > > Among other things, this advice also gives an impression to people not > familiar with C* (e.g. me) that it is too fragile and needs substantial > manual intervention. > > > > Appreciate all the feedback and details that you have been sharing. > >

unsubscribe

2017-02-28 Thread Benjamin Roth
-- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: Is periodic manual repair necessary?

2017-02-27 Thread Benjamin Roth
’t care for it to be corrected as then the data will be > automatically purged because of TTL. > > > > In such a situation, do I need to have a periodic (weekly?) manual/batch > read_repair process? > > > > Thanks, > > Jayesh Thakrar > > -- Benjamin Roth Prokurist J

Re: Which compaction strategy when modeling a dumb set

2017-02-27 Thread Benjamin Roth
already been processed for deduplication. > > It works well enough, but I'm wondering which compaction strategy I should > use. I expect maybe 1% or less of events will end up duplicated (thus not > generating an insert), so the workload will probably be 50% writes 50% read. > &

Re: Understanding of proliferation of sstables during a repair

2017-02-26 Thread Benjamin Roth
Too many open files. Which is 100k by default and we had >40k sstables. Normally the are around 500-1000. Am 27.02.2017 02:40 schrieb "Seth Edwards" <s...@pubnub.com>: > This makes a lot more sense. What does TMOF stand for? > > On Sun, Feb 26, 2017 at 1:01 P

Re: Understanding of proliferation of sstables during a repair

2017-02-26 Thread Benjamin Roth
i-entropy mechanisms to achieve consistency instead. Is > this generally true? > > > Thanks! > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: High disk io read load

2017-02-24 Thread Benjamin Roth
It was only the schema change. 2017-02-24 19:18 GMT+01:00 kurt greaves <k...@instaclustr.com>: > How many CFs are we talking about here? Also, did the script also kick off > the scrubs or was this purely from changing the schemas? > ​ > -- Benjamin Roth Prokurist Jaumo Gm

Re: Does C* coordinator writes to replicas in same order or different order?

2017-02-21 Thread Benjamin Roth
ynchronous replication for each update." > > http://cassandra.apache.org/ > > 2. Is it Guaranteed that C* coordinator writes data in the same order to > all the replicas (either sync or async)? > > Thanks, > kant > > On Tue, Feb 21, 2017 at 1:23 AM, Benjamin R

Re: Does C* coordinator writes to replicas in same order or different order?

2017-02-21 Thread Benjamin Roth
ly for record 1 and record 2 are they guaranteed to be >>>>> inserted in the same order across replicas? (Whatever order coordinator >>>>> may >>>>> choose is fine but I want the same order across all replicas and with >>>>> async >>&g

Re: High disk io read load

2017-02-20 Thread Benjamin Roth
Hah! Found the problem! After setting read_ahead to 0 and compression chunk size to 4kb on all CFs, the situation was PERFECT (nearly, please see below)! I scrubbed some CFs but not the whole dataset, yet. I knew it was not too few RAM. Some stats: - Latency of a quite large CF:

Re: Count(*) is not working

2017-02-20 Thread Benjamin Roth
and 1423 tombstone cells for query SELECT * FROM >>>>> keysace.table WHERE token(id) > token(test:ODP0144-0883E-022R-002/047-052) >>>>> LIMIT 100 (see tombstone_warn_threshold) >>>>> >>>>> Read 100 live rows and 1435 tombstone cells for query SELECT * FROM >>>>> keysace.table WHERE token(id) > token(test:2565-AMK-2) LIMIT 100 (see >>>>> tombstone_warn_threshold) >>>>> >>>>> Read 96 live rows and 1385 tombstone cells for query SELECT * FROM >>>>> keysace.table WHERE token(id) > token(test:-2220-UV033/04) LIMIT 100 (see >>>>> tombstone_warn_threshold). >>>>> >>>>> >>>>> >>>>> >>>>> Can you please help me to get the total count of the table. >>>>> >>>>> -- >>>>> Selvam Raman >>>>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து" >>>>> >>>>> >>>> >>>> >>>> -- >>>> Selvam Raman >>>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து" >>>> >>>> >>>> >> >> > > -- > Sorry this was sent from mobile. Will do less grammar and spell check than > usual. > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: Cassandra blob vs base64 text

2017-02-20 Thread Benjamin Roth
You could save space when storing your data (base64-)decoded as blobs. 2017-02-20 13:38 GMT+01:00 Oskar Kjellin : > We currently have some cases where we store base64 as a text field instead > of a blob (running version 2.0.17). > I would like to move these to blob but

Re: High disk io read load

2017-02-19 Thread Benjamin Roth
ced. > > Regards, > > On Sun, Feb 19, 2017 at 7:47 PM, Edward Capriolo <edlinuxg...@gmail.com> > wrote: > >> >> >> On Sat, Feb 18, 2017 at 3:35 PM, Benjamin Roth <benjamin.r...@jaumo.com> >> wrote: >> >>> We are talking about a read

Re: High disk io read load

2017-02-18 Thread Benjamin Roth
started up and accepts requests. I also diffed sysctl of the both machines. No significant differences. Only CPU-related, random values and some hashes differ. 2017-02-18 21:49 GMT+01:00 Benjamin Roth <benjamin.r...@jaumo.com>: > 256 tokens: > > root@cas9:/sys/block/dm-0# blockdev --re

Re: High disk io read load

2017-02-18 Thread Benjamin Roth
--report for a normal node and 512 token > node. > > Regards, > > On Sun, Feb 19, 2017 at 2:07 AM, Benjamin Roth <benjamin.r...@jaumo.com> > wrote: > >> cat /sys/block/sda/queue/read_ahead_kb >> => 8 >> >> On all CS nodes. Is that what you mea

Re: High disk io read load

2017-02-18 Thread Benjamin Roth
cat /sys/block/sda/queue/read_ahead_kb => 8 On all CS nodes. Is that what you mean? 2017-02-18 21:32 GMT+01:00 Bhuvan Rawal <bhu1ra...@gmail.com>: > Hi Benjamin, > > What is the disk read ahead on both nodes? > > Regards, > Bhuvan > > On Sun, Feb 19, 2017 at 1

Re: High disk io read load

2017-02-18 Thread Benjamin Roth
red with puppet, they share the same roles and no manual "optimizations" are applied. So I can't imagine, a different configuration is responsible for it. 2017-02-18 21:28 GMT+01:00 Benjamin Roth <benjamin.r...@jaumo.com>: > This is status of the largest KS of these both nodes: > UN

Re: High disk io read load

2017-02-18 Thread Benjamin Roth
greaves <k...@instaclustr.com>: > what's the Owns % for the relevant keyspace from nodetool status? > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Di

Re: sasi index question (read timeout on many selects)

2017-02-17 Thread Benjamin Roth
void them is based on the issues that have been filed in > Jira. Benjamin Roth is one of the only people talking about his MV usage, > and has filed a few JIRAs discussing their problems when bootstrapping new > nodes, as well as issues repairing. > > https://issues.apache.org/jira/brows

Re: High disk io read load

2017-02-17 Thread Benjamin Roth
gt; Wellington, NZ > @zznate > > CTO > Apache Cassandra Consulting > http://www.thelastpickle.com > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: sasi index question (read timeout on many selects)

2017-02-16 Thread Benjamin Roth
tasymail.de> wrote: > >> >> >> On 16.02.2017 14:30, DuyHai Doan wrote: >> > Why indexing BLOB data ? It does not make any sense >> >> My partition key is a secure hash sum, I don't index a blob. >> >> >> >> >> > -- Benjamin Roth Proku

Re: High disk io read load

2017-02-15 Thread Benjamin Roth
Erm sorry, forgot to mention. In this case "cas10" is Node A with 512 tokens and "cas9" Node B with 256 tokens. 2017-02-16 6:38 GMT+01:00 Benjamin Roth <benjamin.r...@jaumo.com>: > It doesn't really look like that: > https://cl.ly/2c3Z1u2k0u2I > > Thats t

Re: High disk io read load

2017-02-15 Thread Benjamin Roth
is double the data. More read > repair chances. More load can swing it's way during node failures etc. > > On Wednesday, February 15, 2017, Benjamin Roth <benjamin.r...@jaumo.com> > wrote: > >> Hi there, >> >> Following situation in cluster with 10 nodes: &

High disk io read load

2017-02-15 Thread Benjamin Roth
due to the wait states. Has anybody experienced similar situations? Any hints how to analyze or optimize this - I mean 128GB cache for 460GB payload is not that few. I am pretty sure that not the whole dataset of 460GB is "hot". -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Weh

Re: How does cassandra achieve Linearizability?

2017-02-10 Thread Benjamin Roth
clock drift as much as >>>> > possible, but public NTP servers are good enough for most >>>> > uses. https://www.instaclustr.com/bl >>>> og/2015/11/05/apache-cassandra-synchronization/ >>>> > <https://www.instaclustr.com/blog/20

Re: cassandra user request log

2017-02-10 Thread Benjamin Roth
If you want to audit write operations only, you could maybe use CDC, this is a quite new feature in 3.x (I think it was introduced in 3.9 or 3.10) 2017-02-10 10:10 GMT+01:00 vincent gromakowski < vincent.gromakow...@gmail.com>: > tx > > 2017-02-10 10:01 GMT+01:00 Benjamin R

Re: cassandra user request log

2017-02-10 Thread Benjamin Roth
ta. Maybe > there is another way ? > > 2017-02-10 9:46 GMT+01:00 Benjamin Roth <benjamin.r...@jaumo.com>: > >> On a cluster with just a little bit load, that would cause zillions of >> petabytes of logs (just roughly ;)). I don't think this is viable. >> There are many many JMX

Re: cassandra user request log

2017-02-10 Thread Benjamin Roth
-10 9:42 GMT+01:00 vincent gromakowski <vincent.gromakow...@gmail.com >: > Hi all, > Is there any way to trace user activity at the server level to see which > user is accessing which data ? Do you thin it would be simple to implement ? > Tx > -- Benjamin Roth

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Benjamin Roth
Ok now I REALLY got it :) Thanks Sylvain! 2017-02-09 11:42 GMT+01:00 Sylvain Lebresne <sylv...@datastax.com>: > On Thu, Feb 9, 2017 at 10:52 AM, Benjamin Roth <benjamin.r...@jaumo.com> > wrote: > >> Ok got it. >> >> But it's interesting that this is suppo

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Benjamin Roth
ed batches. One of my colleagues did quite a bit of > benchmarking on this topic for his talk at last year’s C* summit: > http://www.slideshare.net/DataStax/microbatching- > highperformance-writes-adam-zegelin-instaclustr-cassandra-summit-2016 > > On Thu, 9 Feb 2017 at 20:52 Benjamin

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Benjamin Roth
t > around with a batch if you need atomicity. > > Which is not saying it will never be and shouldn't be supported btw, there > is something to be said for the consistency of the CQL language in general. > But it's why no-one took time to do it so far. > > On Thu, Feb 9, 2017 at 1

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Benjamin Roth
tatements > with one statement per PK tuple would be roughly equivalent? And probably > no more complex to generate in the client? > > On Thu, 9 Feb 2017 at 20:22 Benjamin Roth <benjamin.r...@jaumo.com> wrote: > >> Maybe that makes it clear: >> >> DELETE FROM ks.c

Re: DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Benjamin Roth
; Are you looking this to be equivalent to (PK1=1 AND PK2=2) or are you > looking for (PK1 IN (1,2) AND PK2 IN (1,2)) or something else? > > Cheers > Ben > > On Thu, 9 Feb 2017 at 20:09 Benjamin Roth <benjamin.r...@jaumo.com> wrote: > >> Hi Guys, >> >> CQL s

DELETE/SELECT with multi-column PK and IN

2017-02-09 Thread Benjamin Roth
to have it for daily business, IMHO it's a waste of resources to run multiple queries just to fetch a bunch of records by a PK. Thanks in advance for any reply -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880

Re: Why does CockroachDB github website say Cassandra has no Availability on datacenter failure?

2017-02-07 Thread Benjamin Roth
Ask for forgiveness not for permission if you do marketing ;) Am 07.02.2017 13:11 schrieb "Kant Kodali" : > lol. But seriously are they even allowed to say something that is not true > about another product ? > > On Tue, Feb 7, 2017 at 4:05 AM, kurt greaves

Re: CS process killed by kernel OOM

2017-02-06 Thread Benjamin Roth
; > > I do recommend reporting it, it will help others avoid encountering the > same problem if it gets fixed. > > On 02/06/2017 03:07 PM, Benjamin Roth wrote: > > Thanks for the reply. We got rid of the OOMs by increasing > vm.min_free_kbytes, it's default of approx 90mb is maybe

Re: CS process killed by kernel OOM

2017-02-06 Thread Benjamin Roth
or not. It also was my first thought but in the end the main thing is, it works again and it does with more mibn_free_kbytes 2017-02-06 11:53 GMT+01:00 Avi Kivity <a...@scylladb.com>: > > On 01/26/2017 07:36 AM, Benjamin Roth wrote: > > Hi there, > > We installed 2 new nodes these

RE: Is it possible to have a column which can hold any data type (for inserting as json)

2017-02-01 Thread Benjamin Roth
02.02.2017 05:30 schrieb "Rajeswari Menon" <rajeswar...@thinkpalm.com>: > Yes. Is there any way to define value to accept any data type as the json > value data may vary? Or is there any way to do the same without defining a > schema? > > > > Regards, > &

RE: Is it possible to have a column which can hold any data type (for inserting as json)

2017-02-01 Thread Benjamin Roth
; > "id": 1, > >"address":"", > >"datatype":"DOUBLE", > >"name":"Longitude", > >"attributes":{ > > "ID":"1" > >}, > >"cate

Re: Is it possible to have a column which can hold any data type (for inserting as json)

2017-01-31 Thread Benjamin Roth
gt; ); > > > > When I try to insert the data as JSON I got the error : *Error decoding > JSON value for value: Expected a UTF-8 string, but got a Double: 1.390692*. > The message is clear that a double value cannot be inserted to text column. > The real issue is that the value ca

Re: Time series data model and tombstones

2017-01-28 Thread Benjamin Roth
Maybe trace your queries to see what's happening in detail. Am 28.01.2017 21:32 schrieb "John Sanda" : Thanks for the response. This version of the code is using STCS. gc_grace_seconds was set to one day and then I changed it to zero since RF = 1. I understand that expired

Re: Disc size for cluster

2017-01-26 Thread Benjamin Roth
F of 3 and SizeTiered Compaction Strategy. > Is it correct, that SizeTiered Compaction Strategy needs (in the worst > case) 50% free disc space during compaction? > > So this would then result in a cluster of 3TB x 3 x 2 == 18 TB of raw > storage? > > Thanks and Regards > Raphael

CS process killed by kernel OOM

2017-01-25 Thread Benjamin Roth
anyone encountered sth like this before? -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: [Multi DC] Old Data Not syncing from Existing cluster to new Cluster

2017-01-24 Thread Benjamin Roth
es of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > > *From:* Benjamin Roth [mailto:benjamin.r...@jaumo.com] > *Sent:* Tues

Re: [Multi DC] Old Data Not syncing from Existing cluster to new Cluster

2017-01-24 Thread Benjamin Roth
Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > > *From:* Benjamin Roth [mailto:benjamin.r...@jaumo.com] > *Sent:* Tuesday, January 24, 2017 8:55 PM > *To:*

Re: [Multi DC] Old Data Not syncing from Existing cluster to new Cluster

2017-01-24 Thread Benjamin Roth
> > > > *Thanks & Regards,* > *Abhishek Kumar Maheshwari* > *+91- 805591 <+91%208%2005591> (Mobile)* > > Times Internet Ltd. | A Times of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not prin

Re: Huge size of system.batches table after dropping an incomplete Materialized View

2017-01-23 Thread Benjamin Roth
was created >> and a rolling restart to stop these errors. >> >> Now I see huge size of system.batches table on one of the nodes. It seems >> related to issues mentioned above since last modification timestamps of the >> sstable files inside system/batches is same as wh

Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down.

2017-01-23 Thread Benjamin Roth
> FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > > *From:* Benjamin Roth [mailto:benjamin.r...@jaumo.com] > *Sent:* Monday, January 23, 2017 5:05 PM > *T

Re: Getting Error while Writing in Multi DC mode when Remote Dc is Down.

2017-01-23 Thread Benjamin Roth
08%2005591> (Mobile)* > > Times Internet Ltd. | A Times of India Group Company > > FC - 6, Sector 16A, Film City, Noida, U.P. 201301 | INDIA > > *P** Please do not print this email unless it is absolutely necessary. > Spread environmental awareness.* > > > We th

Re: Huge size of system.batches table after dropping an incomplete Materialized View

2017-01-22 Thread Benjamin Roth
as when I tried to drop the MV. > > Some insight and suggestions regarding it will be very helpful. I will > like to know if i can safely truncate the table, rm the files or any other > approach to clean it up? > > Thanks. > -- Benjamin Roth Prokurist Jaumo GmbH · www.jaumo.com Wehrstraße 46 · 73035 Göppingen · Germany Phone +49 7161 304880-6 · Fax +49 7161 304880-1 AG Ulm · HRB 731058 · Managing Director: Jens Kammerer

Re: parallel processing - splitting data

2017-01-19 Thread Benjamin Roth
different, so Im not sure how to > do the split. > > e.g. from node 1 > > Token ranges - start:-5144720537407094184 end:-5129226025397315327 > > This token range isn't returned by node 2, 3 or 4. > > Thanks again > > Frank > > On 19 January 2017 at 12:19, Benj

  1   2   >