Re: stress testing & lab provisioning tools

Hey Jon, It's awesome to see that you're reviving both these projects! I was eager to get my hands on an updated version of tlp-cluster with up to date AMIs  tlp-stress is by far the best Cassandra stress tool I've worked with, and I recommend everyone to test easy-cass-stress and build

Re: Switching to Incremental Repair

Hi Sebastian, That's a feature we need to implement in Reaper. I think disallowing the start of the new incremental repair would be easier to manage than pausing the full repair that's already running. It's also what I think I'd expect as a user. I'll create an issue to track this. Le sam. 3

Re: state of incremental repairs in cassandra 3.x

Hi James, I'd recommend to upgrade to 4.0.1 if you intend to use incremental repair. The changes from CASSANDRA-9143 are massive and couldn't be backported to the 3.11 branch. When moving to incremental, and in order to limit anticompaction

Re: Backup cassandra and restore. Best practices

Yes, Minio is supported by Medusa through the S3 compatible backend. I reckon we need to update the docs with a guide on setting up those backends, but it's pretty much the same as ceph s3 rgw in configuring your medusa.ini : - use s3_compatible as storage backend - set the host, port and region

Re: Anti Compactions while running repair

gt; Manish > > On Mon, Nov 9, 2020 at 11:17 AM Alexander DEJANOVSKI < > adejanov...@gmail.com> wrote: > >> Hi Manish, >> >> Anticompaction is the same whether you run full or incremental repair. >> >> >> Le ven. 6 nov. 2020 à 04:37, manish khan

Re: Issue with anti-compaction while running full repair with -pr option

Hi, You have two options to disable anticompaction when running full repair: - add the list of DCs using the --dc flag (even if there's just a single DC in your cluster) - Use subrange repair, which is done by tools such as Reaper (it can be challenging to do it yourself on a vnode cluster).

Re: Anti Compactions while running repair

Hi Manish, Anticompaction is the same whether you run full or incremental repair. Le ven. 6 nov. 2020 à 04:37, manish khandelwal a écrit : > In documentation it is given that while running incremental repairs, anti > compaction is done which results in repaired and unrepaired sstables. Since

Re: Tool for schema upgrades

I second Alex's recommendation. We use https://github.com/patka/cassandra-migration to manage schema migrations in Reaper and it has a consensus feature to prevent concurrent migrations from clashing. Cheers, Alex Le jeu. 8 oct. 2020 à 19:10, Alex Ott a écrit : > Hi > > Look at

Re: How to predict time to complete for nodetool repair

Also Reaper will skip the anticompaction phase which you might be going through with nodetool (depending on your version of Cassandra). That'll reduce the overall time spent on repair and will remove some compaction pressure. But as Erick said, unless you have past repairs to rely on and a stable

Re: How to elect a normal node to a seed node

tstrap, so they won't stream data in on their first start. Aside from that, any node can become a seed node at anytime. Just update the seed list on all nodes, roll restart the cluster and you'll have a new set of seed nodes. - Alexander Dejanovski France @alexanderdeja Consultant Apache

Re: What is "will be anticompacted on range" ?

Hi, Full repair triggers anticompaction as well. Only subrange repair doesn't trigger anticompaction, and in 4.0, AFAIK, full repairs won't involve anticompaction anymore. Cheers, Le lun. 10 févr. 2020 à 19:17, Krish Donald a écrit : > Thanks Jeff, But we are running repair using below

Cassandra Reaper 2.0 was released

<http://cassandra-reaper.io/>. Cheers, - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: execute is faster than execute_async?

you an idea on how this can be implemented. Sending hundreds/thousands of queries without waiting for a result will DDoS the cluster, so you should always implement some throttling. Cheers, ----- Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting h

Medusa : a new OSS backup/restore tool for Apache Cassandra

, in place restores (same cluster) and remote restores (remote cluster) whether or not the topologies match or not. More details in our latest blog post : https://thelastpickle.com/blog/2019/11/05/cassandra-medusa-backup-tool-is-open-source.html Hope you'll enjoy using it, - Alexander

Re: about remaining data after adding a node

the sstables are fully expired. Cheers, - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com On Thu, Sep 5, 2019 at 11:33 AM Eunsu Kim wrote: > Thank you for your response. > > > > I’m using TimeWindowCom

Re: Rebuilding a node without clients hitting it

t shows up as down, it will rely on hints to get the writes. If it shows as joining, it will get the writes while streaming is ongoing. Cheers, - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com On Tue, Aug 6, 2019 a

Re: What really happened during repair?

Hi Jeff, Anticompaction only runs before repair in the upcoming 4.0. In all other versions of Cassandra, it runs at the end of repair sessions. My understanding from other messages Martin sent to the ML was that he was already running full repair not incremental, which before 4.0 will also

Re: Repair failed and crash the node, how to bring it back?

3.11.4 IMHO as 3.0 offers less performance than 3.11) and use Reaper <http://cassandra-reaper.io/> to handle/schedule repairs. Cheers, - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com On Thu, Aug 1, 2019 at

Re: Repair failed and crash the node, how to bring it back?

)_/¯ - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com On Wed, Jul 31, 2019 at 3:51 PM Martin Xue wrote: > Hi, > > I am running repair on production, started with one of 6 nodes in the > cluster (3 nodes in ea

Re: Repair / compaction for 6 nodes, 2 DC cluster

es as unrepaired, restart Cassandra). Cheers, ----- Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com On Wed, Jul 31, 2019 at 3:53 PM Martin Xue wrote: > Sorry ASAD, don't have chance, still bogged down with the

Re: Tombstones not getting purged

y reported by the docs as an "intensive process" (not more than a repair though). ----- Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com On Thu, Jun 20, 2019 at 9:17 AM Alexander Dejanovski wrote: > My bad on dat

Re: Tombstones not getting purged

are sticking around. Your best shot here will be a major compaction of that table, since it doesn't seem so big. Remember to use the --split-output flag on the compaction command to avoid ending up with a single SSTable after that. Cheers, - Alexander Dejanovski France @alexanderdeja

Re: Tombstones not getting purged

e to be purged is that there is no SSTable outside the compaction that would possibly contain the partition and that would have older timestamps. Is this a followup on your previous issue where you were trying to perform a major compaction on an LCS table? - Alexander Dejanovski France @ale

Re: Speed up compaction

mance in ways I cannot predict, and should be attempted only if you really need to perform this major compaction and cannot wait for it to go through at the current pace. Cheers, - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpic

Re: TWCS and tombstone purging

> Hey guys, > > > > Can someone give me some idea or link some good material for determining a > good / aggressive tombstone strategy? I want to make sure my tombstones are > getting purged as soon as possible to reclaim disk. > > > > Thanks > -- ---

Re: Changing existing Cassandra cluster from single rack configuration to multi racks configuration

y high in comparison to other nodes in > rac1. > > So until both racks have equal number of nodes and we run nodetool cleaup, > the data will not be equally distributed. > > > > > > On Wed, Mar 6, 2019 at 5:50 PM Alexander Dejanovski < > a...@thelastpickle.com> w

Re: Changing existing Cassandra cluster from single rack configuration to multi racks configuration

know what is the recommended process to change an existing cluster > with single racks configuration to multi rack configuration. > > > I want to introduce 3 racks with 2 nodes in each rack. > > > Regards > Manish > > -- - Alexander Dejanovski France @

Re: [EXTERNAL] Re: Question on changing node IP address

y) > > > > I do put UNKNOWN as the default DC so that any missed node easily appears > in its own unused DC. > > > > > > Sean Durity > > > > *From:* Alexander Dejanovski > *Sent:* Wednesday, February 27, 2019 4:43 AM > *To:* user@cassandra.apac

Re: Question on changing node IP address

ill this method cause problems? > > Thanks! > > > On Wed, Feb 27, 2019 at 12:18 PM Alexander Dejanovski < > a...@thelastpickle.com> wrote: > >> You'll be fine with the SimpleSnitch (which shouldn't be used either >> because it doesn't allow a cluster to use multip

Re: Question on changing node IP address

the cluster. On Wed, Feb 27, 2019 at 10:52 AM wxn...@zjqunshuo.com wrote: > I'm using SimpleSnitch. I have only one DC. Is there any problem to follow > the below procedure? > > -Simon > > *From:* Alexander Dejanovski > *Date:* 2019-02-27 16:07 > *To:* user > *Subject

Re: Question on changing node IP address

ndra clusters that use the PropertyFileSnitch. It's been > working fine, but you've kinda scared me :-) > Why is it dangerous to use? > If I decide to change the snitch, is it seamless or is there a specific > procedure one must follow? > > Thanks! > > > On W

Re: Question on changing node IP address

ed in system > keyspace in data directory, and is stable across restarts. > > -- > Alex > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: [EXTERNAL] Availability issues for write/update/read workloads (up to 100s downtime) in case of a Cassandra node failure

rs, viruses, e.g., worms, trojan horses, etc., or other > items of a destructive nature, which may be contained in this attachment > and shall not be liable for direct, indirect, consequential or special > damages in connection with this e-mail message or its attachment. > > -- >

Re: Compacting more than the actual used space

ging the compaction throughput to 0 (removing the > throttling), had no impacts in the current compaction. Do new compaction > throughput values only come into effect when a new compaction kicks in? > > Cheers > > Pedro Gordo > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Cassandra | Cross Data Centre Replication Status

plicating on the Mum region's nodes. > However even after 2 weeks I do not see historical data to be replicated, > but new data being written on Sgp region is present in Mum region as well. > >>>> > >>>> Any help or suggestions to debug this issue will be highly > appreciated. > >>>> > >>>> Regards > >>>> Akshay Bhardwaj > >>>> +91-97111-33849 <+91%2097111%2033849> > >>>> > >>>> > >>> > >>> > >>> -- > >>> Jon Haddad > >>> http://www.rustyrazorblade.com > >>> twitter: rustyrazorblade > >>> > >>> > >> > >> > > > -- > Best Regards, > Kiran.M.K. > > - > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: rolling version upgrade, upgradesstables, and vulnerability window

gradesstables is a node-local concern that doesn't impact streaming or >> node replacement or other situations since cassandra can read old version >> sstables and new sstables would simply be the new format. >> >> - >> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: user-h...@cassandra.apache.org >> >> -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: rolling version upgrade, upgradesstables, and vulnerability window

sstables as needed, and that > upgradesstables is a node-local concern that doesn't impact streaming or > node replacement or other situations since cassandra can read old version > sstables and new sstables would simply be the new format. > -- - Alexa

Re: Best compaction strategy

ionStrategy (LCS) >2. SizeTieredCompactionStrategy (STCS) >3. TimeWindowCompactionStrategy (TWCS) > > > -- > Raman Gugnani > > 8588892293 <(858)%20889-2293> > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Multi dc reaper

3xos1D6CqJXvUWxhX%3DS4ahZFQpfNGLQ%40mail.gmail.com > <https://groups.google.com/d/msgid/tlp-apache-cassandra-reaper-users/CAHEGkNMRpWnU7MvUsiN3xos1D6CqJXvUWxhX%3DS4ahZFQpfNGLQ%40mail.gmail.com?utm_medium=email_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Large partitions

ing by DataStax). I would like to share it with my team. Did anyone > come across this information? If yes, can you please share it? > > Thanks! > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: High IO and poor read performance on 3.11.2 cassandra cluster

ts -H ks.xyz <http://ks.xyz>;* >> Total number of tables: 89 >> >> Keyspace : ks >> Read Count: 1439722 >> Read Latency: 1.8982509581710914 ms >> Write Count: 4222811 >> Write Latency: 0.016324778684151386 ms >> Pending Flushes: 0 >> Table: xyz >> SSTable count: 1036 >> SSTables in each level: [1, 10, 116/100, 909, 0, 0, 0, 0, >> 0] >> Space used (live): 187.09 GiB >> Space used (total): 187.09 GiB >> Space used by snapshots (total): 0 bytes >> Off heap memory used (total): 783.93 MiB >> SSTable Compression Ratio: 0.3238726404414842 >> Number of partitions (estimate): 447095605 >> Memtable cell count: 306194 >> Memtable data size: 20.59 MiB >> Memtable off heap memory used: 0 bytes >> Memtable switch count: 7 >> Local read count: 1440322 >> Local read latency: 6.785 ms >> Local write count: 1408204 >> Local write latency: 0.021 ms >> Pending flushes: 0 >> Percent repaired: 0.0 >> Bloom filter false positives: 19 >> Bloom filter false ratio: 0.3 >> Bloom filter space used: 418.2 MiB >> Bloom filter off heap memory used: 418.19 MiB >> Index summary off heap memory used: 307.75 MiB >> Compression metadata off heap memory used: 57.99 MiB >> Compacted partition minimum bytes: 150 >> Compacted partition maximum bytes: 1916 >> Compacted partition mean bytes: 1003 >> Average live cells per slice (last five minutes): 20.0 >> Maximum live cells per slice (last five minutes): 20 >> Average tombstones per slice (last five minutes): 1.0 >> Maximum tombstones per slice (last five minutes): 1 >> Dropped Mutations: 0 bytes >> >> -- >> >> regards, >> Laxmikant Upadhyay >> >> -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: URGENT: disable reads from node

plication should be changed for this. > > > On Wednesday, August 29, 2018 2:41 PM, kurt greaves > wrote: > > > Note that you'll miss incoming writes if you do that, so you'll be > inconsistent even after the repair. I'd say best to just query at QUORUM > until you can finish repairs. > &g

Re: URGENT: disable reads from node

t problem we were forced to delete commit logs from > one of nodes. > > Now repair is running, but meanwhile some reads bring no data (RF=2) > > Can this node be excluded from reads queries? And that all reads will be > redirected to other node in the ring? > > > Thank

Re: Incremental repair

repair by -pr option only. > > Question:Is incremental repair is the default repair for cassandra 3.11.2 > version. > > Thanks, > Prachi > > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Compaction throughput vs. number of compaction threads?

copy or use it, or > disclose it to anyone else. If you received it in error please notify us > immediately and then destroy it. Dynatrace Austria GmbH (registration > number FN 91482h) is a company registered in Linz whose registered office > is at 4040 Linz, Austria, Freistädterstraße 313 > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Question About Reaper

; We need to finish repair soon. > > On Mon, May 21, 2018 at 10:53 AM Alexander Dejanovski < > a...@thelastpickle.com> wrote: > >> Hi Subri, >> >> Reaper might indeed be your best chance to reduce the overhead of vnodes >> there. >> The latest

Re: Question About Reaper

>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> We have a cluster with 144 nodes( 3 datacenter) with 256 Vnodes . >>>>>> When we tried to start repairs from opscenter then it showed >>>>>> 1.9Million ranges to repair . >>>>>> And even after doing compaction and strekamthroughput to 0 , >>>>>> opscenter is not able to help us much to finish repair in 9 days >>>>>> timeframe . >>>>>> >>>>>> What is your thought on Reaper ? >>>>>> Do you think , Reaper might be able to help us in this scenario ? >>>>>> >>>>>> Thanks >>>>>> Surbhi >>>>>> >>>>>> >>>>>> -- >>>> Jon Haddad >>>> http://www.rustyrazorblade.com >>>> twitter: rustyrazorblade >>>> >>>> >>>> >>> >>> -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Nodetool repair multiple dc

uy exploring cassandra reaper ..does anyone has used that > in prod? > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Many SSTables only on one node

e run "repair -full" regulary for this keyspace (every 24 hours on each > node), because gc_grace_seconds is set to 24 hours. > > Should we consider increasing compaction throughput and > "concurrent_compactors" (as recommended for SSDs) to keep > "CompactionExe

Re: Many SSTables only on one node

ut I'm wondering, what was the cause? How prevent it from repeating? > > -- > Best Regards, > Dmitry Simonov > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Cassandra upgrade from 2.2.8 to 3.10

an you guys please let me know if this approach would work? I’m >> concerned if having mixed version on Cassandra nodes may cause any issues >> like in streaming data/sstables from existing DC to newly created third DC >> with version 3.10 installed, will nodes in DC3 join the cluster

Re: Node won't start

Should I move this sstable out of the way to > let it start? will it rebuild/refresh size estimates if I remove that > folder? > > thanks > -B > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: need to reclaim space with TWCS

> Hi Alexander, Thanks for your response! I'll give it a shot. >> >> On Sat, Jan 20, 2018 at 10:22 AM Alexander Dejanovski < >> a...@thelastpickle.com> wrote: >> >>> Hi Brian, >>> >>> You should definitely set unchecked_tombstone_compac

Re: need to reclaim space with TWCS

an I expect TWCS/C* to reclaim the space from those >> SSTables with 0.89* droppable tombstones? Or do I (can I?) manually >> delete these files and will c* just ignore the overlapping data and treat >> as tombstoned? >> >> >> >> What else should/could be d

Re: Alter composite column

t;>>> 2018-01-12 16:53 GMT+01:00 Joel Samuelsson <samuelsson.j...@gmail.com>: >>>> >>>>> Hi, >>>>> >>>>> I have an older system (C* 2.1) using Thrift tables on which I want to >>>>> alter a column composite. Right now it looks like (int, int, string) but I >>>>> want it to be (int, int, string, string). Is it possible to do this on a >>>>> live cluster without deleting the old data? Can you point me to some >>>>> documentation about this? I can't seem to find it any more. >>>>> >>>>> Best regards, >>>>> Joel >>>>> >>>> >>>> >>> >> > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Alter composite column

tem (C* 2.1) using Thrift tables on which I want to >> alter a column composite. Right now it looks like (int, int, string) but I >> want it to be (int, int, string, string). Is it possible to do this on a >> live cluster without deleting the old data? Can you point me to some >>

Re: New token allocation and adding a new DC

use the automatic > token allocation when we need to scale it out. This seems to be the only > supported way to use it anyway. > > Regards, > -- > Alex > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: TWCS and autocompaction

crontab nodetool disableautocompaction? > > Thanks > > On 16-01-2018 11:35, Alexander Dejanovski wrote: > > Hi, > > The overlaps you're seeing on time windows aren't due to automatic > compactions, but to read repairs. > You must be reading at quorum or local_quorum which can

Re: New token allocation and adding a new DC

ebuild: > https://issues.apache.org/jira/browse/CASSANDRA-7032 > Nor do I find any open tickets that would discuss the topic. > > Is it reasonable to open an issue for that or is there some obvious > blocker? > > Thanks, > -- > Alex > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Too many tombstones using TTL

on > as read latency and disk usage are okey? Are tombstones affect repair time > (using reaper)? > > Thanks. > > > On Tue, Jan 16, 2018 at 11:32 AM, Alexander Dejanovski < > a...@thelastpickle.com> wrote: > >> Hi, >> >> could you be more specific

Re: TWCS and autocompaction

; > > --------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: vnodes: high availability

s getting worse with increase of > vnodes_per_node/Nnode ratio. > > Please, correct me if I'm wrong. > > > How would the situation differ from this example by DataStax, if we had a > real-life 6-nodes cluster with 8 vnodes on each node? > > > Regards, > > Kyrill > > &

Re: Too many tombstones using TTL

ing I can do about it except delete records by key > instead of expiring. > > > On Fri, Jan 12, 2018 at 7:30 PM, Alexander Dejanovski < > a...@thelastpickle.com> wrote: > >> Hi, >> >> As DuyHai said, different TTLs could theoretically be set for different >

Re: vnodes: high availability

. 2018 à 17:55, Alexander Dejanovski <a...@thelastpickle.com> a écrit : > Hi Kyrylo, > > the situation is a bit more nuanced than shown by the Datastax diagram, > which is fairly theoretical. > If you're using SimpleStrategy, there is no rack awareness. Since vnode > distr

Re: vnodes: high availability

uster with following parameters: > > - 50 nodes in the cluster > > - RF=3 > > - vnodes=256 per node > > - CL for some queries = QUORUM > > - endpoint_snitch = SimpleSnitch > > > Is it correct that 2 any nodes down will cause unavailability of a > keyrange at CL=QUORUM? > > > Regards, > > Kyrill > > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Too many tombstones using TTL

>> { "name" : "c2", "deletion_info" : { "local_delete_time" : >>> "2018-01-10T13:29:25Z" } >>> }, >>> { "name" : "c3", "deletion_info" : { "local_delete_time" : >>> "2018-01-10T13:29:25Z" } >>> } >>> ] >>> } >>> ] >>> } >>> ] >>> >>> The question is why Cassandra creates a tombstone for every column >>> instead of single tombstone per row? >>> >>> In production environment I have a table with ~30 columns and It gives >>> me a warning for 30k tombstones and 300 live rows. It is 30 times more then >>> it could be. >>> Can this behavior be tuned in some way? >>> >>> Thanks. >>> >>> -- >>> Best regards, >>> Python_Max. >>> >> >> > > > -- > Best regards, > Python_Max. > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Full repair caused disk space increase issue

d be deleted after during > repair, but for some unknown reason, the repair process failed to delete > them. > -rw-r--r-- 1 root root 237G Dec 31 12:48 lb-123800-big-Data.db > -rw-r--r-- 1 root root 243G Dec 31 12:48 lb-123801-big-Data.db > > C* version is 2.2.8 with STCS. Any ideas? > > Cheers, > -Simon > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Deleted data comes back on node decommission

;> Do yo think that extra data which node is not responsible for can lead to >> zombie data? >> >> >> On 13.12.17 18:43, Jeff Jirsa wrote: >> >>> Did you run cleanup before you shrank the cluster? >>> >>> >> -- >> >> Best Regards, >> Python_Max. >> >> >> - >> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org >> For additional commands, e-mail: user-h...@cassandra.apache.org >> happen > > > -- > > Best Regards, > Python_Max. > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Huge system.batches table after joining a node (Cassandra 3.11.1)

Just a heads up that (in case you missed it) MVs were retroactively marked as experimental and that a large part of the community considers they should not be used in production. On Thu, Dec 7, 2017 at 4:53 PM Alexander Dejanovski <a...@thelastpickle.com> wrote: > Yes, MVs use batch

Re: Huge system.batches table after joining a node (Cassandra 3.11.1)

Lorenz < christian.lor...@webtrekk.com> wrote: > Hi Alexander, > > > > yes we use MV’s. The size of the batch table is around 10GB on the > existing nodes. Also seems pretty high. > > So is this table (also) used to process MV building? > > > > Regards, >

Re: Huge system.batches table after joining a node (Cassandra 3.11.1)

(100GB) which is about 1/3 of the nodes size. > > Is it safe to truncate the table? > > > > Regards, > > Christian > > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: DC aware failover

ing I want to test it. > > Thank you for your help > > > On Nov 16, 2017 10:35 AM, "Alexander Dejanovski" <a...@thelastpickle.com> > wrote: > > Hi, > > The policy is used in production at least in my former company. > > I can help if you have is

Re: DC aware failover

you know about this > project or do you know other projects that provide same kind of > functionality. > > Thanks... > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: High IO Util using TimeWindowCompaction

fault_time_to_live = 0 > > AND gc_grace_seconds = 3600 > > AND max_index_interval = 2048 > > AND memtable_flush_period_in_ms = 0 > > AND min_index_interval = 128 > > AND read_repair_chance = 0.0 > > AND speculative_retry = 'NONE'; > > > Thanks, > Kurt > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: STCS leaving sstables behind

fixed nodetool status from > reporting a wrong Load (nearly 2TB per node instead à 300Gb) => we are > loading some data for a week now, it seems that this can happen sometimes > > If anyone ever experienced that kind of behavior I'd be glad to know > whether it is OK or not, I'd like to avoid manually triggering JMX > UserDefinedCompaction ;) > > Thank you > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Not marking node down due to local pause

> I am seeing some but not a whole lot of dropped mutations. nodetool > tpstats looks ok. > > The growing number of SSTables really makes me think this is an I/O issue. > Casssandra is running in a kubernetes cluster using a SAN which is another > reason I suspect I/O. > >

Re: GC/CPU increase after upgrading to 3.0.14 (from 2.1.18)

lt;https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313=gmail=g> >> >> The contents of this e-mail are intended for the named addressee only. It >> contains information that may be confidential. Unless you are the named >> addressee or an authorized designee, you may not copy or use it, or >> disclose it to anyone else. If you received it in error please notify us >> immediately and then destroy it. Dynatrace Austria GmbH (registration >> number FN 91482h) is a company registered in Linz whose registered office >> is at 4040 Linz, Austria, Freistädterstraße 313 >> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313=gmail=g> >> The contents of this e-mail are intended for the named addressee only. It >> contains information that may be confidential. Unless you are the named >> addressee or an authorized designee, you may not copy or use it, or >> disclose it to anyone else. If you received it in error please notify us >> immediately and then destroy it. Dynatrace Austria GmbH (registration >> number FN 91482h) is a company registered in Linz whose registered office >> is at 4040 Linz, Austria, Freistädterstraße 313 >> <https://maps.google.com/?q=4040+Linz,+Austria,+Freist%C3%A4dterstra%C3%9Fe+313=gmail=g> >> > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Multi-node repair fails after upgrading to 3.0.14

> 2017-09-18 07:59:17 repair finished > > > > > > If running the above nodetool call sequentially on all nodes, repair > finishes without printing a stack trace. > > > > The error message and stack trace isn’t really useful here. Any further > ideas/experiences? &g

Re: Multi-node repair fails after upgrading to 3.0.14

356976885], > (-1193448110686154165,-1161640137086921883], > (-3356304907368646189,-3346460884208327912], > (3466596314109623830,346814432669172], > (-9050241313548454460,-9005441616028750657], > (402227699082311580,407458511300218383]]] Validation failed in /FAKE.33.64 > >

Re: Multi-node repair fails after upgrading to 3.0.14

air.ValidationTask.treesReceived(ValidationTask.java:68) > ~[apache-cassandra-3.0.14.jar:3.0.14] > > at > org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:178) > ~[apache-cassandra-3.0.14.jar:3.0.14] > > at > org.apache.cas

Re: Regular dropped READ messages

we process each > user update: > - SELECT from a "master" slug to get the fields we need > - from that, compute a list of slugs the user had and a list of slugs > the user should have (for example if he changes timezone we have to update > the slug) > - delete the user fr

Re: Regular dropped READ messages

almost always correspond to times > where our schedules SELECTs are happening. That narrows the scope a little, > but still. > > Anyway, I'd appreciate any information about troubleshooting this scenario. > Thanks. > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: AWS Cassandra backup/Restore tools

/www.linkedin.com/in/srinivm/> [image: > facebook.png] <https://www.facebook.com/datastax> [image: twitter.png] > <https://twitter.com/mani_srini> [image: g+.png] > <https://plus.google.com/+Datastax/about> > <http://feeds.feedburner.com/datastax> > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Cassandra 3.10 has partial partition key search but does it result in a table scan?

ext, >> > primary key((a,b), c) >> > ); >> > >> > Now I can do select * from hello where a='foo' allow filtering;// This >> works in 3.10 but I wonder if this query results in table scan and if so is >> there any way to limit such that I get max b? >> > >> > Thanks! >> >> > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: manual deletes with TWCS

>> How problematic is it to perform deletes when using TWCS? I am currently >> using TWCS and have some new use cases for performing deletes. So far I >> have avoided performing deletes, but I am wondering what issues I might run >> into. >> >> >> - John >> >> >> > > > -- > > - John > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: cassandra OOM

t; # By default, ConcGCThreads is 1/4 of ParallelGCThreads. > > # Setting both to the same value can reduce STW durations. > > #-XX:ConcGCThreads=16 > > > > ### GC logging options -- uncomment to enable > > > > #-XX:+PrintGCDetails > > #-XX:+PrintGCDateStamps > > #-X

Re: cassandra OOM

the sender immediately, delete it from your system and > destroy any copies. You may not further disclose or distribute this email > or its attachments. > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: How can I scale my read rate?

that case, with RF= 4 instead of 3, with several clients accessing keys > same key ranges, a coordinator could pick up one node to handle the request > in 4 replicas instead of picking up one node in 3 , thus having > more "workers" to handle a request ? > > Am I wrong here ? > > Thank you for the clarification > > > -- > best, > Alain > > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Help with data modelling (from MySQL to Cassandra)

A, even if we are > querying by Primary keys, the second query will have 100k+ primary key id’s > in the WHERE clause, and the second solution looks like an anti pattern in > cassandra. > > Could anyone give any advice how would we create a model for our use case? > > Thank you in advance, > Zoltan. > > > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Disk full during new node bootstrap

ve this? > > Run compaction for nodes in that group(to which I'm going to add new node, > as data streams to new nodes from nodes of group to which it is added) > > OR > > Boootstrap/add 2(multiple nodes) at a time? > > > Please suggest better way to fix this. > &

Re: Why compacting process uses more data that is expected

a. > > > Hello, > > I believe that output of compactionstats shows you the size of > *uncompressed* data. Can you check (with nodetool tablestats) your > compression ratio? > > -- > Alex > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Reaper repair seems to "hang"

l.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > Hope it helps! > > Regards, > Bhuvan > > According to > https://medium.com/@mlowicki/cassandra-reaper-introduction-ed73410492bf#.f0erygqpk > : > > Segment runner has protection mechanism

Re: Reaper repair seems to "hang"

: 0.9 > * Incremental: true > > Don't want to swamp you with more details or unnecessary logs, especially > as I'd have to sanitize them before sending them out, so please let me know > if there is anything else I can provide, and I'll do my best to get it to > you. > > ​K

Re: Exception thrown on repair after change was made to existing cassandra type

adPoolExecutor.java:1142) > ~[na:1.8.0_60]* > * at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > [na:1.8.0_60]* > * at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]* > > On the node /x.x.x.y > > Do you any suggestion? > Thank you in advance, > Robert > > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: repair -pr in crontab

inish times of repairs, and if the runs were > successful? Does the "nodetool repair" command is guaranteed to exit > only after the repair is finished and does it return a status code to a > shell? > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Out of memory and/or OOM kill on a cluster

13854232 > Maximum tombstones per slice (last five minutes): 22602.0 > > - regarding swap, it's not disabled anywhere, I must say we never really > thought about it. Does it provide a significant benefit ? > > Thanks for your help, really appreciated ! > > On Mon, Nov 21, 20

Re: Cassandra reaper

t; Thanks again for the help > > > On Tue, Nov 1, 2016 at 12:26 PM, Jai Bheemsen Rao Dhanwada < > jaibheem...@gmail.com> wrote: > > ok thank you, > I will try and update you. > > On Tue, Nov 1, 2016 at 10:57 AM, Alexander Dejanovski < > a...@thelastpickle.com&g

Re: Out of memory and/or OOM kill on a cluster

is around 4.6MB, 98% around 2MB. Could the 1% here really have that much impact ? We do write a lot to the biggest table and read quite often too, however I have no way to know if that big partition is ever read. On Mon, Nov 21, 2016, at 01:09 PM, Alexander Dejanovski wrote: Hi Vincent, one of

Re: Out of memory and/or OOM kill on a cluster

mission > control but it didn't really help; it's possible I missed it because I have > no idea what to look for exactly. > > Anyone have some advice for troubleshooting this ? > > Thanks. > > -- - Alexander Dejanovski France @alexanderdeja Consultant Apache Cassandra Consulting http://www.thelastpickle.com

Re: Cassandra 3.6 Repair issue with Reaper

=mobileAppLp_campaign=android> > [image: > A] > <https://itunes.apple.com/in/app/snapdeal-mobile-shopping/id721124909?ls=1=8_source=mobileAppLp_campaign=ios> > [image: > W] > <http://www.windowsphone.com/en-in/store/app/snapdeal/ee17fccf-40d0-4a59-80a3-04da4

  1   2   >