Re: Consistency with Datacenter switch

2020-03-15 Thread manish khandelwal
one dc to local quorum in the other dc > without losing any writes? > > The easiest way to do this strictly correctly is to take the latency hit > and do quorum while you run repair, then you can switch to local quorum on > the other side. > > A few more notes inline > > >

Consistency with Datacenter switch

2020-03-15 Thread manish khandelwal
While switching over datacenters, there is a chance of mutation drop because of which inconsistency may occur. To avoid inconsistency we can do following : Monitor and if require then run repair 1. Monitor tpstats in all nodes. If dropped message count is 0, it can be inferred no mutation

Re: [EXTERNAL] Cassandra 3.11.X upgrades

2020-03-03 Thread manish khandelwal
Should upgradesstables not be run after every node is upgraded? If we need to rollback then we will not be able to downgrade sstables to older version. Regards Manish On Tue, Mar 3, 2020 at 11:26 PM Hossein Ghiyasi Mehr wrote: > It's more safe to upgrade one node before upgrading another node

Re: Corrupt SSTable Cassandra 3.11.2

2020-02-25 Thread manish khandelwal
Thanks all for your support. I executed the discussed process (barring repair, as table was read for reporting only) and it worked fine in production. Regards Manish >

Re: Corrupt SSTable Cassandra 3.11.2

2020-02-13 Thread manish khandelwal
rictly correct to do is treat every corrupt sstable > exception as a failed host, and replace it just like you would a failed > host. > > > On Thu, Feb 13, 2020 at 10:55 PM manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> Thanks Erick >> >> I w

Re: Corrupt SSTable Cassandra 3.11.2

2020-02-13 Thread manish khandelwal
Thanks Erick I would like to explain how data resurrection can take place with single SSTable deletion. Consider this case of table with Levelled Compaction Strategy 1. Data A written a long time back. 2. Data A is deleted and tombstone is created. 3. After GC grace tombstone is purgeable. 4.

Re: Corrupt SSTable Cassandra 3.11.2

2020-02-13 Thread manish khandelwal
Hi Erick Thanks for your quick response. I have attached the full stacktrace which show exception during validation phase of table repair. I would like to know what will be "ordinary hammer" in this case. Do you want to suggest that deleting only corrupt sstable file ( in this case

Re: Corrupt SSTable Cassandra 3.11.2

2020-02-13 Thread manish khandelwal
ompany/datastax> > <https://www.facebook.com/datastax> <https://twitter.com/datastax> > <http://feeds.feedburner.com/datastax> <https://github.com/datastax/> > > <https://www.datastax.com/accelerate> > > > > On Fri, 14 Feb 2020 at 04:39, manish

Corrupt SSTable Cassandra 3.11.2

2020-02-13 Thread manish khandelwal
Hi I see a corrupt SSTable in one of my keyspace table on one node. Cluster is 3 nodes with replication 3. Cassandra version is 3.11.2. I am thinking on following lines to resolve the corrupt SSTable issue. 1. Run nodetool scrub. 2. If step 1 fails, run offline sstabablescrub. 3. If step 2 fails,

Re: sstableloader: How much does it actually need?

2020-02-06 Thread manish khandelwal
Yes you will have all the data in two nodes provided there is no mutation drop at node level or data is repaired For example if you data A,B,C and D. with RF=3 and 4 nodes (node1, node2, node3 and node4) Data A is in node1, node2 and node3 Data B is in node2, node3, and node4 Data C is in node3,

Re: Cassandra2.0.14 : Obsolete files not being deleted after compaction

2020-01-22 Thread manish khandelwal
dead: session failed* Does anyone remember any JIRA ticket related to obsolete sstables not being deleted after compaction? Regards Manish On Wed, Jan 22, 2020 at 11:37 AM Jeff Jirsa wrote: > > > On Tue, Jan 21, 2020 at 8:58 PM manish khandelwal < > manishkhandelwa...@

Re: Cassandra2.0.14 : Obsolete files not being deleted after compaction

2020-01-21 Thread manish khandelwal
ll the files for any given sequence number ie data, index, toc > etc > > Regards, > > Nitan > > Cell: 510 449 9629 > > On Jan 21, 2020, at 5:36 AM, manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >  > Hi Team > > I am observing some obs

Cassandra2.0.14 : Obsolete files not being deleted after compaction

2020-01-21 Thread manish khandelwal
Hi Team I am observing some obsolete files in Cassandra 2.0.14 which are already compacted but not removed from the system after compaction. As per CASSANDRA-7872 , after GC grace period has passed the sstables are open for read again and can

Question regarding bloom filters

2019-10-28 Thread manish khandelwal
Hi All I was reading about the read path. It says bloom filters are read from memory. But bloom filters can become very large so they may occupy lot of memory. So are all bloom filters loaded in memory on startup or are loaded with read requests. Regards Manish

Re: Cassandra Repair question

2019-10-18 Thread manish khandelwal
No it will only cover primary ranges of nodes on single rac. Repair with -pr option is to be run on all nodes in a rolling manner. Regards Manish On 19 Oct 2019 10:03, "Krish Donald" wrote: > Hi Cassandra experts, > > > We are on Cassandra 3.11.1. > > We have to run repairs for a big cluster.

Best practices while designing backup storage system for big Cassandra cluster

2019-03-28 Thread manish khandelwal
Hi I would like to know is there any guideline for selecting storage device (disk type) for Cassandra backups. As per my current observation, NearLine (NL) disk on SAN slows down significantly while copying backup files (taking full backup) from all node simultaneously. Will using SSD disk

Changing existing Cassandra cluster from single rack configuration to multi racks configuration

2019-03-06 Thread manish khandelwal
We have a 6 node Cassandra cluster in which all the nodes are in same rack in a dc. We want to take advantage of "multi rack" cluster (example: parallel upgrade on all the nodes in same rack without downtime). I would like to know what is the recommended process to change an existing cluster with

<    1   2