Re: compaction trigger after every fix interval

Hi Prerna Compactions are triggered automatically based on the compaction strategy. Since you are seeing compactions triggered every 6 hours, the thing that can be happening is you have such a kind of traffic where you have lots of writes every 6 hours. PS: Please use the user mailing list

Re: Question Regarding Cassandra-19336

It looks a crtirical bug for setup with multi DC using high number of vnodes and running full repair with -PR option, since number of parallel repair sessions can be as high as number of vnodes. Thus it can fill up memory causing pom or direct buffer memory oom. It should get prioritized for

Full repair with -pr option getting stuck on Cassandra 4.0.10

In a two datacenter cluster (11 nodes each) we are seeing repair getting stuck. Issue is when repair is triggered on a particular keyspace repair session is lost and cassandra never returns for that particular session. There are no "WARN" or "ERROR" logs in Cassandra logs. No message dropped seen

Re: Switching to Incremental Repair

They(incremental and full repairs) are required to run separately at different times. You need to identify a schedule, for example, running incremental repairs every week for 3 weeks and then run full repair in the 4th week. Regards Manish On Sat, Feb 3, 2024 at 7:29 AM Kristijonas Zalys wrote:

Re: Over streaming in one node during repair.

; "sstablemetadata" and "sstabledump" commands handy. > > > On 23/01/2024 18:07, manish khandelwal wrote: > > In one of our two datacenter setup(3+3), one Cassndra node is getting lot > of data streamed from other nodes during repair to the extent that it

Over streaming in one node during repair.

In one of our two datacenter setup(3+3), one Cassndra node is getting lot of data streamed from other nodes during repair to the extent that it fills up and ends with full disk. I am not able to understand what could be the reason that this node is misbehaving in the cluster. Cassandra version is

Re: Upgrade from 3.11.x to 4.1.3

ing upgrades is not supported; please ensure > it is disabled before beginning the upgrade and re-enable after. > > – Scott > > On Jan 6, 2024, at 10:25 PM, manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >  > Hi > > Is Cassandra upgrade from

Upgrade from 3.11.x to 4.1.3

Hi Is Cassandra upgrade from Cassandra 3.11.x to Cassandra 4.1.3 is supported. NEWS.txt has a general guideline that Snapshotting is fast (especially if you have JNA installed) and takes effectively zero disk space until you start compacting the live data files again. Thus, best practice is to

Re: Materialized View inconsistency issue

>From 4.0.11's cassandra.yaml *## EXPERIMENTAL FEATURES ### Enables materialized view creation on this node.# Materialized views are considered experimental and are not recommended for production use.enable_materialized_views: false* So I

Re: Repair errors

What logs of /172.16.20.16:7000 say when repair failed. It indicates "validation failed". Can you check system.log for /172.16.20.16:7000 and see what they say. Looks like you have some issue with *doc/origdoc, probably some corrupt sstable. *Try to run repair for individual table and see for

Re: Cassandra with RHEL 8.6

Micah Green did you apply any tuning parameters also to RHEL apart from the solution provided. As I can see some nodes swapping still. Regards Manish On Sat, Jul 29, 2023 at 5:52 PM manish khandelwal < manishkhandelwa...@gmail.com> wrote: > Thanks Micah Green. Got the same resolu

Restoring tables with counter columns

Hi All Is there any special consideration to take while restoring tables with 'counter' type columns? For restoration I wipe out data directory and replace with sstables from backup directory. Regards Manish

Re: Cassandra with RHEL 8.6

785021. This is obtrusive > enough that our linux engineering team decided to push a patch through to > all of our teams. Once we updated the appropriate setting per the link I > shared, all swap issues went away. > > On Thu, Jul 27, 2023 at 5:33 AM manish khandelwal < > manishkh

Re: Cassandra with RHEL 8.6

uide/sysctl/vm.html?highlight=swappiness#swappiness > > [4] > https://www.ibm.com/docs/en/linux-on-systems?topic=settings-setting-swappiness-parameter-zero > > > On 27. Jul 2023, at 10:14, manish khandelwal > wrote: > > Has anybody seen issues running Cassandra on RHEL 8.6. I

Cassandra with RHEL 8.6

Has anybody seen issues running Cassandra on RHEL 8.6. I am seeing performance dip while running Cassandra on RHEL8.6. Also with RHEL I am seeing swapping happening in spite of a lot of free memory available. In RHEL 7.9 we did not see any swapping with same workload and same RPM of Cassandra.

Re: Backporting CASSANDRA-18560 to Cassandra 4.0.10

hat is possible. 4.0.10 was already released. We can not > backport anything to what is already released. I believe you need to update > to 4.0.11. > > Regards > > ________ > From: manish khandelwal > Sent: Monday, July 17, 2023 11:40 &g

Backporting CASSANDRA-18560 to Cassandra 4.0.10

I see a critical bug https://issues.apache.org/jira/browse/CASSANDRA-18507 fixed in Cassandra 4.0.10. But also see that one critical bug https://issues.apache.org/jira/browse/CASSANDRA-18560 introduced and that is going to be fixed in 4.0.11. Can fix of

Re: 4.0 upgrade

Yes repairs are prohibited in mixed version cluster. If you want to monitor please disable repairs till complete upgrade is finished On Sat, Jul 8, 2023, 01:21 Runtian Liu wrote: > Hi, > > We are upgrading our Cassandra clusters from 3.0.27 to 4.0.6 and we > observed some error related to

Re: Impact of column names on storage

I think it is verbose to make it readable in sstableoutput. Column names should not have any impact of final space usage. On Mon, Jun 12, 2023 at 6:25 PM Dimpal Gurabani wrote: > Hi all, > > We have a table with 15 columns and ~1M rows. Looking at the output of the > sstabledump tool, it seems

Re: Issue in Logging ASF Cassandra JIRA Account

Have you checked in spam mail folder? Sometimes it lands up there. On Mon, May 8, 2023, 20:52 ranju goel wrote: > Hi, > > > > I am getting incorrect username/password error message while logging in > ASF Cassandra JIRA . > > username - rcass > > > After this, I requested for resetting password

Re: Is cleanup is required if cluster topology changes

You can replace the node directly why to add a node and decommission the another node. Just replace the node with the new node and your topology remains the same so no need to run the cleanup . On Fri, May 5, 2023 at 10:26 AM Jaydeep Chovatia wrote: > We use STCS, and our experience with

Re: Cleanup

There is no advantage of running cleanup if no new nodes are introduced. So cleanup time should remain same when adding new nodes. Cleanup is a local to node so network bandwidth should have no effect on reducing cleanup time. Dont ignore cleanup as it can cause you disks occupied without any

JIRA account creation request

Hi I would like to get my JIRA account created as I would like to contribute. Here are my details email address : manishkhandelwa...@gmail.com If any other information is needed please let me know. Regards Manish

Re: Query for Cassandra Driver

Hi Deepti I think you can reach out to https://groups.google.com/a/lists.datastax.com/g/cpp-driver-user. Regards Manish On Fri, Dec 23, 2022 at 12:52 PM Deepti Sharma S via user < user@cassandra.apache.org> wrote: > Hello Team, > > > > Could you please help in answering below query. > > > > >

Re: Query regarding EOS for Cassandra version 3.11.13

3.11.x versions will be maintained till May July 2023. Please refer https://cassandra.apache.org/_/download.html On Thu, Dec 15, 2022, 20:55 Pranav Kumar (EXT) via user < user@cassandra.apache.org> wrote: > Hi Team, > > > > Could you please help us to know when version 3.11.13 is going to be

Re: Table with 'compact storage' is not shown in "describe table" output in cqlsh

he warning. Regards Manish On Tue, Sep 27, 2022 at 3:22 PM manish khandelwal < manishkhandelwa...@gmail.com> wrote: > Hi All > > As I understand there was a plan to drop *Compact Storage* support with > *Cassandra > 4* but later few issues were identified which resulted in

Table with 'compact storage' is not shown in "describe table" output in cqlsh

Hi All As I understand there was a plan to drop *Compact Storage* support with *Cassandra 4* but later few issues were identified which resulted in continued support for Compact Storage in Cassandra 4. My cluster with a few old "compact storage" tables was able to come up with Cassandra 4.0.5.

Re: Local read request going across DC

Thanks a lot Erick for your help and support. Regards Manish On Wed, Sep 21, 2022, 19:12 Erick Ramirez wrote: > Just to circle back here, I've reviewed the trace output and it shows > multiple requests fired off for lookups on the `roles` table which > indicated to me that the default

Re: Local read request going across DC

Thanks Erick for the response. read_repair_chance is 0. Can speculative_retry cause this? We have that set at 99 percentile. Regards Manish On Wed, Sep 21, 2022 at 11:17 AM Erick Ramirez wrote: > It sounds like read-repair chance is enabled on the table. Check the table > schema for a

Local read request going across DC

Hi All In one of our cluster, read request with consistency "LOCAL_QUORUM" is going across DC. When we run query setting CONSISTECY to LOCAL_QUORUM in cqlsh, with tracing on, we see READ and digest request sent across to nodes on other DC. I have checked gossipinfo, peers table, nodetool status.

Re: Wrong Consistency level seems to be used

Consistency can also be set on a statement basis. So please check in your code that you might be setting consistency 'ALL' for some queries. On Thu, Jul 21, 2022 at 7:23 PM pwozniak wrote: > Hi, > > we have the following code (java driver): > > cluster =

Re: Query around Data Modelling

Table1 should be fine if some column values are not entered than Cassandra will not create entry for them so partiton will almost be same in both cases. On Thu, Jun 23, 2022, 07:08 MyWorld wrote: > Hi all, > > Just a small query around data Modelling. > Suppose we have to design the data model

Re: Cassandra 4.0 upgrade from Cassandra 3x

You can see upgrading instructions here https://github.com/apache/cassandra/blob/cassandra-4.0.2/NEWS.txt. On Fri, Feb 11, 2022 at 2:52 AM Abdul Patel wrote: > Hi > apart from standard upgrade process any thing specific needs ti be > handled separately for this upgrade process? > > Any changes

Re: Query timed out after PT2M

It maybe the case you have lots of tombstones in this table which is making reads slow and timeouts during bulk reads. On Fri, Feb 4, 2022, 03:23 Joe Obernberger wrote: > So it turns out that number after PT is increments of 60 seconds. I > changed the timeout to 96, and now I get PT16M

Re: Hanging repairs in Cassandra

a TCP connection in such short time, unless the number of > connections is very large and generally short lived (which often see in web > servers). > On 24/01/2022 13:03, manish khandelwal wrote: > > Hi All > > Thanks for the suggestions. The issue was *tcp_keepalive_time* ha

Re: Hanging repairs in Cassandra

e to draw upon the community’s knowledge of a current release of the > database. > > — Scott > > On Jan 21, 2022, at 8:51 PM, manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >  > Hi All > > After going through the system.logs, I still see sometimes t

Handshaking version by Cassandra nodes

When a node does handshaking? I see log of handshaking messages in my logs, what do they signify? Regards Manish

Re: Hanging repairs in Cassandra

me behavior. >> >> Regards >> Manish >> >> On Tue, Jan 18, 2022, 22:50 Bowen Song wrote: >> >>> The link was related to Cassandra 1.2, and it was 9 years ago. Cassandra >>> was full of bugs at that time, and it has improved a l

Re: Hanging repairs in Cassandra

repair, I had to either keep retrying or break down the token ranges > into smaller subsets and use the "-st" and "-et" parameters. Since then > I've switched to use Cassandra Reaper and have never had similar issues. > > > On 19/01/2022 02:22, manish khandelw

Re: Hanging repairs in Cassandra

since then. For > that reason, I would rather not compare the issue you have with some 9 > years old issues someone else had. > > > On 18/01/2022 16:11, manish khandelwal wrote: > > I am not sure what is happening but it has happened thrice. It is > happening that merkle tre

Re: Hanging repairs in Cassandra

f you > accidentally ran "nodetool repair" on two nodes and both involve the same > table, or if you cancelled and then restarted a "nodetool repair" on a node > without waiting or killing the remannings of the first repair session on > other nodes. > On 18/01/2022 11

Re: Hanging repairs in Cassandra

ue is easily reproducible, I would re-run the repair with a > reduce scope (such as table and token range) to get less logs related to > the repair session. Less logs means less time spend on reading and > analysing them. > > Hope this helps. > On 18/01/2022 10:03, manish khand

Hanging repairs in Cassandra

I have a Cassandra 3.11.2 cluster with two DCs. While running repair , I am observing the following behavior. I am seeing that node is not able to receive merkle tree from one or two nodes. Also I am able to see that the missing nodes did send the merkle tree but it was not received. This make

Re: How does a node decide where each of its vnodes will be replicated to?

Just to add on to your response: *num_tokens* define the number of vnodes a node can have. Default is 256. *Initial token* range is predefined (For murmur -2**63 to 2**63-1) So if you have one node in (does not make sense) cluster with num_tokens as 256 then you will have 256vnodes. Scaling up

Re: [UPGRADATION] Apache Cassandra from version 3.0.9 to 4.0.0

Totally agree with Jeff and Bowen there. Don't try to achieve something faster by cutting corners. Migration to GCP from physical DC should be done on the same versions. On Mon, Sep 6, 2021 at 2:11 PM Bowen Song wrote: > Hello Ashish, > > > I'm slightly worried about this: > > *Since I won't be

Re: Long GC pauses during repair

eap on 1 node > to see. or go GC1 if it is easy way. > > Thanks, > Jim > > On Tue, Aug 3, 2021 at 3:33 AM manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> Long GC (1 seconds /2 seconds) pauses seen during repair on the >> coordinator. Runnin

Long GC pauses during repair

Long GC (1 seconds /2 seconds) pauses seen during repair on the coordinator. Running full repair with partition range option. GC collector is CMS and heap is 14G. Cluster is 7+7. Cassandra version is 3.11.2. Not much traffic when repair is running. What could be the probable cause of long gc

Re: High memory usage during nodetool repair

Missed the heap part, not sure why is that happening On Tue, Aug 3, 2021 at 8:59 AM manish khandelwal < manishkhandelwa...@gmail.com> wrote: > mmap is used for faster reads and as you guessed right you might see read > performance degradation. If you are seeing high memory usage a

Re: High memory usage during nodetool repair

mmap is used for faster reads and as you guessed right you might see read performance degradation. If you are seeing high memory usage after repairs due to mmaped files, the only way to reduce the memory usage is to trigger some other process which requires memory. *mmapped* files use buffer/cache

Re: Reduce num_tokens on single node cluster

Single node does not make any sense in Cassandra. It should not make any difference with a single node with whatever number of tokens you start with. You can change your tokens on the test server and see what happens. On Fri, Jul 30, 2021 at 5:53 PM Maxim Parkachov wrote: > Hi everyone, > > I

Re: Issue with native protocol

Have you tried restarting your application? It should renegotiate the protoco during handshake and should resolve the issue. On Wed, Jul 28, 2021 at 3:06 PM Srinivas Polamarasetty < srinivas.polamarase...@logmein.com> wrote: > Query shows 4 but App team also getting below error. They were not

Re: Storing user activity logs

I concur with Eliot view. Only way you can reduce partition size is by tweaking your partition key. Here with user_id as partition key, partition size depends on the activity of the user. For a superactive user it can become large in no time. After changing the key migration of old data to the

Re: Number of DCs in Cassandra

We are planning to go with 5 DCs with RF of 3 in each. All DCs will have > reads and writes. Most queries are done at LOCAL_QUORUM. > A very few Simple and CAS queries (<0.1%) will be done at QUORUM > consistency. > > On Wed, Jul 14, 2021 at 12:19 PM manish khandelwal < > m

Re: Number of DCs in Cassandra

I don't think there is any restriction on the number of data centers. So technically you can add as many data centers you want. Performance depends on how you use your cluster. For example, one of your data centers could be read only, or is there traffic on all the data centers. On Wed, Jul 14,

Re: How to remove tombstones in a levelled compaction table in Cassandra 2.1.16?

note: that when you > first run this IF any sstables are above the tombstone_ratio setting for > that table they will be compacted, this may cause extra load on the cluster. > > Vytenis > ... always do your own research and verify what people say. :) > > On Mon, Jul 5, 2021 at 10:11 PM manish k

Re: How to remove tombstones in a levelled compaction table in Cassandra 2.1.16?

ht of using major compaction but for LCS that was introduced in >> Cassandra 2.2. Also user defined compactions dont work on LCS tables. >> >> >> >> Regards >> >> Manish Khandelwal >> > > If it's tombstones specifically you'll be able to

How to remove tombstones in a levelled compaction table in Cassandra 2.1.16?

and queries are getting timed out. Am I right in my thinking? What is the possible way to get out of this? I thought of using major compaction but for LCS that was introduced in Cassandra 2.2. Also user defined compactions dont work on LCS tables. Regards Manish Khandelwal

Re: Suggestions on Running UpgradeSSTables

One should avoid doing things (apart from normal traffic) in mixed mode. The best approach is to upgrade the nodes as fast as possible and then do other activities. Regards On Thu, May 20, 2021 at 10:44 AM Jai Bheemsen Rao Dhanwada < jaibheem...@gmail.com> wrote: > Thanks for the suggestions.

Setting read_ahead_kb in Kubernetes environment

Hi I am struggling to find way to set read_ahead_kb for Cassandra in our Kubernetes environment. Any way by which we can achieve this? My issue is that since PV is mounted in a container and it can have any name depending on partition number. Also wanted to know where it will be set, inside the

Re: Best strategy to run repair

n Mon, 22 Mar 2021 at 20:28, manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> Also try to use Cassandra reaper (as Kane also mentioned) for subrange >> repair. Doing subrange repair yourself may lead to a lot of trouble as >> calculating correct subranges is

Re: Best strategy to run repair

Also try to use Cassandra reaper (as Kane also mentioned) for subrange repair. Doing subrange repair yourself may lead to a lot of trouble as calculating correct subranges is not an easy task. On Tue, Mar 23, 2021 at 3:38 AM Kane Wilson wrote: > -pr on all nodes takes much longer as you'll do

Re: Anti Compactions while running repair

Thanks Alex On Mon, Nov 9, 2020 at 12:36 PM Alexander DEJANOVSKI wrote: > Only sstables at unrepaired state go through anticompaction. > > Le lun. 9 nov. 2020 à 07:01, manish khandelwal < > manishkhandelwa...@gmail.com> a écrit : > >> Thanks Alex. >> &g

Re: Issue with anti-compaction while running full repair with -pr option

Pushpendra, Probably you can read all the data using spark with Consistency level ALL for repairing the data. Regards Manish On Mon, Nov 9, 2020 at 11:31 AM Alexander DEJANOVSKI wrote: > Hi, > > You have two options to disable anticompaction when running full repair: > > - add the list of DCs

Re: Anti Compactions while running repair

cremental repair. > > > Le ven. 6 nov. 2020 à 04:37, manish khandelwal < > manishkhandelwa...@gmail.com> a écrit : > >> In documentation it is given that while running incremental repairs, anti >> compaction is done which results in repaired and unrepaired ss

Anti Compactions while running repair

In documentation it is given that while running incremental repairs, anti compaction is done which results in repaired and unrepaired sstables. Since anti compaction also runs with full repair and primary range repairs, I have the following question: Is anti compaction different in case of full

Re: Anti-entropy repair with a 4 node cluster replication factor 4

If you run full repair then it should be fine, since all the replicas are present on all the nodes. If you are using -pr option then you need to run on all the nodes. On Tue, Oct 27, 2020 at 4:11 PM Fred Al wrote: > Hello! > Running Cassandra 2.2.9 with a 4 node cluster with replication factor

Re: Cassandra scale-out with no traffic on newly joined nodes

Can you add new host as contact points and see if traffic lands on them or not? Also you can verify new nodes are added in system.peers of host name which you are giving as contact points On Tue, Sep 8, 2020 at 11:27 AM Sandeep Nethi wrote: > Yes, all nodes are UN and no issues identified.

Re: Rollback Cassandra after 1 node upgrade

Thanks Alex for the reply. On Sat, Sep 5, 2020 at 3:09 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Sat, Sep 5, 2020 at 5:55 AM manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> Hi >> >> We have been forced into rolling

Re: Rollback Cassandra after 1 node upgrade

Around 1TB On Sat, Sep 5, 2020 at 9:45 AM Aakash Pandhi wrote: > How much data to restore and repair on that node? > > Sincerely, > > Aakash Pandhi > > > On Friday, September 4, 2020, 11:08:56 PM CDT, manish khandelwal < > manishkhandelwa...@gmail.com> wrote:

Re: Rollback Cassandra after 1 node upgrade

3.11.2 to 2.1.16 On Sat, Sep 5, 2020 at 9:27 AM Surbhi Gupta wrote: > Hi Manish, > > Please provide both versions. > > Thanks > Surbhi > > On Fri, Sep 4, 2020 at 8:55 PM manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> Hi >> >&g

Rollback Cassandra after 1 node upgrade

Hi We have been forced into rolling back our Cassandra after 1 node upgrade. The node was upgraded 10 days ago. We have the backup of the old data. Strategy one which we are thinking : 1. Rollback to old binaries and configuration. 2. Restore the old data from backup. 3. Run Repair. Another

Re: efficient delete over a "wide" table?

> > Would there be a difference (in terms of generated tombstones) if I would > > a) issue delete one-by-one like > DELETE FROM ... WHERE ... uuid = 'a' > DELETE FROM ... WHERE ... uuid = 'b' > ... > DELETE FROM ... WHERE ... uuid = 'z' > > or > > b) issue delete in a group fashion like > DELETE

Re: CVE-2020-13946 Apache Cassandra RMI Rebind Vulnerability

Hi Sam Is there any alternative to avoid this vulnerability? Like upgrade to specific JVM version. Regards Manish On Tue, Sep 1, 2020 at 8:03 PM Sam Tunnicliffe wrote: > CVE-2020-13946 Apache Cassandra RMI Rebind Vulnerability > > Versions Affected: > All versions prior to: 2.1.22, 2.2.18,

Re: Getting Invalid Yaml exception after we upgraded java version

Pushpendra You can check this Datastax post . Looks like a similar error. You can run cassandra-stress utility to bypass the yaml parser. Regards Manish On Fri, Aug 21, 2020 at 11:29 AM

Re: Why a READ REPAIR ?

Hi Tobias READ2 will not be blocked by READ repair of READ1. Regards Manish On Tue, Aug 11, 2020 at 6:02 PM Tobias Eriksson wrote: > Thanx Erick, > > Perhaps this is super obvious but I need a confirmation as you say “…not > subsequent reads for other data unrelated to the read being

Re: Connect java application to Cassandra in Kubernetes

Erick, even if we give service as a contact point datastax driver will calculate IPs of topology to connect the cluster. So I think as John proposed in answer to similar question, after driver initialization in case of total outage of Cassandra cluster, bouncing of application is the only option.

Re: Cassandra on K8S

other nodes IP* Regards Manish On Tue, Aug 4, 2020 at 10:07 AM manish khandelwal < manishkhandelwa...@gmail.com> wrote: > But again if > Some Cassandra node (pod) with particular IP X is down, > Second Cassandra node (pod) tries to take the IP X of first Cassandra node, > Sec

Re: Cassandra on K8S

ondition of which container starts first. The k8s scheduler and > kubelet daemon assign IPs to pods. > > On Mon, Aug 3, 2020 at 11:14 PM manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> I have started reading about how to deploy Cassandra with K8S. But a

Re: Cassandra on K8S

en and how this IP swap can occur?* Regards Manish On Mon, Jul 6, 2020 at 10:40 PM Jeff Jirsa wrote: > > > On Mon, Jul 6, 2020 at 10:01 AM manish khandelwal < > manishkhandelwa...@gmail.com> wrote: > >> I have started reading about how to deploy Cassandra with K8S. But

Re: Replacing a Cassandra node in K8S

e correct that the volume will get remounted. > If however you are using a local or hostPath volume, then it will be lost > and you will want to restart Cassandra with the -Dcassandra.replace_address > option. > > On Mon, Jul 27, 2020 at 5:17 AM manish khandelwal < > manishkhandelwa...@

Replacing a Cassandra node in K8S

Hi Team I was wondering how Cassandra node is replaced if one of the worker node fails in k8s. My understanding is that since PVCs are remounted to their volume mounts, no matter where the pods are rescheduled (any node), so replacing a node will not be a issue only ip will get changed.

Re: Corrupt sstables_activity

I did a POC on CCM, removed sstable_activity sstables files from directory after stopping the node. Restarted the node, sstable_activity table was generated again. You can verify it in your test environment and see if node is working fine without any issues. Important thing is to validate any step

Cassandra on K8S

I have started reading about how to deploy Cassandra with K8S. But as I read more I feel there are a lot of challenges in running Cassandra on K8s. Some of the challenges which I feel are 1. POD IPs identification - If the pods go down and when they come up their IPs change, how is it handled as

Re: Implication of having similar private IPs in two datacenters

I did a small poc for this *(similar rpc_address for both data centers) . *My observations: 1. rpc_address is used by a client( datastax java driver) in my case to connect to cassandra nodes. 2. Driver is keeping a list of contact points which it is updating using system.peers table 3. Having two

Re: Cassandra upgrade from 3.11.3 -> 3.11.6

r/bootstrap) with different file versions is >>>> usually a problem. Running a mixed version cluster is fine – for the time >>>> you are doing the upgrade. I would not stay on mixed versions for any >>>> longer than that. It takes more time, but I separa

Re: Implication of having similar private IPs in two datacenters

The nodes within one datacenter will talk on private IPs and for communication across data center public ips are used. Does that answer your question? or you are asking in terms of network layer. As per this link

Re: Cassandra upgrade from 3.11.3 -> 3.11.6

Rightly said by Surbhi, it is not good to scale with mixed versions as debugging issues will be very difficult. Better to upgrade first and then scale. Regards On Wed, Jun 24, 2020 at 11:20 AM Surbhi Gupta wrote: > In case of any issue, it gets very difficult to debug when we have > multiple

Implication of having similar private IPs in two datacenters

Are there any implication if I have similar set of private IPs in two data centers. For example if I have A,B and C as private IPs in one data center and I define A,B and C as my private IPs in other data center, does it impact the cluster? Regards

Re: Consistency level shows as null in Java driver

This is how getConsistencyLevel method is implemented. This method returns consistencylevel of the query or null if no consistency level has been set using setConsistencyLevel. Regards Manish On Fri, Jun 12, 2020 at 3:43 PM Manu Chadha wrote: > Hi > > In my Cassandra Java driver code, I am

Re: Do I understand the working of RF and CL correctly?

Yes Manu you are correct in your understanding. Consistency level defines minimum number of nodes that must acknowledge a read or write operation before the operation can be considered as successful. In your case, since two nodes were up and you queried with CL 1, your request was successful. That

Re: TEST Cluster corrupt after removenode. how to restore

Even when one table is corrupt your repair will fail. To handle this case without data loss you could replace the complete node ( safest and sureshot option) On 19 May 2020 20:20, "Leena Ghatpande" wrote: > One of the tables SStable got corrupted on all nodes. But repairs were > failing for

Re: Can repair run when rebuild command is running on one of the node

Thanks Erick for quick response On Tue, May 12, 2020 at 9:11 AM Erick Ramirez wrote: > It isn't ideal to be running repairs while you're adding a new DC. When > you add nodes with auto_bootstrap: false, they immediately join the ring > but without data on them (which is by design). If repairs

Can repair run when rebuild command is running on one of the node

Hi Team I have a 5 node data center. If I create another datacenter, and run rebuild on nodes of node datacenter. While the rebuild command is running on one of the node of new datacenter can our schedule repair run along with it. I am using repair -pr Regards Manish

Re: Add a new node of 3.11.5 in a 3.11.0 Cassandra Cluster

You should not bootstrap with mixed cluster. First you should upgrade and then bootstrap a new node. If you are not able to upgrade due to disk space constraints while running upgradesstable, then 1. Reduce the number of threads for upgradesstable (--jobs parameter). By default it is 2. You can

Re: Impact of setting low value for flag -XX:MaxDirectMemorySize

I am running spark (max heap 4G) and a java application (4G) with my Cassandra server (8G). After heavy loading, if I run a spark process some main memory is pushed into swap. But if a restart Cassandra and execute the spark process memory is not pushed into the swap. Idea behind asking the

Impact of setting low value for flag -XX:MaxDirectMemorySize

What will be the impact of setting the value of XX:MaxDirectMemorySize to some low value. Currently default value for off heap is equal to heap memory. I saw this open ticket discussing this but could not infer much from it. https://issues.apache.org/jira/browse/CASSANDRA-10930 Regards Manish

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

To add on to my previous email, I think we get this error when actual off heap memory breaches "off heap memory limit" as mentioned in this JIRA <https://issues.apache.org/jira/browse/CASSANDRA-10930>. On Mon, Mar 30, 2020 at 3:10 PM manish khandelwal < manishkhandelwa...@gma

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

I have already updated that property to mmap_index_only. And in pmap I could see only index files are loaded into memory not the SSTables. I feel this is some to do with direct memory buffer. There is a opened JIRA ticket https://issues.apache.org/jira/browse/CASSANDRA-10930 as well regarding

Re: High value of "All Time Blocked" for "Native-Transport-Requests"

Thanks for the response. With continuous runs I am observing my off heap memory increasing and *memory usage remains high even if no traffic is running*. Sometimes a node is hung with following error: ERROR [ReadStage-5] 2020-03-12 20:47:46,406 JVMStabilityInspector.java:74 - OutOfMemory error

High value of "All Time Blocked" for "Native-Transport-Requests"

I am working with Cassandra 3.11.2. In nodetool tpstats output I am observing high number in "All Time Blocked" column for "Native-Transport-Requests" pool. I have two questions 1. Does a high number of "ALL Time Blocked" count impact client latency? 2. Does it increase off head memory overtime?

  1   2   >