Re: Secure data

2018-08-01 Thread Rahul Reddy
Hello, Any one tried aws ec2 volume encryption for Cassandra instances? On Tue, Jul 31, 2018, 12:25 PM Rahul Reddy wrote: > Hello, > > I'm trying to find a good document on to enable encryption for Apache > Cassandra (not on dse) tables and commilogs and store the keystore in km

Secure data

2018-07-31 Thread Rahul Reddy
Hello, I'm trying to find a good document on to enable encryption for Apache Cassandra (not on dse) tables and commilogs and store the keystore in kms or vault. If any of you already configured please direct me to documentation for it.

Re: Secure data

2018-08-01 Thread Rahul Reddy
gt; On Wed, Aug 1, 2018 at 12:36 PM Jeff Jirsa wrote: >>> >>>> EBS encryption worked well on gp2 volumes (never tried it on any others) >>>> >>>> -- >>>> Jeff Jirsa >>>> >>>> >>>> On Aug 1, 2018, at 7:57 AM, Rah

Datastax encryption with kms

2018-09-04 Thread Rahul Reddy
Hello, Has anyone done the dse cassandra stable/ commitlog tde encryption saving the keys in kms or vault instead of kmip. If it's possible please do let me know

Sub range repair

2019-01-01 Thread Rahul Reddy
Hello, Is it possible to find subrange needed for repair in Apache Cassandra like dse which uses dsetool list_subranges like below doc https://docs.datastax.com/en/archived/datastax_enterprise/4.8/datastax_enterprise/srch/srchRepair.html?hl=repair

Re: Tombstones in memtable

2019-02-23 Thread Rahul Reddy
2019, at 8:37 AM, Kenneth Brotman > wrote: > > Can we see the histogram? Why wouldn’t you at times have that many > tombstones? Makes sense. > > > > Kenneth Brotman > > > > *From:* Rahul Reddy [mailto:rahulreddy1...@gmail.com > ] > *Sent:* Thursday,

Re: Tombstones in memtable

2019-02-23 Thread Rahul Reddy
8192M" HEAP_NEWSIZE="800M" ``` On Sat, Feb 23, 2019, 10:15 PM Rahul Reddy wrote: > Thanks Jeff, > > Since low writes and high reads most of the time data in memtables only. > When I noticed intially issue no stables on disk everything in memtable > only. > > On Sat,

Re: Tombstones in memtable

2019-02-23 Thread Rahul Reddy
1.00 42.51152.32 124 1 ``` 3 node in dc1 and 3 node in dc2 cluster. With instanc type aws ec2 m4.xlarge On Sat, Feb 23, 2019, 7:47 PM Jeff Jirsa wrote: > Would also be good to see your schema (anonymized if needed) and the > select quer

Re: Tombstones in memtable

2019-02-23 Thread Rahul Reddy
bit > (given the sizes in your cfstats I’d probably go to 4K instead of 64k), and > maybe consider LCS or TWCS instead of STCS (Which is appropriate depends on > a lot of factors, but STCS is probably causing a fair bit of unnecessary > compactions and probably is very slow to expire d

Re: Tombstones in memtable

2019-02-23 Thread Rahul Reddy
Reads increase on all most all nodes same is the case with CPU. it's goes high on all nodes On Sat, Feb 23, 2019, 11:04 PM Kenneth Brotman wrote: > When the CPU utilization spikes from 5-10% to 50%, how many nodes does it > happen to at the same time? > > > > *

Re: Tombstones in memtable

2019-02-23 Thread Rahul Reddy
one problems with that table / that select. > > -- > Jeff Jirsa > > > On Feb 23, 2019, at 5:55 PM, Rahul Reddy wrote: > > Changing gcgs didn't help > > CREATE KEYSPACE ksname WITH replication = {'class': > 'NetworkTopologyStrategy', 'dc1': '3', 'dc2': '3'} AND durable_wri

Re: Tombstones in memtable

2019-02-24 Thread Rahul Reddy
blem are you solving here? If you’re serving reads > and sitting under 50% cpu, it’s not clear to me what you’re trying to fix. > Tombstones scanned won’t matter for your table, so if that’s your only > concern, I’d ignore it. > > > > -- > Jeff Jirsa > > > On Feb 23, 201

Tombstones in memtable

2019-02-21 Thread Rahul Reddy
We have small table records are about 5k . All the inserts comes as 4hr ttl and we have table level ttl 1 day and gc grace seconds has 3 hours. We do 5k reads a second during peak load. During the peak load seeing Alerts for tomstone scanned histogram reaching million. Cassandra version 3.11.1.

Maximum memory usage

2019-02-06 Thread Rahul Reddy
Hello, I see maximum memory usage alerts in my system.log couple of times in a day as INFO. So far I haven't seen any issue with db. Why those messages are logged in system.log do we have any impact for reads/writes with those warnings? And what nerd to be looked INFO [RMI TCP

Re: Maximum memory usage

2019-02-10 Thread Rahul Reddy
Thanks Kenneth, 110mb is the biggest partition in our db On Sun, Feb 10, 2019, 9:55 AM Kenneth Brotman wrote: > Rahul, > > > > Those partitions are tiny. Could you give us the table histograms for the > biggest tables. > > > > Thanks, > > > > K

Re: Maximum memory usage

2019-02-10 Thread Rahul Reddy
> very often or logging a OOM. > > Dinesh > > > On Wednesday, February 6, 2019, 6:19:42 AM PST, Rahul Reddy < > rahulreddy1...@gmail.com> wrote: > > > Hello, > > I see maximum memory usage alerts in my system.log couple of times in a > day as INFO. So far I

Re: Maximum memory usage

2019-02-10 Thread Rahul Reddy
us the “nodetool tablehistograms” > > > > Kenneth Brotman > > > > *From:* Rahul Reddy [mailto:rahulreddy1...@gmail.com] > *Sent:* Wednesday, February 06, 2019 6:19 AM > *To:* user@cassandra.apache.org > *Subject:* Maximum memory usage > > > > Hello, > >

Re: Maximum memory usage

2019-02-10 Thread Rahul Reddy
On Wednesday, February 6, 2019, 6:19:42 AM PST, Rahul Reddy < > rahulreddy1...@gmail.com> wrote: > > > Hello, > > I see maximum memory usage alerts in my system.log couple of times in a > day as INFO. So far I haven't seen any issue with db. Why those messages > ar

Cassandra.log

2019-02-10 Thread Rahul Reddy
Hello, I'm using Cassandra 3.11.1 and trying to change the name of cassandra.log(which is generate during startup of Cassandra) file. Can someone point me to configuration file where it is configured . System.log/debug.log located in logback.xml but not the Cassandra.log. Thanks

Re: Cassandra.log

2019-02-11 Thread Rahul Reddy
Thank you On Sun, Feb 10, 2019, 5:46 PM Sri Rathan Rangisetti < srirathan.rangise...@gmail.com> wrote: > It will be part of cassandra startup script, of you are using RHEL its > located at /etc/inti.d/cassandra > > > Regards > Sri Rathan > > On Sun, Feb 10, 2019

Sstableloader

2019-05-28 Thread Rahul Reddy
Hello, Does sstableloader works between datastax and Apache cassandra. I'm trying to migrate dse 5.0.7 to Apache 3.11.1 ?

Re: Sstableloader

2019-05-30 Thread Rahul Reddy
> On Wed, May 29, 2019, 5:56 PM Nitan Kainth wrote: > > If cassandra version is same, it should work > > > > Regards, > > Nitan > > Cell: 510 449 9629 > > > On May 28, 2019, at 4:21 PM, Rahul Reddy wrote: > > Hello, > > > > Does sstableloader works between datastax and Apache cassandra. I'm trying > to migrate dse 5.0.7 to Apache 3.11.1 ? > >

Adding new DC with different version of Cassandra

2019-07-01 Thread Rahul Reddy
Hello All, We have datastax Cassandra cluster which uses 3.11.0 and we want to add new DC with apache Cassandra 3.11.3. we tried doing the same and data got streamed to new DC. Since we are able to stream data any other issues we need to consider. Is it because of same type of sstables used in

Re: Adding new DC with different version of Cassandra

2019-07-01 Thread Rahul Reddy
3.11.0 > > > On Jul 1, 2019, at 3:25 AM, Rahul Reddy > wrote: > > > > Hello All, > > > > We have datastax Cassandra cluster which uses 3.11.0 and we want to add > new DC with apache Cassandra 3.11.3. we tried doing the same and data got > streamed to new DC. Sinc

New column

2019-08-18 Thread Rahul Reddy
Hello, We have a table and want to add column and select based on existing entire primary key plus new column using allow filtering. Since my where clause has all the primary key + new column does the allow filtering scan only the partions which are listed or does it has to scan whole table? What

Re: New column

2019-08-19 Thread Rahul Reddy
> > On Sun, Aug 18, 2019, 12:58 PM Rahul Reddy > wrote: > >> Hello, >> >> We have a table and want to add column and select based on existing >> entire primary key plus new column using allow filtering. Since my where >> clause has all the primary key + new

Re: Cassandra copy command

2019-08-21 Thread Rahul Reddy
consequentially repaired if data not match) as they > should. > > What copy command you are talking about precisely, why you cant use just > repair? > > On Wed, 21 Aug 2019 at 12:14, Rahul Reddy > wrote: > > > > Hello, > > > > I have 3 datacenters . Want

Cassandra copy command

2019-08-21 Thread Rahul Reddy
Hello, I have 3 datacenters . Want to make sure record count is same in all dc's . If I run copy command node1 in dc1 does it get the data from only dc1? Nodetool cfstats I'm seeing discrepancies in partitions count is it because we didn't run cleanup after adding few nodes and remove them?. To

Re: Cassandra copy command

2019-08-21 Thread Rahul Reddy
pair. But if you still want to do it, > use Spark for it. > > Jean Carlo > > "The best way to predict the future is to invent it" Alan Kay > > > On Wed, Aug 21, 2019 at 1:51 PM Rahul Reddy > wrote: > >> Yep I did run rebuild on each new node >> >

Re: Cassandra copy command

2019-08-21 Thread Rahul Reddy
ttps://docs.datastax.com/en/archived/cassandra/3.0/cassandra/operations/opsAddDCToCluster.html > > On Wed, 21 Aug 2019 at 12:57, Rahul Reddy > wrote: > > > > Hi sefan, > > > > I'm adding new DC3 to exiting cluster and see discripencies couple of > millions

Re: New column

2019-08-22 Thread Rahul Reddy
a cluster up, use the >> --host to point to it.If you don't have a cluster up, you can spin one >> up in AWS in about 5-10 minutes using our tools: >> https://thelastpickle.com/tlp-cluster/ >> >> Happy testing! >> Jon >> >> >> On Mon, Aug 19, 2019

Assassinate or decommission?

2019-07-30 Thread Rahul Reddy
Hello, While removing old data center any specific reason to use assassinate instead of decommission ? https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/operations/opsDecomissionDC.html

Re: Assassinate or decommission?

2019-07-30 Thread Rahul Reddy
tpickle.com/blog/2018/09/18/assassinate.html > > Rahul Reddy schrieb am Di., 30. Juli 2019, > 11:18: > >> Hello, >> >> While removing old data center any specific reason to use assassinate >> instead of decommission ? >> >> https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/operations/opsDecomissionDC.html >> >> >> >> >> >> >> >>

Cassandra node down metric

2019-07-30 Thread Rahul Reddy
Hello, I'm using jmx metric node org_apache_cassandra_net_failuredetector_downendpointcount to monitor number of Cassandra nodes down. For any reason (aws schedule retirement) we decommission Cassandra node this metric shows the node down for 72 hours until the gossip clearead. We want keep

Re: Jmx metrics shows node down

2019-07-29 Thread Rahul Reddy
< oleksandr.shul...@zalando.de> wrote: > On Mon, Jul 29, 2019 at 1:21 PM Rahul Reddy > wrote: > >> >> Decommissioned 2 nodes from cluster nodetool status doesn't list the >> nodes as expected but jmx metrics shows still those 2 nodes has down. >> Nodetool gossip shows

Jmx metrics shows node down

2019-07-29 Thread Rahul Reddy
Hello, Decommissioned 2 nodes from cluster nodetool status doesn't list the nodes as expected but jmx metrics shows still those 2 nodes has down. Nodetool gossip shows the 2 nodes in Left state. Why does my jmx still shows those nodes down even after 24 hours. Cassandra version 3.11.3 ? Anything

Re: Jmx metrics shows node down

2019-07-29 Thread Rahul Reddy
And also system.peers table doesn't have the information on old nodes only ghost nodes to be there in JMX On Mon, Jul 29, 2019, 7:39 AM Rahul Reddy wrote: > We removed many times nodes from a cluster but never seen the jmx metric > down stay for 72 hours. So it has to be completely r

Snitch

2019-07-17 Thread Rahul Reddy
Hello, All our infrastructure is in aws but we still use gossippropertyfilesnitch. We have two datacenters us_east_1 and us_west_2 trying to add us_east_1_analytics in the same aws DC as us_east_1and us_west_2_analytics in same aws DC us_west_2. Tried to use dc_suffix it seems to be helpful only

Rebooting one Cassandra node caused all the application nodes go down

2019-07-19 Thread Rahul Reddy
Here , We have 6 nodes each in 2 data centers us-east-1 and us-west-2 . We have RF 3 and cl set to local quorum. And gossip snitch. All our instance are c5.2xlarge and data files and comit logs are stored in gp2 ebs. C5 instance type had a bug which aws asked to set nvme_timeout to higher

Re: Rebooting one Cassandra node caused all the application nodes go down

2019-07-19 Thread Rahul Reddy
gt; reboot is same as that of rest of the nodes in the cluster. > 2. Java version is consistent across all nodes in the cluster. > > Do check and revert > > Thanks > > On Fri, 19 Jul, 2019, 9:19 PM Rahul Reddy, > wrote: > >> Here , >> >> We have 6 nodes each in 2

Re: Rebooting one Cassandra node caused all the application nodes go down

2019-07-19 Thread Rahul Reddy
l 19, 2019, at 12:32 PM, ZAIDI, ASAD A wrote: > > “aws asked to set nvme_timeout to higher number in etc/grub.conf.” > > > > Did you ask AWS if setting higher value is real solution to bug - Is there > not any patch available to address the bug? - just curios to know >

Re: Rebooting one Cassandra node caused all the application nodes go down

2019-07-19 Thread Rahul Reddy
> >> Regards, >> >> Nitan >> >> Cell: 510 449 9629 >> >> On Jul 19, 2019, at 12:32 PM, ZAIDI, ASAD A wrote: >> >> “aws asked to set nvme_timeout to higher number in etc/grub.conf.” >> >> >> >> Did you ask AWS if setti

Dse keypsaces in apache cluster

2019-07-24 Thread Rahul Reddy
Hello, I have 2 data centers with dse Cassandra added new DC with apache Cassandra . Dse_perf,dse_systen,dse_security,dse_leases keypsaces created as well . Can we delete them ?

Re: Dse keypsaces in apache cluster

2019-07-24 Thread Rahul Reddy
or info on their product, and be > very careful. Also note I don't have specific experience with your exact > scenario. > > (My experience with Datastax is version 4.6-5.0) > > On Wed, Jul 24, 2019, 10:34 AM Rahul Reddy > wrote: > >> Hello, >> >> I have 2

Re: Aws instance stop and star with ebs

2019-11-05 Thread Rahul Reddy
ntain confidential and legally > privileged information. If you are not the intended recipient, do not copy > or disclose its content, but please reply to this email immediately and > highlight the error to the sender and then immediately delete the message. > > > On Wed, 6 Nov 2019

Re: Aws instance stop and star with ebs

2019-11-05 Thread Rahul Reddy
[RequestResponseStage-7] 2019-11-05 15:16:27,166 Gossiper.java:1019 - InetAddress /nodewhichitwasshowing down is now UP what is causing delay for 10mins to be able to say that node is reachable On Wed, Oct 30, 2019, 8:37 AM Rahul Reddy wrote: > And also aws ec2 stop and start comes with new insta

Re: Aws instance stop and star with ebs

2019-11-06 Thread Rahul Reddy
tes, that sounds suspiciously like blocking on a default TCP > socket timeout. > > > > *From: *Rahul Reddy > *Reply-To: *"user@cassandra.apache.org" > *Date: *Wednesday, November 6, 2019 at 12:12 PM > *To: *"user@cassandra.apache.org" > *Subject: *Re: Aw

Re: Aws instance stop and star with ebs

2019-11-06 Thread Rahul Reddy
lt;==> > Made weak by time and fate, but strong in will, > To strive, to seek, to find, and not to yield. > Ulysses - A. Lord Tennyson > > *Daemeon C.M. Reiydelle* > > *email: daeme...@gmail.com * > *San Francisco 1.415.501.0198/Skype daemeon.c.m.reiydelle* > >

Re: Aws instance stop and star with ebs

2019-10-30 Thread Rahul Reddy
And also aws ec2 stop and start comes with new instance with same ip and all our file systems are in ebs mounted fine. Does coming new instance with same ip cause any gossip issues? On Tue, Oct 29, 2019, 6:16 PM Rahul Reddy wrote: > Thanks Alex. We have 6 nodes in each DC with RF=3 with

Re: Aws instance stop and star with ebs

2019-11-06 Thread Rahul Reddy
imeout as I recall. > > You may be forced to tcpdump the interface(s) to see where the chatter is. > Out of curiosity, when you restart the node, have you snapped the jvm's > memory to see if e.g. heap is even in use? > > > On Tue, Nov 5, 2019 at 7:03 PM Rahul Reddy > wro

Aws instance stop and star with ebs

2019-10-29 Thread Rahul Reddy
Hello, We have our infrastructure on aws and we use ebs storage . And aws was retiring on of the node. Since our storage was persistent we did nodetool drain and stopped and start the instance . This caused 500 errors in the service. We have local_quorum and rf=3 why does stopping one instance

Re: Aws instance stop and star with ebs

2019-10-29 Thread Rahul Reddy
determine what caused the Cassandra unavialbe though log says its stared and listening ? On Tue, Oct 29, 2019, 4:44 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Tue, Oct 29, 2019 at 9:34 PM Rahul Reddy > wrote: > >> >> We have our infrastructure on

"Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB"

2019-12-01 Thread Rahul Reddy
Hello, We are seeing memory usage reached 512 mb and cannot allocate 1MB. I see this because file_cache_size_mb by default set to 512MB. Datastax document recommends to increase the file_cache_size. We have 32G over all memory allocated 16G to Cassandra. What is the recommended value in my

Re: "Maximum memory usage reached (512.000MiB), cannot allocate chunk of 1.000MiB"

2019-12-02 Thread Rahul Reddy
ite heavy. Real time > applications are different than time series data environments and ... . > > > > On Sun, Dec 1, 2019 at 7:09 PM Rahul Reddy > wrote: > >> Hello, >> >> We are seeing memory usage reached 512 mb and cannot allocate 1MB. I see >

Null values in sasi indexed column

2020-02-19 Thread Rahul Reddy
Hello, We have a requirement to query based on a column which is not part of priamry key. We are planning to create sasi index . Before creating index want to confirm how would be index behaviour if it has 10% of data will be null and data distribution across the nodes 6-6 nodes wth RF 3? Please

Ec2 instance transient network issues caused 500 errors

2019-12-30 Thread Rahul Reddy
Hello, We have our cassandra cluster running on aws and we have 2 dc’s 6 and 6 nodes in both regions with RF=3 and cassandra version 3.11.3 . One of the ec2 instance had issues with network issues for 9 minutes. Since it was network issue neither ec2 was down nor cassandra down. But this

Connections limit on gocql

2020-05-26 Thread Rahul Reddy
Hello, Our service code is written in goland uses gocql to connect todb and we have 30 node cluster. And we have service connections coming from kubernettes pods when we scaleup pods to 500 for each pod we are getting connections 1500 per each node. We are not seeing any issues now but wanted to

Memory decline

2020-06-18 Thread Rahul Reddy
Hello, Im seeing continuous decline in memory on a Cassandra instance used to have 20g free memory 15 days back and now its 15g and continue to go down. Same instance it caused the cassandra instance crash before. Can you please give me some pointers to look for which is causing continuous

Re: Memory decline

2020-06-19 Thread Rahul Reddy
ve seen leak. If you’ve configured to have >> swap=0, then you end up in a position where even if the memory usage is >> stale, nothing can push the stale pages out of the way. >> >> >> >> *From: *Rahul Reddy >> *Reply-To: *"user@cassandra.apache.org

Re: Log4j vulnerability

2021-12-13 Thread Rahul Reddy
Hello, I see this jar log4j-over-slf4j-1.7.7.jar does it have any impact on it? Why that jar is used for ? On Sat, Dec 11, 2021 at 12:45 PM Brandon Williams wrote: > https://issues.apache.org/jira/browse/CASSANDRA-5883 > > As that ticket shows, Apache Cassandra has never used log4j2. > >