Re: Cassandra failed to start when trying to move to multi-node

> initial_token is commented. > > The server has been just installed. > Thanks for your help > > 2017-08-01 17:01 GMT-03:00 Nitan Kainth <ni...@bamlabs.com>: > >> If it is blank, you can use bootstrap: true. >> >> What is num_tokens and initial token values? >&g

Re: Cassandra failed to start when trying to move to multi-node

If it is blank, you can use bootstrap: true. What is num_tokens and initial token values? > On Aug 1, 2017, at 2:42 PM, Lucas Alvarez wrote: > > Hi, I'm trying to configure Cassandra as a cluster with two nodes. When > trying to simple start the first node just changing

Re: private interface for interdc messaging

You can setup broadcast address for the IP on which Nodes will communicate with each other. You network team can setup routing table from source to target. We had similar setup done in one of my previous project where we segregated network between application and C* nodes communication. > On

Re: private interface for interdc messaging

Yes. Because that's the ip used for internode communication Sent from my iPhone > On Jul 7, 2017, at 10:52 AM, CPC <acha...@gmail.com> wrote: > > Hi Nitan, > > Do you mean setting broadcast_address to private network would suffice? > >> On 7 July 2

Re: Starting Cassandrs after restore of Data - get error

Jonathan, Make sure initial tokens have values from back up cluster i.e. 256 tokens. It is possible to have typo. > On Jul 7, 2017, at 9:14 AM, Jonathan Baynes > wrote: > > Hi again, > > Trying to restart my nodes after restoring snapshot data, initial tokens

Re: Starting Cassandrs after restore of Data - get error

lusters match I've checked 3 Times and diff'd it as well. Would > file format have any affect I'm amending on windows machine and returning the > file back to Linux > > Thanks > J > > Sent from my iPhone > > On 7 Jul 2017, at 17:43, Nitan Kainth <ni...@

Re: Simulating Read and WriteTimeouts programatically

You can set timeout value in cqlshrc file. But you will have to overload the cluster to take these into effect. Sent from my iPhone > On Jul 6, 2017, at 10:08 AM, Charulata Sharma (charshar) > wrote: > > Hi All, > > Recently I learnt that in C* driver versions

Re: first node in a cluster - should it be a seed node

No, any node can be seed node. But to start the cluster and nodes addition you need some node as seed node. Make sure to include at least one node from each DC as seed. DO NOT make all nodes as seed. > On Jul 14, 2017, at 10:08 AM, Vikram Goyal G > wrote: > >

Re: private interface for interdc messaging

; dc1 public => dc2 private > > traffic. We we want is: > > dc1 private => dc2 private > > Any idea? Also could it be related to > https://issues.apache.org/jira/browse/CASSANDRA-12673. Any idea? > >> On 7 July 2017 at 21:51, CPC <acha...@gmail.com> wrote: >

write time for nulls is not consistent

Hi, We see that null columns have writetime(column) populated for few columns and shows null for few other. Is it any bug or something else? CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy', 'us-east': '2'} AND durable_writes = true; CREATE TABLE test.t ( a

Re: write time for nulls is not consistent

or b. > Since there's no value, there's also no writetime. > > On Tue, Jul 18, 2017 at 12:15 PM Nitan Kainth <ni...@bamlabs.com > <mailto:ni...@bamlabs.com>> wrote: > Hi, > > We see that null columns have writetime(column) populated for few columns and > s

Re: write time for nulls is not consistent

don't > know how to answer the question if you're not actually showing how to > reproduce the problem. > > > On Tue, Jul 18, 2017 at 12:24 PM Nitan Kainth <ni...@bamlabs.com > <mailto:ni...@bamlabs.com>> wrote: > Jonathan, > > Please notice last rows wi

Re: write time for nulls is not consistent

Jeff, It is really strange, look at below log, I inserted your data and then few additional; finally, the issue is reproduced: [cqlsh 5.0.1 | Cassandra 3.0.10.1443 | DSE 5.0.4 | CQL spec 3.4.0 | Native protocol v4] Use HELP for help. cqlsh> CREATE KEYSPACE test WITH replication =

read/write request counts and write size of each write

Hi, We are trying to figure our highest read/write operations/second in our 9x9 cluster. I checked node tool tablestats but it shows from the time of start of node. I would like to see by second. Also, how can i find out size of each right request?

Re: write time for nulls is not consistent

Thank you Jeff. Please share once, the Patch is available for us to apply. > On Jul 27, 2017, at 2:12 PM, Jeff Jirsa <jji...@apache.org> wrote: > > > > On 2017-07-20 08:46 (-0700), "Jeff Jirsa"<jji...@apache.org> wrote: >> >> >> On 20

Re: read/write request counts and write size of each write

Thank you very much Kurt. I am not a java guy, need one small help. I initiated JMX connection but I am getting some exception: java -jar ~/jmxterm-1.0-alpha-4-uber.jar --url localhost:7199 Welcome to JMX terminal. Type "help" for available commands. $>run -b

Re: read/write request counts and write size of each write

11:50 AM, Roger Warner <rwar...@pandora.com> wrote: > > You need to also have the mx4j jar in your Cassandra lib directory. Double > checking you did that – its not included with the distro.You have to > download it. > > http://mx4j.sourceforge.net/ <http://mx4j.

Re: write time for nulls is not consistent

Same output Kurt, [cqlsh 5.0.1 | Cassandra 3.0.10.1443 | DSE 5.0.4 | CQL spec 3.4.0 | Native protocol v4] Use HELP for help. cqlsh> select a, writetime(b) from test.t; a | writetime(b) ---+-- z | null a | null c | null e |

Re: Adding a New Node With The Same IP of an Old Node

Try rolling restart of cluster as solution for schema version mismatch. Sent from my iPhone > On Jul 4, 2017, at 8:31 AM, Shalom Sagges wrote: > > Hi Experts, > > My plan is to upgrade the C* nodes' OS from Centos6 to Centos7. > Since an upgrade wasn't recommended I

Re: Linux version update on DSE

<https://overview.mail.yahoo.com/mobile/?.src=Android> > On Tue, Jun 27, 2017 at 11:23 PM, Nitan Kainth > <ni...@bamlabs.com> wrote: > Anuj, > > We did that in past, even if data was not removed, replace_node caused data > streaming. So changing IP is simplest and safes

Re: Cassandra Cluster Expansion Criteria

Ideally you should maintain 50% disk space. SLA and Node load is also very important to make the decision. > On Jun 29, 2017, at 6:45 AM, ZAIDI, ASAD A wrote: > > Hello Folks, > > I’m on Cassandra 2.2.8 cluster with 14 nodes , each with around 2TB of data > volume. I’m

Re: Seed gossip version will not connect with that version

Restart could help with hung gossip but check you network as possible root cause for this. Sent from my iPhone > On Jul 5, 2017, at 7:23 AM, Jean Carlo wrote: > > Hello > > I have repairs that hangs because this problem > > WARN

Re: read/write request counts and write size of each write

Thank you Kurt. I got read/write requests answered. Getting write request size is my unresolved question :( I m sure, it's a common requirement, anybody have some solution. > On Jul 25, 2017, at 6:23 PM, kurt greaves wrote: > > You will need to use jmx to collect

Re: AWS Cassandra backup/Restore tools

Also , Opscenter backup/restore does not work for large databases Sent from my iPhone > On May 11, 2017, at 3:41 PM, Blake Eggleston wrote: > > OpsCenter 6.0 and up don't work with Cassandra. > >> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote: >>

Re: sstablesplit - status

You can try running major compaction to get rid of duplicate data and deleted data. But will be the routine for future. > On May 17, 2017, at 10:23 AM, Jan Kesten wrote: > > me patt

Amazon linux upgrade

Hi, We are planning to upgrade Amazon linux to 2017.03. Can someone please point us to compatibility matrix for linux with C*? We are currently at C* 3.0.10.1443 and Linux: NAME="Amazon Linux AMI” VERSION=“2016.03" Thank you

Re: sstablesplit - status

and that is likely to never get compacted > without running major compaction again. And therefore not recommended for > production system. > > Hannu > >> On 17 May 2017, at 19:46, Nitan Kainth <ni...@bamlabs.com >> <mailto:ni...@bamlabs.com>> wrot

Re: Long running compaction on huge hint table.

uja <saluj...@gmail.com> wrote: > > Hi Nitan, > > Thanks for response. > > Yes, I could see mutation drops and increase count in system.hints. Is there > any way , i can proceed to truncate hints like using nodetool truncatehints. > > > Regards, > Varun S

Re: Long running compaction on huge hint table.

stem.hints. Is there > any way , i can proceed to truncate hints like using nodetool truncatehints. > > > Regards, > Varun Saluja > >> On 16 May 2017 at 17:52, Nitan Kainth <ni...@bamlabs.com> wrote: >> Do you see mutation drops? >> Select

Re: Range deletes, wide partitions, and reverse iterators

Hannu, How can you read a partition in reverse? Sent from my iPhone > On May 16, 2017, at 9:20 AM, Hannu Kröger wrote: > > Well, I’m guessing that Cassandra doesn't really know if the range tombstone > is useful for this or not. > > In many cases it might be that the

Read timeouts

Hi, We see read timeouts intermittently. Mostly after they have occurred. Timeouts are not consistent and does not occur in 100s at a moment. 1. Does read timeout considered as Dropped Mutation? 2. What is best way to nail down exact issue of scattered timeouts? Thank you.

Re: Long running compaction on huge hint table.

preciate your help. > > Can anyone suggest parameter change or something which can help in this > situation. > > Regards, > Varun > > Sent from my iPhone > >> On 16-May-2017, at 7:31 PM, Nitan Kainth <ni...@bamlabs.com> wrote: >> >> If

Re: Read timeouts

Thank you Jeff. We are at Cassandra 3.0.10 Will look forward to upgrade or enable driver logging. > On May 16, 2017, at 11:44 AM, Jeff Jirsa <jji...@apache.org> wrote: > > > > On 2017-05-16 08:53 (-0700), Nitan Kainth <ni...@bamlabs.com> wrote: >>

Re: Range deletes, wide partitions, and reverse iterators

stef...@gmail.com>> wrote: >> >> That is another way to see the question: are reverse iterators range >> tombstone aware? Yes. >> That is why I am puzzled by this afore-mentioned behavior. >> I would expect them to handle this case more gracefully. >> >

Re: Unsuccessful back-up and restore with differing counts

As Jonathan mentioned, if you are using vnodes , you should back up nodetool output and assign same token to nodes and then copy corresponding sstables. If using, initial token, then assign same value for subsequent node. Sstable loader should work independently, not sure why you are getting

Re: Long running compaction on huge hint table.

Do you see mutation drops? Select count from system.hints; is it increasing? Sent from my iPhone > On May 16, 2017, at 5:52 AM, varun saluja wrote: > > Hi Experts, > > We are facing issue on production cluster. Compaction on system.hint table is > running from last 2

Re: Linux version update on DSE

you are using same configs and if you have hard > coded the token(s), you use the same. > > Hannu > >> On 26 Jun 2017, at 23.24, Nitan Kainth <ni...@sleepiqlabs.com> wrote: >> >> Hi, >> >> We are planning to update linux for C* nodes version 3.0. A

Re: Linux version update on DSE

n, upgrade the C* version, then start it back up. >> >> >> Jon >> >> On Mon, Jun 26, 2017 at 3:56 PM Nitan Kainth <ni...@bamlabs.com >> <mailto:ni...@bamlabs.com>> wrote: >> It's vnodes. We will add to replace new ip in yaml as

Linux version update on DSE

Hi, We are planning to update linux for C* nodes version 3.0. Anybody has steps who did it recent past. Here are draft steps, we are thinking: 1. Create new node. It might have a different IP address. 2. Detach mounts from existing node 3. Attach mounts to new Node 4. Start C*

Re: Linux version update on DSE

Anuj, We did that in past, even if data was not removed, replace_node caused data streaming. So changing IP is simplest and safest option. > On Jun 27, 2017, at 12:43 PM, Anuj Wadehra wrote: > > replace_address_first_boot

Re: Linux version update on DSE

up. >> >> Try it and let us know if it works for you. >> >> Thanks >> Anuj >> >> >> >> On Tue, Jun 27, 2017 at 4:56 AM, Nitan Kainth >> <ni...@bamlabs.com <mailto:ni...@bamlabs.com>> wrote: >> Right, we are just upgr

Re: Amazon linux upgrade

st tarball) > > > Best regards, Vladimir Yudovin, > Winguzone <https://winguzone.com/?from=list> - Cloud Cassandra Hosting > > > ---- On Mon, 15 May 2017 14:34:56 -0400 Nitan Kainth <ni...@bamlabs.com> > wrote > > Hi, > > We are pla

Re: Cassandra Node Density thresholds

Asad, Ideally you should keep 50% free disk space. Yes, right now you have good option to scale horizontally or add more disk space if you can. Adding more nodes will give you more scalability for reads/writes and disk space will just give you more capacity for current load. Sent from my

Re: Long running compaction on huge hint table.

action pendings is also dropped today. Because we have to redump table > again once cluster is stable. > > Regards, > Varun > > Sent from my iPhone > >> On 16-May-2017, at 6:59 PM, Nitan Kainth <ni...@bamlabs.com> wrote: >> >> Yes but it means data has to

Re: Range deletes, wide partitions, and reverse iterators

Thank you Stefano > On May 16, 2017, at 10:56 AM, Stefano Ortolani <ostef...@gmail.com> wrote: > > No, because C* has reverse iterators. > > On Tue, May 16, 2017 at 4:47 PM, Nitan Kainth <ni...@bamlabs.com > <mailto:ni...@bamlabs.com>> wrote: > If the dat

Re: Replication issue with Multi DC setup in cassandra

; > On Tue, May 16, 2017 at 2:19 PM, Nitan Kainth <ni...@bamlabs.com > <mailto:ni...@bamlabs.com>> wrote: > Do you see data on other DC or just directory structure? Directory structure > would populate because it is DDL but inserts shouldn’t populate, ideally. > >

Re: Replication issue with Multi DC setup in cassandra

Do you see data on other DC or just directory structure? Directory structure would populate because it is DDL but inserts shouldn’t populate, ideally. > On May 16, 2017, at 3:19 PM, suraj pasuparthy > wrote: > > elp me fig

Re: Local_serial >> Adding nodes

Is the streaming still going on? Timeouts are in reads or writes? Sent from my iPhone > On Jun 6, 2017, at 9:50 PM, vasu gunja wrote: > > Hi All, > > We are having 2 DC setup each consists of 20 odd nodes and recently we > decided to add 6 more nodes to DC1. We are

Re: Local_serial >> Adding nodes

What is in system log? Does it show new tokens allocated? nodetool netstats > On Jun 7, 2017, at 6:50 AM, vasu gunja <vasu.no...@gmail.com> wrote: > > Yeah all of them are in UJ state.. > > Thanks, > Vasu > > On Jun 7, 2017, at 6:32 AM, Nitan Kainth &l

Re: Tool to manage cassandra

you can try atlas from netflix, it is open source. > On Jun 16, 2017, at 8:52 AM, Surbhi Gupta wrote: > > If u are using dse then u can use opscenter > > On Fri, Jun 16, 2017 at 6:01 AM Ram Bhatia > wrote: >

Re: Node replacement strategy with AWS EBS

Steps are good Rutvij. Step 1 is not mandatory. We snapshot EBS volume and then restored on new node. How are you re-attaching EBS volume without snapshot? I > On Jun 13, 2017, at 10:21 AM, Rutvij Bhatt wrote: > > Hi! > > We're running a Cassandra cluster on AWS. I want

Re: Node replacement strategy with AWS EBS

Hannu, "Cassandra automatically realizes that have just effectively changed IP address” —> are you sure C* will take care of IP change as is? How will it know which token range to be assigned to this new IP address? > On Jun 13, 2017, at 10:51 AM, Hannu Kröger wrote: > >

Re: Node replacement strategy with AWS EBS

82 > <https://issues.apache.org/jira/browse/CASSANDRA-8382> > > If you copy all cassandra data, you are on the safe side. Good point in the > links is that if you have IP addresses in topolgy or other files, then update > those as well. > > Hannu > > On 13 June

Re: Cassandra Schema version mismatch

tified Architect / Cassandra MVP > > Pythian - Love your data > > rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: > linkedin.com/in/carlosjuzarterolo > <http://linkedin.com/in/carlosjuzarterolo> > Mobile: +351 918 918 100 > www.pythian.com &l

Cassandra Schema version mismatch

Hi Experts, We found schema version mismatch in our cluster. We fixed it by bouncing C* on nodes where version was mismatched. Can someone suggest, what are the possible reasons for this? We are trying to figure out the root cause. thank you!

Re: Cassandra Schema version mismatch

rds, > > Carlos Juzarte Rolo > Cassandra Consultant / Datastax Certified Architect / Cassandra MVP > > Pythian - Love your data > > rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin: > linkedin.com/in/carlosjuzarterolo > Mobile: +351 918 918 100 > www.p

Re: Smart Table creation for 2D range query

Make metadata as partition key and x,y as part of partition key i.e. Primary key. It should work Sent from my iPhone > On May 5, 2017, at 2:40 PM, Lydia wrote: > > Hi all, > > I am new to Apache Cassandra and I would like to get some advice on how to > tackle a

Re: Cassandra Schema version mismatch

No, just two nodes have mismatch out of 18 nodes. We upgraded long back. Sent from my iPhone > On May 5, 2017, at 5:17 PM, Jeff Jirsa <jji...@apache.org> wrote: > > > >> On 2017-05-05 11:00 (-0700), Nitan Kainth <ni...@bamlabs.com> wrote: >> Hi Exp

Re: Cassandra Schema version mismatch

t; > Sent from my iPhone > >> On May 5, 2017, at 3:17 PM, Jeff Jirsa <jji...@apache.org> wrote: >> >> >> >>> On 2017-05-05 11:00 (-0700), Nitan Kainth <ni...@bamlabs.com> wrote: >>> Hi Experts, >>> >>> We foun

Re: Impact on latency with larger memtable

Larger memtable means mor time during flushes and larger heap means longer GC pauses. You can see these in system log Sent from my iPhone > On May 24, 2017, at 11:31 AM, preetika tyagi wrote: > > Hi, > > I'm experimenting with memtable/heap size on my Cassandra

Re: Reg:- Install / Configure Cassandra on 2 DCs with 3 nodes

Nandan, Use one node from each DC in seeds parameter on all nodes. Use right DC names and DC suffix to ensure correctly identify. Use network topology and RF on all keyspaces for both DCs Please post specific questions if you have any. Regards, Nitan K. Cassandra and Oracle Architect/SME

Re: Manual Compaction in cassandra

There will be one table after major compaction for all existing tables. But all new sstable will keep on creating while compaction is in progress. Sent from my iPhone > On Oct 4, 2017, at 7:50 AM, Akshit Jain wrote: > > If Size Tiered Compaction Strategy is used, then

Re: split one keyspace from one cluster to another

Modify your code to write to both clusters and migrate existing data in the background. Sent from my iPhone > On Oct 16, 2017, at 7:31 AM, Peng Xiao <2535...@qq.com> wrote: > > Dear All, > > We'd like to migrate one keyspace from one cluster to another,the keyspace is > about 100G. > If we

Re: Restore cassandra snapshots

Sstableloader is good for small dataset, for bigger snapshots restore is a better option Sent from my iPhone > On Oct 16, 2017, at 7:28 AM, Jean Carlo wrote: > > HI, > > Yes of course, you can use sstableloader from every sstable to your new > cluster. Actually

Re: Restore cassandra snapshots

shot - do you > suggest using sstableloader 2) What do you mean by "restore option" - do > you mean copying snapshot dir directly to the nodes of the new cluster ? > > On Mon, Oct 16, 2017 at 6:19 PM, Nitan Kainth <nitankai...@gmail.com> > wrote: > >> Ss

Re: Schema Mismatch Issue in Production

Rolling restart after making DDL changes saves us. We saw this because of race condition in our app servers, but it could happen for various other reasons like node overloaded, network etc Sent from my iPhone > On Oct 12, 2017, at 3:46 AM, Carlos Rolo wrote: > > Which

Re: Tuning bootstrap new node

Do not stop compaction, you will end up with thousands of sstables. You increase stream throughput from default 200 to a heifer value if your network can handle it. Sent from my iPhone > On Oct 31, 2017, at 6:35 AM, Peng Xiao <2535...@qq.com> wrote: > > Can we stop the compaction during the

Re: Academic paper about Cassandra database compaction

Hi Lucas, I am not able to download. can you share as attachment in email? Regards, Nitan K. Cassandra and Oracle Architect/SME Datastax Certified Cassandra expert Oracle 10g Certified On Mon, May 14, 2018 at 9:12 AM, Lucas Benevides < lu...@maurobenevides.com.br> wrote: > Dear community, >

Re: Switching to TWCS

Is old data TTLed already? If not, then I don't think TWCS will know when to delete data. My understanding about TWCS is, data has to be written with TTL. (Please correct me, if wrong) Regards, Nitan K. Cassandra and Oracle Architect/SME Datastax Certified Cassandra expert Oracle 10g Certified

Re: what happens for concurrent node add/removal?

Seed node takes care of token re-assignment. But why do you even wanna do that? Add the node first and then remove. Sent from my iPhone > On Apr 28, 2018, at 5:10 AM, Jinhua Luo wrote: > > HI All, > > For example, if at the same time, two nodes joining and there nodes

Re: Insert-only application repair

If you have RF>CL then Repair needs to be run to make sure data is in sync. Sent from my iPhone > On May 12, 2018, at 3:54 AM, onmstester onmstester > wrote: > > > In an insert-only use case with TTL (6 months), should i run this command, > every 5-7 days on all the

Log application Queries

Hi, I would like to log all C* queries hitting cluster. Could someone please tell me how can I do it at cluster level? Will nodetool setlogginglevel work? If so, please share example with library name. C* version 3.11

Re: Log application Queries

s used for changing the > logging level as below but not for slow query . > >- ALL >- TRACE >- DEBUG >- INFO >- WARN >- ERROR >- OFF > > > > On 25 May 2018 at 08:24, Nitan Kainth <nitankai...@gmail.com> wrote: > >> Hi,

Re: Log application Queries

nly valid for below : > > >- org.apache.cassandra >- org.apache.cassandra.db >- org.apache.cassandra.service.StorageProxy > > > On 25 May 2018 at 09:01, Nitan Kainth <nitankai...@gmail.com> wrote: > >> Thanks Surbhi. I found another way. I used nod

Re: Log application Queries

assandra 4.0 > > > > On Fri, May 25, 2018 at 8:24 AM, Nitan Kainth <nitankai...@gmail.com> > wrote: > >> Hi, >> >> I would like to log all C* queries hitting cluster. Could someone please >> tell me how can I do it at cluster level? >> Will nodetool setl

Re: Log application Queries

t; Oracle 10g Certified >> >> On Fri, May 25, 2018 at 12:15 PM, Surbhi Gupta <surbhi.gupt...@gmail.com> >> wrote: >> >>> nodeool setlogginglevel is only valid for below : >>> >>> >>>- org.apache.cassandra >>>- org.

Re: repair in C* 3.11.2 and anticompactions

Jean, Have you considered nodetool repair -pr (primary range) OR reaper? With Reaper you can throttle repair load on system. These two uses ranges anyway, so you may not run into anti-compaction. Regards, Nitan K. Cassandra and Oracle Architect/SME Datastax Certified Cassandra expert Oracle 10g

Re: Snapshot SSTable modified??

Jeff, Shouldn't Snapshot get consistent state of sstables? -tmp file shouldn't impact backup operation right? Regards, Nitan K. Cassandra and Oracle Architect/SME Datastax Certified Cassandra expert Oracle 10g Certified On Wed, May 23, 2018 at 6:26 PM, Jeff Jirsa wrote: >

Re: How to identify which table causing Maximum Memory usage limit

thanks Martin. 99 percentile of all tables are even size. Max is always higher in all tables. The question is, How do I identify, which table is throwing this "Maximum memory usage reached (512.000MiB)" usage message? On Mon, Jun 11, 2018 at 5:59 AM, Martin Mačura wrote: > Hi, > we've had

Re: How to identify which table causing Maximum Memory usage limit

Sorry, I didn't mean to high jack the thread. But I have seen similar issues and ignore it always because it wasn't really causing any issues. But I am really curious on how to find these. On Mon, Jun 11, 2018 at 9:45 AM, Nitan Kainth wrote: > thanks Martin. > > 99 percentile of a

Re: Read Latency Doubles After Shrinking Cluster and Never Recovers

Did you run cleanup too? On Mon, Jun 11, 2018 at 10:16 AM, Fred Habash wrote: > I have hit dead-ends every where I turned on this issue. > > We had a 15-node cluster that was doing 35 ms all along for years. At > some point, we made a decision to shrink it to 13. Read latency rose to > near 70

Re: Read Latency Doubles After Shrinking Cluster and Never Recovers

d it have impacted read latency the > fact that some nodes still have sstables that they no longer need? > > Thanks > > ---- > Thank you > > From: Nitan Kainth > Sent: Monday, June 11, 2018 10:18 AM > To: user@cassandra.apache.org > Subject: Re: Rea

Re: Caused by: java.lang.RuntimeException: org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 0 responses.

Check logs in your nodes, there could be GC or other issues causing timeouts. This can happen for various reasons, network blips, node down, node over loaded etc. On Thu, Jun 7, 2018 at 6:16 AM, Saurav Suman wrote: > Hi Team, > > > I am getting below error in all the nodes of cassandra . Can

Re: Cassandra 3.0.X migarte to VPC

Riccardo, Simplest method can be is to add VPC as an additional datacenter to existing cluster; once New DC has synced up all data, just switch over your application. Only caveat is that there should be enough network bandwidth between EC2 and VPC. Horizontal scaling will help to some extent.

Re: Cassandra 3.0.X migarte to VPC

Jeff, In this case, if Riccardo is adding new DC, then he can pickup new snitch protocol right? On Thu, Jun 7, 2018 at 12:15 PM, Jeff Jirsa wrote: > > > > On Thu, Jun 7, 2018 at 9:12 AM, Nitan Kainth > wrote: > >> Riccardo, >> >> Simplest method c

Re: Cassandra 3.0.X migarte to VPC

discovered. > > If there were other snitches involved (especially SimpleSnitch), things get > much more complicated. > > > > >> On Thu, Jun 7, 2018 at 9:18 AM, Nitan Kainth wrote: >> Jeff, >> >> In this case, if Riccardo is adding new DC, then he ca

Re: Timestamp on hints file and system.hints table data

e cleaned up. > > On 14 June 2018 at 01:54, Nitan Kainth wrote: > >> Kurt, >> >> No node is down for months. And yes, I am surprised to look at Unix >> timestamp on files. >> >> >> >> On Jun 13, 2018, at 6:41 PM, kurt greaves wrote: >>

Re: Options to replace hardware of the cluster

Both are good options, a new data center in same cluster will work faster as you can stream multiple nodes. Or you can add 8 nodes to existing data center and then decommission one node at a time but this is lengthy process including cleanup after decommission. One node at a time is equally

Re: Restoring snapshot

ica could be found” but the question is, why is it giving > the name of keyspace K2 when I was restoring only K1(It should have given > warning for K1). > > From: Nitan Kainth [mailto:nitankai...@gmail.com] > Sent: Wednesday, June 13, 2018 4:31 PM > To: user@cassandra.apache.o

Timestamp on hints file and system.hints table data

Hi, I observed a strange behavior about stored hints. Time stamp of hints file shows several months old. I deleted them and saw new hints files created with same old date. Why is that? Also, I see hints files on disk but if I query system.hints table, it shows 0 rows. Why system.hints is not

Re: Restoring snapshot

Verify dc name and replication factor in create keyspace command in new cluster. Sent from my iPhone > On Jun 13, 2018, at 2:40 AM, > wrote: > > Dear Community, > > I took a snapshot from a node which was part of a 2 node cluster. There were > 2 keyspaces in that cluster K1 and K2. I took

Re: Timestamp on hints file and system.hints table data

ix timestamp in the file name? > Is there a node that's been down for several months? > >> On Wed., 13 Jun. 2018, 23:41 Nitan Kainth, wrote: >> Hi, >> >> I observed a strange behavior about stored hints. >> >> Time stamp of hints file shows several months o

Re: how to immediately delete tombstones

You can compact selective sstables using jmx Call. Sent from my iPhone > On Jun 2, 2018, at 12:04 AM, onmstester onmstester > wrote: > > Thanks for your replies > But my current situation is that i do not have enough free disk for my > biggest sstable, so i could not run major compaction or

Re: copy sstables while cassandra is running

Yeah you are right Pradeep. I overlooked the point there. Sent from my iPhone > On Jun 24, 2018, at 1:10 PM, Pradeep Chhetri wrote: > > I doubt mv will run instantly because copy is across two different filesystems > >> On Sun, 24 Jun 2018 at 9:26 PM, Nitan Kainth wrote:

Re: copy sstables while cassandra is running

To be safe you could follow below prices on each node one at a time: Stop Cassandra Move sstable— mv will do it instantly Start Cassandra If you do it online and a read trust comes for sane data that is being moved will fail. > On Jun 23, 2018, at 11:23 PM, onmstester onmstester > wrote: > >

Re: Best approach to Replace existing 8 smaller nodes in production cluster with New 8 nodes that are bigger in capacity, without a downtime

You can also create a new DC and then terminate old one. Sent from my iPhone > On Feb 20, 2018, at 2:49 PM, Kyrylo Lebediev wrote: > > Hi, > Consider using this approach, replacing nodes one by one: >

Re: Best approach to Replace existing 8 smaller nodes in production cluster with New 8 nodes that are bigger in capacity, without a downtime

if there is a chance to get rid of streaming, it worth doing as > > usually direct data copy (not by means of C*) is more effective and less > > troublesome. > > > > Regards, > > Kyrill > > > > > > Fro

Re: Discrepancy in nodetool status

Try telnet on your listen port. It must be network issue due to port or firewall issue. Sent from my iPhone > On Dec 22, 2017, at 5:28 PM, sat wrote: > > > > Hi, > > We have 3 nodes in cluster, we rebooted one of the cassandra VM, we noticed > nodetool status

default_time_to_live vs TTL on insert statement

Hi, As per document: https://docs.datastax.com/en/cql/3.3/cql/cql_using/useExpireExample.html - You can set a default TTL for an entire table by setting the table's default_time_to_live

Re: default_time_to_live vs TTL on insert statement

property by specifying a custom value for each CQL statement > > The behavior is correct, nothing wrong here > > On Wed, Jul 11, 2018 at 7:31 PM, Nitan Kainth > wrote: > >> Hi, >> >> As per document: https://docs.datastax.com/en/cql/3.3/cql/cql_ >

  1   2   >