Re: Cassandra GC tuning

2022-09-19 Thread Patrick McFadin
GC tuning may seem like it's the best move, but more than likely, that is just the smoke from the real fire. Can you go more into your configuration? Memory. CPU. DIsk. Many times, GC is what shows up when running out of disk bandwidth or some other process eating up resources. Patrick On Mon,

Re: Codec not found for list

2022-09-19 Thread Sébastien Rebecchi
Please ignore that message, I found my mistake, it was clear in the error message, I forgot to collect the Java stream as List... :) Sorry for the disturbance Le lun. 19 sept. 2022 à 11:08, Sébastien Rebecchi < sebastien.rebec...@gmail.com> a écrit : > Hello, > > I have a table where I store

Re: Local reads metric

2022-09-18 Thread Jeff Jirsa
Yes > On Sep 17, 2022, at 10:46 PM, Gil Ganz wrote: > >  > Hey > Do reads that come from a read repair are somehow counted as part of the > local read metric? > i.e > org.apache.cassandra.metrics.Table... : > ReadLatency.1m_rate > > Version is 4.0.4 > > Gil

Re: Ldap/AD Authentication

2022-09-15 Thread Slater, Ben
Hi Our LDAP Authenticator for Cassandra is open source, Apache 2.0 licenced and free to use: https://github.com/instaclustr/cassandra-ldap We do offer support for it as part of our Cassandra support contracts but you don’t need to buy support to use it. Cheers Ben -- Ben Slater Chief

Re: TimeWindowCompactionStrategy Operational Concerns

2022-09-15 Thread Jeff Jirsa
If you were able to generate old data offline, using something like the CQLSSTableWriter class, you can add that to the cluster (either via streaming or nodetool import), that would maintain the TWCS invariant. That said, with https://issues.apache.org/jira/browse/CASSANDRA-13418 , IF you're

Re: Hints not being sent from 3.0/3.11 to 4.0?

2022-09-14 Thread Morten A. Iversen via user
Hi, I have finally had time to test this a bit. And the issue seems to be there with Cassandra 3.11.13 as well as 3.0.27. Hinted handoff from these versions to version 4.0.6 does not seem to work. The way I tested this was by setting up a 2 node cluster, 1 node in each of dc1 and dc2. Install

Re: Using zstd compression on Cassandra 3.x

2022-09-13 Thread onmstester onmstester via user
I patched this on 3.11.2 easily: 1. build jar file from src and put in cassandra/lib directory 2. restart cassandra service 3. alter table using compression zstd and rebuild sstables But it was in a time when 4.0 was not available yet and after that i upgraded to 4.0 immidiately. Sent

Re: Using zstd compression on Cassandra 3.x

2022-09-12 Thread Eunsu Kim
Thank you for your response. I'll consider upgrading to 4.x. > 2022. 9. 13. 오후 2:41, Dinesh Joshi 작성: > > Is there something preventing you from upgrading to 4.0? It is backward > compatible with 3.0 so clients don’t need to change. > > If you don’t want to absolutely upgrade you can

Re: Using zstd compression on Cassandra 3.x

2022-09-12 Thread Dinesh Joshi
Is there something preventing you from upgrading to 4.0? It is backward compatible with 3.0 so clients don’t need to change. If you don’t want to absolutely upgrade you can extract the implementation from 4.0 and use it. I would advise against this path though as zstd implementation is

Re: Bootstrap data streaming order

2022-09-12 Thread Jeff Jirsa
me order? > > > > *From:* Gil Ganz > *Sent:* Monday, September 12, 2022 8:50 AM > *To:* user@cassandra.apache.org > *Subject:* Re: Bootstrap data streaming order > > > > EXTERNAL > > I can understand why the number of nodes sending at once might be > interesting somehow, but

Re: Bootstrap data streaming order

2022-09-12 Thread Jeff Jirsa
that same order? > > From: Gil Ganz > Sent: Monday, September 12, 2022 8:50 AM > To: user@cassandra.apache.org > Subject: Re: Bootstrap data streaming order > > EXTERNAL > > I can understand why the number of nodes sending at once might be interesting > somehow,

Re: Bootstrap data streaming order

2022-09-12 Thread Gil Ganz
September 12, 2022 8:50 AM > *To:* user@cassandra.apache.org > *Subject:* Re: Bootstrap data streaming order > > > > EXTERNAL > > I can understand why the number of nodes sending at once might be > interesting somehow, but why would the order of the nodes matter? > >

RE: Bootstrap data streaming order

2022-09-12 Thread Marc Hoppins
nodetool display the status in that same order? From: Gil Ganz Sent: Monday, September 12, 2022 8:50 AM To: user@cassandra.apache.org Subject: Re: Bootstrap data streaming order EXTERNAL I can understand why the number of nodes sending at once might be interesting somehow, but why would the order

Re: Bootstrap data streaming order

2022-09-12 Thread Gil Ganz
s send data at once and what determines that limit? > > The usual kind of questions. > > > > -Original Message- > From: Dinesh Joshi > Sent: Friday, September 9, 2022 9:14 AM > To: user@cassandra.apache.org > Subject: Re: Bootstrap data streaming order >

RE: Bootstrap data streaming order

2022-09-09 Thread Marc Hoppins
@cassandra.apache.org Subject: Re: Bootstrap data streaming order EXTERNAL The data is requested asynchronously from peers. There is some logic to select the peers however there isn’t a set order for data delivery. Why do you ask? > > On Sep 8, 2022, at 11:35 PM, Marc Hoppins wrote: > > Hulloa

Re: Bootstrap data streaming order

2022-09-09 Thread Dinesh Joshi
The data is requested asynchronously from peers. There is some logic to select the peers however there isn’t a set order for data delivery. Why do you ask? > > On Sep 8, 2022, at 11:35 PM, Marc Hoppins wrote: > > Hulloa all, > > Can anyone shed light on the order which nodes will deliver

Re: Compaction task priority

2022-09-06 Thread onmstester onmstester via user
Using nodetool stop -id COMPACTION_UUID(reported in compactionstats), also you could figure it out with nodetool help stop Sent using https://www.zoho.com/mail/ On Mon, 05 Sep 2022 10:18:52 +0430 Gil Ganz wrote --- onmstester  - How can you stop a specific compaction task?

Re: Compaction task priority

2022-09-04 Thread Gil Ganz
onmstester - How can you stop a specific compaction task? stop command stops all compactions of a given type (would be nice to be able to stop specific one). Jim - in my case the solution was actually to limit concurrent compactors, not increase it. Too many tasks caused the server to slow down

Re: Compaction task priority

2022-09-02 Thread Jim Shaw
if capacity allowed, increase compaction_throughput_mb_per_sec as 1st tuning, and if still behind, increase concurrent_compactors as 2nd tuning. Regards, Jim On Fri, Sep 2, 2022 at 3:05 AM onmstester onmstester via user < user@cassandra.apache.org> wrote: > Another thing that comes to my

Re: Compaction task priority

2022-09-02 Thread onmstester onmstester via user
Another thing that comes to my mind: increase minimum sstable count to compact from 4 to 32 for the big table that won't be read that much, although you should watch out for too many sstables count. Sent using https://www.zoho.com/mail/ On Fri, 02 Sep 2022 11:29:59 +0430

Re: Compaction task priority

2022-09-02 Thread onmstester onmstester via user
I was there too! and found nothing to work around it except stopping big/unnecessary compactions manually (using nodetool stop) whenever they appears by some shell scrips (using crontab) Sent using https://www.zoho.com/mail/ On Fri, 02 Sep 2022 10:59:22 +0430 Gil Ganz wrote ---

Re: netty connection reset by peer errors in logs

2022-09-01 Thread Gil Ganz
Reason I would like to suppress it is I think this is due to network disconnects we know are happening, and looks like it's not going to change. Since it doesn't happen that often, and not causing a real issue , I would like to have cleaner logs if possible. On Thu, Sep 1, 2022 at 9:55 AM Erick

Re: netty connection reset by peer errors in logs

2022-09-01 Thread Erick Ramirez
That error message indicates that 2 nodes are unable to communicate with each other over the internode (gossip) port. It makes no sense to suppress it since it's an indication that there's a problem that you need to address. Cheers!

RE: Erroneous node. - node is not a member of the

2022-08-24 Thread Marc Hoppins
CassandraRoleManager.java:370 - Setup task failed with error, rescheduling Am I going to have to remove this node and try again? -Original Message- From: Marc Hoppins Sent: Wednesday, August 24, 2022 11:59 AM To: user@cassandra.apache.org Subject: RE: Erroneous node. - node is not a member

RE: Erroneous node. - node is not a member of the

2022-08-24 Thread Marc Hoppins
Sent: Wednesday, August 24, 2022 9:35 AM To: user@cassandra.apache.org Subject: RE: Erroneous node. - node is not a member of the Also, I just had some changes made to the cass.yml config so thought that is I rolling restart the nodes it might help the problem. Now I have a startup problem

RE: Erroneous node. - node is not a member of the

2022-08-24 Thread Marc Hoppins
in the cluster. How can I remove this from the cluster/ring so that I can re-add? Marc

Re: Hints not being sent from 3.0 to 4.0?

2022-08-23 Thread Jim Shaw
Is it over max hint window ? if over, better to do a full repair. check table system.hints, do you see rows ? As I remember, during upgrade, transactions will store in hints until other cluster have done upgrade, so for safety, change default 3 hours hint window to long time just before

Re: Cassandra 4.0 upgrade - Upgradesstables

2022-08-21 Thread Jim Shaw
Though it is not required to run upgradesstables, but upgradesstables -a will re-write the file to kick out tombstones, in sizeTieredcompaction, the largest files may stay a long time to wait for the next compaction to kick out tombstones. So it really depends, to run it or not, usually upgrades

Re: cell vs row timestamp tie resolution

2022-08-21 Thread Jim Shaw
Andrey: cassandra every cell has a timestamp, select writetime (..) can see the timestamp, cassandra merge cells when compaction, when read, sort by timestamp. for you example, if you left pad the writetime to column value (writetime + cell value), then sort, shall return what you see

Re: Question about num_tokens

2022-08-18 Thread Elliott Sims
I'm not sure I entirely agree with the docs there, as they don't quite match my experiences, but it's going to depend a lot on your specific needs and other parts of the configuration. I think data distribution with low num_tokens is generally considered to be less of a problem with larger

Re: Cassandra 4.0 upgrade - Upgradesstables

2022-08-16 Thread Jai Bheemsen Rao Dhanwada
Thank you On Tue, Aug 16, 2022 at 11:48 AM C. Scott Andreas wrote: > No downside at all for 3.x -> 4.x (however, Cassandra 3.x reading 2.1 > SSTables incurred a performance hit). > > Many users of Cassandra don't run upgradesstables after 3.x -> 4.x > upgrades at all. It's not necessary to run

Re: Cassandra 4.0 upgrade - Upgradesstables

2022-08-16 Thread C. Scott Andreas
No downside at all for 3.x -> 4.x (however, Cassandra 3.x reading 2.1 SSTables incurred a performance hit).Many users of Cassandra don't run upgradesstables after 3.x -> 4.x upgrades at all. It's not necessary to run until a hypothetical future time if/when support for reading Cassandra 3.x

Re: Question about num_tokens

2022-08-16 Thread Jai Bheemsen Rao Dhanwada
Thanks for the response and details. I am just curious about the below statement mentioned in the doc. I am pretty confident that my clusters are going to grow to 100+ nodes (same DC or combining all DCs). I am just concerned that the doc says it is *not recommended for clusters over 50 nodes*.

Re: Cassandra 4.0 upgrade - Upgradesstables

2022-08-16 Thread Jai Bheemsen Rao Dhanwada
Thank you Erick, > it is going to be single-threaded by default so it will take a while to get through all the sstables on dense nodes Is there any downside if the upgradesstables take longer (example 1-2 days), other than I/O? Also when is the upgradesstable get triggered? after every node is

Re: Cassandra 4.0 upgrade - Upgradesstables

2022-08-16 Thread Erick Ramirez
As convenient as it is, there are a few caveats and it isn't a silver bullet. The automatic feature will only kick in if there are no other compactions scheduled. Also, it is going to be single-threaded by default so it will take a while to get through all the sstables on dense nodes. In

Re: Understanding multi region read query and latency

2022-08-09 Thread Bowen Song via user
Adding sleep to solve racing conditions is a bad practice, and should be avoided if possible. Instead, use read and write CL that guarantees strong consistency when it is required/needed. On 09/08/2022 23:49, Jim Shaw wrote: Raphael:    Have you found  root cause ? If not, here are a few

Re: Understanding multi region read query and latency

2022-08-09 Thread Jim Shaw
Raphael: Have you found root cause ? If not, here are a few tips, based on what I experienced before, but may not be same as your case, just hope it is helpful. 1) app side called wrong code module get the cql from system.prepared_statements cql statement is helpful to developers to search

RE: RPM Installation on RHEL7 broken

2022-08-09 Thread Amit Patel via user
Thank you Yakir From: Stéphane Alleaume Sent: 09 August 2022 14:21 To: Yakir Gibraltar Cc: user@cassandra.apache.org; Amit Patel Subject: Re: RPM Installation on RHEL7 broken CAUTION: This email comes from outside Euroclear! Be vigilant! Thanks you very much :-) Kind regards Stéphane Le

Re: RPM Installation on RHEL7 broken

2022-08-09 Thread Stéphane Alleaume
he.org> wrote: > >> Hi Stephane, >> >> >> >> I have followed same instruction but new rpm version 4.0.5 is broken(bug) >> and there are no other package on that repo (download base url for rhel) so >> I can not install older stable version. >> >>

Re: RPM Installation on RHEL7 broken

2022-08-09 Thread Yakir Gibraltar
gt; Amit Patel > > > > *From:* Stéphane Alleaume > *Sent:* 09 August 2022 13:32 > *To:* user@cassandra.apache.org; Amit Patel > *Subject:* Re: RPM Installation on RHEL7 broken > > > > *CAUTION:* This email comes from outside Euroclear! Be vigilant! > > Hi >

RE: RPM Installation on RHEL7 broken

2022-08-09 Thread Amit Patel via user
@cassandra.apache.org; Amit Patel Subject: Re: RPM Installation on RHEL7 broken CAUTION: This email comes from outside Euroclear! Be vigilant! Hi Hope it will help : https://cassandra.apache.org/doc/trunk/cassandra/getting_started/installing.html#installing-the-rpm-packages 1. Add the Apache

Re: RPM Installation on RHEL7 broken

2022-08-09 Thread Stéphane Alleaume
Hi Hope it will help : https://cassandra.apache.org/doc/trunk/cassandra/getting_started/installing.html#installing-the-rpm-packages 1. Add the Apache repository of Cassandra to the file /etc/yum.repos.d/cassandra.repo (as the root user). The latest major version is 4.0 and the

Re: Understanding multi region read query and latency

2022-08-07 Thread Stéphane Alleaume
You're right too, this option is not new, sorry. Is this option can be useful ? Le dim. 7 août 2022, 22:18, Bowen Song via user a écrit : > Do you mean "nodetool settraceprobability"? This is not exactly new, I > remember it was available on Cassandra 2.x. > On 07/08/2022 20:43, Stéphane

Re: Understanding multi region read query and latency

2022-08-07 Thread Bowen Song via user
Do you mean "nodetool settraceprobability"? This is not exactly new, I remember it was available on Cassandra 2.x. On 07/08/2022 20:43, Stéphane Alleaume wrote: I think perhaps you already know but i read you can now trace only a % of all queries, i will look to retrieve the name of this

Re: Understanding multi region read query and latency

2022-08-07 Thread Stéphane Alleaume
Thanks a lot Scott, i didn't knew this fact. Kind regards Stéphane Le dim. 7 août 2022, 19:31, C. Scott Andreas a écrit : > > but still as I understand the documentation the read repair should not > be in the blocking path of a query ? > > Read repair is in the blocking read path for the

Re: Understanding multi region read query and latency

2022-08-07 Thread Stéphane Alleaume
I think perhaps you already know but i read you can now trace only a % of all queries, i will look to retrieve the name of this fonctionnality (in new Cassandra release). Hope it will help Kind regards Stéphane Le dim. 7 août 2022, 20:26, Raphael Mazelier a écrit : > > "Read repair is in the

Re: Understanding multi region read query and latency

2022-08-07 Thread Raphael Mazelier
> "Read repair is in the blocking read path for the query, yep" OK interesting. This is not what I understood from the documentation. And I use localOne level consistency. I enabled tracing (see in the attachment of my first msg)/ but I didn't see read repair in the trace (and btw I tried to

Re: Understanding multi region read query and latency

2022-08-07 Thread C. Scott Andreas
> but still as I understand the documentation the read repair should not be in the blocking path of a query ?Read repair is in the blocking read path for the query, yep. At quorum consistency levels, the read repair must complete before returning a result to the client to ensure the data returned

Re: Understanding multi region read query and latency

2022-08-07 Thread Stéphane Alleaume
Read repair chance ? Le dim. 7 août 2022, 19:25, Raphael Mazelier a écrit : > Nope. And what really puzzle me is in the trace we really show the > difference between queries. The fast queries only request read from one > replicas, while slow queries request from multiple replicas (and not only

Re: Understanding multi region read query and latency

2022-08-07 Thread Raphael Mazelier
Nope. And what really puzzle me is in the trace we really show the difference between queries. The fast queries only request read from one replicas, while slow queries request from multiple replicas (and not only local to the dc). On 07/08/2022 14:02, Stéphane Alleaume wrote: Hi Is there

Re: Understanding multi region read query and latency

2022-08-06 Thread Raphael Mazelier
Well answering to myself this is not related to read_repair chance. Settings them to 0 change also nothing. So the question remains: why from time to time C* want to make multiple read on a non local dc ? On 06/08/2022 12:31, Raphael Mazelier wrote: Well I tried (but already have some

Re: Understanding multi region read query and latency

2022-08-06 Thread Raphael Mazelier
Well I tried (but already have some whiteListFilter) it changed nothing but it's more convenient that using whiteListFilter (speeding up the connection time). So still from time to time (dedanding of the frequency of my requests) I got slow request when I notice in the trace that c* try to

Re: Understanding multi region read query and latency

2022-08-05 Thread Bowen Song via user
The  DCAwareRoundRobinPolicy/TokenAwareHostPolicy controlls which Cassandra coordinator node the client sends queries to, not the nodes it connects to, nor the nodes that performs the actual read. A client sends a CQL read query to a coordinator node, and the coordinator node parses the CQL

Re: Understanding multi region read query and latency

2022-08-05 Thread Jim Shaw
I remember gocql.DataCentreHostFilter was used. try add it to see whether will read local DC only in your case ? Thanks, James On Fri, Aug 5, 2022 at 2:40 PM Raphael Mazelier wrote: > Hi Cassandra Users, > > I'm relatively new to Cassandra and first I have to say I'm really > impressed by

Re: unsubscribe

2022-08-04 Thread Bowen Song via user
Please send an email to "user-unsubscr...@cassandra.apache.org" to unsubscribe from this mailing list. On 04/08/2022 18:29, Dathan Vance Pattishall wrote: unsubscribe

RE: Service shutdown

2022-08-04 Thread Marc Hoppins
ser@cassandra.apache.org Subject: Re: Service shutdown EXTERNAL Generally speaking, I've seen Cassandra process stopping for the following reasons: OOM killer JVM OOM Received a signal, such as SIGTERM and SIGKILL File IO error when disk_failure_policy or commit_failure_policy is set to die Hardware iss

Re: Service shutdown

2022-08-04 Thread Bowen Song via user
Generally speaking, I've seen Cassandra process stopping for the following reasons: OOM killer JVM OOM Received a signal, such as SIGTERM and SIGKILL File IO error when disk_failure_policy or commit_failure_policy is set to die Hardware issues, such as memory corruption,

Re: Cassandra Client compatibility

2022-08-02 Thread Erick Ramirez
In the context of driver compatibility, Cassandra "3.0+" means C* 3.0 and newer releases which include C* 3.0.x, 3.11.x, 4.0.x and [soon] 4.1.x. To answer your question directly, version 2.6 of the C++ driver works with C* 3.11.11 but we don't recommend you use it since it's an ancient release

Re: Wrong Consistency level seems to be used

2022-07-21 Thread Jim Shaw
My experience to debug this kind of issue is to turn on trace. The nice thing in cassandra is: you can turn on trace only on 1 node and with a small percentage, i.e. nodetool settraceprobability 0.05 --- only run on 1 node. Hope it helps. Regards, James On Thu, Jul 21, 2022 at 2:50 PM

Re: Wrong Consistency level seems to be used

2022-07-21 Thread Tolbert, Andy
I'd bet the JIRA that Paul is pointing to is likely what's happening here. I'd look for read repair errors in your system logs or in your metrics (if you have easy access to them). There are operations that can happen during the course of a query being executed that may happen at different

Re: Export as csv

2022-07-21 Thread Arvinder Dhillon
Yup, copy worked. Thanks. On Thu, Jul 21, 2022 at 10:55 AM Nitan Kainth wrote: > Use copy command or dsbulk > > > > > > Regards, > > Nitan K. > > 510-449-9629 > > > > > > *From: *Arvinder Dhillon > *Date: *Thursday, July 21, 2022 at 12:52 PM > *To: *user@cassandra.apache.org > *Subject:

Re: Export as csv

2022-07-21 Thread Nitan Kainth
Use copy command or dsbulk Regards, Nitan K. 510-449-9629 From: Arvinder Dhillon Date: Thursday, July 21, 2022 at 12:52 PM To: user@cassandra.apache.org Subject: Export as csv What tool do I use to dump a columns of a table having 40K partitions into a csv file? I tried caqsh with CAPTURE,

Re: Wrong Consistency level seems to be used

2022-07-21 Thread Paul Chandler
I came across this problem a few years ago, and had long conversations with Datastax support about it. In my case it turns out that the error message is misleading and I was pointed to the ticket: https://issues.apache.org/jira/browse/CASSANDRA-14715 I con’t remember much about it now, but

Re: Wrong Consistency level seems to be used

2022-07-21 Thread pwozniak
Yes, I did it. Nothing like this in my code. Consistency level is set only in one place (shown below). On 7/21/22 4:08 PM, manish khandelwal wrote: Consistency can also be set on a statement basis. So please check in your code that you might be setting consistency 'ALL' for some queries. On

Re: Wrong Consistency level seems to be used

2022-07-21 Thread Bowen Song via user
It doesn't make any sense to see consistency level ALL if the code is not explicitly using it. My best guess is somewhere in the code the consistency level was overridden. On 21/07/2022 14:52, pwozniak wrote: Hi, we have the following code (java driver): cluster

Re: Wrong Consistency level seems to be used

2022-07-21 Thread manish khandelwal
Consistency can also be set on a statement basis. So please check in your code that you might be setting consistency 'ALL' for some queries. On Thu, Jul 21, 2022 at 7:23 PM pwozniak wrote: > Hi, > > we have the following code (java driver): > > cluster =

Re: Adding nodes

2022-07-20 Thread Bowen Song via user
To unsubscribe, please send an email to user-unsubscr...@cassandra.apache.org On 20/07/2022 18:34, emmanuel warreng wrote: Unsubscribe On Thu, Jul 7, 2022, 16:49 Marc Hoppins wrote: Hi all, Cluster of 2 DC and 24 nodes DC1 (RF3) = 12 nodes, 16 tokens each DC2 (RF3) = 12

Re: Adding nodes

2022-07-20 Thread emmanuel warreng
Unsubscribe On Thu, Jul 7, 2022, 16:49 Marc Hoppins wrote: > Hi all, > > Cluster of 2 DC and 24 nodes > > DC1 (RF3) = 12 nodes, 16 tokens each > DC2 (RF3) = 12 nodes, 16 tokens each > > Adding 12 more nodes to DC1: I installed Cassandra (version is the same > across all nodes) but, after the

Re: Adding nodes

2022-07-12 Thread Jeff Jirsa
not a practical response as any business is > unlikely to be spending speculative money. > > > > *From:* Jeff Jirsa > *Sent:* Tuesday, July 12, 2022 4:43 PM > *To:* cassandra > *Cc:* Bowen Song > *Subject:* Re: Adding nodes > > > > EXTERNAL &g

Re: Adding nodes

2022-07-12 Thread Bowen Song via user
to suggest that folk head off a pay for a course when there are ‘pre-sales’ questions is not a practical response as any business is unlikely to be spending speculative money. *From:*Jeff Jirsa *Sent:* Tuesday, July 12, 2022 4:43 PM *To:* cassandra *Cc:* Bowen Song *Subject:* Re: Adding nodes

RE: Adding nodes

2022-07-12 Thread Marc Hoppins
’ questions is not a practical response as any business is unlikely to be spending speculative money. From: Jeff Jirsa Sent: Tuesday, July 12, 2022 4:43 PM To: cassandra Cc: Bowen Song Subject: Re: Adding nodes EXTERNAL On Tue, Jul 12, 2022 at 7:27 AM Marc Hoppins mailto:marc.hopp...@eset.com

Re: Adding nodes

2022-07-12 Thread Jeff Jirsa
On Tue, Jul 12, 2022 at 7:27 AM Marc Hoppins wrote: > > I was asking the questions but no one cared to answer. > This is probably a combination of "it is really hard to answer a question with insufficient data" and your tone. Nobody here gets paid to help you solve your company's problems

Re: Adding nodes

2022-07-12 Thread Jeff Jirsa
n Song via user > *Sent:* Tuesday, July 12, 2022 12:29 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Adding nodes > > > > EXTERNAL > > For RF=2 in your DC1, you will not be able to achieve both strong > consistency and single point of failure tolerance within that

RE: Adding nodes

2022-07-12 Thread Durity, Sean R via user
, the whole DC is useless. Sean R. Durity INTERNAL USE From: Marc Hoppins Sent: Tuesday, July 12, 2022 8:49 AM To: user@cassandra.apache.org; Bowen Song Subject: [EXTERNAL] RE: Adding nodes The data guys want 2 copies of data in DC1 and that data to be replicated offsite to DC1 for 1 copy (DR

Re: Adding nodes

2022-07-12 Thread Bowen Song via user
 or whatever, it is a serious limitation as not every business/organisation is going to have multiple racks available. *From:*Bowen Song via user *Sent:* Monday, July 11, 2022 8:57 PM *To:* user@cassandra.apache.org *Subject:* Re: Adding nodes EXTERNAL I've noticed the joining node has a different

RE: Adding nodes

2022-07-12 Thread Marc Hoppins
/organisation is going to have multiple racks available. From: Bowen Song via user Sent: Monday, July 11, 2022 8:57 PM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL I've noticed the joining node has a different rack than the rest of the nodes, is this intended? Will you add all new

Re: Adding nodes

2022-07-11 Thread Bowen Song via user
Obernberger *Sent:* Monday, July 11, 2022 1:29 PM *To:* user@cassandra.apache.org *Subject:* Re: Adding nodes EXTERNAL I too came from HBase and discovered adding several nodes at a time doesn't work.  Are you absolutely sure that the clocks are in sync across the nodes?  This has bitten me several

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
should be run after adding all nodes to reduce unnecessary strain on the cluster. From: Marc Hoppins Sent: Monday, July 11, 2022 2:15 PM To: user@cassandra.apache.org Subject: RE: Adding nodes All clocks are fine. Why would time synch would affect whether or not a node appears in the nodetool statu

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
Obernberger Sent: Monday, July 11, 2022 1:29 PM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL I too came from HBase and discovered adding several nodes at a time doesn't work. Are you absolutely sure that the clocks are in sync across the nodes? This has bitten me several

Re: Adding nodes

2022-07-11 Thread Joe Obernberger
day, July 11, 2022 11:56 AM *To:* user@cassandra.apache.org *Subject:* Re: Adding nodes EXTERNAL Checking on multiple nodes won't help if the joining node suffers from any of the issues I described, as it will likely be flipping up and down frequently, and the existing nodes in the cluster may

Re: Adding nodes

2022-07-11 Thread Bowen Song via user
al=0.03 secs] *From:*Bowen Song via user *Sent:* Monday, July 11, 2022 11:56 AM *To:* user@cassandra.apache.org *Subject:* Re: Adding nodes EXTERNAL Checking on multiple nodes won't help if the joining node suffers from any of the issues I described, as it will likely be flipping up and dow

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
NEW NODE manually. No seconds. No pauses. No timeouts. Just waiting. From: Bowen Song via user Sent: Monday, July 11, 2022 12:13 PM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL How long doe it take to add a new node? I'm 100% sure neither 90s nor 120s is the answer

Re: Adding nodes

2022-07-11 Thread Bowen Song via user
. Therefore, if I know when my first node is finished, I will have an idea of how long before I check for the when subsequent nodes can be joined. *From:*Bowen Song via user *Sent:* Monday, July 11, 2022 11:25 AM *To:* user@cassandra.apache.org *Subject:* Re: Adding nodes EXTERNAL Sleeping/pausing

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
ser=0.22 sys=0.00, real=0.03 secs] From: Bowen Song via user Sent: Monday, July 11, 2022 11:56 AM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL Checking on multiple nodes won't help if the joining node suffers from any of the issues I described, as it will likely be flipping

Re: Adding nodes

2022-07-11 Thread Bowen Song via user
in the cluster, has been so for several weeks, and is also functioning without error. *From:*Bowen Song via user *Sent:* Monday, July 11, 2022 11:40 AM *To:* user@cassandra.apache.org *Subject:* Re: Adding nodes EXTERNAL A node in joining state can disappearing from the cluster from other

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
Sent: Monday, July 11, 2022 11:40 AM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL A node in joining state can disappearing from the cluster from other nodes' perspective if the joining node stops sending/receiving gossip messages to other nodes. This can happen when

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
can be joined. From: Bowen Song via user Sent: Monday, July 11, 2022 11:25 AM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL Sleeping/pausing for a fixed amount of time between operations at best is a hack to workaround an unknown issue, but it's almost always better

Re: Adding nodes

2022-07-11 Thread Bowen Song via user
up but from the original seed node, it does not appear in the nodetool status. Can anyone shed any light on this phenomena? *From:*Marc Hoppins *Sent:* Monday, July 11, 2022 10:02 AM *To:* user@cassandra.apache.org *Cc:* Bowen Song *Subject:* RE: Adding nodes Well then… I left this on Friday

Re: Adding nodes

2022-07-11 Thread Bowen Song via user
*To:* cassandra *Cc:* Bowen Song *Subject:* Re: Adding nodes EXTERNAL Having a node UJ but not sending/receiving other streams is an invalid state (unless 4.0 moved the streaming data out of netstats? I'm not 100% sure, but I'm 99% sure it should be there). It likely stopped the bootstrap

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
up but from the original seed node, it does not appear in the nodetool status. Can anyone shed any light on this phenomena? From: Marc Hoppins Sent: Monday, July 11, 2022 10:02 AM To: user@cassandra.apache.org Cc: Bowen Song Subject: RE: Adding nodes Well then… I left this on Friday (still

RE: Adding nodes

2022-07-11 Thread Marc Hoppins
which is unlikely to cause problems? After all, in the modern world of big (how big is big?) data, 600G per node is far less than the real BIG big-data. Marc From: Jeff Jirsa Sent: Friday, July 8, 2022 5:46 PM To: cassandra Cc: Bowen Song Subject: Re: Adding nodes EXTERNAL Having a node UJ

Re: Adding nodes

2022-07-08 Thread Jeff Jirsa
issue of not being able to add more nodes. > > -Original Message- > From: Bowen Song via user > Sent: Friday, July 8, 2022 11:47 AM > To: user@cassandra.apache.org > Subject: Re: Adding nodes > > EXTERNAL > > > I would assume that's 85 GB (i.e. gigabytes) then. Which

Re: Adding nodes

2022-07-08 Thread Bowen Song via user
To:user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL I would assume that's 85 GB (i.e. gigabytes) then. Which is approximately 79 GiB (i.e. gibibytes). This still sounds awfully slow - less than 1MB/s over a full day (24 hours). You said CPU and network aren't the bottleneck. Have

RE: Adding nodes

2022-07-08 Thread Marc Hoppins
None of this addresses the issue of not being able to add more nodes. -Original Message- From: Bowen Song via user Sent: Friday, July 8, 2022 11:47 AM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL I would assume that's 85 GB (i.e. gigabytes) then. Which

Re: Adding nodes

2022-07-08 Thread Bowen Song via user
within 2minutes. It is hardly practical in production. -Original Message- From: Bowen Song via user Sent: Thursday, July 7, 2022 8:43 PM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL 86Gb (that's gigabits, which is 10.75GB, gigabytes) took an entire day seems

RE: Adding nodes

2022-07-08 Thread Marc Hoppins
, 2022 8:43 PM To: user@cassandra.apache.org Subject: Re: Adding nodes EXTERNAL 86Gb (that's gigabits, which is 10.75GB, gigabytes) took an entire day seems obviously too long. I would check the network bandwidth, disk IO and CPU usage and find out what is the bottleneck. On 07/07/2022 15:48

Re: Adding nodes

2022-07-07 Thread Bowen Song via user
86Gb (that's gigabits, which is 10.75GB, gigabytes) took an entire day seems obviously too long. I would check the network bandwidth, disk IO and CPU usage and find out what is the bottleneck. On 07/07/2022 15:48, Marc Hoppins wrote: Hi all, Cluster of 2 DC and 24 nodes DC1 (RF3) = 12

Re: Adding nodes

2022-07-07 Thread Jeff Jirsa
What version are you using? When you run `nodetool netstats` on the joining node, what is the output? How much data is there per node (presumably more than 86G)? On Thu, Jul 7, 2022 at 7:49 AM Marc Hoppins wrote: > Hi all, > > Cluster of 2 DC and 24 nodes > > DC1 (RF3) = 12 nodes, 16 tokens

RE: Strange Garbage Collections results in service impact

2022-07-06 Thread Michail Kotsiouros via user
Hello Cassandra community, I have restarted one of the Cassandra nodes that the issue was observed and the GC has stopped being invoked so frequently and causing the service impact. The rest of the Cassandra nodes in the same datacenter that were not restarted still have this issue. What is

Re: Query around Data Modelling -2

2022-07-01 Thread Bowen Song via user
From:* Bowen Song *Sent:* Friday, July 1, 2022 08:48 *To:* user@cassandra.apache.org *Subject:* Re: Query around Data Modelling -2 This message was sent from outside the company. Please do not click links or open attachments unless you recognise the sourc

<    2   3   4   5   6   7   8   9   10   11   >