Re: Convert single node C* to cluster (rebalancing problem)

On 2017-06-16 10:31 (-0700), John Hughes wrote: > Hi Affan, > > Others can likely speak to this more authoritatively I am sure, but with a > RF of 1x, I would not expect it to rebalance. Now if you were 4 nodes and a > RF of 2x I would expect it to. > Even with an RF

Re: Convert single node C* to cluster (rebalancing problem)

what you are experiencing can be cleared up by >>>>> repair/compaction/cleanup. Also, what are your outputs when you call out >>>>> specific keyspaces? Do the numbers get more even? >>>>> >>>>> Cheers, >>>>> >>>>>

Re: Convert single node C* to cluster (rebalancing problem)

c9 1c >>>>>>> UN XX.XX.XX.XX 17.17 GiB 256 51.3% >>>>>>> d2a65c51-087d-48de-ae1f-a41142eb148d 1b >>>>>>> UN XX.XX.XX.XX 26.15 GiB 256 52.4% >>>>>>> acf5dd34-5b81

Re: Convert single node C* to cluster (rebalancing problem)

t; 6c8842dd-a966-467c-a7bc-bd6269ce3e7e 1a >>>>>> UN XX.XX.XX.XX 24.39 GiB 256 49.8% >>>>>> fd92525d-edf2-4974-8bc5-a350a8831dfa 1a >>>>>> UN XX.XX.XX.XX 23.8 GiB 256 48.7% >>>>>> bdc597c0-718c-4ef6-b3ef-7785110a99

Re: Convert single node C* to cluster (rebalancing problem)

;> UN XX.XX.XX.XX 23.8 GiB 256 48.7% >>>>> bdc597c0-718c-4ef6-b3ef-7785110a9923 1b >>>>> >>>>> Though maybe part of what you are experiencing can be cleared up by >>>>> repair/compaction/cleanup. Also, what are your outputs when you call out >>

Re: Convert single node C* to cluster (rebalancing problem)

;> On Mon, Jun 12, 2017 at 5:22 AM Akhil Mehra <akhilme...@gmail.com> >>>> wrote: >>>> >>>>> auto_bootstrap is true by default. Ensure its set to true. On startup >>>>> look at your logs for your auto_bootstrap value. Look at the node

Re: Convert single node C* to cluster (rebalancing problem)

Look at the node >>>> configuration line in your log file. >>>> >>>> Akhil >>>> >>>> On Mon, Jun 12, 2017 at 6:18 PM, Junaid Nasir <jna...@an10.io> wrote: >>>> >>>>> No, I didn't set it (left it at default

Re: Convert single node C* to cluster (rebalancing problem)

> >>>>> Did you make sure auto_bootstrap property is indeed set to [true] >>>>> when you added the node? >>>>> >>>>> >>>>> >>>>> *From:* Junaid Nasir [mailto:jna...@an10.io] >>>>> *Sent:* Mon

Re: Convert single node C* to cluster (rebalancing problem)

AD A <az1...@att.com> wrote: >>> >>>> Did you make sure auto_bootstrap property is indeed set to [true] when >>>> you added the node? >>>> >>>> >>>> >>>> *From:* Junaid Nasir [mailto:jna...@an10.io] >>>&

Re: Convert single node C* to cluster (rebalancing problem)

gt;>> *From:* Junaid Nasir [mailto:jna...@an10.io] >>> *Sent:* Monday, June 05, 2017 6:29 AM >>> *To:* Akhil Mehra <akhilme...@gmail.com> >>> *Cc:* Vladimir Yudovin <vla...@winguzone.com>; user@cassandra.apache.org >>> *Subject:* Re: Convert single

Re: Convert single node C* to cluster (rebalancing problem)

gt;> *Sent:* Monday, June 05, 2017 6:29 AM >> *To:* Akhil Mehra <akhilme...@gmail.com> >> *Cc:* Vladimir Yudovin <vla...@winguzone.com>; user@cassandra.apache.org >> *Subject:* Re: Convert single node C* to cluster (rebalancing problem) >> >> >> >>

Re: Convert single node C* to cluster (rebalancing problem)

ent:* Monday, June 05, 2017 6:29 AM > *To:* Akhil Mehra <akhilme...@gmail.com> > *Cc:* Vladimir Yudovin <vla...@winguzone.com>; user@cassandra.apache.org > *Subject:* Re: Convert single node C* to cluster (rebalancing problem) > > > > not evenly, i have s

RE: Convert single node C* to cluster (rebalancing problem)

Subject: Re: Convert single node C* to cluster (rebalancing problem) not evenly, i have setup a new cluster with subset of data (around 5gb). using the configuration above I am getting these results Datacenter: datacenter1 === Status=Up/Down |/ State=Normal/Leaving/Jo

Re: Convert single node C* to cluster (rebalancing problem)

not evenly, i have setup a new cluster with subset of data (around 5gb). using the configuration above I am getting these results Datacenter: datacenter1 === Status=Up/Down |/ State=Normal/Leaving/Joining/Moving -- Address Load Tokens Owns (effective) Host

Re: Convert single node C* to cluster (rebalancing problem)

So now the data is evenly balanced in both nodes? Refer to the following documentation to get a better understanding of the roc_address and the broadcast_rpc_address https://www.instaclustr.com/demystifying-cassandras-broadcast_address/

Re: Convert single node C* to cluster (rebalancing problem)

I am able to get it working. I added a new node with following changes #rpc_address:0.0.0.0 rpc_address: 10.128.1.11 #rpc_broadcast_address:10.128.1.11 rpc_address was set to 0.0.0.0, (I ran into a problem previously regarding remote connection and made these changes

Re: Convert single node C* to cluster (rebalancing problem)

Did you run "nodetool cleanup" on first node after second was bootstrapped? It should clean rows not belonging to node after tokens changed. Best regards, Vladimir Yudovin, Winguzone - Cloud Cassandra Hosting On Wed, 31 May 2017 03:55:54 -0400 Junaid Nasir jna...@an10.io wrote

Re: Convert single node C* to cluster (rebalancing problem)

When you bootstrapped the node for the first time did you see log similar to the following: INFO [main] 2017-06-01 07:19:45,199 StorageService.java:1435 - JOINING: waiting for schema information to complete INFO [main] 2017-06-01 07:19:45,250 StorageService.java:1435 - JOINING: schema

Re: Convert single node C* to cluster (rebalancing problem)

sorry that was an old notetool status output (old cluster with a few configuration changes). new configuration is -- Address Load Tokens Owns (effective) Host ID Rack UN 10.128.1.2 270.75 GiB 256 52.6% e4427611-c247-42ee-9404-371e177f5f17

Re: Convert single node C* to cluster (rebalancing problem)

Hi Junaid, I noticed in the log files that data is being streamed from/to 10.128.1.2 but the address of your original node is 10.128.0.7. Are there any other Cassandra nodes on your local network the same cluster name. Regards, Akhil > 10.128.1.2 > On 31/05/2017, at 10:15 PM, Junaid Nasir

Re: Convert single node C* to cluster (rebalancing problem)

after nodetool repair on new node following log appears in cassandra log INFO [StreamConnectionEstablisher:1] 2017-05-31 10:07:23,496 > StreamSession.java:266 - [Stream #f0eb9670-45e8-11e7-a17b-81e9a18c6eac] > Starting streaming to /10.128.1.2 > INFO [StreamConnectionEstablisher:1] 2017-05-31

Re: Convert single node C* to cluster (rebalancing problem)

Cassandra version is 3.10, and yes its not a production server. i have seen some warnings in logs saying token exist on both servers. other than that nothing. if you need any more settings/details please ask. thank you for your time On Wed, May 31, 2017 at 2:04 PM, Oleksandr Shulgin <

Re: Convert single node C* to cluster (rebalancing problem)

On Wed, May 31, 2017 at 9:55 AM, Junaid Nasir wrote: > Cassandra ensure that adding or removing nodes are very easy and that load > is balanced between nodes when a change is made. but it's not working in my > case. > I have a single node C* deployment (with 270 GB of data) and

Convert single node C* to cluster (rebalancing problem)

Cassandra ensure that adding or removing nodes are very easy and that load is balanced between nodes when a change is made. but it's not working in my case. I have a single node C* deployment (with 270 GB of data) and want to load balance the data on multiple nodes, I followed this guide

Re: cluster rebalancing…

On Tue, Jul 22, 2014 at 7:03 PM, Kevin Burton bur...@spinn3r.com wrote: So , shouldn't it be easy to rebalance a cluster? I'm not super excited to type out 200 commands to move around individual tokens. That's why vnodes exist? Before vnodes, the only sane option was to double your cluster

cluster rebalancing…

So , shouldn't it be easy to rebalance a cluster? I'm not super excited to type out 200 commands to move around individual tokens. I realize that this isn't a super easy solution, and that there are probably 2-3 different algorithms to pick here… but having this be the only option doesn't seem

Re: cluster rebalancing…

You don't need to specify tokens. The new node gets them automatically. On Jul 22, 2014, at 7:03 PM, Kevin Burton bur...@spinn3r.com wrote: So , shouldn't it be easy to rebalance a cluster? I'm not super excited to type out 200 commands to move around individual tokens. I realize

Re: cluster rebalancing…

ok.. I think I get what's happening. This node is still joining the cluster. It wasn't totally clear that it was still joining as the only indicator is the little J ... On Tue, Jul 22, 2014 at 7:09 PM, Jonathan Haddad jonathan.had...@gmail.com wrote: You don't need to specify tokens. The new