Starting Cassandrs after restore of Data - get error

2017-07-07 Thread Jonathan Baynes
n pasting in the tokens originally I got the an error, but this was due to the spacing between the tokens. That error has been resolved I'm just left with this one? Any ideas Thanks J Jonathan Baynes DBA Tradeweb Europe Limited Moor Place * 1 Fore Street Avenue * London EC2Y 9DT P +44 (0)20 7

Re: Starting Cassandrs after restore of Data - get error

2017-07-07 Thread Jonathan Baynes
t;> wrote: Jonathan, Make sure initial tokens have values from back up cluster i.e. 256 tokens. It is possible to have typo. On Jul 7, 2017, at 9:14 AM, Jonathan Baynes <jonathan.bay...@tradeweb.com<mailto:jonathan.bay...@tradeweb.com>> wrote: Hi again, Trying to restart my nodes

Move Production data to Development Cluster

2017-07-07 Thread Jonathan Baynes
is so many ways its getting confusing.. what's right? Can anyone give me some pointers as to the best route to migrate data from cluster to cluster? The Documentation is so vague and not detailed enough? Do I copy the snapshots from all the nodes? Do I just work on one node at a time? Any suggestion

RE: Cassandra issue “Unable to gossip with any seeds”

2017-07-04 Thread Jonathan Baynes
Hi Bhargav I had this recently when making changes to a cluster, and It was down to the Listen_address & broadcast_address. Check your cassandra.yaml and make sure that your listen_address/broadcast_address and "seeds" values match. Make sure the Data folder is empty as well. I would go

RE: C* 3 node issue -Urgent

2017-08-23 Thread Jonathan Baynes
The cqlsh image say bad credentials. Just confirming that you have the correct username/password when logging on. By turing on authentication I am assuming you mean using the PasswordAuthenticator instead of the AllowAllAuthenticator in the yaml. Cheers, Akhil On 23/08/2017, at 8:59 PM, Jonat

C* 3 node issue -Urgent

2017-08-23 Thread Jonathan Baynes
:2197) ~[guava-18.0.jar:na] ... 38 common frames omitted Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level LOCAL_ONE I have no idea what to do, so any help would be amazing right now.. Thanks J Jonathan Baynes DBA Tradeweb Europe Limited M

RE: C* 3 node issue -Urgent

2017-08-23 Thread Jonathan Baynes
of the AllowAllAuthenticator in the yaml. Cheers, Akhil On 23/08/2017, at 8:59 PM, Jonathan Baynes <jonathan.bay...@tradeweb.com<mailto:jonathan.bay...@tradeweb.com>> wrote: I will also mention I am on: C* 3.0.11 Linux Oracle red hat 7.1 Java 1.8.0.31 Python 2.7 From: Jonathan Baynes Sen

RE: C* 3 node issue -Urgent

2017-08-23 Thread Jonathan Baynes
I will also mention I am on: C* 3.0.11 Linux Oracle red hat 7.1 Java 1.8.0.31 Python 2.7 From: Jonathan Baynes Sent: 23 August 2017 09:47 To: 'user@cassandra.apache.org' Cc: Stewart Allman Subject: C* 3 node issue -Urgent Hi Everyone. I need the communities help here. I have attempted

Cassandra Setup Question

2017-08-23 Thread Jonathan Baynes
, but we want the data to replicate across both DCs , so that in the event of a DC failure the second DC acts as our DR DC and has all the data as well. The Consistency will be Quorum, but replication Factor does that need to be 3 or 6. Thanks J Jonathan Baynes DBA Tradeweb Europe Limited Moor

RE: C* 3 node issue -Urgent

2017-08-23 Thread Jonathan Baynes
@Kurt, You have hit the nail on the head. Whilst it was an issue with the cassandra-env.sh file, and I fixed this by overwriting it with the default file, essentially rolling it back, which got me back in, the larger issue is with the system_auth db it is a REPL factor of 1 and thus as stated

Removing Columns from production table

2017-08-18 Thread Jonathan Baynes
Hi Is there anything I need to do after dropping a column and adding in a column to flush cassandra of the changes? We are experiencing issues with our front end application, and the developers are asking if the issue was caused by the change in schema, as I've done the drop and add of a

RE: Understanding Messages in the Debug.log

2017-09-21 Thread Jonathan Baynes
Ljo=T5vuzfn8ElJgnkkZ4sQTkQUMwN-_GEOacqRbeVayQUQ=> -- Jacob Shadix On Thu, Sep 21, 2017 at 4:56 AM, Jonathan Baynes <jonathan.bay...@tradeweb.com<mailto:jonathan.bay...@tradeweb.com>> wrote: Good Morning Community, Can anyone help me understand what these messages (and I have a lot o

RE: Understanding Messages in the Debug.log

2017-09-21 Thread Jonathan Baynes
for correlation with other entries in the debug log, or start inspecting the heap/thread pools with some combination of tpstats and dumping the heap and looking at the running threads may give you an indication of what’s going on. -- Jeff Jirsa On Sep 21, 2017, at 4:37 AM, Jonathan Baynes

Understanding Messages in the Debug.log

2017-09-21 Thread Jonathan Baynes
be great. Or if they can be ignored, how do I supress these? Thanks J Jonathan Baynes DBA Tradeweb Europe Limited Moor Place * 1 Fore Street Avenue * London EC2Y 9DT P +44 (0)20 77760988 * F +44 (0)20 7776 3201 * M +44 (0) xx jonathan.bay...@tradeweb.com [cid:image001.jpg@01CD26AD

RE: system_auth replication factor in Cassandra 2.1

2017-08-30 Thread Jonathan Baynes
I recently came across an issue where by my user Keyspace was replicated by 3 (I have 3 nodes) but my system_auth was default to 1, we also use authentication, I then lost 2 of my nodes and because authentication wasn’t replicated I couldn’t log in. Once I resolved the issue, and got the

Issue with New Production Cluster

2017-09-29 Thread Jonathan Baynes
connecting. Checked the PID is up for cassandra (it is) Any idea how I can debug the cause of this further? Thanks J Jonathan Baynes DBA Tradeweb Europe Limited Moor Place * 1 Fore Street Avenue * London EC2Y 9DT P +44 (0)20 77760988 * F +44 (0)20 7776 3201 * M +44 (0) xx jo

RE: Issue with New Production Cluster

2017-09-29 Thread Jonathan Baynes
immediately, does it reconnect? Anything in the logs? What you describe is atypical - timeouts on queries can (and will) happen occasionally under load, but timeout on connect is atypical. Any sign of networking issues/slowness/dns problems/etc? On Fri, Sep 29, 2017 at 6:43 AM, Jonathan Baynes

Re: Removing Columns from production table

2017-08-18 Thread Jonathan Baynes
Thanks Jeff I have a horrible feeling it may be. I'll get the errors from the dev guys Monday and email the group, hopefully I can tie this down. Thanks Sent from my iPhone > On 18 Aug 2017, at 17:24, Jeff Jirsa wrote: > > Cassandra-13004

RE: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory

2017-08-31 Thread Jonathan Baynes
Can you tell us what version you are on? -Original Message- From: qf zhou [mailto:zhouqf2...@gmail.com] Sent: 31 August 2017 10:52 To: user@cassandra.apache.org Subject: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory I am running a cluster with 3

RE: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory

2017-08-31 Thread Jonathan Baynes
: Cassandra - Nodes can't restart due to java.lang.OutOfMemoryError: Direct buffer memory I am usingCassandra 3.9 with cqlsh 5.0.1. 在 2017年8月31日,下午5:54,Jonathan Baynes <jonathan.bay...@tradeweb.com<mailto:jonathan.bay...@tradeweb.com>>

RE: node restart causes application latency

2018-02-15 Thread Jonathan Baynes
I’m following this intensely as the discussion is important to me as well in understanding the restarting of a node correctly, currently (3.0.11) I run Disable thrift, disable gossip then drain, from reading this If I understand correctly is there any point in doing anything other than drain?

Tracing cql code being run through the drive

2018-02-22 Thread Jonathan Baynes
had any experience in doing this? Thanks in advance. J Jonathan Baynes DBA Tradeweb Europe Limited Moor Place * 1 Fore Street Avenue * London EC2Y 9DT P +44 (0)20 77760988 * F +44 (0)20 7776 3201 * M +44 (0)7884111546 jonathan.bay...@tradeweb.com [cid:image001.jpg@01CD26AD.4165F110]<h

Node fails to start - error in Log (CassandraDaemon.java:710 - Exception encountered during startup)

2018-02-23 Thread Jonathan Baynes
) Thanks J Jonathan Baynes DBA Tradeweb Europe Limited Moor Place * 1 Fore Street Avenue * London EC2Y 9DT P +44 (0)20 77760988 * F +44 (0)20 7776 3201 * M +44 (0)7884111546 jonathan.bay...@tradeweb.com [cid:image001.jpg@01CD26AD.4165F110]<http://www.tradeweb.com/> fol

RE: How do you monitoring Cassandra Cluster?

2018-06-21 Thread Jonathan Baynes
I’m using Quest Foglight, very good if you have budget for it?? Jonathan Baynes DBA Tradeweb Europe Limited Moor Place • 1 Fore Street Avenue • London EC2Y 9DT P +44 (0)20 77760988 • F +44 (0)20 7776 3201 • M +44 (0)7884111546 jonathan.bay...@tradeweb.com [cid:image001.jpg@01CD26AD

RE: JOB | Permanent Java Development Manager (Montreal, Canada)

2018-08-01 Thread Jonathan Baynes
Agree we don't need this. -Original Message- From: Tobin Landricombe [mailto:to...@sensatus.com] Sent: 01 August 2018 16:21 To: user@cassandra.apache.org Subject: Re: JOB | Permanent Java Development Manager (Montreal, Canada) Spam. Let this one in and the channel will fill up with them.

RE: Cassandra recommended server uptime?

2018-07-17 Thread Jonathan Baynes
I haven’t restarted a node (service) unless there are changes made to config, and for restarting the server its incredibly rare. In fact since Production has been up I think ive restrted the servers twice (both for DR occurences) in a Year. There is very little need to reboot the servers and

Nodetool Repair --full

2018-03-17 Thread Jonathan Baynes
Hi Community, Can someone confirm, as the documentation out on the web is so contradictory and vague. Nodetool repair -full if I call this, do I need to run this on ALL my nodes or is just the once sufficient? Thanks J Jonathan Baynes DBA Tradeweb Europe Limited Moor Place * 1 Fore Street

Rolling back Cassandra upgrades (tarball)

2018-09-30 Thread Jonathan Baynes
Hi community, Wonder if you can help me. I have a cluster on v3.0.11 I am planning to upgrade this to 3.10. Is rolling back the binaries a viable solution? From reading blogs and such, I’ve read that downgrading is a no-no. Whilst it is possible literally and in theory possible, many of the

RE: openjdk for cassandra production cluster

2018-10-10 Thread Jonathan Baynes
I’m using OpenJDK on version 3.10 (rehl 7.3) in Production and have the warning in the logs. I have a stable system, I think you can ignore it. From: Jonathan Haddad [mailto:j...@jonhaddad.com] Sent: 10 October 2018 15:46 To: user@cassandra.apache.org Subject: Re: openjdk for cassandra

RE: Error during truncate: Cannot achieve consistency level ALL , how to fix it

2018-09-19 Thread Jonathan Baynes
What RF is your system_auth keyspace? If its one, match it to the user keyspace, and restart the node. From: sha p [mailto:shatestt...@gmail.com] Sent: 19 September 2018 11:49 To: user@cassandra.apache.org Subject: Error during truncate: Cannot achieve consistency level ALL , how to fix it Hi