Re: Re: Re: stream failed when bootstrap

2018-06-27 Thread kurt greaves
Yeah, but you only really need to drain, restart Cassandra one by one. Not that the others will hurt, but they aren't strictly necessary. On 28 June 2018 at 05:38, dayu wrote: > Hi kurt, a rolling restart means run disablebinary, disablethrift, > disablegossip, drain, > stop cassandra and start

Re:Re: Re: stream failed when bootstrap

2018-06-27 Thread dayu
Hi kurt, a rolling restart means run disablebinary, disablethrift, disablegossip, drain, stop cassandra and start cassandra command one by one, right? Only one node is executed at a time Dayu At 2018-06-28 11:37:43, "kurt greaves" wrote: Best off trying a rolling restart. On 28 June 201

Re: Re: stream failed when bootstrap

2018-06-27 Thread kurt greaves
Best off trying a rolling restart. On 28 June 2018 at 03:18, dayu wrote: > the output of nodetool describecluster > Cluster Information: > Name: online-xxx > Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch > Partitioner: org.apache.cassandra.dht.Murmur3Partitioner > Schema versions: >

Re:Re: stream failed when bootstrap

2018-06-27 Thread dayu
the output of nodetool describecluster Cluster Information: Name: online-xxx Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch Partitioner: org.apache.cassandra.dht.Murmur3Partitioner Schema versions: c3f00d61-1ad7-3702-8703-af2a29e401c1: [10.136.71.43] 0568e8c1-48ba-3fb0-bb3c-462438978d

Re: stream failed when bootstrap

2018-06-27 Thread Jeff Jirsa
You can sometimes bounce your way through it (or use nodetool resetlocalschema if it’s a single node that’s wrong), but there are some edge cases from which it’s very hard to recover What’s the output of nodetool describecluster? If you select from the schema tables, do you see that CFID on any

Re:Re: stream failed when bootstrap

2018-06-27 Thread dayu
That sound reasonable, I have seen schema mismatch error before. So any advise to deal with schema mismatches? Dayu At 2018-06-28 09:50:37, "Jeff Jirsa" wrote: >That log message says you did: > > CF 53f6d520-2dc6-11e8-948d-ab7caa3c8c36 was dropped during streaming > >If you’re absolutely sure yo

Re: stream failed when bootstrap

2018-06-27 Thread Jeff Jirsa
That log message says you did: CF 53f6d520-2dc6-11e8-948d-ab7caa3c8c36 was dropped during streaming If you’re absolutely sure you didn’t, you should look for schema mismatches in your cluster -- Jeff Jirsa > On Jun 27, 2018, at 7:49 PM, dayu wrote: > > CF 53f6d520-2dc6-11e8-948d-ab7caa3c

Re:Re: stream failed when bootstrap

2018-06-27 Thread dayu
Hi Jeff Thanks for your help. But I didn't remove any tables during bootstrap. I have tried 3 times, still got this error BTW the cassandra version is 3.0.10. dayu ERROR [STREAM-IN-/10.136.71.33] 2018-06-28 08:51:33,330 StreamSession.java:534 - [Stream #712021f0-7a25-11e8-b622-e7402f

Re: stream failed when bootstrap

2018-06-27 Thread Jeff Jirsa
Try again and this time don’t remove tables during bootstrap (the streaming code doesn’t handle removing tables very well). -- Jeff Jirsa > On Jun 27, 2018, at 7:30 PM, dayu wrote: > > Hi everyone > I am joining a new node to a cluster. but it failed even if I use > nodetool bootstrap r

stream failed when bootstrap

2018-06-27 Thread dayu
Hi everyone I am joining a new node to a cluster. but it failed even if I use nodetool bootstrap resume. I found error in log. what can I do? Thanks ! Dayu

Re: Maximum SSTable size

2018-06-27 Thread Jeff Jirsa
That’s a target and not a maximum, and it can go over that even on LCS with wide partitions In general, the practical limit is hundreds of gigabytes (I’ve seen 500-600 a lot, but not more than a terabyte very often). You’ll typically hit other issues and need to tune some params before you hit

Re: [External] Maximum SSTable size

2018-06-27 Thread Tom van der Woerdt
I’ve had SSTables as big as 11TB. It works, read performance is fine. But, compaction is hell, because you’ll need twice that in disk space and it will take many hours 🙂 Avoid large SSTables unless you really know what you’re doing. LCS is a great default for almost every workload, especially if y

RE: Maximum SSTable size

2018-06-27 Thread ZAIDI, ASAD A
With Leveled compaction strategy , you can set target size with sstable_size_in_mb attribute however actual size can still be larger than target given large partition size. Thanks/Asad From: Lucas Benevides [mailto:lu...@m

Re: Is it ok to add more than one node to a exist cluster

2018-06-27 Thread kurt greaves
To clarify, this is after the node reaches NORMAL state. don't bootstrap other nodes while one is in JOINING unless you really know what you are doing and aren't using vnodes. On Wed., 27 Jun. 2018, 21:02 Abdul Patel, wrote: > Theres always an 2 minute rule ..after adding one node wait for 2 min

Maximum SSTable size

2018-06-27 Thread Lucas Benevides
Hello Community, Is there a maximum SSTable Size? If there is not, does it go up to the maximum Operational System values? Thanks in advance, Lucas Benevides

Re: Is it ok to add more than one node to a exist cluster

2018-06-27 Thread Abdul Patel
Theres always an 2 minute rule ..after adding one node wait for 2 minutes before addding second node. On Wednesday, June 27, 2018, dayu wrote: > Thanks for your reply, kurt. > > another question, Can I bootstrap a new node when some node is in Joining > state ? Or I should wait until Joining nod

Re:Re: Is it ok to add more than one node to a exist cluster

2018-06-27 Thread dayu
Thanks Abdul ! dayu At 2018-06-27 18:02:08, "Abdul Patel" wrote: Theres always an 2 minute rule ..after adding one node wait for 2 minutes before addding second node. On Wednesday, June 27, 2018, dayu wrote: Thanks for your reply, kurt. another question, Can I bootstrap a new node whe

Re:Re: Is it ok to add more than one node to a exist cluster

2018-06-27 Thread dayu
Thanks for your reply, kurt. another question, Can I bootstrap a new node when some node is in Joining state ? Or I should wait until Joining node becoming Normal ? Dayu At 2018-06-27 17:50:34, "kurt greaves" wrote: Don't bootstrap nodes simultaneously unless you really know what you're d

Re: Is it ok to add more than one node to a exist cluster

2018-06-27 Thread kurt greaves
Don't bootstrap nodes simultaneously unless you really know what you're doing, and you're using single tokens. It's not straightforward and will likely lead to data loss/inconsistencies. This applies for all current versions. On 27 June 2018 at 10:21, dayu wrote: > Hi, > I have read a warnin

Is it ok to add more than one node to a exist cluster

2018-06-27 Thread dayu
Hi, I have read a warning of not Simultaneously bootstrapping more than one new node from the same rack in version 2.1 link My cassandra cluster version is 3.0.10. So I wonder to know Is it ok to add more than one node to a exist cluster in 3.0.10. Thanks! Dayu