Even with a low amount of vnodes you're asking for a bad time. Even if you
managed to get down to 2 vnodes per node, you're still likely to include
double the amount of nodes in any streaming/repair operation which will
likely be very problematic for incremental repairs, and you still won't be
I believe you are able to get away with just altering the keyspace to
include both DC's even before the DC exists, and then adding your nodes to
that new DC using the algorithm. Note you'll probably want to take the
opportunity to reduce the number of vnodes to something reasonable. Based
off
Sure. That takes the project from awesome to 10X awesome. I absolutely would
be willing to do that. Thanks Kurt!
Regarding your comment on the keyspaces, I agree. There should be a few simple
examples one way or the other that can be duplicated and observed, and then an
example to
Thank you for this valuable info, Jon.
I guess both you and Alex are referring to improved vnodes allocation method
https://issues.apache.org/jira/browse/CASSANDRA-7032 which was implemented in
3.0.
Based on your info and comments in the ticket it's really a bad idea to have
small number of
Sounds like a great idea. Probably would be valuable to add to the official
docs as an example set up if you're willing.
Only thing I'd add is that you should have keyspaces that replicate over
only a subset of DC's, plus one/some replicated to all DC's
On 17 Jan. 2018 03:26, "Kenneth Brotman"
In certain versions (2.2 specifically) I’ve seen a massive performance hit from
the extra logging in some very specific circumstances. In the case I looked at
it was due to the added overhead of reflection. The issue I found was resolved
in 3.0 (I think), but I always disable DEBUG logging
Hi,
Do you guys enable debug log in production? Is it recommended?
By default, the cassandra log level is set to debug:
https://github.com/apache/cassandra/blob/trunk/conf/logback.xml#L100
We’re using 3.0.x, which generates lots of Gossip messages:
FailureDetector.java:456 - Ignoring interval
The ticket I was referring to is the following :
https://issues.apache.org/jira/browse/CASSANDRA-13418
It's been merged in 3.11.1, so just make sure you enable
unsafe_aggressive_sstable_expiration and you'll evict expired SSTables
regardless of overlaps (and IMHO it's totally safe to do this).
Do
We’ve used 32 tokens pre 3.0. It’s been a mixed result due to the randomness.
There’s going to be some imbalance, the amount of imbalance depends on luck,
unfortunately.
I’m interested to hear your results using 4 tokens, would you mind letting the
ML know your experience when you’ve done
Hi,
My read_repair_chance is 0 (AND read_repair_chance = 0.0)
When I bootstrap a new node there is around 700 sstables, but after auto
compaction the number drop to around 100.
I'm using C* 3.11.1. To solve the problem I've already changed to
'unchecked_tombstone_compaction': 'true'. Now should
Agree with you, Jon.
Actually, this cluster was configured by my 'predecessor' and [fortunately for
him] we've never met :)
We're using version 2.1.15 and can't upgrade because of legacy Netflix Astyanax
client used.
Below in the thread Alex mentioned that it's recommended to set vnodes to a
While all the token math is helpful, I have to also call out the elephant in
the room:
You have not correctly configured Cassandra for production.
If you had used the correct endpoint snitch & network topology strategy, you
would be able to withstand the complete failure of an entire
I've begun working on a reference project intended to provide guidance on
configuring and operating a modest Cassandra cluster of about 18 nodes
suitable for the economic study, demonstration, experimentation and testing
of a Cassandra cluster.
The slender cluster would be designed to be as
This looks like the post-POODLE commit:
https://issues.apache.org/jira/browse/CASSANDRA-10508
I think you might just set 'TLS' as in the example to use the JVM's
preferred TLS protocol version.
--
Michael
On 01/16/2018 08:13 AM, Tommy Stendahl wrote:
> Hi,
>
> I have problems upgrading a
On Tue, Jan 16, 2018 at 4:16 PM, Alexander Dejanovski <
a...@thelastpickle.com> wrote:
> Hi Oleksandr,
>
> if bootstrap is disabled, it will only skip the streaming phase but will
> still go through token allocation and thus should use the new algorithm.
> The algorithm won't try to spread data
Thanks for a very helpful reply.
Will try to refactor the code accordingly.
On Tue, Jan 16, 2018 at 4:36 PM, Alexander Dejanovski <
a...@thelastpickle.com> wrote:
> I would not plan on deleting data at the row level as you'll end up with a
> lot of tombstones eventually (and you won't even
Hi Oleksandr,
if bootstrap is disabled, it will only skip the streaming phase but will
still go through token allocation and thus should use the new algorithm.
The algorithm won't try to spread data based on size on disk but it will
try to spread token ownership as evenly as possible.
The
Hello,
We want to add a new rack to an existing cluster (a new Availability Zone
on AWS).
Currently we have 12 nodes in 2 racks with ~4 TB data per node. We also
want to have bigger number of smaller nodes. In order to minimize the
streaming we want to add a new DC which will span 3 racks and
I would not plan on deleting data at the row level as you'll end up with a
lot of tombstones eventually (and you won't even notice them).
It's not healthy to allow that many tombstones to be read, and while your
latency may fit your SLA now, it may not in the future.
Tombstones are going to create
Hi,
I have problems upgrading a cluster from 3.0.14 to 3.11.1 but when I
upgrade the first node it fails to gossip.
I have server encryption enabled on all nodes with this setting:
server_encryption_options:
internode_encryption: all
keystore:
Hello.
I was planning to remove a row (not partition).
Most of the tombstones are seen in the use case of geographic grid with X:Y
as partition key and object id (timeuuid) as clustering key where objects
could be temporary with TTL about 10 hours or fully persistent.
When I select all objects
Alex, thanks for detailed explanation.
> The more nodes you have, the smaller will be the subset of data that cannot
> achieve quorum (so your outage is not as bad as when you have a small number
> of nodes
Okay, let's say we lost 0.5% of keyrange [for specified CL]. Critically
important
Hi,
The overlaps you're seeing on time windows aren't due to automatic
compactions, but to read repairs.
You must be reading at quorum or local_quorum which can perform foreground
read repair in case of digest mismatch.
You can set unchecked_tombstone_compaction to true if you want to perform
Hi list,
My settings:
AND compaction = {'class':
'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
'compaction_window_size': '4', 'compaction_window_unit': 'HOURS',
'enabled': 'true', 'max_threshold': '64', 'min_threshold': '2',
'tombstone_compaction_interval': '15000',
Hi Kyrylo,
high availability can be interpreted in many ways, and comes with some
tradeoffs with consistency when things go wrong.
A few considerations here :
- The more nodes you have, the smaller will be the subset of data that
cannot achieve quorum (so your outage is not as bad as when
...to me it sounds like 'C* isn't that highly-available by design as it's
declared'.
More nodes in a cluster means higher probability of simultaneous node failures.
And from high-availability standpoint, looks like situation is made even worse
by recommended setting vnodes=256.
Need to do
Hi,
could you be more specific about the deletes you're planning to perform ?
This will end up moving your problem somewhere else as you'll be generating
new tombstones (and if you're planning on deleting rows, be aware that row
level tombstones aren't reported anywhere in the metrics, logs and
Hi.
Thank you very much for detailed explanation.
Seems that there is nothing I can do about it except delete records by key
instead of expiring.
On Fri, Jan 12, 2018 at 7:30 PM, Alexander Dejanovski <
a...@thelastpickle.com> wrote:
> Hi,
>
> As DuyHai said, different TTLs could theoretically
28 matches
Mail list logo
