Re: Cassandra Recommended System Settings
The TCP settings are basically "how much RAM to use to buffer data for TCP
sessions, per session", which translates roughly to maximum TCP window
size. You can actually calculate approximately what you need by just
multiplying bandwidth and latency (10,000,000,000bps * .0001s * 1GB/8Gb =
125KB buffer needed to fill the pipe). In practice, I'd double or triple
the max setting vs the calculated value. The suggested value from Datastax
is 16MB, which doesn't seem like a lot, but if you have 1, connections
that could lead to up to 16GB of RAM being dedicated to TCP buffers.
As an example, my traffic in and out of Cassandra is within a local 10Gb
network. I use "409687380 6291456", but that's not particularly
highly-tuned for Cassandra specifically (that is, it's a value also used by
hosts that talk to the outside internet with much higher latency).
On Mon, Oct 21, 2019 at 1:53 PM Sergio wrote:
> Thanks Elliott!
>
> How do you know if there is too much RAM used for those settings?
>
> Which metrics do you keep track of?
>
> What would you recommend instead?
>
> Best,
>
> Sergio
>
> On Mon, Oct 21, 2019, 1:41 PM Elliott Sims wrote:
>
>> Based on my experiences, if you have a new enough kernel I'd strongly
>> suggest switching the TCP scheduler algorithm to BBR. I've found the rest
>> tend to be extremely sensitive to even small amounts of packet loss among
>> cluster members where BBR holds up well.
>>
>> High ulimits for basically everything are probably a good idea, although
>> "unlimited" may not be purely optimal for all cases.
>> The TCP keepalive settings are probably only necessary for traffic
>> buggy/misconfigured firewalls, but shouldn't really do any harm on a modern
>> fast network.
>>
>> The TCP memory settings are pretty aggressive and probably result in
>> unnecessary RAM usage.
>> The net.core.rmem_default/net.core.wmem_default settings are overridden
>> by the TCP-specific settings as far as I know, so they're not really
>> relevant/helpful for Cassandra
>> The net.ipv4.tcp_rmem/net.ipv4.tcp_wmem max settings are pretty
>> aggressive. That works out to something like 1Gbps with 130ms latency per
>> TCP connection, but on a local LAN with latencies <1ms it's enough buffer
>> for over 100Gbps per TCP session. A much smaller value will probably make
>> more sense for most setups.
>>
>>
>> On Mon, Oct 21, 2019 at 10:21 AM Sergio
>> wrote:
>>
>>>
>>> Hello!
>>>
>>> This is the kernel that I am using
>>> Linux 4.16.13-1.el7.elrepo.x86_64 #1 SMP Wed May 30 14:31:51 EDT 2018
>>> x86_64 x86_64 x86_64 GNU/Linux
>>>
>>> Best,
>>>
>>> Sergio
>>>
>>> Il giorno lun 21 ott 2019 alle ore 07:30 Reid Pinchback <
>>> rpinchb...@tripadvisor.com> ha scritto:
>>>
I don't know which distro and version you are using, but watch out for
surprises in what vm.swappiness=0 means. In older kernels it means "only
use swap when desperate". I believe that newer kernels changed to have 1
mean that, and 0 means to always use the oomkiller. Neither situation is
strictly good or bad, what matters is what you intend the system behavior
to be in comparison with whatever monitoring/alerting you have put in
place.
R
On 10/18/19, 9:04 PM, "Sergio Bilello"
wrote:
Message from External Sender
Hello everyone!
Do you have any setting that you would change or tweak from the
below list?
sudo cat /proc/4379/limits
Limit Soft Limit Hard Limit
Units
Max cpu time unlimitedunlimited
seconds
Max file size unlimitedunlimited
bytes
Max data size unlimitedunlimited
bytes
Max stack sizeunlimitedunlimited
bytes
Max core file sizeunlimitedunlimited
bytes
Max resident set unlimitedunlimited
bytes
Max processes 3276832768
processes
Max open files1048576 1048576
files
Max locked memory unlimitedunlimited
bytes
Max address space unlimitedunlimited
bytes
Max file locksunlimitedunlimited
locks
Max pending signals unlimitedunlimited
signals
Max msgqueue size unlimitedunlimited
bytes
Max nice priority 00
Max realtime priority 00
Max realtime timeout unlimitedunlimited
us
These are the sysctl settings
default['cassandra']['sysctl'] = {
Re: Cassandra Recommended System Settings
Sergio, if you do some online searching about ‘bufferbloat’ in networking,
you’ll find the background to help explain what motivates networking changes.
Actual investigation of network performance can get a bit gnarly. The TL;DR
summary is that big buffers function like big queues, and thus attempts to
speed up throughput can cause things stuck in a queue to have higher latency.
With very fast networks, there isn’t as much need to have big buffers. Imagine
having a coordinator node waiting to respond to a query but can’t because a
bunch of merkel trees are sitting in the tcp buffer waiting to be sent out.
Sometimes total latency doesn’t fairly measure actual effort to do the work,
some of that can be time spent sitting waiting in the buffer to be shipped out
back to the client.
From: Sergio
Reply-To: "user@cassandra.apache.org"
Date: Monday, October 21, 2019 at 4:54 PM
To: "user@cassandra.apache.org"
Subject: Re: Cassandra Recommended System Settings
Message from External Sender
Thanks Elliott!
How do you know if there is too much RAM used for those settings?
Which metrics do you keep track of?
What would you recommend instead?
Best,
Sergio
On Mon, Oct 21, 2019, 1:41 PM Elliott Sims
mailto:elli...@backblaze.com>> wrote:
Based on my experiences, if you have a new enough kernel I'd strongly suggest
switching the TCP scheduler algorithm to BBR. I've found the rest tend to be
extremely sensitive to even small amounts of packet loss among cluster members
where BBR holds up well.
High ulimits for basically everything are probably a good idea, although
"unlimited" may not be purely optimal for all cases.
The TCP keepalive settings are probably only necessary for traffic
buggy/misconfigured firewalls, but shouldn't really do any harm on a modern
fast network.
The TCP memory settings are pretty aggressive and probably result in
unnecessary RAM usage.
The net.core.rmem_default/net.core.wmem_default settings are overridden by the
TCP-specific settings as far as I know, so they're not really relevant/helpful
for Cassandra
The net.ipv4.tcp_rmem/net.ipv4.tcp_wmem max settings are pretty aggressive.
That works out to something like 1Gbps with 130ms latency per TCP connection,
but on a local LAN with latencies <1ms it's enough buffer for over 100Gbps per
TCP session. A much smaller value will probably make more sense for most
setups.
On Mon, Oct 21, 2019 at 10:21 AM Sergio
mailto:lapostadiser...@gmail.com>> wrote:
Hello!
This is the kernel that I am using
Linux 4.16.13-1.el7.elrepo.x86_64 #1 SMP Wed May 30 14:31:51 EDT 2018 x86_64
x86_64 x86_64 GNU/Linux
Best,
Sergio
Il giorno lun 21 ott 2019 alle ore 07:30 Reid Pinchback
mailto:rpinchb...@tripadvisor.com>> ha scritto:
I don't know which distro and version you are using, but watch out for
surprises in what vm.swappiness=0 means. In older kernels it means "only use
swap when desperate". I believe that newer kernels changed to have 1 mean
that, and 0 means to always use the oomkiller. Neither situation is strictly
good or bad, what matters is what you intend the system behavior to be in
comparison with whatever monitoring/alerting you have put in place.
R
On 10/18/19, 9:04 PM, "Sergio Bilello"
mailto:lapostadiser...@gmail.com>> wrote:
Message from External Sender
Hello everyone!
Do you have any setting that you would change or tweak from the below list?
sudo cat /proc/4379/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimitedunlimitedseconds
Max file size unlimitedunlimitedbytes
Max data size unlimitedunlimitedbytes
Max stack sizeunlimitedunlimitedbytes
Max core file sizeunlimitedunlimitedbytes
Max resident set unlimitedunlimitedbytes
Max processes 3276832768
processes
Max open files1048576 1048576 files
Max locked memory unlimitedunlimitedbytes
Max address space unlimitedunlimitedbytes
Max file locksunlimitedunlimitedlocks
Max pending signals unlimitedunlimitedsignals
Max msgqueue size unlimitedunlimitedbytes
Max nice priority 00
Max realtime priority 00
Max realtime timeout unlimitedunlimitedus
These are the sysctl settings
default['cassandra']['sysctl'] = {
'net.ipv4.tcp_keepalive_time' => 60,
'ne
Re: Cassandra Recommended System Settings
Thanks Elliott!
How do you know if there is too much RAM used for those settings?
Which metrics do you keep track of?
What would you recommend instead?
Best,
Sergio
On Mon, Oct 21, 2019, 1:41 PM Elliott Sims wrote:
> Based on my experiences, if you have a new enough kernel I'd strongly
> suggest switching the TCP scheduler algorithm to BBR. I've found the rest
> tend to be extremely sensitive to even small amounts of packet loss among
> cluster members where BBR holds up well.
>
> High ulimits for basically everything are probably a good idea, although
> "unlimited" may not be purely optimal for all cases.
> The TCP keepalive settings are probably only necessary for traffic
> buggy/misconfigured firewalls, but shouldn't really do any harm on a modern
> fast network.
>
> The TCP memory settings are pretty aggressive and probably result in
> unnecessary RAM usage.
> The net.core.rmem_default/net.core.wmem_default settings are overridden by
> the TCP-specific settings as far as I know, so they're not really
> relevant/helpful for Cassandra
> The net.ipv4.tcp_rmem/net.ipv4.tcp_wmem max settings are pretty
> aggressive. That works out to something like 1Gbps with 130ms latency per
> TCP connection, but on a local LAN with latencies <1ms it's enough buffer
> for over 100Gbps per TCP session. A much smaller value will probably make
> more sense for most setups.
>
>
> On Mon, Oct 21, 2019 at 10:21 AM Sergio wrote:
>
>>
>> Hello!
>>
>> This is the kernel that I am using
>> Linux 4.16.13-1.el7.elrepo.x86_64 #1 SMP Wed May 30 14:31:51 EDT 2018
>> x86_64 x86_64 x86_64 GNU/Linux
>>
>> Best,
>>
>> Sergio
>>
>> Il giorno lun 21 ott 2019 alle ore 07:30 Reid Pinchback <
>> rpinchb...@tripadvisor.com> ha scritto:
>>
>>> I don't know which distro and version you are using, but watch out for
>>> surprises in what vm.swappiness=0 means. In older kernels it means "only
>>> use swap when desperate". I believe that newer kernels changed to have 1
>>> mean that, and 0 means to always use the oomkiller. Neither situation is
>>> strictly good or bad, what matters is what you intend the system behavior
>>> to be in comparison with whatever monitoring/alerting you have put in place.
>>>
>>> R
>>>
>>>
>>> On 10/18/19, 9:04 PM, "Sergio Bilello"
>>> wrote:
>>>
>>> Message from External Sender
>>>
>>> Hello everyone!
>>>
>>>
>>>
>>> Do you have any setting that you would change or tweak from the
>>> below list?
>>>
>>>
>>>
>>> sudo cat /proc/4379/limits
>>>
>>> Limit Soft Limit Hard Limit
>>> Units
>>>
>>> Max cpu time unlimitedunlimited
>>> seconds
>>>
>>> Max file size unlimitedunlimited
>>> bytes
>>>
>>> Max data size unlimitedunlimited
>>> bytes
>>>
>>> Max stack sizeunlimitedunlimited
>>> bytes
>>>
>>> Max core file sizeunlimitedunlimited
>>> bytes
>>>
>>> Max resident set unlimitedunlimited
>>> bytes
>>>
>>> Max processes 3276832768
>>> processes
>>>
>>> Max open files1048576 1048576
>>> files
>>>
>>> Max locked memory unlimitedunlimited
>>> bytes
>>>
>>> Max address space unlimitedunlimited
>>> bytes
>>>
>>> Max file locksunlimitedunlimited
>>> locks
>>>
>>> Max pending signals unlimitedunlimited
>>> signals
>>>
>>> Max msgqueue size unlimitedunlimited
>>> bytes
>>>
>>> Max nice priority 00
>>>
>>> Max realtime priority 00
>>>
>>> Max realtime timeout unlimitedunlimited
>>> us
>>>
>>>
>>>
>>> These are the sysctl settings
>>>
>>> default['cassandra']['sysctl'] = {
>>>
>>> 'net.ipv4.tcp_keepalive_time' => 60,
>>>
>>> 'net.ipv4.tcp_keepalive_probes' => 3,
>>>
>>> 'net.ipv4.tcp_keepalive_intvl' => 10,
>>>
>>> 'net.core.rmem_max' => 16777216,
>>>
>>> 'net.core.wmem_max' => 16777216,
>>>
>>> 'net.core.rmem_default' => 16777216,
>>>
>>> 'net.core.wmem_default' => 16777216,
>>>
>>> 'net.core.optmem_max' => 40960,
>>>
>>> 'net.ipv4.tcp_rmem' => '4096 87380 16777216',
>>>
>>> 'net.ipv4.tcp_wmem' => '4096 65536 16777216',
>>>
>>> 'net.ipv4.ip_local_port_range' => '1 65535',
>>>
>>> 'net.ipv4.tcp_window_scaling' => 1,
>>>
>>>'net.core.netdev_max_backlog' => 2500,
>>>
>>>'net.core.somaxconn' => 65000,
>>>
>>> 'vm.max_map_count' => 1048575,
>>>
>>> 'vm.swappiness' => 0
>>>
>>> }
>>>
>>>
>>>
>>> Am I missing something else?
>>>
>>>
>>>
>>> Do you have any experience to configure CENTOS 7
>>>
>>> for
>>>
>>> JAVA HUGE PAGES
>>>
>>>
>>>
Re: Cassandra Recommended System Settings
Based on my experiences, if you have a new enough kernel I'd strongly
suggest switching the TCP scheduler algorithm to BBR. I've found the rest
tend to be extremely sensitive to even small amounts of packet loss among
cluster members where BBR holds up well.
High ulimits for basically everything are probably a good idea, although
"unlimited" may not be purely optimal for all cases.
The TCP keepalive settings are probably only necessary for traffic
buggy/misconfigured firewalls, but shouldn't really do any harm on a modern
fast network.
The TCP memory settings are pretty aggressive and probably result in
unnecessary RAM usage.
The net.core.rmem_default/net.core.wmem_default settings are overridden by
the TCP-specific settings as far as I know, so they're not really
relevant/helpful for Cassandra
The net.ipv4.tcp_rmem/net.ipv4.tcp_wmem max settings are pretty
aggressive. That works out to something like 1Gbps with 130ms latency per
TCP connection, but on a local LAN with latencies <1ms it's enough buffer
for over 100Gbps per TCP session. A much smaller value will probably make
more sense for most setups.
On Mon, Oct 21, 2019 at 10:21 AM Sergio wrote:
>
> Hello!
>
> This is the kernel that I am using
> Linux 4.16.13-1.el7.elrepo.x86_64 #1 SMP Wed May 30 14:31:51 EDT 2018
> x86_64 x86_64 x86_64 GNU/Linux
>
> Best,
>
> Sergio
>
> Il giorno lun 21 ott 2019 alle ore 07:30 Reid Pinchback <
> rpinchb...@tripadvisor.com> ha scritto:
>
>> I don't know which distro and version you are using, but watch out for
>> surprises in what vm.swappiness=0 means. In older kernels it means "only
>> use swap when desperate". I believe that newer kernels changed to have 1
>> mean that, and 0 means to always use the oomkiller. Neither situation is
>> strictly good or bad, what matters is what you intend the system behavior
>> to be in comparison with whatever monitoring/alerting you have put in place.
>>
>> R
>>
>>
>> On 10/18/19, 9:04 PM, "Sergio Bilello"
>> wrote:
>>
>> Message from External Sender
>>
>> Hello everyone!
>>
>>
>>
>> Do you have any setting that you would change or tweak from the below
>> list?
>>
>>
>>
>> sudo cat /proc/4379/limits
>>
>> Limit Soft Limit Hard Limit
>> Units
>>
>> Max cpu time unlimitedunlimited
>> seconds
>>
>> Max file size unlimitedunlimited
>> bytes
>>
>> Max data size unlimitedunlimited
>> bytes
>>
>> Max stack sizeunlimitedunlimited
>> bytes
>>
>> Max core file sizeunlimitedunlimited
>> bytes
>>
>> Max resident set unlimitedunlimited
>> bytes
>>
>> Max processes 3276832768
>> processes
>>
>> Max open files1048576 1048576
>> files
>>
>> Max locked memory unlimitedunlimited
>> bytes
>>
>> Max address space unlimitedunlimited
>> bytes
>>
>> Max file locksunlimitedunlimited
>> locks
>>
>> Max pending signals unlimitedunlimited
>> signals
>>
>> Max msgqueue size unlimitedunlimited
>> bytes
>>
>> Max nice priority 00
>>
>> Max realtime priority 00
>>
>> Max realtime timeout unlimitedunlimitedus
>>
>>
>>
>> These are the sysctl settings
>>
>> default['cassandra']['sysctl'] = {
>>
>> 'net.ipv4.tcp_keepalive_time' => 60,
>>
>> 'net.ipv4.tcp_keepalive_probes' => 3,
>>
>> 'net.ipv4.tcp_keepalive_intvl' => 10,
>>
>> 'net.core.rmem_max' => 16777216,
>>
>> 'net.core.wmem_max' => 16777216,
>>
>> 'net.core.rmem_default' => 16777216,
>>
>> 'net.core.wmem_default' => 16777216,
>>
>> 'net.core.optmem_max' => 40960,
>>
>> 'net.ipv4.tcp_rmem' => '4096 87380 16777216',
>>
>> 'net.ipv4.tcp_wmem' => '4096 65536 16777216',
>>
>> 'net.ipv4.ip_local_port_range' => '1 65535',
>>
>> 'net.ipv4.tcp_window_scaling' => 1,
>>
>>'net.core.netdev_max_backlog' => 2500,
>>
>>'net.core.somaxconn' => 65000,
>>
>> 'vm.max_map_count' => 1048575,
>>
>> 'vm.swappiness' => 0
>>
>> }
>>
>>
>>
>> Am I missing something else?
>>
>>
>>
>> Do you have any experience to configure CENTOS 7
>>
>> for
>>
>> JAVA HUGE PAGES
>>
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html-23CheckJavaHugepagessettings=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=b6lGdbtv1SN9opBsIOFRT6IX6BroMW-8Tudk9qEh3bI=
>>
>>
>>
>> OPTIMIZE SSD
>>
>>
>>
Re: Cassandra Recommended System Settings
Hello!
This is the kernel that I am using
Linux 4.16.13-1.el7.elrepo.x86_64 #1 SMP Wed May 30 14:31:51 EDT 2018
x86_64 x86_64 x86_64 GNU/Linux
Best,
Sergio
Il giorno lun 21 ott 2019 alle ore 07:30 Reid Pinchback <
rpinchb...@tripadvisor.com> ha scritto:
> I don't know which distro and version you are using, but watch out for
> surprises in what vm.swappiness=0 means. In older kernels it means "only
> use swap when desperate". I believe that newer kernels changed to have 1
> mean that, and 0 means to always use the oomkiller. Neither situation is
> strictly good or bad, what matters is what you intend the system behavior
> to be in comparison with whatever monitoring/alerting you have put in place.
>
> R
>
>
> On 10/18/19, 9:04 PM, "Sergio Bilello" wrote:
>
> Message from External Sender
>
> Hello everyone!
>
>
>
> Do you have any setting that you would change or tweak from the below
> list?
>
>
>
> sudo cat /proc/4379/limits
>
> Limit Soft Limit Hard Limit
> Units
>
> Max cpu time unlimitedunlimited
> seconds
>
> Max file size unlimitedunlimited
> bytes
>
> Max data size unlimitedunlimited
> bytes
>
> Max stack sizeunlimitedunlimited
> bytes
>
> Max core file sizeunlimitedunlimited
> bytes
>
> Max resident set unlimitedunlimited
> bytes
>
> Max processes 3276832768
> processes
>
> Max open files1048576 1048576
> files
>
> Max locked memory unlimitedunlimited
> bytes
>
> Max address space unlimitedunlimited
> bytes
>
> Max file locksunlimitedunlimited
> locks
>
> Max pending signals unlimitedunlimited
> signals
>
> Max msgqueue size unlimitedunlimited
> bytes
>
> Max nice priority 00
>
> Max realtime priority 00
>
> Max realtime timeout unlimitedunlimitedus
>
>
>
> These are the sysctl settings
>
> default['cassandra']['sysctl'] = {
>
> 'net.ipv4.tcp_keepalive_time' => 60,
>
> 'net.ipv4.tcp_keepalive_probes' => 3,
>
> 'net.ipv4.tcp_keepalive_intvl' => 10,
>
> 'net.core.rmem_max' => 16777216,
>
> 'net.core.wmem_max' => 16777216,
>
> 'net.core.rmem_default' => 16777216,
>
> 'net.core.wmem_default' => 16777216,
>
> 'net.core.optmem_max' => 40960,
>
> 'net.ipv4.tcp_rmem' => '4096 87380 16777216',
>
> 'net.ipv4.tcp_wmem' => '4096 65536 16777216',
>
> 'net.ipv4.ip_local_port_range' => '1 65535',
>
> 'net.ipv4.tcp_window_scaling' => 1,
>
>'net.core.netdev_max_backlog' => 2500,
>
>'net.core.somaxconn' => 65000,
>
> 'vm.max_map_count' => 1048575,
>
> 'vm.swappiness' => 0
>
> }
>
>
>
> Am I missing something else?
>
>
>
> Do you have any experience to configure CENTOS 7
>
> for
>
> JAVA HUGE PAGES
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html-23CheckJavaHugepagessettings=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=b6lGdbtv1SN9opBsIOFRT6IX6BroMW-8Tudk9qEh3bI=
>
>
>
> OPTIMIZE SSD
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html-23OptimizeSSDs=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=c0S3S3V_0YHVMx2I-pyOh24MiQs1D-L73JytaSw648M=
>
>
>
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=PZFG6SXF6dL5LRJ-aUoidHnnLGpKPbpxdKstM8M9JMk=
>
>
>
> We are using AWS i3.xlarge instances
>
>
>
> Thanks,
>
>
>
> Sergio
>
>
>
> -
>
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
>
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>
>
>
>
>
Re: Cassandra Recommended System Settings
I don't know which distro and version you are using, but watch out for
surprises in what vm.swappiness=0 means. In older kernels it means "only use
swap when desperate". I believe that newer kernels changed to have 1 mean
that, and 0 means to always use the oomkiller. Neither situation is strictly
good or bad, what matters is what you intend the system behavior to be in
comparison with whatever monitoring/alerting you have put in place.
R
On 10/18/19, 9:04 PM, "Sergio Bilello" wrote:
Message from External Sender
Hello everyone!
Do you have any setting that you would change or tweak from the below list?
sudo cat /proc/4379/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimitedunlimitedseconds
Max file size unlimitedunlimitedbytes
Max data size unlimitedunlimitedbytes
Max stack sizeunlimitedunlimitedbytes
Max core file sizeunlimitedunlimitedbytes
Max resident set unlimitedunlimitedbytes
Max processes 3276832768
processes
Max open files1048576 1048576 files
Max locked memory unlimitedunlimitedbytes
Max address space unlimitedunlimitedbytes
Max file locksunlimitedunlimitedlocks
Max pending signals unlimitedunlimitedsignals
Max msgqueue size unlimitedunlimitedbytes
Max nice priority 00
Max realtime priority 00
Max realtime timeout unlimitedunlimitedus
These are the sysctl settings
default['cassandra']['sysctl'] = {
'net.ipv4.tcp_keepalive_time' => 60,
'net.ipv4.tcp_keepalive_probes' => 3,
'net.ipv4.tcp_keepalive_intvl' => 10,
'net.core.rmem_max' => 16777216,
'net.core.wmem_max' => 16777216,
'net.core.rmem_default' => 16777216,
'net.core.wmem_default' => 16777216,
'net.core.optmem_max' => 40960,
'net.ipv4.tcp_rmem' => '4096 87380 16777216',
'net.ipv4.tcp_wmem' => '4096 65536 16777216',
'net.ipv4.ip_local_port_range' => '1 65535',
'net.ipv4.tcp_window_scaling' => 1,
'net.core.netdev_max_backlog' => 2500,
'net.core.somaxconn' => 65000,
'vm.max_map_count' => 1048575,
'vm.swappiness' => 0
}
Am I missing something else?
Do you have any experience to configure CENTOS 7
for
JAVA HUGE PAGES
https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html-23CheckJavaHugepagessettings=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=b6lGdbtv1SN9opBsIOFRT6IX6BroMW-8Tudk9qEh3bI=
OPTIMIZE SSD
https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html-23OptimizeSSDs=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=c0S3S3V_0YHVMx2I-pyOh24MiQs1D-L73JytaSw648M=
https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_dse_5.1_dse-2Dadmin_datastax-5Fenterprise_config_configRecommendedSettings.html=DwIBaQ=9Hv6XPedRSA-5PSECC38X80c1h60_XWA4z1k_R1pROA=OIgB3poYhzp3_A7WgD7iBCnsJaYmspOa2okNpf6uqWc=zke-WpkD1c6Qt1cz8mJG0ZQ37h8kezqknMSnerQhXuU=PZFG6SXF6dL5LRJ-aUoidHnnLGpKPbpxdKstM8M9JMk=
We are using AWS i3.xlarge instances
Thanks,
Sergio
-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org
Cassandra Recommended System Settings
Hello everyone!
Do you have any setting that you would change or tweak from the below list?
sudo cat /proc/4379/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimitedunlimitedseconds
Max file size unlimitedunlimitedbytes
Max data size unlimitedunlimitedbytes
Max stack sizeunlimitedunlimitedbytes
Max core file sizeunlimitedunlimitedbytes
Max resident set unlimitedunlimitedbytes
Max processes 3276832768processes
Max open files1048576 1048576 files
Max locked memory unlimitedunlimitedbytes
Max address space unlimitedunlimitedbytes
Max file locksunlimitedunlimitedlocks
Max pending signals unlimitedunlimitedsignals
Max msgqueue size unlimitedunlimitedbytes
Max nice priority 00
Max realtime priority 00
Max realtime timeout unlimitedunlimitedus
These are the sysctl settings
default['cassandra']['sysctl'] = {
'net.ipv4.tcp_keepalive_time' => 60,
'net.ipv4.tcp_keepalive_probes' => 3,
'net.ipv4.tcp_keepalive_intvl' => 10,
'net.core.rmem_max' => 16777216,
'net.core.wmem_max' => 16777216,
'net.core.rmem_default' => 16777216,
'net.core.wmem_default' => 16777216,
'net.core.optmem_max' => 40960,
'net.ipv4.tcp_rmem' => '4096 87380 16777216',
'net.ipv4.tcp_wmem' => '4096 65536 16777216',
'net.ipv4.ip_local_port_range' => '1 65535',
'net.ipv4.tcp_window_scaling' => 1,
'net.core.netdev_max_backlog' => 2500,
'net.core.somaxconn' => 65000,
'vm.max_map_count' => 1048575,
'vm.swappiness' => 0
}
Am I missing something else?
Do you have any experience to configure CENTOS 7
for
JAVA HUGE PAGES
https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/config/configRecommendedSettings.html#CheckJavaHugepagessettings
OPTIMIZE SSD
https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/config/configRecommendedSettings.html#OptimizeSSDs
https://docs.datastax.com/en/dse/5.1/dse-admin/datastax_enterprise/config/configRecommendedSettings.html
We are using AWS i3.xlarge instances
Thanks,
Sergio
-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org
