Logan B created CLOUDSTACK-6473:
-----------------------------------
Summary: Debian 7 Virtual Router ip_conntrack_max not set at boot
Key: CLOUDSTACK-6473
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-6473
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Virtual Router
Affects Versions: 4.3.0
Environment: XenServer 6.2
CloudStack 4.3.0
Debian 7 SystemVM/Virtual Router
Reporter: Logan B
Fix For: 4.3.1
The Problem:
The Debian 7 Virtual Router VMs for XenServer experiences intermittent
connectivity problems. This affects all VMs behind the virtual router in
various ways: SSH failures, Apache connections fail, etc.
This issue also affects various function within CloudStack that attempt to
connect to the Virtual Router (updating firewall rules, NAT, etc.)
The Cause:
It appears that the issues is being caused by a low default limit for the
net.ipv4.netfilter.ip_conntrack_max sysctl. The issue can be easily diagnosed
in /var/log/messages:
Apr 22 15:45:34 r-5602-VM kernel: [ 1085.117498] nf_conntrack: table full,
dropping packet.
Apr 22 15:45:34 r-5602-VM kernel: [ 1085.133095] nf_conntrack: table full,
dropping packet.
Apr 22 15:45:34 r-5602-VM kernel: [ 1085.145440] nf_conntrack: table full,
dropping packet.
The default setting for ip_conntrack_max is '3796':
# sysctl net.ipv4.netfilter.ip_conntrack_max
net.ipv4.netfilter.ip_conntrack_max = 3796
As per /etc/sysctl.conf this setting should be '1000000':
net.ipv4.netfilter.ip_conntrack_max=1000000
It would appear that this setting is not being correctly applied when the
virtual router boots.
The Solution:
- A temporary workaround is to manually set the ip_conntrack_max sysctl to the
correct value:
# sysctl -w net.ipv4.netfilter.ip_conntrack_max=1000000
It's likely that this sysctl is being run at boot before the module is loaded,
so it doesn't take effect. There are various solutions suggested around the
web, any of which should work fine.
To resolve this problem a new System VM template should be created. I'm
assuming this can be done in between CloudStack releases. I know there is
supposed to be a new template released to fix the HeartBleed vulnerability, so
this would be a good fix to include with that updated template.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
