Spent some time debugging this and I found some interesting bits. I was able to reproduce this by deploying a xenial-queens environment with VXLAN and the OVS firewall. Investigating this, here is what I found:
1) This module is first loaded on the compute nodes when libvirt-bin is installed. This package's postinst script creates the default libvirt network, so when libvirt service is enabled it will create some rules on iptables for this network, which will load the nf_conntrack_ipv4 module. 2) At some point during the configuration of the environment, this default network is destroyed (I'm still investigating who is doing this, but makes sense since nova/neutron won't use the default libvirt network), so those iptables rules won't be added anymore on libvirt service startup, so what was previously loading the module, won't do it. 3) Neutron relies on conntrack for the OVS firewall to work. It's on the documentation: https://docs.openstack.org/newton/networking-guide /config-ovsfwdriver.html 4) As pointed on the bug description, OVS complains whenever the module is not loaded, so we can assume that it's not its responsibility to load it. In my opinion this is something that neutron-ovs-agent should be loading, since the OVS firewall requires conntrack to work and OVS complains that it is not loaded. It would be interesting to see how (if) this works on different openstack deployments. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1834213 Title: After kernel upgrade, nf_conntrack_ipv4 module unloaded, no IP traffic to instances Status in OpenStack neutron-openvswitch charm: Confirmed Status in linux package in Ubuntu: Confirmed Bug description: With an environment running Xenial-Queens, and having just upgraded the linux-image-generic kernel for MDS patching, a few of our hypervisor hosts that were rebooted (3 out of 100) ended up dropping IP (tcp/udp) ingress traffic. It turns out that nf_conntrack module was loaded, but nf_conntrack_ipv4 was not loading, and the traffic was being dropped by this rule: table=72, n_packets=214989, priority=50,ct_state=+inv+trk actions=resubmit(,93) The ct_state "inv" means invalid conntrack state, which the manpage describes as: The state is invalid, meaning that the connection tracker couldn’t identify the connection. This flag is a catch- all for problems in the connection or the connection tracker, such as: • L3/L4 protocol handler is not loaded/unavailable. With the Linux kernel datapath, this may mean that the nf_conntrack_ipv4 or nf_conntrack_ipv6 modules are not loaded. • L3/L4 protocol handler determines that the packet is malformed. • Packets are unexpected length for protocol. It appears that there may be an issue when patching the OS of a hypervisor not running instances may fail to update initrd to load nf_conntrack_ipv4 (and/or _ipv6). I couldn't find anywhere in the charm code that this would be loaded unless the charm's "harden" option is used on nova-compute charm (see charmhelpers contrib/host templates). It is unset in our environment, so we are not using any special module probing. Did nf_conntrack_ipv4 get split out from nf_conntrack in recent kernel upgrades or is it possible that the charm should define a modprobe file if we have the OVS firewall driver configured? To manage notifications about this bug go to: https://bugs.launchpad.net/charm-neutron-openvswitch/+bug/1834213/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
