> On Sep 22, 2016, at 6:53 PM, Jesse Gross <je...@kernel.org> wrote: > > I think this isn't actually related to tunnels - ovn-controller > deletes the tunnel ports that it creates when it exits gracefully. > Plus, it seems like vport-geneve was unloaded successfully. >
That is right, unloading of vport-geneve happens successfully.
> When an OVS vport is created, a reference is taken on the
> corresponding vport-*.ko module. However, in the case of internal
> ports created by the datapath, the internal device vport is actually
> part of the main OVS module and so we are taking a reference on
> ourselves. As a result, the only way to unload the module is to delete
> all of the datapaths. It seems like this is what is happening here and
> is what force-reload-kmod is working around.
>
> Obviously, there's no benefit in taking a reference on ourselves, so
> we could just check for this and not take a reference. Combined with
> the fact that lightweight tunnels eliminates the need for vport
> modules that are not part of OVS (on new enough kernels), then
> unloading the module would be enough to gracefully cleanup everything.
Very interesting, Jesse. In my little experiment, I create no logical switches
or any other config, so you are likely talking about datapath in terms of
the connectivity between the ovn-controller and the ovsdb server that is
managing the southbound database?!?
All the steps I took are basically in the email, but I'd be glad to test/try
anything you propose.
Thanks!
-- flaviof
>
> On Thu, Sep 22, 2016 at 7:59 AM, Guru Shetty <g...@ovn.org> wrote:
>> I think it usually happens because the geneve tunnel exists in the kernel
>> datapath. The right way to do this is via '/etc/init.d/openvswitch-switch
>> force-reload-kmod'
>>
>> On 22 September 2016 at 01:46, Flavio Fernandes <fla...@flaviof.com> wrote:
>>>
>>> [cc: Jesse]
>>>
>>> Hi folks,
>>>
>>> While playing with branch-2.6, I've noticed that something may not be
>>> cleaned properly
>>> in the kernel with openvswitch module; [only] when ovn was configured.
>>>
>>> What I observe [below] is that as soon as I configure info used by
>>> ovn-controller to
>>> connect to sb db, I am unable to unload the ovs kernel module.
>>>
>>> Any ideas on what commands I could use to further determine what could be
>>> causing
>>> this? I did not try it on master, but suspect the issue exists there as
>>> well.
>>>
>>> Thanks,
>>>
>>> -- flaviof
>>>
>>> ==
>>>
>>> $ lsb_release -sc
>>> trusty
>>>
>>> cd ~/ovs.git
>>>
>>> ./boot.sh && \
>>> ./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc \
>>> --enable-ssl --with-linux=/lib/modules/$(uname -r)/build
>>> && \
>>> make -j$(nproc) && \
>>> sudo make install ; echo $?
>>>
>>> for x in libcrc32c nf_conntrack nf_nat nf_nat_ipv6 nf_nat_ipv4
>>> nf_defrag_ipv4 gre ; do \
>>> sudo modprobe $x ; \
>>> done
>>>
>>> sudo insmod ./datapath/linux/openvswitch.ko && \
>>> sudo insmod ./datapath/linux/vport-geneve.ko && \
>>> echo ok
>>>
>>> sudo cp ./debian/openvswitch-switch.init /etc/init.d/openvswitch-switch
>>>
>>> sudo /etc/init.d/openvswitch-switch start
>>> sudo /usr/share/openvswitch/scripts/ovn-ctl restart_northd
>>> sudo /usr/share/openvswitch/scripts/ovn-ctl restart_controller
>>>
>>> #### insert commands that configure conf.db with OVN related commands
>>> #### to make issue happen :/
>>>
>>> sudo /usr/share/openvswitch/scripts/ovn-ctl stop_controller
>>> sudo /usr/share/openvswitch/scripts/ovn-ctl stop_northd
>>> sudo /etc/init.d/openvswitch-switch stop
>>>
>>> # as you can see there is nothing related to ovs running...
>>> $ ps aux | grep "ov[sn]"
>>> $
>>>
>>> $ sudo rmmod vport_geneve
>>> $
>>>
>>> # Happy case...
>>> $ # this works fine if commands that configure conf.db with ovn are not
>>> used...
>>> $ sudo rmmod openvswitch
>>> $ sudo lsmod | grep openv
>>> $
>>>
>>> ==
>>>
>>> # However, not as nice if conf.db is configured with encaps info:
>>> $ sudo rmmod openvswitch
>>> rmmod: ERROR: Module openvswitch is in use
>>>
>>> $ sudo lsmod | grep openv
>>> openvswitch 253098 2
>>> gre 13796 1 openvswitch
>>> nf_defrag_ipv4 12758 2 openvswitch,nf_conntrack_ipv4
>>> nf_nat_ipv4 13263 1 openvswitch
>>> nf_defrag_ipv6 34768 2 openvswitch,nf_conntrack_ipv6
>>> nf_nat_ipv6 13279 1 openvswitch
>>> nf_nat 21841 3 openvswitch,nf_nat_ipv4,nf_nat_ipv6
>>> nf_conntrack 97201 6
>>> openvswitch,nf_nat,nf_nat_ipv4,nf_nat_ipv6,nf_conntrack_ipv4,nf_conntrack_ipv6
>>> libcrc32c 12644 1 openvswitch
>>>
>>> $ sudo modprobe --show-depends openvswitch
>>> insmod /lib/modules/3.13.0-95-generic/kernel/lib/libcrc32c.ko
>>> insmod /lib/modules/3.13.0-95-generic/kernel/net/ipv4/gre.ko
>>> insmod /lib/modules/3.13.0-95-generic/kernel/net/ipv4/ip_tunnel.ko
>>> insmod /lib/modules/3.13.0-95-generic/kernel/drivers/net/vxlan.ko
>>> insmod
>>> /lib/modules/3.13.0-95-generic/kernel/net/openvswitch/openvswitch.ko
>>>
>>>
>>> #### commands that configure conf.db with OVN related commands to make
>>> rmmod fail
>>>
>>> HOST_IP=$(ip -4 addr show eth0 | grep -oP "(?<=inet ).*(?=/)")
>>> OVN_DB_IP=$HOST_IP
>>> OVN_SB_DB_PORT=${2-6642}
>>> OVN_SB_REMOTE="tcp:${OVN_DB_IP}:${OVN_SB_DB_PORT}"
>>> sudo ovs-vsctl --no-wait set open_vswitch .
>>> external-ids:ovn-bridge="br-int"
>>> sudo ovs-vsctl --no-wait set open_vswitch .
>>> external-ids:ovn-encap-ip="$HOST_IP"
>>> sudo ovs-vsctl --no-wait set open_vswitch .
>>> external-ids:ovn-encap-type="geneve"
>>> sudo ovs-vsctl --no-wait set open_vswitch .
>>> external-ids:ovn-remote="$OVN_SB_REMOTE"
>>>
>>>
>>>
>>> _______________________________________________
>>> discuss mailing list
>>> discuss@openvswitch.org
>>> http://openvswitch.org/mailman/listinfo/discuss
>>>
>>
>>
>> _______________________________________________
>> discuss mailing list
>> discuss@openvswitch.org
>> http://openvswitch.org/mailman/listinfo/discuss
>>
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss
