Hi Jan/Pradeep, Thanks for flagging this issue, I hadn't considered it before. I agree that this needs to be investigated further.
If it's of use I have reverted the per port mempool patches manually for testing with the head of master so that both approaches can be checked against these type of bugs. https://github.com/istokes/ovs/tree/mempool_revert I can start updating documentation also record these issues if they are not fixed in time for the release. Thanks Ian From: Jan Scheurich [mailto:[email protected]] Sent: Monday, January 29, 2018 1:58 PM To: Venkatesan Pradeep <[email protected]>; Stokes, Ian <[email protected]> Cc: Ilya Maximets <[email protected]>; [email protected] Subject: RE: OVS 2.9 Mempool Issue Hi Pradeep, Yes, this is for sure a bug in the current OVS mempool handling (and possibly in many other DPDK applications). In the past it was perhaps not so critical because shared mempools rarely got released, but with per port mempools it becomes a real threat. And as you say, we do not have any test experience with long-running OVS-DPDK and frequent creation/deletion of vhostuser ports. @Ian: This is also something that should be noted as a known bug in OVS 2.9.0 release documentation if we can't fix it in time for the release. Regards, Jan From: Venkatesan Pradeep Sent: Monday, 29 January, 2018 14:42 To: Jan Scheurich <[email protected]<mailto:[email protected]>>; Stokes, Ian <[email protected]<mailto:[email protected]>> Subject: RE: OVS 2.9 Mempool Issue Hi Ian, There is one more mempool issue that needs to be evaluated. I had brought it up in my post yesterday and Ilya also confirmed it today. When a mempool is deleted (as a consequence of port deletion or configuration change) there is no attempt made to free the mbufs sitting on other port's tx queues. When they get freed up eventually by the driver (in a lazy fashion depending on driver specific threshold) there is potential for corruption/crash. This problem exists even with the shared mempool model but this situation would occur only when the last reference to the mempool goes away. For instance, if all ports share the same MTU the situation will never occur. In some very limited tests that I did, mempool deletion did not cause any obvious problems but it was not an exhaustive test To verify the problem actually exists, I setup a magic number in the mempool, cleared it when the mempool is deleted and validated the magic number when an mbuf is freed. Sure enough, the validation check failed when the mempool was deleted. Given that vhostuser ports keep getting added and deleted as part of VM provisioning it is quite likely that this situation will be hit often. Perhaps there are some safeguards (inbuilt or incidental) that prevented the corruption/crash but I think it is important to understand if we can live without addressing the issue. Thanks, Pradeep _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
