Hi Vlad, On 2019-10-22 10:17 a.m., Vlad Buslov wrote:
Currently, significant fraction of CPU time during TC filter allocation is spent in percpu allocator. Moreover, percpu allocator is protected with single global mutex which negates any potential to improve its performance by means of recent developments in TC filter update API that removed rtnl lock for some Qdiscs and classifiers. In order to significantly improve filter update rate and reduce memory usage we would like to allow users to skip percpu counters allocation for specific action if they don't expect high traffic rate hitting the action, which is a reasonable expectation for hardware-offloaded setup. In that case any potential gains to software fast-path performance gained by usage of percpu-allocated counters compared to regular integer counters protected by spinlock are not important, but amount of additional CPU and memory consumed by them is significant.
Great to see this becoming low hanging on the fruit tree after your improvements. Note: had a discussion a few years back with Eric D.(on Cc) when i was trying to improve action dumping; what you are seeing was very visible when doing a large batch creation of actions. At the time i was thinking of amortizing the cost of that mutex in a batch action create i.e you ask the per cpu allocator to alloc a batch of the stats instead of singular. I understand your use case being different since it is for h/w offload. If you have time can you test with batching many actions and seeing the before/after improvement? Note: even for h/w offload it makes sense to first create the actions then bind to filters (in my world thats what we end up doing). If we can improve the first phase it is a win for both s/w and hw use cases. Question: Given TCA_ACT_FLAGS_FAST_INIT is common to all actions would it make sense to use Could you have used a TLV in the namespace of TCA_ACT_MAX (outer TLV)? You will have to pass a param to ->init(). cheers, jamal
