Hi Elias, On Tue, Dec 19, 2017 at 02:23:21PM +0100, Elias Abacioglu wrote: > Hi, > > I recently bought a solarflare NIC with (ScaleOut) Onload / OpenOnload to > test it with HAproxy. > > Have anyone tried running haproxy with solarflare onload functions? > > After I started haproxy with onload, this started spamming on the kernel > log: > Dec 12 14:11:54 dflb06 kernel: [357643.035355] [onload] > oof_socket_add_full_hw: 6:3083 ERROR: FILTER TCP 10.3.54.43:4147 > 10.3.20.116:80 failed (-16) > Dec 12 14:11:54 dflb06 kernel: [357643.064395] [onload] > oof_socket_add_full_hw: 6:3491 ERROR: FILTER TCP 10.3.54.43:39321 > 10.3.20.113:80 failed (-16) > Dec 12 14:11:54 dflb06 kernel: [357643.081069] [onload] > oof_socket_add_full_hw: 3:2124 ERROR: FILTER TCP 10.3.54.43:62403 > 10.3.20.30:445 failed (-16) > Dec 12 14:11:54 dflb06 kernel: [357643.082625] [onload] > oof_socket_add_full_hw: 3:2124 ERROR: FILTER TCP 10.3.54.43:62403 > 10.3.20.30:445 failed (-16) > > And this in haproxy log: > Dec 12 14:12:07 dflb06 haproxy[21145]: Proxy ssl-relay reached system > memory limit at 9931 sockets. Please check system tunables. > Dec 12 14:12:07 dflb06 haproxy[21146]: Proxy ssl-relay reached system > memory limit at 9184 sockets. Please check system tunables. > Dec 12 14:12:07 dflb06 haproxy[21145]: Proxy HTTP reached system memory > limit at 9931 sockets. Please check system tunables. > Dec 12 14:12:07 dflb06 haproxy[21145]: Proxy HTTP reached system memory > limit at 9931 sockets. Please check system tunables. > > > Apparently I've hit the max hardware filter limit on the card. > Does anyone here have experience in running haproxy with onload features?
I've never got any report of any such test, though in the past I thought it would be nice to run such a test, at least to validate the perimeter covered by the library (you're using it as LD_PRELOAD, that's it ?). > Mind sharing insights and advice on how to get a functional setup? I really don't know what can reasonably be expected from code trying to partially bypass a part of the TCP stack to be honnest. From what I've read a long time ago, onload might be doing its work in a not very intrusive way but judging by your messages above I'm having some doubts now. Have you tried without this software, using the card normally ? I mean, 2 years ago I had the opportunity to test haproxy on a dual-40G setup and we reached 60 Gbps of forwarded traffic with all machines in the test bench reaching their limits (and haproxy reaching 100% as well), so for me that proves that the TCP stack still scales extremely well and that while such acceleration software might make sense for a next generation NIC running on old hardware (eg: when 400 Gbps NICs start to appear), I'm really not convinced that it makes any sense to use them on well supported setups like 2-4 10Gbps links which are very common nowadays. I mean, I managed to run haproxy at 10Gbps 10 years ago on a core2-duo! Hardware has evolved quite a bit since :-) Regards, Willy
