Hi Elias,
On Tue, Dec 19, 2017 at 02:23:21PM +0100, Elias Abacioglu wrote:
Hi,
I recently bought a solarflare NIC with (ScaleOut) Onload /
OpenOnload to
test it with HAproxy.
Have anyone tried running haproxy with solarflare onload
functions?
After I started haproxy with onload, this started spamming on the
kernel
log:
Dec 12 14:11:54 dflb06 kernel: [357643.035355] [onload]
oof_socket_add_full_hw: 6:3083 ERROR: FILTER TCP 10.3.54.43:4147
[1]
10.3.20.116:80 [2] failed (-16)
Dec 12 14:11:54 dflb06 kernel: [357643.064395] [onload]
oof_socket_add_full_hw: 6:3491 ERROR: FILTER TCP 10.3.54.43:39321
[3]
10.3.20.113:80 [4] failed (-16)
Dec 12 14:11:54 dflb06 kernel: [357643.081069] [onload]
oof_socket_add_full_hw: 3:2124 ERROR: FILTER TCP 10.3.54.43:62403
[5]
10.3.20.30:445 [6] failed (-16)
Dec 12 14:11:54 dflb06 kernel: [357643.082625] [onload]
oof_socket_add_full_hw: 3:2124 ERROR: FILTER TCP 10.3.54.43:62403
[5]
10.3.20.30:445 [6] failed (-16)
And this in haproxy log:
Dec 12 14:12:07 dflb06 haproxy[21145]: Proxy ssl-relay reached
system
memory limit at 9931 sockets. Please check system tunables.
Dec 12 14:12:07 dflb06 haproxy[21146]: Proxy ssl-relay reached
system
memory limit at 9184 sockets. Please check system tunables.
Dec 12 14:12:07 dflb06 haproxy[21145]: Proxy HTTP reached system
memory
limit at 9931 sockets. Please check system tunables.
Dec 12 14:12:07 dflb06 haproxy[21145]: Proxy HTTP reached system
memory
limit at 9931 sockets. Please check system tunables.
Apparently I've hit the max hardware filter limit on the card.
Does anyone here have experience in running haproxy with onload
features?
I've never got any report of any such test, though in the past I
thought
it would be nice to run such a test, at least to validate the
perimeter
covered by the library (you're using it as LD_PRELOAD, that's it ?).
Mind sharing insights and advice on how to get a functional setup?
I really don't know what can reasonably be expected from code trying
to
partially bypass a part of the TCP stack to be honnest. From what
I've
read a long time ago, onload might be doing its work in a not very
intrusive way but judging by your messages above I'm having some
doubts
now.
Have you tried without this software, using the card normally ? I
mean,
2 years ago I had the opportunity to test haproxy on a dual-40G
setup
and we reached 60 Gbps of forwarded traffic with all machines in the
test bench reaching their limits (and haproxy reaching 100% as
well),
so for me that proves that the TCP stack still scales extremely well
and that while such acceleration software might make sense for a
next
generation NIC running on old hardware (eg: when 400 Gbps NICs start
to appear), I'm really not convinced that it makes any sense to use
them on well supported setups like 2-4 10Gbps links which are very
common nowadays. I mean, I managed to run haproxy at 10Gbps 10 years
ago on a core2-duo! Hardware has evolved quite a bit since :-)
Regards,
Willy