Hi Emeric,
On 5/7/19 1:53 PM, Emeric Brun wrote:
On 5/7/19 1:24 PM, Marcin Deranek wrote:
Hi Emeric,
On 5/7/19 11:44 AM, Emeric Brun wrote:
Hi Marcin,>>>>>> As I use HAProxy 1.8 I had to adjust the patch (see attachment for end
result). Unfortunately after applying the patch there is no change in behavior: we still leak /dev/usdm_drv
descriptors and have "stuck" HAProxy instances after reload..
Regards,
Could you perform a test recompiling the usdm_drv and the engine with this
patch, it applies on QAT 1.7 but I've no hardware to test this version here.
It should fix the fd leak.
It did fix fd leak:
# ls -al /proc/2565/fd|fgrep dev
lr-x------ 1 root root 64 May 7 13:15 0 -> /dev/null
lrwx------ 1 root root 64 May 7 13:15 7 -> /dev/usdm_drv
# systemctl reload haproxy.service
# ls -al /proc/2565/fd|fgrep dev
lr-x------ 1 root root 64 May 7 13:15 0 -> /dev/null
lrwx------ 1 root root 64 May 7 13:15 8 -> /dev/usdm_drv
# systemctl reload haproxy.service
# ls -al /proc/2565/fd|fgrep dev
lr-x------ 1 root root 64 May 7 13:15 0 -> /dev/null
lrwx------ 1 root root 64 May 7 13:15 9 -> /dev/usdm_drv
But there are still stuck processes :-( This is with both patches included: for
QAT and HAProxy.
Regards,
Marcin Deranek
Thank you Marcin! Anyway it's was also a bug.
Could you process a 'show fds' command on a stucked process adding the patch in
attachement.
I did apply this patch and all previous patches (QAT + HAProxy
ssl_free_engine). This is what I got after 1st reload:
show proc
#<PID> <type> <relative PID> <reloads> <uptime>
8025 master 0 1 0d 00h03m25s
# workers
31269 worker 1 0 0d 00h00m39s
31270 worker 2 0 0d 00h00m39s
31271 worker 3 0 0d 00h00m39s
31272 worker 4 0 0d 00h00m39s
# old workers
9286 worker [was: 1] 1 0d 00h03m25s
9287 worker [was: 2] 1 0d 00h03m25s
9288 worker [was: 3] 1 0d 00h03m25s
9289 worker [was: 4] 1 0d 00h03m25s
@!9286 show fd
13 : st=0x05(R:PrA W:pra) ev=0x01(heopI) [lc] cache=0
owner=0x23eaae0 iocb=0x4877c0(mworker_accept_wrapper) tmask=0x1 umask=0x0
16 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x4e1ab0 iocb=0x4e1ab0(thread_sync_io_handler)
tmask=0xffffffffffffffff umask=0x0
20 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1601b840 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
21 : st=0x22(R:pRa W:pRa) ev=0x00(heopi) [lc] cache=0
owner=0x1f0ec4f0 iocb=0x4ce6e0(conn_fd_handler) tmask=0x1 umask=0x0
cflg=0x00241300 fe=GLOBAL mux=PASS mux_ctx=0x22ad8630
1412 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bab1f30 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1413 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x247e5bc0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1414 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x18883650 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1415 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x14476c10 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1416 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11a27850 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1418 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x12008230 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1419 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bb0a570 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1420 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11c94790 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1421 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1449e050 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1422 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1f00c150 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1423 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x15f40550 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1424 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x124b6340 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1425 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11fe4500 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1426 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11c70a60 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1427 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x12572540 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1428 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1249a420 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1430 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11b224a0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1431 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x14f668e0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1432 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1448a630 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1433 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x14f32010 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1434 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1588ed80 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1435 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1efb3e50 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1436 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x10f4cc40 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1437 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bac59b0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1439 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x144b1a70 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1440 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1170a380 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1441 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bad93f0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1442 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bb27ca0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1443 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x158233b0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1444 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x124ba940 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1445 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x15f65850 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1446 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1ab4c9e0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1447 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11e2a7b0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1448 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x16923e40 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1449 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x15e156c0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1450 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1585f040 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1451 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11d0c0f0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1452 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bb00860 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1454 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bb1df90 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1455 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11b16850 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1460 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x115ffe30 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1461 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x16936f10 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1462 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x15fbf350 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1463 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1efd1630 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1465 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1bacf6d0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1467 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11079580 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1468 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11e425d0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1469 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x144a7d60 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1472 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x23e6c10 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1474 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x158beac0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1476 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1270e190 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1480 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11f10960 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1484 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x124a4b40 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1488 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11b461d0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1490 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x11643280 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1492 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x215945c0 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1499 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x14f68b30 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1500 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x19e59970 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1503 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x1fc7b710 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
1508 : st=0x05(R:PrA W:pra) ev=0x00(heopi) [lc] cache=0
owner=0x16e7cc90 iocb=0x4f4d50(ssl_async_fd_free) tmask=0x1 umask=0x0
Regards,
Marcin Deranek
