Thanks guys, I will try all of those and report back. First up, full mpool output. I just noticed ble_gap_update pool, which I don't remember seeing before. How come there is only one of those - is that per connection, or total concurrent updates?
1491016 Mempools: 1491016 name blksz cnt free min 1491018 msys_1 88 96 20 2 1491019 ble_hci_ram_evt_hi_pool 72 32 31 22 1491021 ble_hci_ram_evt_lo_pool 72 32 22 0 1491023 ble_hs_hci_ev_pool 16 64 51 22 1491024 ble_hs_conn_pool 100 32 29 24 1491026 ble_l2cap_chan_pool 28 96 87 72 1491027 ble_l2cap_sig_proc_pool 20 1 1 1 1491029 ble_att_svr_prep_entry_pool 12 16 16 13 1491031 ble_gap_update 24 1 1 0 1491032 ble_gattc_proc_pool 56 32 1 0 1491034 cmd_pool 64 64 58 53 1491035 peer_pool 76 32 29 19 1491037 peer_op_pool 28 64 63 53 1491039 peer_addr_pool 32 48 47 40 1491040 rcpt_pool 40 64 0 0 1491042 ble_att_svr_entry_pool 20 29 0 0 1491043 ble_gatts_clt_cfg_pool 8 33 29 24 Cheers, simon On Mon, Feb 5, 2018 at 5:39 PM, Christopher Collins <[email protected]> wrote: > I agree with Łukasz - enabling `BLE_HS_DEBUG` is a good start. > > I spent some time looking at the code, but I'm afraid I don't see any > possible culprits. > > I have a few requests: > > 1. Could you please send the full output of the `mpool` command (i.e., all > the mempools, not just gattc_procs and mbufs)? > > 2. Also, if your app has the `sys/stats/full` dependency, and you have > enabled `STATS_NAMES`, could you please send the following commands to > the device: > > stat ble_hs > stat ble_att > stat ble_gattc > stat ble_gatts > stat ble_l2cap > stat ble_ll > stat ble_ll_conn > > If you don't have stats enabled, it might be owrthwhile to enable them > so they'll be available next time you replicate the issue. > > Thanks, > Chris > > On Sat, Feb 03, 2018 at 02:17:33PM -0800, Simon Ratner wrote: > > Hey devs, > > > > I am seeing the number of available ble_gattc_proc structs in > > ble_gattc_proc_pool dwindling over time, suspect they are not always > being > > freed (see bottom of the email for mpool output). > > > > This seems to be happening under heavy client load, where a significant > > fraction of the clients may be failing or timing out. The radio is > > nrf52832, and the procs in use are: mtu, disc_svc_uuid, disc_chr_uuid, > > disc_all_dscs, read, write, indicate_custom. There may be an mbuf leak > > associated with this, since i see those dwindle too -- but i have not > > completely eliminated application error for that one yet. > > > > I am trying to isolate the culprit, but in the meantime, if someone has > an > > idea as to which bit of code is most likely to be responsible, or ran > into > > something similar, any information would be appreciated. Scope of change > is > > 3742e67..eb3d264 (yes, i know that's a lot). > > > > Thanks! > > > > mpool output over 3 hours: > > > > ticks name blksz cnt free min > > 037579 ble_gattc_proc_pool 56 32 32 27 > > 062221 ble_gattc_proc_pool 56 32 32 27 > > 068531 ble_gattc_proc_pool 56 32 32 27 > > 084960 ble_gattc_proc_pool 56 32 32 27 > > 087382 ble_gattc_proc_pool 56 32 31 27 > > 108489 ble_gattc_proc_pool 56 32 32 27 > > 128580 ble_gattc_proc_pool 56 32 32 27 > > 136642 ble_gattc_proc_pool 56 32 32 27 > > 147967 ble_gattc_proc_pool 56 32 31 27 > > 148889 ble_gattc_proc_pool 56 32 32 27 > > 161142 ble_gattc_proc_pool 56 32 31 27 > > 162518 ble_gattc_proc_pool 56 32 32 27 > > 173439 ble_gattc_proc_pool 56 32 31 27 > > 173766 ble_gattc_proc_pool 56 32 31 27 > > 263767 ble_gattc_proc_pool 56 32 26 22 > > 264659 ble_gattc_proc_pool 56 32 27 22 > > 265966 ble_gattc_proc_pool 56 32 26 22 > > 266472 ble_gattc_proc_pool 56 32 26 22 > > 270542 ble_gattc_proc_pool 56 32 27 22 > > 274803 ble_gattc_proc_pool 56 32 25 22 > > 286099 ble_gattc_proc_pool 56 32 25 22 > > 286484 ble_gattc_proc_pool 56 32 23 22 > > 297022 ble_gattc_proc_pool 56 32 23 22 > > 304540 ble_gattc_proc_pool 56 32 25 20 > > 309070 ble_gattc_proc_pool 56 32 25 20 > > 318542 ble_gattc_proc_pool 56 32 26 20 > > 359797 ble_gattc_proc_pool 56 32 16 11 > > 361184 ble_gattc_proc_pool 56 32 16 11 > > 364989 ble_gattc_proc_pool 56 32 16 11 > > 367769 ble_gattc_proc_pool 56 32 15 11 > > 371913 ble_gattc_proc_pool 56 32 16 11 > > 384139 ble_gattc_proc_pool 56 32 14 11 > > 384907 ble_gattc_proc_pool 56 32 14 11 > > 388905 ble_gattc_proc_pool 56 32 15 11 > > 396090 ble_gattc_proc_pool 56 32 15 11 > > 405044 ble_gattc_proc_pool 56 32 14 11 > > 405902 ble_gattc_proc_pool 56 32 15 11 > > 408975 ble_gattc_proc_pool 56 32 15 11 > > 417053 ble_gattc_proc_pool 56 32 14 11 > > 422387 ble_gattc_proc_pool 56 32 13 11 > > 428185 ble_gattc_proc_pool 56 32 15 11 > > 429465 ble_gattc_proc_pool 56 32 14 11 > > 440093 ble_gattc_proc_pool 56 32 13 11 > > 440546 ble_gattc_proc_pool 56 32 14 11 > > 448875 ble_gattc_proc_pool 56 32 13 10 > > 450181 ble_gattc_proc_pool 56 32 12 10 > > 459837 ble_gattc_proc_pool 56 32 13 10 > > 468516 ble_gattc_proc_pool 56 32 14 10 > > 476963 ble_gattc_proc_pool 56 32 12 10 > > 477853 ble_gattc_proc_pool 56 32 14 10 > > 480811 ble_gattc_proc_pool 56 32 12 10 > > 486291 ble_gattc_proc_pool 56 32 14 10 > > 487289 ble_gattc_proc_pool 56 32 14 10 > > 492067 ble_gattc_proc_pool 56 32 14 10 > > 498029 ble_gattc_proc_pool 56 32 14 10 > > 501499 ble_gattc_proc_pool 56 32 12 10 > > 502159 ble_gattc_proc_pool 56 32 14 10 > > 505527 ble_gattc_proc_pool 56 32 14 10 > > 510637 ble_gattc_proc_pool 56 32 14 10 > > 515632 ble_gattc_proc_pool 56 32 14 8 > > 517661 ble_gattc_proc_pool 56 32 13 8 > > 536317 ble_gattc_proc_pool 56 32 11 8 > > 537171 ble_gattc_proc_pool 56 32 11 8 > > 559742 ble_gattc_proc_pool 56 32 10 6 > > 593844 ble_gattc_proc_pool 56 32 9 5 > > 597221 ble_gattc_proc_pool 56 32 10 5 > > 609749 ble_gattc_proc_pool 56 32 10 5 > > 614919 ble_gattc_proc_pool 56 32 10 5 > > 617287 ble_gattc_proc_pool 56 32 9 5 > > 617944 ble_gattc_proc_pool 56 32 9 5 > > 618695 ble_gattc_proc_pool 56 32 9 5 > > 619334 ble_gattc_proc_pool 56 32 7 5 > > 627599 ble_gattc_proc_pool 56 32 9 5 > > 629590 ble_gattc_proc_pool 56 32 8 5 > > 632973 ble_gattc_proc_pool 56 32 9 5 > > 638943 ble_gattc_proc_pool 56 32 8 5 > > 639313 ble_gattc_proc_pool 56 32 9 5 > > 643202 ble_gattc_proc_pool 56 32 9 5 > > 644098 ble_gattc_proc_pool 56 32 8 5 > > 645687 ble_gattc_proc_pool 56 32 9 5 > > 649758 ble_gattc_proc_pool 56 32 8 5 > > 650897 ble_gattc_proc_pool 56 32 8 4 > > 651717 ble_gattc_proc_pool 56 32 9 4 > > 661057 ble_gattc_proc_pool 56 32 8 4 > > 704396 ble_gattc_proc_pool 56 32 6 3 > > 751324 ble_gattc_proc_pool 56 32 2 0 > > 757149 ble_gattc_proc_pool 56 32 0 0 > > 765214 ble_gattc_proc_pool 56 32 1 0 > > 826451 ble_gattc_proc_pool 56 32 0 0 > > 827167 ble_gattc_proc_pool 56 32 1 0 > > 836772 ble_gattc_proc_pool 56 32 1 0 > > 837603 ble_gattc_proc_pool 56 32 1 0 > > 841727 ble_gattc_proc_pool 56 32 1 0 > > 899918 ble_gattc_proc_pool 56 32 1 0 > > 973821 ble_gattc_proc_pool 56 32 1 0 > > 981766 ble_gattc_proc_pool 56 32 0 0 > > 994802 ble_gattc_proc_pool 56 32 1 0 > > 997527 ble_gattc_proc_pool 56 32 0 0 > > 1094434 ble_gattc_proc_pool 56 32 1 0 > > 1112118 ble_gattc_proc_pool 56 32 1 0 > > 1133815 ble_gattc_proc_pool 56 32 1 0 > > 1303219 ble_gattc_proc_pool 56 32 1 0 > > 1361911 ble_gattc_proc_pool 56 32 0 0 > > 1374568 ble_gattc_proc_pool 56 32 1 0 > > 1378516 ble_gattc_proc_pool 56 32 1 0 > > 1392138 ble_gattc_proc_pool 56 32 0 0 > > 1438400 ble_gattc_proc_pool 56 32 1 0 > > 1441345 ble_gattc_proc_pool 56 32 1 0 > > 1488771 ble_gattc_proc_pool 56 32 0 0 > > 1491032 ble_gattc_proc_pool 56 32 1 0 > > 1501575 ble_gattc_proc_pool 56 32 1 0 >
