[dpdk-dev] Memory requirement calculation to create mempools with external memory

[Bao-Long Tran]

Hi,

My goal is to create mbuf pools using external memory with rte_malloc_heap_*().
The initial implementation goes like this:
- Create an empty heap rte_malloc_heap_create()
- Calculate memory required
- Provision and add the calculated single memory chunk to 
  the heap rte_malloc_heap_memory_add()
- Create mbuf pool from the heap

The problematic step is the calculation step. AFAIK there's no generic way to 
know how much memory is required to create a particular mempool. The example 
I found in testpmd (see below) is quite pessimistic as stated, which leads to 
up to 1G overprovision if you're using 1G hugepages.

I wonder under what circumstances can I make this memory requirement 
calculation deterministic. I.e., let's say I have total control over the 
underlying hugepages, including page size, iova, alignment, contiguousness 
(both iova and va), etc, how can I know exactly how much memory to allocate?

Anatoly, I don't quite understand your comment in the code below about the 
hdr_mem provision in particular. Why do we need to give hdr_mem special 
treatment? Further explanation is much appreciated.

Thanks,
BL

/* extremely pessimistic estimation of memory required to create a mempool */
static int
calc_mem_size(uint32_t nb_mbufs, uint32_t mbuf_sz, size_t pgsz, size_t *out)
{
        unsigned int n_pages, mbuf_per_pg, leftover;
        uint64_t total_mem, mbuf_mem, obj_sz;

        /* there is no good way to predict how much space the mempool will
         * occupy because it will allocate chunks on the fly, and some of those
         * will come from default DPDK memory while some will come from our
         * external memory, so just assume 128MB will be enough for everyone.
         */
        uint64_t hdr_mem = 128 << 20;

        /* account for possible non-contiguousness */
        obj_sz = rte_mempool_calc_obj_size(mbuf_sz, 0, NULL);
        if (obj_sz > pgsz) {
                TESTPMD_LOG(ERR, "Object size is bigger than page size\n");
                return -1;
        }

        mbuf_per_pg = pgsz / obj_sz;
        leftover = (nb_mbufs % mbuf_per_pg) > 0;
        n_pages = (nb_mbufs / mbuf_per_pg) + leftover;

        mbuf_mem = n_pages * pgsz;

        total_mem = RTE_ALIGN(hdr_mem + mbuf_mem, pgsz);

        if (total_mem > SIZE_MAX) {
                TESTPMD_LOG(ERR, "Memory size too big\n");
                return -1;
        }
        *out = (size_t)total_mem;

        return 0;
}