Hi Anatoly,
On Wednesday 07 March 2018 10:26 PM, Anatoly Burakov wrote: > Before, we were aggregating multiple pages into one memseg, so the > number of memsegs was small. Now, each page gets its own memseg, > so the list of memsegs is huge. To accommodate the new memseg list > size and to keep the under-the-hood workings sane, the memseg list > is now not just a single list, but multiple lists. To be precise, > each hugepage size available on the system gets one or more memseg > lists, per socket. > > In order to support dynamic memory allocation, we reserve all > memory in advance. As in, we do an anonymous mmap() of the entire > maximum size of memory per hugepage size, per socket (which is > limited to either RTE_MAX_MEMSEG_PER_TYPE pages or > RTE_MAX_MEM_PER_TYPE gigabytes worth of memory, whichever is the > smaller one), split over multiple lists (which are limited to > either RTE_MAX_MEMSEG_PER_LIST memsegs or RTE_MAX_MEM_PER_LIST > gigabytes per list, whichever is the smaller one). > > So, for each hugepage size, we get (by default) up to 128G worth > of memory, per socket, split into chunks of up to 32G in size. > The address space is claimed at the start, in eal_common_memory.c. > The actual page allocation code is in eal_memalloc.c (Linux-only > for now), and largely consists of copied EAL memory init code. > > Pages in the list are also indexed by address. That is, for > non-legacy mode, in order to figure out where the page belongs, > one can simply look at base address for a memseg list. Similarly, > figuring out IOVA address of a memzone is a matter of finding the > right memseg list, getting offset and dividing by page size to get > the appropriate memseg. For legacy mode, old behavior of walking > the memseg list remains. > > Due to switch to fbarray and to avoid any intrusive changes, > secondary processes are not supported in this commit. Also, one > particular API call (dump physmem layout) no longer makes sense > and was removed, according to deprecation notice [1]. > > In legacy mode, nothing is preallocated, and all memsegs are in > a list like before, but each segment still resides in an appropriate > memseg list. > > The rest of the changes are really ripple effects from the memseg > change - heap changes, compile fixes, and rewrites to support > fbarray-backed memseg lists. > > [1] http://dpdk.org/dev/patchwork/patch/34002/ > > Signed-off-by: Anatoly Burakov <anatoly.bura...@intel.com> > --- Thanks for good work!. Few observations: # Noticed performance regression for thunderx platform for l3fwd application, drops by 3%. git bisect shows this changeset is offending commit. I'm still investigating reason for perf-dip.. Would like to know - have you noticed any regression on x86 platform? # In next version, pl. make sure that individual patch builds successfully. Right now, Some patches are dependent on other, leads to build break, observed while git-bisecting. Few examples are: >>fa71cdef6963ed795fdd7e7f35085170bb300e39 >>1037fcd989176c5cc83db6223534205cac469765 >> befdec10759d30275a17a829919ee45228d91d3c >> 495e60f4e02af8a344c0f817a60d1ee9b9322df4 [above commits are from your github repo..] # Nits: Perhaps you may club all below comits into one single patch, as changes are identical... that way you'd reduce patch count by few less. 9a1e2a7bd9f6248c680ad3e444b6f173eb92d457 net/vmxnet3: use contiguous allocation for DMA memory 46388b194cd559b5cf7079e01b04bf67a99b64d7 net/virtio: use contiguous allocation for DMA memory a3d2eb10bd998ba3ae3a3d39adeaff38d2e53a9d net/qede: use contiguous allocation for DMA memory 6f16b23ef1f472db475edf05159dea5ae741dbf8 net/i40e: use contiguous allocation for DMA memory f9f7576eed35cb6aa50793810cdda43bcc0f4642 net/enic: use contiguous allocation for DMA memory 2af6c33009b8008da7028a351efed2932b1a13d0 net/ena: use contiguous allocation for DMA memory 18003e22bd7087e5e2e03543cb662d554f7bec52 net/cxgbe: use contiguous allocation for DMA memory 59f79182502dcb3634dfa3e7b918195829777460 net/bnx2x: use contiguous allocation for DMA memory f481a321e41da82ddfa00f5ddbcb42fc29e6ae76 net/avf: use contiguous allocation for DMA memory 5253e9b757c1855a296656d939f5c28e651fea69 crypto/qat: use contiguous allocation for DMA memory 297ab037b4c0d9d725aa6cfdd2c33f7cd9396899 ethdev: use contiguous allocation for DMA memory Thanks.
