On Fri, 9 Nov 2018 14:03:25 +0000 "Burakov, Anatoly" <anatoly.bura...@intel.com> wrote:
> On 09-Nov-18 12:20 PM, Burakov, Anatoly wrote: > > On 09-Nov-18 9:23 AM, jianmingfan wrote: > >> --- fix coding style of the previous patch > >> > >> During procless startup, dpdk invokes clear_hugedir() to unlink all > >> hugepage files under /dev/hugepages. Then in map_all_hugepages(), > >> it invokes mmap to allocate and zero all the huge pages as configured > >> in /sys/kernel/mm/hugepages/xxx/nr_hugepages. > >> > >> This cause startup process extreamly slow with large size of huge page > >> configured. > >> > >> In our use case, we usually configure as large as 200GB hugepages in our > >> router. It takes more than 50s each time dpdk process startup to clear > >> the pages. > >> > >> To address this issue, user can turn on --reuse-map switch. With it, > >> dpdk will check the validity of the exiting page cache under > >> /dev/hugespages. If valid, the cache will be reused not deleted, > >> so that the os doesn't need to zero the pages again. > >> > >> However, as there are a lot of users ,e.g. rte_kni_alloc, rely on the > >> os zeor page behavior. To keep things work, I add memset during > >> malloc_heap_alloc(). This makes sense due to the following reason. > >> 1) user often configure hugepage size too large to be used by the > >> program. > >> In our router, 200GB is configured, but less than 2GB is actually used. > >> 2) dpdk users don't call heap allocation in performance-critical path. > >> They alloc memory during process bootup. > >> > >> Signed-off-by: Jianming Fan <fanjianm...@jd.com> > >> --- > > > > I believe this issue is better solved by actually fixing all of the > > memory that DPDK leaves behind. We already have rte_eal_cleanup() call > > which will deallocate any EAL-allocated memory that have been reserved, > > and an exited application should free any memory it was using so that > > memory subsystem could free it back to the system, thereby not needing > > any cleaning of hugepages at startup. > > > > If your application does not e.g. free its mempools on exit, it should > > :) Chances are, the problem will go away. The only circumstance where > > this may not work is if you preallocated your memory using > > -m/--socket-mem flag. > > > > To clarify - all of the above is only applicable to 18.05 and beyond. > The map_all_hugepages() function only gets called in the legacy mem > init, so this patch solves a problem that does not exist on recent DPDK > versions in the first place - faster initialization is one of the key > reasons why the new memory subsystem was developed. Applications crash and need to be restarted. It is a mistake to assume mempool is in any valid state on startup.