On 2024-09-18 12:11, Jerin Jacob wrote:
On Thu, Sep 12, 2024 at 8:52 PM Jerin Jacob <jerinjac...@gmail.com> wrote:
On Thu, Sep 12, 2024 at 7:11 PM Morten Brørup <m...@smartsharesystems.com>
wrote:
From: Jerin Jacob [mailto:jerinjac...@gmail.com]
Sent: Thursday, 12 September 2024 15.17
On Thu, Sep 12, 2024 at 2:40 PM Morten Brørup <m...@smartsharesystems.com>
wrote:
+#define LCORE_BUFFER_SIZE (RTE_MAX_LCORE_VAR * RTE_MAX_LCORE)
Considering hugepages...
Lcore variables may be allocated before DPDK's memory allocator
(rte_malloc()) is ready, so rte_malloc() cannot be used for lcore variables.
And lcore variables are not usable (shared) for DPDK multi-process, so the
lcore_buffer could be allocated through the O/S APIs as anonymous hugepages,
instead of using rte_malloc().
The alternative, using rte_malloc(), would disallow allocating lcore
variables before DPDK's memory allocator has been initialized, which I think
is too late.
I thought it is not. A lot of the subsystems are initialized after the
memory subsystem is initialized.
[1] example given in documentation. I thought, RTE_INIT needs to
replaced if the subsystem called after memory initialized (which is
the case for most of the libraries)
The list of RTE_INIT functions are called before main(). It is not very useful.
Yes, it would be good to replace (or supplement) RTE_INIT_PRIO by something similar,
which calls the list of "INIT" functions at the appropriate time during EAL
initialization.
DPDK should then use this "INIT" list for all its initialization, so the init
function of new features (such as this, and trace) can be inserted at the correct
location in the list.
Trace library had a similar situation. It is managed like [2]
Yes, if we insist on using rte_malloc() for lcore variables, the alternative is
to prohibit establishing lcore variables in functions called through RTE_INIT.
I was not insisting on using ONLY rte_malloc(). Since rte_malloc() can
be called before rte_eal_init)(it will return NULL). Alloc routine can
check first rte_malloc() is available if not switch over glibc.
@Mattias Rönnblom This comment is not addressed in v7. Could you check?
Calling rte_malloc() and depending on it returning NULL if it's too
early in the initialization process sounds a little fragile, but maybe
it's fine.
One issue with lcore-variables-in-huge-pages I've failed to mentioned
this time around this is being discussed is that it would increase
memory usage by something like RTE_MAX_LCORE * 0.5 MB (or more probably
a little more).
In the huge pages case, you can't rely on demand paging to avoid
bringing in unused pages.
That said, I suspect some very latency-sensitive apps lock all pages in
memory, and thus lose out on this OS feature.
I suggest we just leave the first incarnation of lcore variables in
normal pages.
Thanks for the reminder.
