Re: [vpp-dev] Message serialize / deserializer in VPP ( like google protobuf )
Satish, > Thanks Ole for the quick response. > > We are trying to send messages from a control plane process to VPP process > via memif channel. > This is to program the VPP dataplane tables for our product. > > I see some serialize functions in vppinfra/serialize.h file. Are these the > ones you were referring that can come handy for us ? > If so, I hope, we need to integrate this code onto our control plane process > as well. You could, but that wasn’t what I was referring to. > Please let me know if my understanding is correct. You currently have the following CP / DP interfaces in VPP: 1) The VPP Binary API I was suggesting you looked at the VPP binary API. For example the file src/vnet/interface.api contains the interface specification. Serialization here is essentially big endian C-structs. Then with a set of language bindings on the CP side. This is quite similar to gRPC. 1a) The VPP binary API with serialized memory structures in shared memory This is above, but combined with the above serialization code for e.g. larger memory structures. Downside is that it doesn’t work with other transports than shared memory. (E.g. unix domain sockets), and that it doesn’t have a specified data model. 2) Punt paths Punting packets from data plane over ‘normal’ interfaces or over Unix domain punt socket with meta data. 3) Counters and statistics over a separate shared memory interface 4) Host stack / VCL. E.g. for TCP control plane applications using VPP’s TCP implementation Are you suggesting to create a new one? Any reason why you couldn’t use 1.? Best regards, Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11493): https://lists.fd.io/g/vpp-dev/message/11493 Mute This Topic: https://lists.fd.io/mt/28583926/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] VPP API language changes
All,
Here is an update on API progress.
In the long process of making types in the API language more explicit I have a
few changes merged and a few in the pipeline:
- Type aliases. Previously any new user defined type had to be a structure.
With type aliasing one can do
e.g.
typedef u32 interface_index
instead of:
typedef {
u32 index;
} interface_index;
This is merged in 53fffa1d
I have a similar change for the relatively new ip4_address and ip6_address
types, so instead of:
typedef {
u32 address[4];
} ip4_address;
That is now: typedef u32 ip4_address[4];
That’s awaiting some fixes in the Java binding before it can be merged:
https://gerrit.fd.io/r/c/16217/
- New string type
{
u32 client_index;
u32 context;
- u32 length;
- u8 cmd[length];
+ string cmd;
};
The language did not distinguish between byte strings and text strings.
That made it hard for some language bindings and autogenerating code for
printing strings.
This patch is in
https://gerrit.fd.io/r/c/16326/
and awaiting changes in Java binding.
- Bool type
There is also a separate bool type that is intended to replace the use of “u8
is_ip6”, I have merged that for one component so far.
I will try to update the API definitions with new type as I get the chance.
bool and type aliases are backwards compatible (in that they require no change
on the VPP side).
Depending on language binding they might be hidden for the application.
I have also created a new mailing list: vpp-api-...@lists.fd.io.
It can be used for discussion if we don’t want to disturb the main list.
The main idea was for it to be used to receive notifications on API updates.
Both automatically triggered by gerrit (we’re setting this up now for any
change to a .api file) or for “manually” triggered alerts to API users.
Please subscribe.
Any other changes you’d like in the API language?
And please let me know if you have comments on the above.
(And yes it’s on my list to document the language, just procrastination monster
has so far prohibited that from happening).
Cheers,
Ole
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11496): https://lists.fd.io/g/vpp-dev/message/11496
Mute This Topic: https://lists.fd.io/mt/28610110/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] String tests failures
See https://gerrit.fd.io/r/#/c/16352, which computes strlen if necessary for a
precise copy-overlap check. Strncpy_s was always slightly wrong in this regard.
The "make test" failure showed up [only] on aarch64 due to test code stack
variable placement / alignment differences between aarch64 and x86_64.
HTH... Dave
From: Lijian Zhang (Arm Technology China)
Sent: Wednesday, December 5, 2018 4:39 AM
To: Dave Barach (dbarach)
Cc: Juraj Linkeš ; Dave Barach (dbarach)
; Damjan Marion
Subject: String tests failures
Hi Dave,
StringTest is failing on ARM machines.
I narrowed down the problem and commit an internal code review as below.
Once the internal code review process is passed, I'll upstream the patch for
community review.
#define clib_strncpy(d,s,n) strncpy_s_inline(d,CLIB_STRING_MACRO_MAX,s,n)
errno_t
strncpy_s (char *__restrict__ dest, rsize_t dmax,
const char *__restrict__ src, rsize_t n);
always_inline errno_t
strncpy_s_inline (char *__restrict__ dest, rsize_t dmax,
const char *__restrict__ src, rsize_t n)
{
u8 bad;
uword low, hi;
rsize_t m;
errno_t status = EOK;
bad = (dest == 0) + (dmax == 0) + (src == 0) + (n == 0);
if (PREDICT_FALSE (bad != 0))
{
/* Not actually trying to copy anything is OK */
if (n == 0)
return EOK;
if (dest == 0)
clib_c11_violation ("dest NULL");
if (src == 0)
clib_c11_violation ("src NULL");
if (dmax == 0)
clib_c11_violation ("dmax 0");
return EINVAL;
}
if (PREDICT_FALSE (n >= dmax))
{
/* Relax and use strnlen of src */
clib_c11_violation ("n >= dmax");
m = clib_strnlen (src, dmax);
if (m >= dmax)
{
/* Truncate, adjust copy length to fit dest */
m = dmax - 1;
status = EOVERFLOW;
}
}
else
-m = n;
+m = clib_strnlen (src, n);
/* Check for src/dst overlap, which is not allowed */
low = (uword) (src < dest ? src : dest);
hi = (uword) (src < dest ? dest : src);
if (PREDICT_FALSE (low + (m - 1) >= hi))
{
clib_c11_violation ("src/dest overlap");
return EINVAL;
}
clib_memcpy_fast (dest, src, m);
dest[m] = '\0';
return status;
}
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11497): https://lists.fd.io/g/vpp-dev/message/11497
Mute This Topic: https://lists.fd.io/mt/28611039/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Message serialize / deserializer in VPP ( like google protobuf )
Thanks Ole. We can use option (1). But, Only thing is, on the CP side, we have some restrictions to have a format either as google protobuf (or) JSON format. Do we have builtin Json serialize/deserialize support in VPP for JSON messages into C structures . If so, can you please point me to some code references. Really appreciate your inputs on this. Thanks & Regards, Satish -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11498): https://lists.fd.io/g/vpp-dev/message/11498 Mute This Topic: https://lists.fd.io/mt/28583926/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Message serialize / deserializer in VPP ( like google protobuf )
Satish, > We can use option (1). > But, Only thing is, on the CP side, we have some restrictions to have a > format either as google protobuf (or) JSON format. > Do we have builtin Json serialize/deserialize support in VPP for JSON > messages into C structures . > If so, can you please point me to some code references. Let’s see: - VAT does create JSON (manually) from API. But please don’t reuse that. - It would be relatively trivial to auto-generate a JSON representation from the Python binding. (but you might not want this in Python for performance reasons) - Shwetha wrote a poc API -> GRPC server a while back. It has been on my list for a while to write a VPP API to gRPC proxy. (help appreciated!) Cheers, Ole-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11499): https://lists.fd.io/g/vpp-dev/message/11499 Mute This Topic: https://lists.fd.io/mt/28583926/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Message serialize / deserializer in VPP ( like google protobuf )
Hi Staish, I assume that since you where looking a C functions for serialize/deserialize you might try the direct C function API in, e.g : ./build-root/install-vpp-native/vpp/include/vapi/interface.api.vapi.h which is generated directly from the .api files. /neale De : au nom de "satish.gu...@gmail.com" Date : mercredi 5 décembre 2018 à 15:56 À : "vpp-dev@lists.fd.io" Objet : Re: [vpp-dev] Message serialize / deserializer in VPP ( like google protobuf ) Thanks Ole. We can use option (1). But, Only thing is, on the CP side, we have some restrictions to have a format either as google protobuf (or) JSON format. Do we have builtin Json serialize/deserialize support in VPP for JSON messages into C structures . If so, can you please point me to some code references. Really appreciate your inputs on this. Thanks & Regards, Satish -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11500): https://lists.fd.io/g/vpp-dev/message/11500 Mute This Topic: https://lists.fd.io/mt/28583926/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] vnet crashes, and problems building debug version (was Re: netlink & router (vppsb or patch->vpp) - help building/running)
Greetings again, Here is more context on the problem I'm seeing. The problem occurs if a large-ish number of IPv4 prefixes are added to the FIB (by way of the netlink and router plugin). If the prefix count is below some threshold (e.g. 50,000 prefixes), things work fine. At some prefix count (haven't narrowed it down to a specific number, but I don't think the actual number is relevant), vnet crashes, in a failure within ip4_mtrie.c. I have been trying to run in debug mode, but am having a lot of difficulty building everything with debug. Basically, the only way I can successfully build everything is to use the script vagrant/build.sh (which does a make pkg-rpm that generates a bunch of rpm files that I then install with yum). Then, I have to rebuild things using the instructions from vppsb/router/README.md (doing 4 symlinks and various make iterations, and THEN having to run some of those with a bunch of CFLAGS values just to get it to compile). I don't see any good/easy way to build debug images from this environment, without a LOT of work/investigation on how all the various build components work. Is the problem easy enough to diagnose from a non-symbolic stack dump, or can someone provide details on how to build and run vpp with everything to use gdb, including the plugins for netlink/router, so the problem can be further isolated? I think there's basically some kind of bug related to the fib stuff in vnet, that really needs to be fixed. The box has an unreasonably large amount of memory (128GB, doing nothing but VPP), and I get the same error even if I up the initial heap size by a factor of 2^12 (changing 32<<20 to 32ULL<<32). Please help. Brian (In the following, the buffer space message is likely a consequence of the thread handling netlink messages dying, rather than a cause.) Here's the log messages: > Dec 4 17:08:14 sj2tldnslab09 vnet[19785]: dpdk_pool_create:535: > ioctl(VFIO_IOMMU_MAP_DMA) pool 'dpdk_mbuf_pool_socket0': Inappropriate > ioctl for device (errno 25) > > Dec 4 17:08:14 sj2tldnslab09 vnet[19785]: dpdk_ipsec_process:1026: not > enough DPDK crypto resources, default to OpenSSL > > Dec 4 17:08:16 sj2tldnslab09 vnet[19785]: rtnl_ns_recv:403: Received > notification while in sync. Restart synchronization. > > Dec 4 17:08:16 sj2tldnslab09 vnet[19785]: rtnl_process_read:467: > rtnetlink recv error (31) []: Bad file descriptor > > Dec 4 17:08:58 sj2tldnslab09 vnet[19785]: rtnl_process_read:467: > rtnetlink recv error (27) []: No buffer space available > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: rtnl_process_read:467: > rtnetlink recv error (27) []: No buffer space available > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: received signal SIGABRT, PC > 0x7f043c3c7277 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #0 0x7f043e5c18c5 > 0x7f043e5c18c5 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #1 0x7f043c9716d0 > 0x7f043c9716d0 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #2 0x7f043c3c7277 gsignal > + 0x37 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #3 0x7f043c3c8968 abort + > 0x148 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #4 0x5569eb7900d3 > 0x5569eb7900d3 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #5 0x7f043d0e8512 > vec_resize_allocate_memory + 0x2f2 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #6 0x7f043dd9809f > 0x7f043dd9809f > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #7 0x7f043dd985cd > ip4_fib_mtrie_route_add + 0x17d > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #8 0x7f043e129b08 > fib_entry_src_action_install + 0xb8 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #9 0x7f043e1274a0 > fib_entry_create + 0x70 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #10 0x7f043e11e890 > fib_table_entry_path_add2 + 0x190 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #11 0x7f03f86833fd > add_del_route + 0x34c > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #12 0x7f03f8683594 > netns_notify_cb + 0x8c > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #13 0x7f03f8466e71 > netns_notify + 0x1f3 > > Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #14 0x7f03f84684ed > ns_rcv_route + 0x825 > On Tue, Nov 27, 2018 at 6:17 PM Brian Dickson wrote: > I have been working with the netlink and router plugins, which I was able > to build from the 18.07 tree via the instructions in vppsb/router. > > (NB: trying to build from anything more recent, e.g. 18.10 or 19.01 > breaks, with no obvious easy resolution). > > When running with these plugins, connected with an open source router > (bird version 1.6.4 or 2.02) and with a very small routing table, it works > really really well. > > (I was able to run roughly line-rate 10g even with small packets, and when > using a second host with vpp and the span->pg->pcap to /tmp, didn't lose > any data.) > > However, when trying to load up the routing table, things went sideways, > and it seems to be something netlink-related.(This was using BGP to feed in > 3 copies of the
Re: [vpp-dev] vnet crashes, and problems building debug version (was Re: netlink & router (vppsb or patch->vpp) - help building/running)
Hi Brian, If you’re adding lots of routes, you’ll also need to bump the heap size for the IP FIBs as well as the main heap: https://fdio-vpp.readthedocs.io/en/latest/gettingstarted/users/configuring/startup.html#ip to run in gdb: sudo service vpp stop (or your OS equivalent) make build sudo gdb –args ./build-root/install-vpp_debug-native/vpp/bin/vpp –c plugin_path hope that helps, /neale De : au nom de Brian Dickson Date : mercredi 5 décembre 2018 à 19:31 À : "vpp-dev@lists.fd.io" Objet : [vpp-dev] vnet crashes, and problems building debug version (was Re: netlink & router (vppsb or patch->vpp) - help building/running) Greetings again, Here is more context on the problem I'm seeing. The problem occurs if a large-ish number of IPv4 prefixes are added to the FIB (by way of the netlink and router plugin). If the prefix count is below some threshold (e.g. 50,000 prefixes), things work fine. At some prefix count (haven't narrowed it down to a specific number, but I don't think the actual number is relevant), vnet crashes, in a failure within ip4_mtrie.c. I have been trying to run in debug mode, but am having a lot of difficulty building everything with debug. Basically, the only way I can successfully build everything is to use the script vagrant/build.sh (which does a make pkg-rpm that generates a bunch of rpm files that I then install with yum). Then, I have to rebuild things using the instructions from vppsb/router/README.md (doing 4 symlinks and various make iterations, and THEN having to run some of those with a bunch of CFLAGS values just to get it to compile). I don't see any good/easy way to build debug images from this environment, without a LOT of work/investigation on how all the various build components work. Is the problem easy enough to diagnose from a non-symbolic stack dump, or can someone provide details on how to build and run vpp with everything to use gdb, including the plugins for netlink/router, so the problem can be further isolated? I think there's basically some kind of bug related to the fib stuff in vnet, that really needs to be fixed. The box has an unreasonably large amount of memory (128GB, doing nothing but VPP), and I get the same error even if I up the initial heap size by a factor of 2^12 (changing 32<<20 to 32ULL<<32). Please help. Brian (In the following, the buffer space message is likely a consequence of the thread handling netlink messages dying, rather than a cause.) Here's the log messages: Dec 4 17:08:14 sj2tldnslab09 vnet[19785]: dpdk_pool_create:535: ioctl(VFIO_IOMMU_MAP_DMA) pool 'dpdk_mbuf_pool_socket0': Inappropriate ioctl for device (errno 25) Dec 4 17:08:14 sj2tldnslab09 vnet[19785]: dpdk_ipsec_process:1026: not enough DPDK crypto resources, default to OpenSSL Dec 4 17:08:16 sj2tldnslab09 vnet[19785]: rtnl_ns_recv:403: Received notification while in sync. Restart synchronization. Dec 4 17:08:16 sj2tldnslab09 vnet[19785]: rtnl_process_read:467: rtnetlink recv error (31) []: Bad file descriptor Dec 4 17:08:58 sj2tldnslab09 vnet[19785]: rtnl_process_read:467: rtnetlink recv error (27) []: No buffer space available Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: rtnl_process_read:467: rtnetlink recv error (27) []: No buffer space available Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: received signal SIGABRT, PC 0x7f043c3c7277 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #0 0x7f043e5c18c5 0x7f043e5c18c5 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #1 0x7f043c9716d0 0x7f043c9716d0 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #2 0x7f043c3c7277 gsignal + 0x37 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #3 0x7f043c3c8968 abort + 0x148 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #4 0x5569eb7900d3 0x5569eb7900d3 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #5 0x7f043d0e8512 vec_resize_allocate_memory + 0x2f2 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #6 0x7f043dd9809f 0x7f043dd9809f Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #7 0x7f043dd985cd ip4_fib_mtrie_route_add + 0x17d Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #8 0x7f043e129b08 fib_entry_src_action_install + 0xb8 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #9 0x7f043e1274a0 fib_entry_create + 0x70 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #10 0x7f043e11e890 fib_table_entry_path_add2 + 0x190 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #11 0x7f03f86833fd add_del_route + 0x34c Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #12 0x7f03f8683594 netns_notify_cb + 0x8c Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #13 0x7f03f8466e71 netns_notify + 0x1f3 Dec 4 17:09:07 sj2tldnslab09 vnet[19785]: #14 0x7f03f84684ed ns_rcv_route + 0x825 On Tue, Nov 27, 2018 at 6:17 PM Brian Dickson mailto:brian.peter.dick...@gmail.com>> wrote: I have been working with the netlink and router plugins, which I was able to build from the 18.07 tree via the instructions in vppsb/router. (NB: trying to
[vpp-dev] Heap allocation
Hi, I would like to pre-allocate separate heap space (outside of main heap) and be able to use the returned memory for my application's memory needs (just like malloc()). It seems we can have up to 256 heaps. Have few questions around this: 1) My understanding is that currently the main heap is allocated using clib_mem_init()/clib_mem_init_thread_safe() and the subsequent calls like clib_mem_alloc(size) will carve out memory from within the main heap and returned to clients (similar to glibc malloc()). Is this correct? If so, I would like to know how this is achieved since I don't see the returned pointer from clib_mem_init()/clib_mem_init_thread_safe() is being referenced or used anywhere. 2) Some applications are using mheap_alloc() directly to create new heap. Ex: void *heap; void *oldheap; heap = mheap_alloc (0, 10<<20); oldheap = clib_mem_set_heap (heap); .. .. clib_mem_set_heap (oldheap); << why is this required to write back oldheap into per-cpu heap? What I really like to know is once the heap is allocated how can we use it? Does it imply to have pools/hashes to be created to be able to use it? I'm expecting something which takes void *heap (created using mheap_alloc) and returns the requested memory for a given size from within the heap so that we don't need to use pools/hashes. Ex: void *foobar(void *mheap, size_t size); Is it possible? Thanks, Harish -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#11503): https://lists.fd.io/g/vpp-dev/message/11503 Mute This Topic: https://lists.fd.io/mt/28617718/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Heap allocation
You can have as many heaps as you need. There is no inherent limit. The
constant CLIB_MAX_MHEAPS has to do with the number of distinct thread-id’s
which os_get_thread_index() will return.
This guitar-lick means “temporarily use the supplied heap for all allocations”:
void *oldheap = clib_mem_set_heap (heap);
pool_get() / vec_validate() / clib_mem_alloc() etc. as desired
clib_mem_set_heap(oldheap); // back to the previous heap, usually the shared
global heap...
Required when allocating / free memory in the binary API segment, and in
private heaps.
Note that clib_mem_alloc() and variants mean “allocate memory from the current
per-thread heap,” which is easily changed for a moment as shown above.
Please read through src/vppinfra/{mem.h, dlmalloc.[ch]} carefully and
understand what’s going on before you jump in.
D.
From: vpp-dev@lists.fd.io On Behalf Of Harish Patil
Sent: Wednesday, December 5, 2018 4:31 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Heap allocation
Hi,
I would like to pre-allocate separate heap space (outside of main heap) and be
able to use the returned memory for my application's memory needs (just like
malloc()). It seems we can have up to 256 heaps. Have few questions around this:
1) My understanding is that currently the main heap is allocated using
clib_mem_init()/clib_mem_init_thread_safe() and the subsequent calls like
clib_mem_alloc(size) will carve out memory from within the main heap and
returned to clients (similar to glibc malloc()). Is this correct? If so, I
would like to know how this is achieved since I don't see the returned pointer
from clib_mem_init()/clib_mem_init_thread_safe() is being referenced or used
anywhere.
2) Some applications are using mheap_alloc() directly to create new heap.
Ex:
void *heap;
void *oldheap;
heap = mheap_alloc (0, 10<<20);
oldheap = clib_mem_set_heap (heap);
..
..
clib_mem_set_heap (oldheap); << why is this required to write back oldheap into
per-cpu heap?
What I really like to know is once the heap is allocated how can we use it?
Does it imply to have pools/hashes to be created to be able to use it? I'm
expecting something which takes void *heap (created using mheap_alloc) and
returns the requested memory for a given size from within the heap so that we
don't need to use pools/hashes.
Ex: void *foobar(void *mheap, size_t size);
Is it possible?
Thanks,
Harish
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11504): https://lists.fd.io/g/vpp-dev/message/11504
Mute This Topic: https://lists.fd.io/mt/28617718/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Heap allocation
Hi Dave,
Great, thanks for the quick reply.
Thanks for clarifying the same, I tried out a quick example and it seems
okay.
Harish
On Wed, Dec 5, 2018 at 1:52 PM Dave Barach (dbarach)
wrote:
> You can have as many heaps as you need. There is no inherent limit. The
> constant CLIB_MAX_MHEAPS has to do with the number of distinct thread-id’s
> which os_get_thread_index() will return.
>
>
>
> This guitar-lick means “temporarily use the supplied heap for all
> allocations”:
>
>
>
> void *oldheap = clib_mem_set_heap (heap);
>
> pool_get() / vec_validate() / clib_mem_alloc() etc. as desired
>
> clib_mem_set_heap(oldheap); // back to the previous heap, usually the
> shared global heap...
>
>
>
> Required when allocating / free memory in the binary API segment, and in
> private heaps.
>
>
>
> Note that clib_mem_alloc() and variants mean “allocate memory from the
> current per-thread heap,” which is easily changed for a moment as shown
> above.
>
>
>
> Please read through src/vppinfra/{mem.h, dlmalloc.[ch]} carefully and
> understand what’s going on before you jump in.
>
>
>
> D.
>
>
>
> *From:* vpp-dev@lists.fd.io *On Behalf Of *Harish
> Patil
> *Sent:* Wednesday, December 5, 2018 4:31 PM
> *To:* vpp-dev@lists.fd.io
> *Subject:* [vpp-dev] Heap allocation
>
>
>
> Hi,
>
>
>
> I would like to pre-allocate separate heap space (outside of main heap)
> and be able to use the returned memory for my application's memory needs
> (just like malloc()). It seems we can have up to 256 heaps. Have few
> questions around this:
>
>
>
> 1) My understanding is that currently the main heap is allocated using
> clib_mem_init()/clib_mem_init_thread_safe() and the subsequent calls like
> clib_mem_alloc(size) will carve out memory from within the main heap and
> returned to clients (similar to glibc malloc()). Is this correct? If so, I
> would like to know how this is achieved since I don't see the returned
> pointer from clib_mem_init()/clib_mem_init_thread_safe() is being
> referenced or used anywhere.
>
>
>
> 2) Some applications are using mheap_alloc() directly to create new heap.
>
> Ex:
>
> void *heap;
>
> void *oldheap;
>
> heap = mheap_alloc (0, 10<<20);
>
> oldheap = clib_mem_set_heap (heap);
>
> ..
>
> ..
>
> clib_mem_set_heap (oldheap); << why is this required to write back oldheap
> into per-cpu heap?
>
>
>
> What I really like to know is once the heap is allocated how can we use
> it? Does it imply to have pools/hashes to be created to be able to use it?
> I'm expecting something which takes void *heap (created using mheap_alloc)
> and returns the requested memory for a given size from within the heap so
> that we don't need to use pools/hashes.
>
> Ex: void *foobar(void *mheap, size_t size);
>
> Is it possible?
>
> Thanks,
>
>
>
> Harish
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#11505): https://lists.fd.io/g/vpp-dev/message/11505
Mute This Topic: https://lists.fd.io/mt/28617718/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
