Hello Sergio,
I'm running the following command
$ ./build/helloworld -c fff -n 1
And get the attached log (hope it goes through). Using "-n 2" (I'm not sure how
many channels) gives the same SIGSEGV error.
Here's the configuration:
$ numactl -H
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
node 0 size: 65431 MB
node 0 free: 62040 MB
node distances:
node 0
0: 10
$ cat /proc/meminfo
MemTotal: 65867360 kB
MemFree: 63529276 kB
Buffers: 93996 kB
Cached: 562160 kB
SwapCached: 0 kB
Active: 314816 kB
Inactive: 483752 kB
Active(anon): 144372 kB
Inactive(anon): 28 kB
Active(file): 170444 kB
Inactive(file): 483724 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 12 kB
Writeback: 0 kB
AnonPages: 144184 kB
Mapped: 49004 kB
Shmem: 280 kB
Slab: 77572 kB
SReclaimable: 31580 kB
SUnreclaim: 45992 kB
KernelStack: 2904 kB
PageTables: 7744 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 32421680 kB
Committed_AS: 383316 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 378992 kB
VmallocChunk: 34359352736 kB
HardwareCorrupted: 0 kB
AnonHugePages: 73728 kB
HugePa ges_Total: 500
HugePages_Free: 9
HugePages_Rsvd: 9
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 4096 kB
DirectMap2M: 2027520 kB
DirectMap1G: 65011712 kB
Thanks,
Alain
-----Original Message-----
From: Sergio Gonzalez Monroy [mailto:[email protected]]
Sent: Monday, January 25, 2016 5:50 AM
To: Alain Gautherot <alain at edicogenome.com>; users at dpdk.org
Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory allocation
On 23/01/2016 00:20, Alain Gautherot wrote:
> Hello,
>
> I came across DPDK in a thread @
> http://stackoverflow.com/questions/4401912/linux-contiguous-physical-memory-from-userspace
> (bottom reply from mrsmith) and wanted to see if I can use rte_malloc() to
> allocate large blocks of contiguous physical memory (16GB or even 32GB at
> some point).
>
> The platform I'm working on has an FPGA that shares host memory with the
> x86_64 cores via a QPI link.
> The FPGA crunches data directly from host memory and uses physical addresses
> (mostly a QPI limitation, but it is also dictated by performance
> considerations and the ability to make the best possible use of multiple
> memory controllers).
> The data shared is 16GB or up to 32GB and could be provided as multiple
> descriptors to the FPGA, but that still means that each descriptor is in the
> order of several GBytes each.
> I understand that allocation may fail, but is ok for now, since I'm still in
> the proof-of-concept stage, trying to rule things out.
>
> My sample application attempts to allocate memory by chunks of 100MB like so:
>
> int main(int argc, char **argv)
> {
> int ret;
>
> ret = rte_eal_init(argc, argv);
> if (ret < 0) {
> rte_panic("Cannot init EAL\n");
> }
>
> int i;
> for (i = 1; i <= 100; ++i) {
> size_t allocsize = i * 100*1000*1000;
>
> printf(" Allocating %3.1fGB: ", ((float )i)/10.0f);
> fflush(stdout);
> void* ptr = rte_malloc(NULL, allocsize, 0U);
> if (ptr != NULL) {
> printf("PASS\n");
> rte_free(ptr);
> } else {
> printf("fail\n");
> }
> }
>
> printf("Done\n");
> return 0;
> }
>
> I get a consistent crash @ the 2.2GB mark:
> (gdb) r -c f -n 4
> ...
> EAL: PCI device 0000:06:00.1 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: Not managed by a supported kernel driver, skipped
> Allocating 0.1GB: fail
> Allocating 0.2GB: fail
> ...
> Allocating 2.0GB: fail
> Allocating 2.1GB: fail
> Allocating 2.2GB:
> Program received signal SIGSEGV, Segmentation fault.
> 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880,
> heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
> 61 elem->heap = heap;
> Missing separate debuginfos, use: debuginfo-install
> glibc-2.12-1.149.el6_6.5.x86_64
> (gdb) bt
> ...
> #0 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880,
> heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
> #1 0x00000000004c694e in split_elem (elem=0x7ffff3e00000,
> split_pt=0x800070eaa880) at
> /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:121
> #2 0x00000000004c6bda in malloc_elem_alloc (elem=0x7ffff3e00000,
> size=18446744071614584320, align=64)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:223
> #3 0x00000000004c736e in malloc_heap_alloc (heap=0x7ffff7fe561c, type=0x0,
> size=18446744071614584320, align=64)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_heap.c:167
> #4 0x00000000004c0aa1 in rte_malloc_socket (type=0x0,
> size=18446744071614584320, align=0, socket_arg=-1)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:89
> #5 0x00000000004c0b5b in rte_malloc (type=0x0, size=18446744071614584320,
> align=0) at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:115
> #6 0x000000000041ca6e in main (argc=5, argv=0x7fffffffdd48) at
> /home/alaing/INTEL/dpdk-2.0.0/examples/hugephymem/main.c:66
>
>
> Has anybody seen such an issue?
> Could I be misusing RTE somehow?
>
What options are you running your DPDK app with?
Can you also provide the full initialization log and hugepage info?
Sergio
> Thanks for your time,
> Alain
>
>
> --
> Alain Gautherot
> Edico Genome
>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dpdk_log.txt
URL:
<http://dpdk.org/ml/archives/users/attachments/20160125/3fa0b05c/attachment.txt>
