[resend with enclosed log instead of attachment]
Hello Sergio,
I'm running the following command
$ ./build/helloworld -c fff -n 1
And get the attached log (hope it goes through). Using "-n 2" (I'm not sure how
many channels) gives the same SIGSEGV error.
Here's the configuration:
$ numactl -H
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
node 0 size: 65431 MB
node 0 free: 62040 MB
node distances:
node 0
0: 10
$ cat /proc/meminfo
MemTotal: 65867360 kB
MemFree: 63529276 kB
Buffers: 93996 kB
Cached: 562160 kB
SwapCached: 0 kB
Active: 314816 kB
Inactive: 483752 kB
Active(anon): 144372 kB
Inactive(anon): 28 kB
Active(file): 170444 kB
Inactive(file): 483724 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 12 kB
Writeback: 0 kB
AnonPages: 144184 kB
Mapped: 49004 kB
Shmem: 280 kB
Slab: 77572 kB
SReclaimable: 31580 kB
SUnreclaim: 45992 kB
KernelStack: 2904 kB
PageTables: 7744 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 32421680 kB
Committed_AS: 383316 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 378992 kB
VmallocChunk: 34359352736 kB
HardwareCorrupted: 0 kB
AnonHugePages: 73728 kB
HugePa ges_Total: 500
HugePages_Free: 9
HugePages_Rsvd: 9
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 4096 kB
DirectMap2M: 2027520 kB
DirectMap1G: 65011712 kB
Log:
EAL: Detected lcore 0 as core 0 on socket 0
EAL: Detected lcore 1 as core 1 on socket 0
EAL: Detected lcore 2 as core 2 on socket 0
EAL: Detected lcore 3 as core 3 on socket 0
EAL: Detected lcore 4 as core 4 on socket 0
EAL: Detected lcore 5 as core 5 on socket 0
EAL: Detected lcore 6 as core 0 on socket 0
EAL: Detected lcore 7 as core 1 on socket 0
EAL: Detected lcore 8 as core 2 on socket 0
EAL: Detected lcore 9 as core 3 on socket 0
EAL: Detected lcore 10 as core 4 on socket 0
EAL: Detected lcore 11 as core 5 on socket 0
EAL: Support maximum 128 logical core(s) by configuration.
EAL: Detected 12 lcore(s)
EAL: Setting up memory...
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7fd26c800000 (size = 0x200000)
EAL: Ask a virtual area of 0x35800000 bytes
EAL: Virtual area found at 0x7fd236e00000 (size = 0x35800000)
EAL: Requesting 429 pages of size 2MB from socket 0
EAL: TSC frequency is ~2400001 KHz
EAL: Master lcore 0 is ready (tid=6cd40880;cpuset=[0])
PMD: ENICPMD trace: rte_enic_pmd_init
EAL: lcore 6 is ready (tid=331f7700;cpuset=[6])
EAL: lcore 5 is ready (tid=33bf8700;cpuset=[5])
EAL: lcore 9 is ready (tid=313f4700;cpuset=[9])
EAL: lcore 11 is ready (tid=2fff2700;cpuset=[11])
EAL: lcore 4 is ready (tid=345f9700;cpuset=[4])
EAL: lcore 8 is ready (tid=31df5700;cpuset=[8])
EAL: lcore 1 is ready (tid=363fc700;cpuset=[1])
EAL: lcore 10 is ready (tid=309f3700;cpuset=[10])
EAL: lcore 3 is ready (tid=34ffa700;cpuset=[3])
EAL: lcore 2 is ready (tid=359fb700;cpuset=[2])
EAL: lcore 7 is ready (tid=327f6700;cpuset=[7])
EAL: PCI device 0000:01:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:01:00.1 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL: probe driver: 8086:10fb rte_ixgbe_pmd
EAL: Not managed by a supported kernel driver, skipped
Allocating 0.1GB: PASS
Allocating 0.2GB: PASS
Allocating 0.3GB: PASS
Allocating 0.4GB: fail
Allocating 0.5GB: fail
Allocating 0.6GB: fail
Allocating 0.7GB: fail
Allocating 0.8GB: fail
Allocating 0.9GB: fail
Allocating 1.0GB: fail
Allocating 1.1GB: fail
Allocating 1.2GB: fail
Allocating 1.3GB: fail
Allocating 1.4GB: fail
Allocating 1.5GB: fail
Allocating 1.6GB: fail
Allocating 1.7GB: fail
Allocating 1.8GB: fail
Allocating 1.9GB: fail
Allocating 2.0GB: fail
Allocating 2.1GB: fail
Allocating 2.2GB:
Thanks,
Alain
-----Original Message-----
From: Sergio Gonzalez Monroy [mailto:[email protected]]
Sent: Monday, January 25, 2016 5:50 AM
To: Alain Gautherot <alain at edicogenome.com>; users at dpdk.org
Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory allocation
On 23/01/2016 00:20, Alain Gautherot wrote:
> Hello,
>
> I came across DPDK in a thread @
> http://stackoverflow.com/questions/4401912/linux-contiguous-physical-memory-from-userspace
> (bottom reply from mrsmith) and wanted to see if I can use rte_malloc() to
> allocate large blocks of contiguous physical memory (16GB or even 32GB at
> some point).
>
> The platform I'm working on has an FPGA that shares host memory with the
> x86_64 cores via a QPI link.
> The FPGA crunches data directly from host memory and uses physical addresses
> (mostly a QPI limitation, but it is also dictated by performance
> considerations and the ability to make the best possible use of multiple
> memory controllers).
> The data shared is 16GB or up to 32GB and could be provided as multiple
> descriptors to the FPGA, but that still means that each descriptor is in the
> order of several GBytes each.
> I understand that allocation may fail, but is ok for now, since I'm still in
> the proof-of-concept stage, trying to rule things out.
>
> My sample application attempts to allocate memory by chunks of 100MB like so:
>
> int main(int argc, char **argv)
> {
> int ret;
>
> ret = rte_eal_init(argc, argv);
> if (ret < 0) {
> rte_panic("Cannot init EAL\n");
> }
>
> int i;
> for (i = 1; i <= 100; ++i) {
> size_t allocsize = i * 100*1000*1000;
>
> printf(" Allocating %3.1fGB: ", ((float )i)/10.0f);
> fflush(stdout);
> void* ptr = rte_malloc(NULL, allocsize, 0U);
> if (ptr != NULL) {
> printf("PASS\n");
> rte_free(ptr);
> } else {
> printf("fail\n");
> }
> }
>
> printf("Done\n");
> return 0;
> }
>
> I get a consistent crash @ the 2.2GB mark:
> (gdb) r -c f -n 4
> ...
> EAL: PCI device 0000:06:00.1 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: Not managed by a supported kernel driver, skipped
> Allocating 0.1GB: fail
> Allocating 0.2GB: fail
> ...
> Allocating 2.0GB: fail
> Allocating 2.1GB: fail
> Allocating 2.2GB:
> Program received signal SIGSEGV, Segmentation fault.
> 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880,
> heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
> 61 elem->heap = heap;
> Missing separate debuginfos, use: debuginfo-install
> glibc-2.12-1.149.el6_6.5.x86_64
> (gdb) bt
> ...
> #0 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880,
> heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
> #1 0x00000000004c694e in split_elem (elem=0x7ffff3e00000,
> split_pt=0x800070eaa880) at
> /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:121
> #2 0x00000000004c6bda in malloc_elem_alloc (elem=0x7ffff3e00000,
> size=18446744071614584320, align=64)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:223
> #3 0x00000000004c736e in malloc_heap_alloc (heap=0x7ffff7fe561c, type=0x0,
> size=18446744071614584320, align=64)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_heap.c:167
> #4 0x00000000004c0aa1 in rte_malloc_socket (type=0x0,
> size=18446744071614584320, align=0, socket_arg=-1)
> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:89
> #5 0x00000000004c0b5b in rte_malloc (type=0x0, size=18446744071614584320,
> align=0) at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:115
> #6 0x000000000041ca6e in main (argc=5, argv=0x7fffffffdd48) at
> /home/alaing/INTEL/dpdk-2.0.0/examples/hugephymem/main.c:66
>
>
> Has anybody seen such an issue?
> Could I be misusing RTE somehow?
>
What options are you running your DPDK app with?
Can you also provide the full initialization log and hugepage info?
Sergio
> Thanks for your time,
> Alain
>
>
> --
> Alain Gautherot
> Edico Genome
>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: dpdk_log.txt
URL:
<http://dpdk.org/ml/archives/users/attachments/20160125/3fa0b05c/attachment.txt>
