Hi Alain,
On 25/01/2016 21:02, Alain Gautherot wrote:
> [resend with enclosed log instead of attachment]
>
> Hello Sergio,
>
> I'm running the following command
>
> $ ./build/helloworld -c fff -n 1
>
> And get the attached log (hope it goes through). Using "-n 2" (I'm not sure
> how many channels) gives the same SIGSEGV error.
>
> Here's the configuration:
>
> $ numactl -H
> available: 1 nodes (0)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11
> node 0 size: 65431 MB
> node 0 free: 62040 MB
> node distances:
> node 0
> 0: 10
>
> $ cat /proc/meminfo
> MemTotal: 65867360 kB
> MemFree: 63529276 kB
> Buffers: 93996 kB
> Cached: 562160 kB
> SwapCached: 0 kB
> Active: 314816 kB
> Inactive: 483752 kB
> Active(anon): 144372 kB
> Inactive(anon): 28 kB
> Active(file): 170444 kB
> Inactive(file): 483724 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> SwapTotal: 0 kB
> SwapFree: 0 kB
> Dirty: 12 kB
> Writeback: 0 kB
> AnonPages: 144184 kB
> Mapped: 49004 kB
> Shmem: 280 kB
> Slab: 77572 kB
> SReclaimable: 31580 kB
> SUnreclaim: 45992 kB
> KernelStack: 2904 kB
> PageTables: 7744 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 32421680 kB
> Committed_AS: 383316 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed: 378992 kB
> VmallocChunk: 34359352736 kB
> HardwareCorrupted: 0 kB
> AnonHugePages: 73728 kB
> HugePa ges_Total: 500
> HugePages_Free: 9
> HugePages_Rsvd: 9
> HugePages_Surp: 0
> Hugepagesize: 2048 kB
> DirectMap4k: 4096 kB
> DirectMap2M: 2027520 kB
> DirectMap1G: 65011712 kB
>
>
> Log:
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 5 on socket 0
> EAL: Detected lcore 6 as core 0 on socket 0
> EAL: Detected lcore 7 as core 1 on socket 0
> EAL: Detected lcore 8 as core 2 on socket 0
> EAL: Detected lcore 9 as core 3 on socket 0
> EAL: Detected lcore 10 as core 4 on socket 0
> EAL: Detected lcore 11 as core 5 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 12 lcore(s)
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x200000 bytes
> EAL: Virtual area found at 0x7fd26c800000 (size = 0x200000)
> EAL: Ask a virtual area of 0x35800000 bytes
> EAL: Virtual area found at 0x7fd236e00000 (size = 0x35800000)
> EAL: Requesting 429 pages of size 2MB from socket 0
> EAL: TSC frequency is ~2400001 KHz
> EAL: Master lcore 0 is ready (tid=6cd40880;cpuset=[0])
> PMD: ENICPMD trace: rte_enic_pmd_init
> EAL: lcore 6 is ready (tid=331f7700;cpuset=[6])
> EAL: lcore 5 is ready (tid=33bf8700;cpuset=[5])
> EAL: lcore 9 is ready (tid=313f4700;cpuset=[9])
> EAL: lcore 11 is ready (tid=2fff2700;cpuset=[11])
> EAL: lcore 4 is ready (tid=345f9700;cpuset=[4])
> EAL: lcore 8 is ready (tid=31df5700;cpuset=[8])
> EAL: lcore 1 is ready (tid=363fc700;cpuset=[1])
> EAL: lcore 10 is ready (tid=309f3700;cpuset=[10])
> EAL: lcore 3 is ready (tid=34ffa700;cpuset=[3])
> EAL: lcore 2 is ready (tid=359fb700;cpuset=[2])
> EAL: lcore 7 is ready (tid=327f6700;cpuset=[7])
> EAL: PCI device 0000:01:00.0 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:01:00.1 on NUMA socket 0
> EAL: probe driver: 8086:1521 rte_igb_pmd
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:03:00.0 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: Not managed by a supported kernel driver, skipped
> EAL: PCI device 0000:03:00.1 on NUMA socket 0
> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
> EAL: Not managed by a supported kernel driver, skipped
> Allocating 0.1GB: PASS
> Allocating 0.2GB: PASS
> Allocating 0.3GB: PASS
> Allocating 0.4GB: fail
> Allocating 0.5GB: fail
> Allocating 0.6GB: fail
> Allocating 0.7GB: fail
> Allocating 0.8GB: fail
> Allocating 0.9GB: fail
> Allocating 1.0GB: fail
> Allocating 1.1GB: fail
> Allocating 1.2GB: fail
> Allocating 1.3GB: fail
> Allocating 1.4GB: fail
> Allocating 1.5GB: fail
> Allocating 1.6GB: fail
> Allocating 1.7GB: fail
> Allocating 1.8GB: fail
> Allocating 1.9GB: fail
> Allocating 2.0GB: fail
> Allocating 2.1GB: fail
> Allocating 2.2GB:
>
> Thanks,
> Alain
>
> -----Original Message-----
> From: Sergio Gonzalez Monroy [mailto:sergio.gonzalez.monroy at intel.com]
> Sent: Monday, January 25, 2016 5:50 AM
> To: Alain Gautherot <alain at edicogenome.com>; users at dpdk.org
> Subject: Re: [dpdk-users] Using DPDK for contiguous physical memory allocation
>
> On 23/01/2016 00:20, Alain Gautherot wrote:
>> Hello,
>>
>> I came across DPDK in a thread @
>> http://stackoverflow.com/questions/4401912/linux-contiguous-physical-memory-from-userspace
>> (bottom reply from mrsmith) and wanted to see if I can use rte_malloc() to
>> allocate large blocks of contiguous physical memory (16GB or even 32GB at
>> some point).
>>
>> The platform I'm working on has an FPGA that shares host memory with the
>> x86_64 cores via a QPI link.
>> The FPGA crunches data directly from host memory and uses physical addresses
>> (mostly a QPI limitation, but it is also dictated by performance
>> considerations and the ability to make the best possible use of multiple
>> memory controllers).
>> The data shared is 16GB or up to 32GB and could be provided as multiple
>> descriptors to the FPGA, but that still means that each descriptor is in the
>> order of several GBytes each.
>> I understand that allocation may fail, but is ok for now, since I'm still in
>> the proof-of-concept stage, trying to rule things out.
>>
>> My sample application attempts to allocate memory by chunks of 100MB like so:
>>
>> int main(int argc, char **argv)
>> {
>> int ret;
>>
>> ret = rte_eal_init(argc, argv);
>> if (ret < 0) {
>> rte_panic("Cannot init EAL\n");
>> }
>>
>> int i;
I get warning with this code. It warns of undefined behavior because of
signed integer overflow.
Could you change the above 'int i' to 'size_t i' and run it again?
Sergio
>> for (i = 1; i <= 100; ++i) {
>> size_t allocsize = i * 100*1000*1000;
>>
>> printf(" Allocating %3.1fGB: ", ((float )i)/10.0f);
>> fflush(stdout);
>> void* ptr = rte_malloc(NULL, allocsize, 0U);
>> if (ptr != NULL) {
>> printf("PASS\n");
>> rte_free(ptr);
>> } else {
>> printf("fail\n");
>> }
>> }
>>
>> printf("Done\n");
>> return 0;
>> }
>>
>> I get a consistent crash @ the 2.2GB mark:
>> (gdb) r -c f -n 4
>> ...
>> EAL: PCI device 0000:06:00.1 on NUMA socket 0
>> EAL: probe driver: 8086:1521 rte_igb_pmd
>> EAL: Not managed by a supported kernel driver, skipped
>> Allocating 0.1GB: fail
>> Allocating 0.2GB: fail
>> ...
>> Allocating 2.0GB: fail
>> Allocating 2.1GB: fail
>> Allocating 2.2GB:
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880,
>> heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064)
>> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
>> 61 elem->heap = heap;
>> Missing separate debuginfos, use: debuginfo-install
>> glibc-2.12-1.149.el6_6.5.x86_64
>> (gdb) bt
>> ...
>> #0 0x00000000004c6770 in malloc_elem_init (elem=0x800070eaa880,
>> heap=0x7ffff7fe561c, mz=0x7ffff7fb2c1c, size=2200000064)
>> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:61
>> #1 0x00000000004c694e in split_elem (elem=0x7ffff3e00000,
>> split_pt=0x800070eaa880) at
>> /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:121
>> #2 0x00000000004c6bda in malloc_elem_alloc (elem=0x7ffff3e00000,
>> size=18446744071614584320, align=64)
>> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_elem.c:223
>> #3 0x00000000004c736e in malloc_heap_alloc (heap=0x7ffff7fe561c, type=0x0,
>> size=18446744071614584320, align=64)
>> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/malloc_heap.c:167
>> #4 0x00000000004c0aa1 in rte_malloc_socket (type=0x0,
>> size=18446744071614584320, align=0, socket_arg=-1)
>> at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:89
>> #5 0x00000000004c0b5b in rte_malloc (type=0x0, size=18446744071614584320,
>> align=0) at /home/alaing/INTEL/dpdk-2.0.0/lib/librte_malloc/rte_malloc.c:115
>> #6 0x000000000041ca6e in main (argc=5, argv=0x7fffffffdd48) at
>> /home/alaing/INTEL/dpdk-2.0.0/examples/hugephymem/main.c:66
>>
>>
>> Has anybody seen such an issue?
>> Could I be misusing RTE somehow?
>>
> What options are you running your DPDK app with?
>
> Can you also provide the full initialization log and hugepage info?
>
> Sergio
>> Thanks for your time,
>> Alain
>>
>>
>> --
>> Alain Gautherot
>> Edico Genome
>>
> -------------- next part --------------
> An embedded and charset-unspecified text was scrubbed...
> Name: dpdk_log.txt
> URL:
> <http://dpdk.org/ml/archives/users/attachments/20160125/3fa0b05c/attachment.txt>
