Exactly, you need a configuration line per CPU node node=0 hugepagenumber=1024 node=1 hugepagenumber=1024
Alfredo > On 16 Nov 2016, at 20:30, Spransy, Derek <[email protected]> wrote: > > Hi Alfredo, > > Here's the output: > > Node 0 HugePages_Total: 1024 > Node 0 HugePages_Free: 1024 > Node 0 HugePages_Surp: 0 > Node 1 HugePages_Total: 0 > Node 1 HugePages_Free: 0 > Node 1 HugePages_Surp: 0 > > Our hugepages.conf consists of: > node=0 hugepagenumber=1024 > > Do you think this needs to be added for node 1 as well? > > Thanks, > Derek > From: [email protected] > <[email protected]> on behalf of Alfredo Cardigliano > <[email protected]> > Sent: Tuesday, November 15, 2016 9:01 AM > To: [email protected] > Subject: Re: [Ntop-misc] n2disk dumping with RHEL6 Kernel > 2.6.32-642.4.2.el6.x86_64 > > Hi Derek > this seems to be related to hugepages, could you provide more info about > memory in your system including: > > cat /sys/devices/system/node/node*/meminfo | grep Huge > > Alfredo > >> On 14 Nov 2016, at 15:13, Spransy, Derek <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi Alfredo, >> >> Yes the behavior is the same as before: >> >> $ sudo /usr/local/bin/n2disk10g /etc/n2disk/n2disk-eth5.conf >> 14/Nov/2016 09:04:21 [n2disk.c:4819] Welcome to n2disk10g v.2.6.161111 >> (r4672) [SandyBridge] >> 14/Nov/2016 09:04:21 [n2disk.c:4846] Running on 2 node(s) system with 24 >> core(s). NUMA affinity set to node 1. >> 14/Nov/2016 09:04:21 [n2disk.c:4875] Using PF_RING for packet capture >> 14/Nov/2016 09:04:21 [n2disk.c:4901] WARNING: If you are using standard >> drivers (packet capture via kernel) please disable time-pulse thread >> 14/Nov/2016 09:04:21 [n2disk.c:4904] Multithread support enabled >> 14/Nov/2016 09:04:21 [n2disk.c:5018] Dump files max size is set to 1024 MB >> 14/Nov/2016 09:04:21 [n2disk.c:5035] Buffer memory is set to 3 GB (x 3 pcap >> files) >> 14/Nov/2016 09:04:21 [n2disk.c:5070] Using directory /data1/captures for >> dump files >> 14/Nov/2016 09:04:21 [n2disk.c:5075] No sub-directories will be created >> 14/Nov/2016 09:04:21 [n2disk.c:5080] Up to 43000 files will be written >> before overwriting >> 14/Nov/2016 09:04:21 [n2disk.c:5090] Dump files max duration is set to 600 >> sec >> 14/Nov/2016 09:04:21 [n2disk.c:5106] Dumping data in 0.1 MB chunks >> 14/Nov/2016 09:04:21 [n2disk.c:5149] Index processing memory is set to 847 >> MB (x 3 index files) >> 14/Nov/2016 09:04:24 [n2disk.c:5339] Memory allocated successfully >> 14/Nov/2016 09:04:24 [n2disk.c:3597] Using time pulse timestamps >> 14/Nov/2016 09:04:24 [n2disk.c:3630] Started PF_RING packet reader thread >> for device zc:eth5 >> $ >> >> Running zbalance works fine: >> >> $ sudo /usr/local/bin/zbalance_ipc -i zc:eth5 -c 99 -n 1 -m 1 >> 14/Nov/2016 09:01:24 [zbalance_ipc.c:686] Starting balancer with 1 consumer >> queues.. >> 14/Nov/2016 09:01:24 [zbalance_ipc.c:696] Run your application instances as >> follows: >> 14/Nov/2016 09:01:24 [zbalance_ipc.c:701] pfcount -i zc:99@0 >> 14/Nov/2016 09:01:25 [zbalance_ipc.c:151] ========================= >> 14/Nov/2016 09:01:25 [zbalance_ipc.c:152] Absolute Stats: Recv 354'797 pkts >> (0 drops) - Forwarded 8'190 pkts (346'607 drops) >> 14/Nov/2016 09:01:25 [zbalance_ipc.c:234] ========================= >> 14/Nov/2016 09:01:26 [zbalance_ipc.c:151] ========================= >> 14/Nov/2016 09:01:26 [zbalance_ipc.c:152] Absolute Stats: Recv 714'213 pkts >> (0 drops) - Forwarded 8'190 pkts (706'023 drops) >> 14/Nov/2016 09:01:26 [zbalance_ipc.c:224] Actual Stats: Recv 359'348.08 pps >> (0.00 drops) - Forwarded 0.00 pps (359'348.08 drops) >> 14/Nov/2016 09:01:26 [zbalance_ipc.c:234] ========================= >> 14/Nov/2016 09:01:27 [zbalance_ipc.c:151] ========================= >> 14/Nov/2016 09:01:27 [zbalance_ipc.c:152] Absolute Stats: Recv 1'053'124 >> pkts (0 drops) - Forwarded 8'190 pkts (1'044'926 drops) >> 14/Nov/2016 09:01:27 [zbalance_ipc.c:224] Actual Stats: Recv 338'839.84 pps >> (0.00 drops) - Forwarded 0.00 pps (338'831.84 drops) >> 14/Nov/2016 09:01:27 [zbalance_ipc.c:234] ========================= >> 14/Nov/2016 09:01:28 [zbalance_ipc.c:151] ========================= >> 14/Nov/2016 09:01:28 [zbalance_ipc.c:152] Absolute Stats: Recv 1'410'053 >> pkts (0 drops) - Forwarded 8'190 pkts (1'401'863 drops) >> 14/Nov/2016 09:01:28 [zbalance_ipc.c:224] Actual Stats: Recv 356'860.48 pps >> (0.00 drops) - Forwarded 0.00 pps (356'868.48 drops) >> 14/Nov/2016 09:01:28 [zbalance_ipc.c:234] ========================= >> 14/Nov/2016 09:01:29 [zbalance_ipc.c:151] ========================= >> 14/Nov/2016 09:01:29 [zbalance_ipc.c:152] Absolute Stats: Recv 1'782'541 >> pkts (0 drops) - Forwarded 8'190 pkts (1'774'351 drops) >> 14/Nov/2016 09:01:29 [zbalance_ipc.c:224] Actual Stats: Recv 372'419.84 pps >> (0.00 drops) - Forwarded 0.00 pps (372'419.84 drops) >> 14/Nov/2016 09:01:29 [zbalance_ipc.c:234] ========================= >> >> Here's strace output of the crash, if that helps any: >> >> write(1, "14/Nov/2016 09:06:12 [n2disk.c:3"..., 9314/Nov/2016 09:06:12 >> [n2disk.c:3630] Started PF_RING packet reader thread for device zc:eth5 >> ) = 93 >> stat("/sys/devices/system/node/node0/cpumap", {st_mode=S_IFREG|0444, >> st_size=4096, ...}) = 0 >> stat("/sys/devices/system/node/node1/cpumap", {st_mode=S_IFREG|0444, >> st_size=4096, ...}) = 0 >> stat("/sys/devices/system/node/node2/cpumap", 0x7ffc10d5fa60) = -1 ENOENT >> (No such file or directory) >> stat("/sys/devices/system/node/node0/cpumap", {st_mode=S_IFREG|0444, >> st_size=4096, ...}) = 0 >> stat("/sys/devices/system/node/node1/cpumap", {st_mode=S_IFREG|0444, >> st_size=4096, ...}) = 0 >> stat("/sys/devices/system/node/node2/cpumap", 0x7ffc10d5fa40) = -1 ENOENT >> (No such file or directory) >> set_mempolicy(MPOL_BIND, 0x2d3b3b0, 4) = 0 >> socket(0x1b /* PF_??? */, SOCK_RAW, 768) = 5 >> setsockopt(5, SOL_IP, 0x87 /* IP_??? */, "\0\0\0\0\0\0\0\0", 8) = 0 >> setsockopt(5, SOL_IP, 0x6e /* IP_??? */, "pfring-zc-cluster-0", 19) = 0 >> open("/proc/meminfo", O_RDONLY) = 6 >> fstat(6, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7faed3cf3000 >> read(6, "MemTotal: 16291360 kB\nMemF"..., 1024) = 1024 >> read(6, "e: 1024\nHugePages_Rsvd: "..., 1024) = 174 >> close(6) = 0 >> munmap(0x7faed3cf3000, 4096) = 0 >> open("/proc/mounts", O_RDONLY) = 6 >> fstat(6, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = >> 0x7faed3cf3000 >> read(6, "rootfs / rootfs rw 0 0\nproc /pro"..., 1024) = 1024 >> read(6, "ugepages hugetlbfs rw,relatime 0"..., 1024) = 35 >> open("/mnt/hugepages/pfring_zc_0", O_RDWR|O_CREAT, 0755) = 7 >> flock(7, LOCK_SH) = 0 >> mmap(NULL, 134217728, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_LOCKED, 7, 0) = >> 0x7fb033200000 >> --- SIGBUS {si_signo=SIGBUS, si_code=BUS_ADRERR, si_addr=0x7fb033200018} --- >> +++ killed by SIGBUS (core dumped) +++ >> >> Thanks, >> Derek > > > > This e-mail message (including any attachments) is for the sole use of > the intended recipient(s) and may contain confidential and privileged > information. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution > or copying of this message (including any attachments) is strictly > prohibited. > > If you have received this message in error, please contact > the sender by reply e-mail message and destroy all copies of the > original message (including attachments). > _______________________________________________ > Ntop-misc mailing list > [email protected] <mailto:[email protected]> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <http://listgateway.unipi.it/mailman/listinfo/ntop-misc>
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
