smp_affinity mask was different for input and output ethernet cards, i.e. irq of eth1 was on cpu0, irq of eth2 was on cpu1.
If both irqs were distributed over CPUs (spm_affinity=0xffffff) then the forwarder was working much more stable.
12.12.08, 17:46, "Vladimir"
> Hi
> During the testing of reflector example, I have got following kernel messages like "bash invoked oom-killer" (see below).
> Could you explain this (possibly) leak ? Fixes, ideas ?
> I have searched maillists and seems the leak has been fixed in previous releases.
> Could you help me, Luca ?
> This happened after some intensive tests using pktgen from another machine (about 500 Mbit/s, 64 byte pkts)
> Bash script periodically called "ethtool -S eth1" to get statistics from ethernet card.
> After this message ssh session was aborted. I tried to log in again but the system was hanging after login and before password.
> Neither pings nor logins from console were accepted by it.
> PF_RING module was loaded with transparent mode=0
> BTW, the system hangs on pfcount with PF_RING settings transparent mode=1, TX capturing=1
> --
> My system :
> Fedora 8, Athlon64 Duo, 2 GB RAM, Intel 1000/PRO PT, mainboard ASUS M2N-SLI, sata, LVM
> kernel :
> [...@recv ~]$ uname -a
> Linux recv 2.6.26.6-pfring #1 SMP Thu Nov 27 18:14:24 MSK 2008 i686 athlon i386 GNU/Linux
> driver:
> [...@recv ~]$ ethtool -i eth1
> driver: e1000e
> version: 0.3.3.3-k4
> firmware-version: 5.11-8
> bus-info: 0000:02:00.0
> BR, Vladimir
> -----------------------------------------
> Dec 11 15:17:43 recv kernel: [PF_RING] Welcome to PF_RING 3.9.0
> Dec 11 15:17:43 recv kernel: (C) 2004-08 L.Deri
> Dec 11 15:17:43 recv kernel: NET: Registered protocol family 27
> Dec 11 15:17:43 recv kernel: [PF_RING] Ring slots 4096
> Dec 11 15:17:43 recv kernel: [PF_RING] Slot version 9
> Dec 11 15:17:43 recv kernel: [PF_RING] Capture TX Yes [RX+TX]
> Dec 11 15:17:43 recv kernel: [PF_RING] IP Defragment No
> Dec 11 15:17:43 recv kernel: [PF_RING] Initialized correctly
> Dec 11 15:17:43 recv kernel: [PF_RING] registered /proc/net/pf_ring/
> Dec 11 15:19:01 recv kernel: [PF_RING] successfully allocated 6434816 bytes at 0xf8b9c000
> Dec 11 15:19:01 recv kernel: [PF_RING] allocated 4098 slots [slot_len=1570][tot_mem=643481
> 6]
> Dec 11 15:19:01 recv kernel: device eth1 entered promiscuous mode
> Dec 11 15:19:01 recv kernel: [PF_RING] successfully allocated 6434816 bytes at 0xf91c0000
> Dec 11 15:19:01 recv kernel: [PF_RING] allocated 4098 slots [slot_len=1570][tot_mem=643481
> 6]
> Dec 11 15:19:01 recv kernel: device eth2 entered promiscuous mode
> Dec 11 15:27:44 recv kernel: [PF_RING] removed /proc/net/pf_ring/2074-eth1.0
> Dec 11 15:27:44 recv kernel: [PF_RING] removed /proc/net/pf_ring/2074-eth2.1
> Dec 11 15:28:03 recv kernel: [PF_RING] successfully allocated 6434816 bytes at 0xf8b9c000
> Dec 11 15:28:03 recv kernel: [PF_RING] allocated 4098 slots [slot_len=1570][tot_mem=643481
> 6]
> Dec 11 15:28:04 recv kernel: [PF_RING] successfully allocated 6434816 bytes at 0xf91c0000
> Dec 11 15:28:04 recv kernel: [PF_RING] allocated 4098 slots [slot_len=1570][tot_mem=643481
> 6]
> Dec 11 15:28:36 recv kernel: bash invoked oom-killer: gfp_mask=0x40d0, order=0, oomkilladj=0
> Dec 11 15:28:36 recv kernel: Pid: 1968, comm: bash Not tainted 2.6.26.6-pfring #1
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: =======================
> Dec 11 15:28:36 recv kernel: Mem-info:
> Dec 11 15:28:36 recv kernel: DMA per-cpu:
> Dec 11 15:28:36 recv kernel: CPU 0: hi: 0, btch: 1 usd: 0
> Dec 11 15:28:36 recv kernel: CPU 1: hi: 0, btch: 1 usd: 0
> Dec 11 15:28:36 recv kernel: Normal per-cpu:
> Dec 11 15:28:36 recv kernel: CPU 0: hi: 186, btch: 31 usd: 0
> Dec 11 15:28:36 recv kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Dec 11 15:28:36 recv kernel: HighMem per-cpu:
> Dec 11 15:28:36 recv kernel: CPU 0: hi: 186, btch: 31 usd: 0
> Dec 11 15:28:36 recv kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Dec 11 15:28:36 recv kernel: Active:2666 inactive:1791 dirty:0 writeback:0 unstable:0
> Dec 11 15:28:36 recv kernel: free:290976 slab:218464 mapped:832 pagetables:130 bounce:0
> Dec 11 15:28:36 recv kernel: DMA free:3508kB min:68kB low:84kB high:100kB active:0kB inactive:0kB present:16256kB pages_scanned:0 all_unreclaimable? yes
> Dec 11 15:28:36 recv kernel: lowmem_reserve[]: 0 873 2015 2015
> Dec 11 15:28:36 recv kernel: Normal free:2636kB min:3744kB low:4680kB high:5616kB active:100kB inactive:0kB present:894080kB pages_scanned:100 all_unreclaimable? no
> Dec 11 15:28:36 recv kernel: lowmem_reserve[]: 0 0 9135 9135
> Dec 11 15:28:36 recv kernel: HighMem free:1157760kB min:512kB low:1736kB high:2960kB active:10564kB inactive:7164kB present:1169288kB pages_scanned:0 all_unreclaimable? no
> Dec 11 15:28:36 recv kernel: lowmem_reserve[]: 0 0 0 0
> Dec 11 15:28:36 recv kernel: DMA: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3508kB
> Dec 11 15:28:36 recv kernel: Normal: 374*4kB 35*8kB 9*16kB 3*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 2720kB
> Dec 11 15:28:36 recv kernel: HighMem: 624*4kB 323*8kB 248*16kB 184*32kB 90*64kB 22*128kB 11*256kB 6*512kB 0*1024kB 3*2048kB 274*4096kB = 1157848kB
> Dec 11 15:28:36 recv kernel: 3285 total pagecache pages
> Dec 11 15:28:36 recv kernel: Swap cache: add 0, delete 0, find 0/0
> Dec 11 15:28:36 recv kernel: Free swap = 2031608kB
> Dec 11 15:28:36 recv kernel: Total swap = 2031608kB
> Dec 11 15:28:36 recv kernel: 524000 pages of RAM
> Dec 11 15:28:36 recv kernel: 294624 pages of HIGHMEM
> Dec 11 15:28:36 recv kernel: 8654 reserved pages
> Dec 11 15:28:36 recv kernel: 3951 pages shared
> Dec 11 15:28:36 recv kernel: 0 pages swap cached
> Dec 11 15:28:36 recv kernel: 0 pages dirty
> Dec 11 15:28:36 recv kernel: 0 pages writeback
> Dec 11 15:28:36 recv kernel: 832 pages mapped
> Dec 11 15:28:36 recv kernel: 218464 pages slab
> Dec 11 15:28:36 recv kernel: 130 pages pagetables
> Dec 11 15:28:36 recv kernel: Out of memory: kill process 2044 (bash) score 3011 or a child
> Dec 11 15:28:36 recv kernel: Killed process 2192 (forwarder)
> Dec 11 15:28:36 recv kernel: [PF_RING] removed /proc/net/pf_ring/2192-eth1.2
> Dec 11 15:28:36 recv kernel: bash invoked oom-killer: gfp_mask=0x40d0, order=0, oomkilladj=0
> Dec 11 15:28:36 recv kernel: Pid: 1968, comm: bash Not tainted 2.6.26.6-pfring #1
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: [
> Dec 11 15:28:36 recv kernel: =======================
> Dec 11 15:28:36 recv kernel: Mem-info:
> Dec 11 15:28:36 recv kernel: DMA per-cpu:
> Dec 11 15:28:36 recv kernel: CPU 0: hi: 0, btch: 1 usd: 0
> Dec 11 15:28:36 recv kernel: CPU 1: hi: 0, btch: 1 usd: 0
> Dec 11 15:28:36 recv kernel: Normal per-cpu:
> Dec 11 15:28:36 recv kernel: CPU 0: hi: 186, btch: 31 usd: 4
> Dec 11 15:28:36 recv kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Dec 11 15:28:36 recv kernel: HighMem per-cpu:
> Dec 11 15:28:36 recv kernel: CPU 0: hi: 186, btch: 31 usd: 53
> Dec 11 15:28:36 recv kernel: CPU 1: hi: 186, btch: 31 usd: 0
> Dec 11 15:28:36 recv kernel: Active:2666 inactive:1791 dirty:0 writeback:0 unstable:0
> Dec 11 15:28:36 recv kernel: free:290955 slab:218464 mapped:832 pagetables:130 bounce:0
> Dec 11 15:28:36 recv kernel: DMA free:3508kB min:68kB low:84kB high:100kB active:0kB inactive:0kB present:16256kB pages_scanned:0 all_unreclaimable? yes
> Dec 11 15:28:36 recv kernel: lowmem_reserve[]: 0 873 2015 2015
> Dec 11 15:28:36 recv kernel: Normal free:2636kB min:3744kB low:4680kB high:5616kB active:100kB inactive:0kB present:894080kB pages_scanned:150 all_unreclaimable? yes
> Dec 11 15:28:36 recv kernel: lowmem_reserve[]: 0 0 9135 9135
> Dec 11 15:28:36 recv kernel: HighMem free:1157676kB min:512kB low:1736kB high:2960kB active:10564kB inactive:7164kB present:1169288kB pages_scanned:0 all_unreclaimable? no
> Dec 11 15:28:36 recv kernel: lowmem_reserve[]: 0 0 0 0
> Dec 11 15:28:36 recv kernel: DMA: 1*4kB 0*8kB 1*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3508kB
> Dec 11 15:28:36 recv kernel: Normal: 374*4kB 35*8kB 9*16kB 3*32kB 1*64kB 1*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 2720kB
> Dec 11 15:28:36 recv kernel: HighMem: 593*4kB 323*8kB 248*16kB 184*32kB 90*64kB 22*128kB 11*256kB 6*512kB 0*1024kB 3*2048kB 274*4096kB = 1157724kB
> Dec 11 15:28:36 recv kernel: 3285 total pagecache pages
> Dec 11 15:28:36 recv kernel: Swap cache: add 0, delete 0, find 0/0
> Dec 11 15:28:36 recv kernel: Free swap = 2031608kB
> Dec 11 15:28:36 recv kernel: Total swap = 2031608kB
> Dec 11 15:28:36 recv kernel: [PF_RING] removed /proc/net/pf_ring/2192-eth2.3
> Dec 11 15:28:36 recv kernel: 524000 pages of RAM
> Dec 11 15:28:36 recv kernel: 294624 pages of HIGHMEM
> Dec 11 15:28:36 recv kernel: 5512 reserved pages
> Dec 11 15:28:36 recv kernel: 3852 pages shared
> Dec 11 15:28:36 recv kernel: 0 pages swap cached
> Dec 11 15:28:36 recv kernel: 0 pages dirty
> Dec 11 15:28:36 recv kernel: 0 pages writeback
> Dec 11 15:28:36 recv kernel: 832 pages mapped
> Dec 11 15:28:36 recv kernel: 218464 pages slab
> Dec 11 15:28:36 recv kernel: 130 pages pagetables
> Dec 11 15:28:36 recv kernel: Out of memory: kill process 1967 (sshd) score 2875 or a child
> Dec 11 15:28:36 recv kernel: Killed process 1968 (bash)
> Dec 11 15:34:26 recv kernel: NET: Unregistered protocol family 27
> Dec 11 15:34:26 recv kernel: [PF_RING] removed /proc/net/pf_ring/info
> Dec 11 15:34:26 recv kernel: [PF_RING] removed /proc/net/pf_ring/plugins_info
> Dec 11 15:34:26 recv kernel: [PF_RING] deregistered /proc/net/pf_ring
> Dec 11 15:34:26 recv kernel: [PF_RING] unloaded
> Dec 11 15:38:45 recv kernel: [PF_RING] Welcome to PF_RING 3.9.0
> Dec 11 15:38:45 recv kernel: (C) 2004-08 L.Deri
> Dec 11 15:38:45 recv kernel: NET: Registered protocol family 27
> Dec 11 15:38:45 recv kernel: [PF_RING] Ring slots 4096
> Dec 11 15:38:45 recv kernel: [PF_RING] Slot version 9
> Dec 11 15:38:45 recv kernel: [PF_RING] Capture TX Yes [RX+TX]
> Dec 11 15:38:45 recv kernel: [PF_RING] IP Defragment No
> Dec 11 15:38:45 recv kernel: [PF_RING] Initialized correctly
> Dec 11 15:38:45 recv kernel: [PF_RING] registered /proc/net/pf_ring/
> Dec 11 15:43:12 recv rsyslogd: [origin software="rsyslogd" swVersion="1.19.6" x-pid="1845"][x-configInfo udpReception="No" udpPort="514" tcpReception="No" tcpPort="0"] restart
> Dec 11 15:43:12 recv kernel: rklogd 1.19.6, log source = /proc/kmsg started.
> Dec 11 15:43:12 recv kernel: Initializing cgroup subsys cpuset
> Dec 11 15:43:12 recv kernel: Linux version 2.6.26.6-pfring (k...@recv) (gcc version 4.1.2 20070925 (Red Hat 4.1.2-33)) #1 SMP Thu Nov 27 18:14:24 MSK 2008
> --
> Best regards,
> Vladimir Kukushkin
--
Best regards,
Vladimir Kukushkin
_______________________________________________ Ntop-dev mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-dev
