Re: [pmacct-discussion] segv with memory,sfprobe plugins
Hi Stig,
thanks very much for having reported the issue. This is now solved
in the CVS. I managed to reproduce it.
It was lying in the fact that initialization of the sfprobe plugin
was explicitely disabling the IP fragment handler in pmacctd; this
was causing the IMT plugin, configured with L4 primitives (src_port
for example), to crash because it expects the IP fragment handler
to be there.
The one-liner fix basically avoids sfprobe to turn the IP fragment
handler off in case it was previously turned on (hence you see the
position of the plugins was relevant) as part of operations of a
concurrent plugin.
Cheers,
Paolo
On Mon, Aug 17, 2009 at 08:27:51PM -0700, Stig Thormodsrud wrote:
I'm getting a segv fault when using the following conf file:
s...@io:~/git/pmacct-0.11.4/src$ cat pm.conf
daemonize: false
debug: true
promisc: true
pidfile: /var/run/pmacctd-eth0.pid
imt_path: /tmp/pmacctd-eth0.pipe
aggregate: src_host,dst_host,proto,src_port,dst_port,tos,flows
interface: eth0
!syslog: daemon
pcap_filter: !ether src 00:15:17:0b:d2:16
plugins: memory,sfprobe
sfprobe_agentsubid: 5
sfprobe_receiver: 172.16.117.25:6343
s...@io:~/git/pmacct-0.11.4/src$ sudo ./pmacctd -f pm.conf
INFO ( default/memory ): 131070 bytes are available to address shared
memory segment; buffer size is 132 bytes.
INFO ( default/memory ): Trying to allocate a shared memory segment of
4325244 bytes.
INFO ( default/sfprobe ): Pipe size obtained: 131070 / 49348.
OK ( default/core ): link type is: 1
DEBUG ( default/sfprobe ): Creating sFlow agent.
INFO ( default/sfprobe ): Exporting flows to [172.16.117.25]:6343
INFO ( default/sfprobe ): Sampling at: 1/1
DEBUG ( default/memory ): allocating a new memory segment.
DEBUG ( default/memory ): allocating a new memory segment.
OK ( default/memory ): waiting for data on: '/tmp/pmacctd-eth0.pipe'
DEBUG ( default/memory ): Selecting bucket 16151.
Segmentation fault
In gdb it stops at:
(gdb) run -f pm.conf
Starting program: /home/stig/git/pmacct-0.11.4/src/pmacctd -f pm.conf
[Thread debugging using libthread_db enabled]
INFO ( default/memory ): 131070 bytes are available to address shared
memory segment; buffer size is 132 bytes.
INFO ( default/memory ): Trying to allocate a shared memory segment of
4325244 bytes.
INFO ( default/sfprobe ): Pipe size obtained: 131070 / 49348.
DEBUG ( default/memory ): allocating a new memory segment.
DEBUG ( default/sfprobe ): Creating sFlow agent.
INFO ( default/sfprobe ): Exporting flows to [172.16.117.25]:6343
INFO ( default/sfprobe ): Sampling at: 1/1
DEBUG ( default/memory ): allocating a new memory segment.
OK ( default/memory ): waiting for data on: '/tmp/collect.pipe'
OK ( default/core ): link type is: 1
[New Thread 0xb788fa90 (LWP 23213)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb788fa90 (LWP 23213)]
0x080649f3 in src_port_handler (chptr=0x80c3ce0, pptrs=0xbf90dca8,
data=0xbf90dc6c) at pkt_handlers.c:353
(gdb)
(gdb) where
#0 0x080649f3 in src_port_handler (chptr=0x80c3ce0, pptrs=0xbf90dca8,
data=0xbf90dc6c) at pkt_handlers.c:353
#1 0x0805d218 in exec_plugins (pptrs=0xbf90dca8) at plugin_hooks.c:219
#2 0x08059b72 in pcap_cb (user=0xbf90de8c \031\, pkthdr=0xbf90dd88,
buf=0x883d1ba ) at pmacctd.c:665
#3 0xb7ebbd45 in ?? () from /usr/lib/libpcap.so.0.8
#4 0xbf90de8c in ?? ()
#5 0xbf90dd88 in ?? ()
#6 0x0883d1ba in ?? ()
#7 0x0020 in ?? ()
#8 0xbf90dd74 in ?? ()
#9 0xbf90dd98 in ?? ()
#10 0x in ?? ()
(gdb)
(gdb) p *pptrs
$1 = {pkthdr = 0xbf90dd88, f_agent = 0xb7e52219 SMP, f_header = 0x0,
f_data = 0x1 Address 0x1 out of bounds, f_tpl = 0x0, f_status = 0x1
Address 0x1 out of bounds, idtable = 0x0, bpas_table = 0x756e694c
Address 0x756e694c out of bounds, bta_table = 0xbf90e09c \220\223,
packet_ptr = 0x883d1ba , mac_ptr = 0x883d1ba , l3_proto = 2048,
l3_handler = 0x8059c77 ip_handler, l4_proto = 6, tag = 0, bpas = 0, bta
= 0, bgp_src = 0xb78900f0 \003\210\020ii\r, bgp_dst = 0x1 Address 0x1
out of bounds, bgp_peer = 0x1 Address 0x1 out of bounds, pf = 0,
new_flow = 0 '\0', tcp_flags = 0 '\0', vlan_ptr = 0x0, mpls_ptr = 0x0,
iph_ptr = 0x883d1c8 E, tlh_ptr = 0x29370 Address 0x29370 out of
bounds, payload_ptr = 0x0, class = 0, cst = {tentatives = 20 '\024',
stamp = {tv_sec = 0, tv_usec = 0}, ba = 3213942184, pa = 25312, fa = 240
''}, shadow = 0 '\0', tag_dist = 1 '\001'}
(gdb)
void src_port_handler(struct channels_list_entry *chptr, struct
packet_ptrs *pptrs, char **data)
{
struct pkt_data *pdata = (struct pkt_data *) *data;
if (pptrs-l4_proto == IPPROTO_UDP || pptrs-l4_proto == IPPROTO_TCP)
pdata-primitives.src_port = ntohs(((struct my_tlhdr *)
pptrs-tlh_ptr)-src_port);
else pdata-primitives.src_port = 0;
}
Seems like the problem is dereferencing pptrs-tlh_ptr in
src_port_handler(). If I reverse the plugins to sfprobe,memory or
remove the memory plugin, the it works. Could the
Re: [pmacct-discussion] segv with memory,sfprobe plugins
Great! Thanks for the quick fix.
stig
Hi Stig,
thanks very much for having reported the issue. This is now solved
in the CVS. I managed to reproduce it.
It was lying in the fact that initialization of the sfprobe plugin
was explicitely disabling the IP fragment handler in pmacctd; this
was causing the IMT plugin, configured with L4 primitives (src_port
for example), to crash because it expects the IP fragment handler
to be there.
The one-liner fix basically avoids sfprobe to turn the IP fragment
handler off in case it was previously turned on (hence you see the
position of the plugins was relevant) as part of operations of a
concurrent plugin.
Cheers,
Paolo
On Mon, Aug 17, 2009 at 08:27:51PM -0700, Stig Thormodsrud wrote:
I'm getting a segv fault when using the following conf file:
s...@io:~/git/pmacct-0.11.4/src$ cat pm.conf
daemonize: false
debug: true
promisc: true
pidfile: /var/run/pmacctd-eth0.pid
imt_path: /tmp/pmacctd-eth0.pipe
aggregate: src_host,dst_host,proto,src_port,dst_port,tos,flows
interface: eth0
!syslog: daemon
pcap_filter: !ether src 00:15:17:0b:d2:16
plugins: memory,sfprobe
sfprobe_agentsubid: 5
sfprobe_receiver: 172.16.117.25:6343
s...@io:~/git/pmacct-0.11.4/src$ sudo ./pmacctd -f pm.conf
INFO ( default/memory ): 131070 bytes are available to address shared
memory segment; buffer size is 132 bytes.
INFO ( default/memory ): Trying to allocate a shared memory segment of
4325244 bytes.
INFO ( default/sfprobe ): Pipe size obtained: 131070 / 49348.
OK ( default/core ): link type is: 1
DEBUG ( default/sfprobe ): Creating sFlow agent.
INFO ( default/sfprobe ): Exporting flows to [172.16.117.25]:6343
INFO ( default/sfprobe ): Sampling at: 1/1
DEBUG ( default/memory ): allocating a new memory segment.
DEBUG ( default/memory ): allocating a new memory segment.
OK ( default/memory ): waiting for data on: '/tmp/pmacctd-eth0.pipe'
DEBUG ( default/memory ): Selecting bucket 16151.
Segmentation fault
In gdb it stops at:
(gdb) run -f pm.conf
Starting program: /home/stig/git/pmacct-0.11.4/src/pmacctd -f pm.conf
[Thread debugging using libthread_db enabled]
INFO ( default/memory ): 131070 bytes are available to address shared
memory segment; buffer size is 132 bytes.
INFO ( default/memory ): Trying to allocate a shared memory segment of
4325244 bytes.
INFO ( default/sfprobe ): Pipe size obtained: 131070 / 49348.
DEBUG ( default/memory ): allocating a new memory segment.
DEBUG ( default/sfprobe ): Creating sFlow agent.
INFO ( default/sfprobe ): Exporting flows to [172.16.117.25]:6343
INFO ( default/sfprobe ): Sampling at: 1/1
DEBUG ( default/memory ): allocating a new memory segment.
OK ( default/memory ): waiting for data on: '/tmp/collect.pipe'
OK ( default/core ): link type is: 1
[New Thread 0xb788fa90 (LWP 23213)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb788fa90 (LWP 23213)]
0x080649f3 in src_port_handler (chptr=0x80c3ce0, pptrs=0xbf90dca8,
data=0xbf90dc6c) at pkt_handlers.c:353
(gdb)
(gdb) where
#0 0x080649f3 in src_port_handler (chptr=0x80c3ce0, pptrs=0xbf90dca8,
data=0xbf90dc6c) at pkt_handlers.c:353
#1 0x0805d218 in exec_plugins (pptrs=0xbf90dca8) at
plugin_hooks.c:219
#2 0x08059b72 in pcap_cb (user=0xbf90de8c \031\,
pkthdr=0xbf90dd88,
buf=0x883d1ba ) at pmacctd.c:665
#3 0xb7ebbd45 in ?? () from /usr/lib/libpcap.so.0.8
#4 0xbf90de8c in ?? ()
#5 0xbf90dd88 in ?? ()
#6 0x0883d1ba in ?? ()
#7 0x0020 in ?? ()
#8 0xbf90dd74 in ?? ()
#9 0xbf90dd98 in ?? ()
#10 0x in ?? ()
(gdb)
(gdb) p *pptrs
$1 = {pkthdr = 0xbf90dd88, f_agent = 0xb7e52219 SMP, f_header = 0x0,
f_data = 0x1 Address 0x1 out of bounds, f_tpl = 0x0, f_status = 0x1
Address 0x1 out of bounds, idtable = 0x0, bpas_table = 0x756e694c
Address 0x756e694c out of bounds, bta_table = 0xbf90e09c \220\223,
packet_ptr = 0x883d1ba , mac_ptr = 0x883d1ba , l3_proto = 2048,
l3_handler = 0x8059c77 ip_handler, l4_proto = 6, tag = 0, bpas = 0,
bta
= 0, bgp_src = 0xb78900f0 \003\210\020ii\r, bgp_dst = 0x1 Address
0x1
out of bounds, bgp_peer = 0x1 Address 0x1 out of bounds, pf = 0,
new_flow = 0 '\0', tcp_flags = 0 '\0', vlan_ptr = 0x0, mpls_ptr = 0x0,
iph_ptr = 0x883d1c8 E, tlh_ptr = 0x29370 Address 0x29370 out of
bounds, payload_ptr = 0x0, class = 0, cst = {tentatives = 20 '\024',
stamp = {tv_sec = 0, tv_usec = 0}, ba = 3213942184, pa = 25312, fa =
240
''}, shadow = 0 '\0', tag_dist = 1 '\001'}
(gdb)
void src_port_handler(struct channels_list_entry *chptr, struct
packet_ptrs *pptrs, char **data)
{
struct pkt_data *pdata = (struct pkt_data *) *data;
if (pptrs-l4_proto == IPPROTO_UDP || pptrs-l4_proto ==
IPPROTO_TCP)
pdata-primitives.src_port = ntohs(((struct my_tlhdr *)
pptrs-tlh_ptr)-src_port);
else pdata-primitives.src_port = 0;
}
Seems like the problem is
Re: [pmacct-discussion] segv with memory,sfprobe plugins
Some say that he is a CIA experiment that went wrong, and that he only
eats cheese. All we know is that he's not The Stig.
Stig Thormodsrud wrote:
Great! Thanks for the quick fix.
stig
Hi Stig,
thanks very much for having reported the issue. This is now solved
in the CVS. I managed to reproduce it.
It was lying in the fact that initialization of the sfprobe plugin
was explicitely disabling the IP fragment handler in pmacctd; this
was causing the IMT plugin, configured with L4 primitives (src_port
for example), to crash because it expects the IP fragment handler
to be there.
The one-liner fix basically avoids sfprobe to turn the IP fragment
handler off in case it was previously turned on (hence you see the
position of the plugins was relevant) as part of operations of a
concurrent plugin.
Cheers,
Paolo
On Mon, Aug 17, 2009 at 08:27:51PM -0700, Stig Thormodsrud wrote:
I'm getting a segv fault when using the following conf file:
s...@io:~/git/pmacct-0.11.4/src$ cat pm.conf
daemonize: false
debug: true
promisc: true
pidfile: /var/run/pmacctd-eth0.pid
imt_path: /tmp/pmacctd-eth0.pipe
aggregate: src_host,dst_host,proto,src_port,dst_port,tos,flows
interface: eth0
!syslog: daemon
pcap_filter: !ether src 00:15:17:0b:d2:16
plugins: memory,sfprobe
sfprobe_agentsubid: 5
sfprobe_receiver: 172.16.117.25:6343
s...@io:~/git/pmacct-0.11.4/src$ sudo ./pmacctd -f pm.conf
INFO ( default/memory ): 131070 bytes are available to address shared
memory segment; buffer size is 132 bytes.
INFO ( default/memory ): Trying to allocate a shared memory segment of
4325244 bytes.
INFO ( default/sfprobe ): Pipe size obtained: 131070 / 49348.
OK ( default/core ): link type is: 1
DEBUG ( default/sfprobe ): Creating sFlow agent.
INFO ( default/sfprobe ): Exporting flows to [172.16.117.25]:6343
INFO ( default/sfprobe ): Sampling at: 1/1
DEBUG ( default/memory ): allocating a new memory segment.
DEBUG ( default/memory ): allocating a new memory segment.
OK ( default/memory ): waiting for data on: '/tmp/pmacctd-eth0.pipe'
DEBUG ( default/memory ): Selecting bucket 16151.
Segmentation fault
In gdb it stops at:
(gdb) run -f pm.conf
Starting program: /home/stig/git/pmacct-0.11.4/src/pmacctd -f pm.conf
[Thread debugging using libthread_db enabled]
INFO ( default/memory ): 131070 bytes are available to address shared
memory segment; buffer size is 132 bytes.
INFO ( default/memory ): Trying to allocate a shared memory segment of
4325244 bytes.
INFO ( default/sfprobe ): Pipe size obtained: 131070 / 49348.
DEBUG ( default/memory ): allocating a new memory segment.
DEBUG ( default/sfprobe ): Creating sFlow agent.
INFO ( default/sfprobe ): Exporting flows to [172.16.117.25]:6343
INFO ( default/sfprobe ): Sampling at: 1/1
DEBUG ( default/memory ): allocating a new memory segment.
OK ( default/memory ): waiting for data on: '/tmp/collect.pipe'
OK ( default/core ): link type is: 1
[New Thread 0xb788fa90 (LWP 23213)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb788fa90 (LWP 23213)]
0x080649f3 in src_port_handler (chptr=0x80c3ce0, pptrs=0xbf90dca8,
data="" at pkt_handlers.c:353
(gdb)
(gdb) where
#0 0x080649f3 in src_port_handler (chptr=0x80c3ce0, pptrs=0xbf90dca8,
data="" at pkt_handlers.c:353
#1 0x0805d218 in exec_plugins (pptrs=0xbf90dca8) at
plugin_hooks.c:219
#2 0x08059b72 in pcap_cb (user=0xbf90de8c "\031\"",
pkthdr=0xbf90dd88,
buf=0x883d1ba "") at pmacctd.c:665
#3 0xb7ebbd45 in ?? () from /usr/lib/libpcap.so.0.8
#4 0xbf90de8c in ?? ()
#5 0xbf90dd88 in ?? ()
#6 0x0883d1ba in ?? ()
#7 0x0020 in ?? ()
#8 0xbf90dd74 in ?? ()
#9 0xbf90dd98 in ?? ()
#10 0x in ?? ()
(gdb)
(gdb) p *pptrs
$1 = {pkthdr = 0xbf90dd88, f_agent = 0xb7e52219 "SMP", f_header = 0x0,
f_data = 0x1 Address 0x1 out of bounds, f_tpl = 0x0, f_status = 0x1
Address 0x1 out of bounds, idtable = 0x0, bpas_table = 0x756e694c
Address 0x756e694c out of bounds, bta_table = 0xbf90e09c "\220\223",
packet_ptr = 0x883d1ba "", mac_ptr = 0x883d1ba "", l3_proto = 2048,
l3_handler = 0x8059c77 ip_handler, l4_proto = 6, tag = 0, bpas = 0,
bta
= 0, bgp_src = 0xb78900f0 "\003\210\020ii\r", bgp_dst = 0x1 Address
0x1
out of bounds, bgp_peer = 0x1 Address 0x1 out of bounds, pf = 0,
new_flow = 0 '\0', tcp_flags = 0 '\0', vlan_ptr = 0x0, mpls_ptr = 0x0,
iph_ptr = 0x883d1c8 "E", tlh_ptr = 0x29370 Address 0x29370 out of
bounds, payload_ptr = 0x0, class = 0, cst = {tentatives = 20 '\024',
stamp = {tv_sec = 0, tv_usec = 0}, ba = 3213942184, pa = 25312, fa =
240
''}, shadow = 0 '\0', tag_dist = 1 '\001'}
(gdb)
void src_port_handler(struct channels_list_entry *chptr, struct
packet_ptrs *pptrs, char **data)
{
struct pkt_data *pdata = (struct pkt_data *) *data;
if (pptrs-l4_proto == IPPROTO_UDP || pptrs-l4_proto ==
IPPROTO_TCP)
