On Dec 11, 2010, at 4:35 AM, Vikram Roopchand wrote:
> We have been using Libpcap 1.1.1 heavily
On what operating system are you using it? The code used for capturing is very
different on different OSes.
> and noticed
> something of the following nature. It seems that during a packet read,
> internal buffer to read each segment is allocated a size of "snap_len"
> bytes.
The buffer into which packets are read is allocated in pcap_activate(), not in
the read code. libpcap doesn't allocate a separate buffer for each packet.
> This is okay since we do not know how many bytes will arrive so it
> makes sense to have snap_len number of bytes as buffer to read into. But
> after that it looks that no realloc is done to reduce the buffer to
> the captured length (caplen) size. Is this correct ?
Yes. There's no good reason to do so, as the next packet to be read into the
buffer might be larger.
And this only applies on those platforms where
1) only one packet is read into the buffer at a time
and
2) the packets aren't read from a memory-mapped buffer shared between
the kernel and userland.
If more than one packet is read into the buffer at a time - as is the case, for
example, if your OS has BPF as the capture mechanism (*BSD, Mac OS X, AIX, some
newer OpenSolaris builds), the buffer has to be as big as BPF's internal packet
buffer, which can be, and has to be, much bigger than a single packet.
If it's a memory-mapped buffer, that buffer is also bigger than one packet.
> If this is the
> case, isn't some memory getting wasted each time a segment is caught.
For any given packet, not all the memory allocated for the buffer contains
packet data, so, in that sense, it's "wasted". It might be used for the next
packet.
That "waste" is not cumulative, i.e. it's not as if you lose more and more
memory with each packet that's captured.
> For
> example, in case of Ethernet, If the snap_len was kept to 65535, and the
> ethernet frame can go atmost to 9022 (jumbo),
Actually, the "frame" can be bigger; if the network adapter is doing TCP
reassembly (TCP Segmentation Offloading, or TSO), it might hand a reassembled
chunk of TCP data to the host, so what you're getting wouldn't be a single
Ethernet frame.
> we lose on quite a bit of
> internal buffer each time a segment is caught.
Again, it's not as if memory is lost for each packet captured; there might be
some memory that, during the entire capture, is never used, but that amount of
memory is allocated at the start of the capture, not allocated for every packet
captured.
> I understand there must be
> some performance drop doing realloc, may be that is the reason ?
If we did a realloc() to shrink the buffer after the packet's been read, we
would have to do *another* realloc() to grow it again before reading the next
packet; that would increase the CPU time required to capture, *and* would
potentially fragment memory significantly and waste a lot more memory.-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.
