Something that I've done (although our version of duplicate suppression,
written by a co-worker, just does header compares) is to use
high-entropy bytes in the packet structure to quickly eliminate the
possibility of duplicates, e.g. IP/TCP/UDP checksums, and if your
network card/OS provide it, the FCS at the end of the packet (although
that requires full capture). You only need to compare if the
high-entropy bytes are the same (which usually they are not unless there
really is a duplicate).
Alex,
I do something similar, in that I look at the length first, I think.
I'm only looking at the last 4 packets, so that length the crc if
needed, does the trick.
http://en.wikipedia.org/wiki/Adler-32
Interestingly, the SCTP protocol dropped Adler-32 for a conventional CRC
(see RFC 3309) because the Adler will not give you a good distribution
of entropy across all 32 bits for short packets. I don't know if that's
an issue for your application.
Funny that. I read that when I searched for the link. When I implemented
the dupe check, the results of the MD5 and Adler were virtually
identical. I had a few missed dupes with Adler out of a GB of packets.
Thats acceptable for my application.
Andy
-
This is the tcpdump-workers list.
Visit https://cod.sandelman.ca/ to unsubscribe.
