Replies quoted with --GV-- Have a nice day GV
----- Original Message ----- From: Denis Kourktchan To: [email protected] Sent: Tuesday, August 11, 2009 11:27 AM Subject: Re: [Winpcap-users] How to ensure all packets are captured Dear Mr. Varenni, Thanks for your reply. I just wanted to clarify a few points you have made. (Note: I am using a multicore machine so the multithread is a natural choice and my processing code is definitely not as optimized as your pcap code, but this is a solo project so I don't need to satisfy anyone else) --GV-- Using multiple threads, even if you have an SMP machine, isn't always the best choice. It makes sense if the CPU needed for receiving packets + CPU needed for processing packets is greater than 100% (where by 100% i mean 1 single core fully loaded, if you have 4 cores the total processing time is 400%), and the cost of pushing packets to a queue/linked-list is negligible versus the processing. So for example you have 60%CPU for receiving packets and 5% to push the packets on a queue (on one CPU) and another 60%CPU for processing packets and 5% to pull the packets out of the queue. --GV-- 1. You mention, that I should pre-allocate and reuse buffer space where to store packets and not allocate new buffer. Do you mean something like: PCAPDATANODE myPacketBuffer[amountOfNeededPackets]; // store in a specific memory location predetermined Is PCAPDATANODE packetNode = new PCAPDATANODE(); not a good way to go? --GV-- Allocating your storage (PCAPDATANODE) whenever a packet arrives is a bad strategy because 1. memory allocation is expensive and 2. you will keep allocating and freeing chunks of memory, which is kinda useless in your scenario. --GV-- 2. To prevent overloading, I need to pause capture (not extraction) for a some amounts time according to SYSTEM time settings, is there a command that can facilitate that? Or should I simply stop the adapter, release it, and then reacquire when the time comes to capture (also would this automatically flush the buffer)? At the moment, I merely stopped the extraction but as you might have predicted the capture continued and I simply continued processing where I last left it off. --GV-- I don't understand why you want to pause the capture. Either you just drop drop the packets when let's say your queue is full, or you can stop the capture (i.e. close the pcap_t handle_ and restart it (calling pcap_open_live again). There is no API to pause reception. Another possibility to "virtually" pause reception is to just set a filter that drops almost all the packets. --GV-- Thanks again for you time, - Dennis From: Gianluca Varenni [mailto:[email protected]] Sent: Monday, August 10, 2009 2:50 PM To: [email protected]; [email protected] Subject: Re: [Winpcap-users] How to ensure all packets are captured Moving the processing code in a separate thread may or might not help. If you move the processing on a different thread and use a FIFO/linked list to pass the packets among the threads, you will need to copy the packets in your own allocated buffer. Copying a packet is extremely fast, but it makes sense only if your packet processing code is much more CPU intensive than copying the packet itself. Also, using a different thread makes sense if you run on a multicore machine, where basically you can use both the CPUs. If you go this way, remember to preallocate and reuse the buffers in which you copy the packets, do not allocate a new buffer whenever you receive a packet. Hope this helps GV ----- Original Message ----- From: Denis Kourktchan To: [email protected] Sent: Sunday, August 09, 2009 1:05 PM Subject: [Winpcap-users] How to ensure all packets are captured Hi Everybody, I am writing an application that is intercepting packets from a network and after reading a whole lot of posts regarding lost messages I figured the fault is usually in application taking too long to process a batch of packets which causes the overfilled buffer to replace oldest messages. So I decided that the following setup is the best possible way to ensure that my messages are not lost. 1. Put capture (using pcap_next_ex()) into a different thread then the processing. 2. Store packets immediately upon extraction into a custom written linked list, adding to the end and processing from the front (in different threads) ensuring no mutex controls are necessary and no idle time. Now here is the problem for which I need your expert advice, since the pcap_next_ex() returns pointers to data, I still need to extract them (I use for loop) before I can store which still preoccupies the capture thread and slows down extraction process. So is there a better way to extract packets without this delay? I would appreciate any and all advice regarding the optimal solution architecture wise. Regards, - Dennis _______________________________________________ Winpcap-users mailing list [email protected] https://www.winpcap.org/mailman/listinfo/winpcap-users _______________________________________________ Winpcap-users mailing list [email protected] https://www.winpcap.org/mailman/listinfo/winpcap-users _______________________________________________ Winpcap-users mailing list [email protected] https://www.winpcap.org/mailman/listinfo/winpcap-users
