Re: [casper] Dropped packets during HASHPIPE data acquisition

2020-03-31 Thread Mark Ruzindana
Thanks a lot for the quick responses John and David! I really appreciate it. I will definitely be updating the version of Hashpipe that I currently have on the server as well as ensure that the network tuning is good. I'm currently using the standard "socket()" function, and a switch to packet so

Re: [casper] Dropped packets during HASHPIPE data acquisition

2020-03-31 Thread David MacMahon
Just to expand on John's excellent tips, Hashpipe does lock its shared memory buffers with mlock. These buffers will have the NUMA node affinity of the thread that created them so be sure to pin the threads to the desired core or cores by preceding the thread names on the command line with a -c

Re: [casper] Dropped packets during HASHPIPE data acquisition

2020-03-31 Thread David MacMahon
Hi, Mark, That packet rate should be very manageable. Are you using the standard "socket()" and "recv()" functions or are you using packet sockets? Packet sockets are a more efficient way to get packets from the kernel that bypasses the kernel's IP stack. It's not as efficient as IBVerbs or

Re: [casper] Dropped packets during HASHPIPE data acquisition

2020-03-31 Thread John Ford
Hi Mark. Since the newer version has a script called "hashpipe_irqaffinity.sh" I would think that the most expedient thing to do is to upgrade to the newer version. It's likely to fix some or all of this. That said, there are a lot of things that you can check, and not only the irq affinity, but

[casper] Dropped packets during HASHPIPE data acquisition

2020-03-31 Thread Mark Ruzindana
Hi all, I am fairly new to asking questions on a forum so if I need to provide more details, please let me know. Worth noting that just as I was about to send this out, I checked and I don't have the most recent version of HASHPIPE with hashpipe_irqaffinity.sh among other additions and modificati