Thanks for you help. I will try that as your suggestion.
Cheers! >------------------------------------------------------------------------ ZHANG Laiyu Phone(China) 010-88233078 Cellphone(China) 13681385567 E-mail: [email protected] Address: 19B Yuquan Road,Shijingshan District,Beijing，China Department: Center for Particle Astrophysics Office: Astrophysics Building 205 Institute of High Energy Physics, CAS web: http://www.ihep.cas.cn >------------------------------------------------------------------------ -----原始邮件----- 发件人:"John Ford" <[email protected]> 发送时间:2020-12-23 01:13:14 (星期三) 收件人: LIST <[email protected]> 抄送: 主题: Re: [casper] How to check or modify the network setting on ROACH2 ? Hi. Since you can receive 1500 packets error-free, it seems like your PC is getting more data than it can handle and starts losing packets. As Jack says, the PPC doesn't get involved in it at all. Using small packets like you are using makes for more work on the PC side, as many more interrupts need to be processed. You might try making your packets bigger and using jumbo packets to save on overhead. Some things to try, besides as you said, optimizing your C++ code: Lock all memory in place in the PC with mlockall() If you have a multi-CPU (not multi-core) machine, make sure your processes access the local memory on that CPU by binding them to the CPU and also the interrupt handler CPU. single-processor multicore machines don't have this problem. Tune the buffers in the kernel. There are some mails in this archive with reasonable values. Tune the interface card driver. Some settings like interrupt coalescing will help. You may need a buffering scheme like hashpipe or other packet processing pipeline to smooth out the flow. John On Tue, Dec 22, 2020 at 3:41 AM zhang laiyu <[email protected]> wrote: Hi,Jack I realized a firmware that acquire data using ADC_MKID_4X yellow block and then pack the data into multi 64bits data to 10Gb yellow block in ROACH2.The PC receiver UDP package through 10Gb cable.The ADC_MKID_4x board clock is 512MHz,and the ROACH2 worked on 128MHz. I use part of Tutorial 2 's design to control the tx_valid port and tx_end_of_frame port of the 10Gb yellow block .I set the tx_valid is 1081 and the tx_end_of_frame was set to 1441. The UDP package is 8648 bytes less than the max capacity (8k+512) of 10Gb yellow block . Now,the firmware worked well. I can get about 1500 UDP frames without losing data in the PC server once a time. Because every UDP frame has frame counter,so I can judge that the frames is continuous or not.The firmware was running continuously,and I can get right data from PC during different time. The frame counter is continuous. But when I want to receive much more packages,such as 100000 packages once a time, I found that the fram counter is not continuous, part of the frame was lost.The valid data (ADC data)rate is 6.144Gb/s. (512*10^6*12bit/s) I suspect the reason of losting UDP frams coming from two sources. Frist, the PC server's receive program(C++) need to optimize to realize high frequency and large capacity udp data receiving . Another reason may be coming from the network setting of the linux system in PowerPC on ROACH2, I am not sure. My question is how to check and modidy the network setting of the linux system in ROACH2？ I want to check the max send buffer (MTU )of the PowerPC. Thank you! Cheers! >------------------------------------------------------------------------ ZHANG Laiyu Phone(China) 010-88233078 Cellphone(China) 13681385567 E-mail: [email protected] Address: 19B Yuquan Road,Shijingshan District,Beijing，China Department: Center for Particle Astrophysics Office: Astrophysics Building 205 Institute of High Energy Physics, CAS web: http://www.ihep.cas.cn >------------------------------------------------------------------------ -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/7e0e704.f595.1768a09a04f.Coremail.zhangly%40ihep.ac.cn. -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/CABmH8B-C6nh1HdffxWxq5DLQxFDDJO6Mi1EvS_EvKLL-9-gAyQ%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "[email protected]" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/lists.berkeley.edu/d/msgid/casper/2cc7eff2.fc74.1768dcbcc65.Coremail.zhangly%40ihep.ac.cn.
