Re: [dpdk-users] Round-robin packet distribution
Hi, > Start from reading https://doc.dpdk.org/guides/prog_guide/rte_flow.html > eg Table 12.22 > And here's the painful news, the Mellanox cards we're using do not support RAW flow items, which means I can't make use of the features explained in the table 12.22 > But in my mind, as already mentioned by someone else, I think that > randomly spreading packets across different cores > might not be a good idea. > > If you want packets to be spread across all cores in deterministic but > even way then easiest would be to use > RSS over IP / UDP. As a second iteration I would use Flow director API + > IP / UDP rules *without* RAW items. > Can you elaborate a bit? Perhaps there's some example somewhere on git we can use as reference? Thanks > Paweł > > On 24.08.2020 12:07, Tom Barbette wrote: >> He means bytes. >> >> I think some drivers only allow the flexible bytes to start after the >> actual matching though, eg TCP header. So check for that. I'd contact >> Sprayer authors to ask how they did it if I were you. >> >> Do you have a specific NIC in mind? Mellanox's ones are pretty >> powerful, it may be worth it to have a PM-to-PM meeting asking about >> feature, or CC one of the maintainer. Devs do not always look at all >> mails. >> >> Cheers, >> >> Tom >> >> Le 18/08/2020 à 16:27, Filip Janiszewski a écrit : >>> Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As >>> for the network traffic type, we're capturing financial market traffic >>> over TCP/UDP, nothing really fancy. >>> >>> Thanks >>> >>> Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto: Flexible payload matching (aka RAW in flow API) works up to first 64b of the packet - at least in e1000, ixgbe and i40e. It will be easier if you can provide some details about you network traffic. Paweł On 18.08.2020 14:40, Filip Janiszewski wrote: > Hi, > > We had a look at that, and decided that it might be a bit too > complicated to implement in our SW and will not work in a > performant way > as we might wish, ideally we're looking for a simple approach even if > not ideal.. > > So, I was wondering if we can get at least a "fair" distribution (no > round robin) using rte flow? And BTW, is it even possible to access > the > checksum from this API using for example the pattern matching with an > offset that points to the proper byte location in the packet? (It > seems > it can't be done without modification to the driver..) > > Thanks > > Il 7/25/20 10:28 AM, Tom Barbette ha scritto: >> Hi Filip, >> >> This is not possible, but you may use the idea of Sprayer >> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch >> packets randomly (use RSS on the checksum). >> >> However, in our paper RSS++ >> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) >> we >> show it's nearly always a bad idea because you'll have to share >> state, >> and even for "stateless" function, that leads to a very bad >> locality in >> a firewall as the same rules have to be fetched to L1 to all cores at >> the same time when you receive a burst of similar packets. >> >> Tom >> >> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : >>> Hi, >>> >>> Is there a way in DPDK to configure the NIC to distribute the >>> incoming >>> packets to multiple queues in a round robin fashion? Without taking >>> into >>> account the payload/headers or type of packet, just plain round >>> robin >>> distribution to multiple queues. >>> >>> I'm struggling to obtain a fair mechanism using RSS, perhaps the >>> rte_flow API can do the trick? Any other suggestion? >>> >>> Thanks >>> >> > -- BR, Filip +48 666 369 823
Re: [dpdk-users] Round-robin packet distribution
Yes, I meant bytes. Thanks Tom. If you deal with encrypted traffic you can use the payload as random data and spread it using flow director. Constructing flow rules is painful process and be prepared that you case might not work if it is not coverd by test scenarios in DTS. Also you will have to read documentation of your NIC to see what kind of flow type it might support. Start from reading https://doc.dpdk.org/guides/prog_guide/rte_flow.html eg Table 12.22 But in my mind, as already mentioned by someone else, I think that randomly spreading packets across different cores might not be a good idea. If you want packets to be spread across all cores in deterministic but even way then easiest would be to use RSS over IP / UDP. As a second iteration I would use Flow director API + IP / UDP rules *without* RAW items. Paweł On 24.08.2020 12:07, Tom Barbette wrote: He means bytes. I think some drivers only allow the flexible bytes to start after the actual matching though, eg TCP header. So check for that. I'd contact Sprayer authors to ask how they did it if I were you. Do you have a specific NIC in mind? Mellanox's ones are pretty powerful, it may be worth it to have a PM-to-PM meeting asking about feature, or CC one of the maintainer. Devs do not always look at all mails. Cheers, Tom Le 18/08/2020 à 16:27, Filip Janiszewski a écrit : Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As for the network traffic type, we're capturing financial market traffic over TCP/UDP, nothing really fancy. Thanks Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto: Flexible payload matching (aka RAW in flow API) works up to first 64b of the packet - at least in e1000, ixgbe and i40e. It will be easier if you can provide some details about you network traffic. Paweł On 18.08.2020 14:40, Filip Janiszewski wrote: Hi, We had a look at that, and decided that it might be a bit too complicated to implement in our SW and will not work in a performant way as we might wish, ideally we're looking for a simple approach even if not ideal.. So, I was wondering if we can get at least a "fair" distribution (no round robin) using rte flow? And BTW, is it even possible to access the checksum from this API using for example the pattern matching with an offset that points to the proper byte location in the packet? (It seems it can't be done without modification to the driver..) Thanks Il 7/25/20 10:28 AM, Tom Barbette ha scritto: Hi Filip, This is not possible, but you may use the idea of Sprayer (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch packets randomly (use RSS on the checksum). However, in our paper RSS++ (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we show it's nearly always a bad idea because you'll have to share state, and even for "stateless" function, that leads to a very bad locality in a firewall as the same rules have to be fetched to L1 to all cores at the same time when you receive a burst of similar packets. Tom Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : Hi, Is there a way in DPDK to configure the NIC to distribute the incoming packets to multiple queues in a round robin fashion? Without taking into account the payload/headers or type of packet, just plain round robin distribution to multiple queues. I'm struggling to obtain a fair mechanism using RSS, perhaps the rte_flow API can do the trick? Any other suggestion? Thanks
Re: [dpdk-users] Round-robin packet distribution
He means bytes. I think some drivers only allow the flexible bytes to start after the actual matching though, eg TCP header. So check for that. I'd contact Sprayer authors to ask how they did it if I were you. Do you have a specific NIC in mind? Mellanox's ones are pretty powerful, it may be worth it to have a PM-to-PM meeting asking about feature, or CC one of the maintainer. Devs do not always look at all mails. Cheers, Tom Le 18/08/2020 à 16:27, Filip Janiszewski a écrit : Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As for the network traffic type, we're capturing financial market traffic over TCP/UDP, nothing really fancy. Thanks Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto: Flexible payload matching (aka RAW in flow API) works up to first 64b of the packet - at least in e1000, ixgbe and i40e. It will be easier if you can provide some details about you network traffic. Paweł On 18.08.2020 14:40, Filip Janiszewski wrote: Hi, We had a look at that, and decided that it might be a bit too complicated to implement in our SW and will not work in a performant way as we might wish, ideally we're looking for a simple approach even if not ideal.. So, I was wondering if we can get at least a "fair" distribution (no round robin) using rte flow? And BTW, is it even possible to access the checksum from this API using for example the pattern matching with an offset that points to the proper byte location in the packet? (It seems it can't be done without modification to the driver..) Thanks Il 7/25/20 10:28 AM, Tom Barbette ha scritto: Hi Filip, This is not possible, but you may use the idea of Sprayer (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch packets randomly (use RSS on the checksum). However, in our paper RSS++ (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we show it's nearly always a bad idea because you'll have to share state, and even for "stateless" function, that leads to a very bad locality in a firewall as the same rules have to be fetched to L1 to all cores at the same time when you receive a burst of similar packets. Tom Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : Hi, Is there a way in DPDK to configure the NIC to distribute the incoming packets to multiple queues in a round robin fashion? Without taking into account the payload/headers or type of packet, just plain round robin distribution to multiple queues. I'm struggling to obtain a fair mechanism using RSS, perhaps the rte_flow API can do the trick? Any other suggestion? Thanks
Re: [dpdk-users] Round-robin packet distribution
Do you mean bit or bytes? 'b' refers to bit, maybe you meant bytes? As for the network traffic type, we're capturing financial market traffic over TCP/UDP, nothing really fancy. Thanks Il 8/18/20 3:50 PM, Pawel Wodkowski ha scritto: > Flexible payload matching (aka RAW in flow API) works up to first 64b of > the > packet - at least in e1000, ixgbe and i40e. > > It will be easier if you can provide some details about you network > traffic. > > Paweł > > On 18.08.2020 14:40, Filip Janiszewski wrote: >> Hi, >> >> We had a look at that, and decided that it might be a bit too >> complicated to implement in our SW and will not work in a performant way >> as we might wish, ideally we're looking for a simple approach even if >> not ideal.. >> >> So, I was wondering if we can get at least a "fair" distribution (no >> round robin) using rte flow? And BTW, is it even possible to access the >> checksum from this API using for example the pattern matching with an >> offset that points to the proper byte location in the packet? (It seems >> it can't be done without modification to the driver..) >> >> Thanks >> >> Il 7/25/20 10:28 AM, Tom Barbette ha scritto: >>> Hi Filip, >>> >>> This is not possible, but you may use the idea of Sprayer >>> (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch >>> packets randomly (use RSS on the checksum). >>> >>> However, in our paper RSS++ >>> (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we >>> show it's nearly always a bad idea because you'll have to share state, >>> and even for "stateless" function, that leads to a very bad locality in >>> a firewall as the same rules have to be fetched to L1 to all cores at >>> the same time when you receive a burst of similar packets. >>> >>> Tom >>> >>> Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : Hi, Is there a way in DPDK to configure the NIC to distribute the incoming packets to multiple queues in a round robin fashion? Without taking into account the payload/headers or type of packet, just plain round robin distribution to multiple queues. I'm struggling to obtain a fair mechanism using RSS, perhaps the rte_flow API can do the trick? Any other suggestion? Thanks > -- BR, Filip +48 666 369 823
Re: [dpdk-users] Round-robin packet distribution
Flexible payload matching (aka RAW in flow API) works up to first 64b of the packet - at least in e1000, ixgbe and i40e. It will be easier if you can provide some details about you network traffic. Paweł On 18.08.2020 14:40, Filip Janiszewski wrote: Hi, We had a look at that, and decided that it might be a bit too complicated to implement in our SW and will not work in a performant way as we might wish, ideally we're looking for a simple approach even if not ideal.. So, I was wondering if we can get at least a "fair" distribution (no round robin) using rte flow? And BTW, is it even possible to access the checksum from this API using for example the pattern matching with an offset that points to the proper byte location in the packet? (It seems it can't be done without modification to the driver..) Thanks Il 7/25/20 10:28 AM, Tom Barbette ha scritto: Hi Filip, This is not possible, but you may use the idea of Sprayer (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch packets randomly (use RSS on the checksum). However, in our paper RSS++ (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we show it's nearly always a bad idea because you'll have to share state, and even for "stateless" function, that leads to a very bad locality in a firewall as the same rules have to be fetched to L1 to all cores at the same time when you receive a burst of similar packets. Tom Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : Hi, Is there a way in DPDK to configure the NIC to distribute the incoming packets to multiple queues in a round robin fashion? Without taking into account the payload/headers or type of packet, just plain round robin distribution to multiple queues. I'm struggling to obtain a fair mechanism using RSS, perhaps the rte_flow API can do the trick? Any other suggestion? Thanks
Re: [dpdk-users] Round-robin packet distribution
Hi, We had a look at that, and decided that it might be a bit too complicated to implement in our SW and will not work in a performant way as we might wish, ideally we're looking for a simple approach even if not ideal.. So, I was wondering if we can get at least a "fair" distribution (no round robin) using rte flow? And BTW, is it even possible to access the checksum from this API using for example the pattern matching with an offset that points to the proper byte location in the packet? (It seems it can't be done without modification to the driver..) Thanks Il 7/25/20 10:28 AM, Tom Barbette ha scritto: > Hi Filip, > > This is not possible, but you may use the idea of Sprayer > (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch > packets randomly (use RSS on the checksum). > > However, in our paper RSS++ > (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we > show it's nearly always a bad idea because you'll have to share state, > and even for "stateless" function, that leads to a very bad locality in > a firewall as the same rules have to be fetched to L1 to all cores at > the same time when you receive a burst of similar packets. > > Tom > > Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : >> Hi, >> >> Is there a way in DPDK to configure the NIC to distribute the incoming >> packets to multiple queues in a round robin fashion? Without taking into >> account the payload/headers or type of packet, just plain round robin >> distribution to multiple queues. >> >> I'm struggling to obtain a fair mechanism using RSS, perhaps the >> rte_flow API can do the trick? Any other suggestion? >> >> Thanks >> -- BR, Filip +48 666 369 823
Re: [dpdk-users] Round-robin packet distribution
Hi Filip, This is not possible, but you may use the idea of Sprayer (http://www.gta.ufrj.br/ftp/gta/TechReports/SCC18d.pdf) to dispatch packets randomly (use RSS on the checksum). However, in our paper RSS++ (https://www.diva-portal.org/smash/get/diva2:1371780/FULLTEXT01.pdf) we show it's nearly always a bad idea because you'll have to share state, and even for "stateless" function, that leads to a very bad locality in a firewall as the same rules have to be fetched to L1 to all cores at the same time when you receive a burst of similar packets. Tom Le 24/07/2020 à 12:05, Filip Janiszewski a écrit : Hi, Is there a way in DPDK to configure the NIC to distribute the incoming packets to multiple queues in a round robin fashion? Without taking into account the payload/headers or type of packet, just plain round robin distribution to multiple queues. I'm struggling to obtain a fair mechanism using RSS, perhaps the rte_flow API can do the trick? Any other suggestion? Thanks