[Ntop-misc] pfring filtering fails or not clear
Hello, I’m using PF_RING-6.0.1. I’m trying to develop an application that runs some algorithm consisting on rules. I made some tests using the “pfcount” tester, and unfortunately, I don’t understand the behavior: I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r –m” which AFAIU, adds a wildcard filter for each incoming packet. If I get it correctly, once a rule was added, I should not expect other packets of the same session to receive, and this is not what I’m getting. For example: --- [root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m Adding wildcard filtering rules Using PF_RING v.6.0.1 Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11] # Device RX channels: 6 # Polling threads:1 Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074 18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063] [caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 0 added successfully... 18:52:35.956301616 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596844523] [caplen=128][len=650][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 1 added successfully... 18:52:35.956303262 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596845111] [caplen=128][len=1086][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 2 added successfully... : --- How come, that once rule #0 was added for [10.61.10.9:52311 - 10.70.150.108:60189], I still see such packets in the next lines? Shouldn’t they be filtered by the rule that just as added? (BTW, when I use the command “./pfcount -i eth3 -u 1 -v 1 -r –m” (i.e. –u is 1 rather than 2), the tester uses hash filters, and in this case, I get errors: 18:53:19.052549112 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596847159] [caplen=128][len=1490][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] pfring_add_hash_filtering_rule(1) failed) Any help will be appreciated. Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pfring filtering fails or not clear
Hi Alfredo,
I hope you saw what I've sent last week. Any chance that you will take a
look at it soon?
Thanks,
Amir
On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri akadur...@gmail.com wrote:
Hi Alfredo,
Since the original pcap is huge, I sliced a smaller pcap from it, holding
344 packets.
Link to the tester code (based on pfcount_82599.c):
https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkwzVExNRHJXbmt3/view?usp=sharing
Link to the pcap file (congtaining 344 packets):
https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
Thanks a lot,
Amir
On Wed, Apr 8, 2015 at 4:07 PM, Alfredo Cardigliano cardigli...@ntop.org
wrote:
Hi Amir
could you provide your patched pfcount and a pcap with the packets
matching your rule?
Alfredo
On 08 Apr 2015, at 13:36, Amir Kaduri akadur...@gmail.com wrote:
I'm currently in a situation, that no rules work for me whatsoever (same
configuration I described earlier), and they did work for me a few day ago.
I'm using the pfcount_82599.c tester, and added a single hash rule. The
only thing that does work is the default behavior, when using the API
pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive
normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no
packet arrive to the tester).
This is the part of code:
--
:
rc = pfring_enable_ring(pd);
if (rc0)
printf(pfring_enable_ring() failed. rc=%d\n, rc);
pfring_toggle_filtering_policy(pd, 1); /* Default to allow */
if (1) {
hash_filtering_rule rule;
u_int16_t rule_id = 0;
memset(rule, 0, sizeof(hash_filtering_rule));
rule.proto = 6;
rule.rule_id = rule_id++;
rule.rule_action = dont_forward_packet_and_stop_rule_evaluation;
rule.host4_peer_a = ntohl(inet_addr(10.12.150.231));
rule.port_peer_a = 2489;
rule.host4_peer_b = ntohl(inet_addr(10.61.12.31));
rule.port_peer_b = 139;
rc = pfring_handle_hash_filtering_rule(pd, rule, 1);
if(rc0)
printf(pfring_add_hash_filtering_rule(%d) failed [errno=%d: %s]\n,
rule.rule_id, errno, strerror(errno));
else
printf(pfring_add_hash_filtering_rule(%d) succeeded\n, rule.rule_id);
}
--
I don't get any errors when the following code run, but when I bombard
the machine
with a pcap containing more than 600,000 packets of the specified session
that I've expected to be filtered out,
packets of it still received at the tester.
I'm pretty sure that something goes wrong in pf_ring, but I can't tell
what.
What is the best way to get debug information, other than reading
/var/log/messages?
Thanks,
Amir
On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano cardigli...@ntop.org
wrote:
On 05 Apr 2015, at 16:07, Amir Kaduri akadur...@gmail.com wrote:
Hi,
I think I've made some progress: AFAIU, the packets that I see despite
the rule that supposed to filter them, are packets the receive during the
time interval from rule-set to rule-apply by pfring.
I'll appreciate getting some answers about the following:
1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way
to know which rules-ids are set and which are vacant?
This is because I have to follow the rules-ids when setting them,
but when I purge them, I don't know which of them were removed.
No, unfortunately this is not possible with the current API.
2. Does this API also purges HW rules?
No, It doesn’t.
3. According to the documentation, I know that HW rules have a limit of
32,000. What is the limit for hash rules? IS this limit includes the 32,000
of the HW, or additional to it?
There is no limit to the number of software hash rules.
4. I have a valid rule, but whenever I
call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
pfring_get_hash_filtering_rule_stats() should be used with sw rules to
get stats from kernel plugins (when used), otherwise there is no stats per
rule.
Br
Alfredo
- I've add the stats code to the pfcount_82599 tester
- In /var/log/messages I see the following message that is probably
originated from ring_setsockopt(): kernel: [PF_RING] Found rule but
pluginId 0 is not registered
Thanks,
Amir
On Thu, Apr 2, 2015 at 5:06 PM, Amir Kaduri akadur...@gmail.com wrote:
Hi Alfredo,
Thanks for referring to my question.
I hope the following answers:
[root@CT10K10G]# cat /etc/pf_ring/pfring.conf
min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1
enable_ip_defrag=1
[root@CT10K10G]# cat /proc/net/pf_ring/info
PF_RING Version : 6.0.1 ($Revision: exported$)
Total rings : 0
Standard (non DNA) Options
Ring slots : 1024
Slot version : 15
Capture TX : Yes [RX+TX]
IP Defragment: Yes
Socket Mode
Re: [Ntop-misc] pfring filtering fails or not clear
Hi Pavel, Thanks for your response. I didn't try it yet, I'll consider trying it, but currently my goal is to evaluate pf_ring C/C++ package to see if it fits my needs, in comparison to other packages. This is since I need to develop a kind of business-logic application using it. I really want to be able to add/remove HW filtering rules using pf_ring's APIs. Thanks, Amir On Wed, Apr 29, 2015 at 10:57 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello, Amir! Have you tried manual configuration of hardware filters for 82599? You could try do it with ethtool: https://github.com/FastVPSEestiOu/fastnetmon/wiki/Traffic-filtration-using-NIC-capabilities-on-wire-speed-(10GE,-14Mpps) From my point of view ethtool interface is more flexible for hardware filter management. But I can't find any API for C/C++ for it. On Wed, Apr 29, 2015 at 10:50 AM, Amir Kaduri akadur...@gmail.com wrote: Hi, After testing software filtering rules, I've tried hardware filtering rules. Unfortunately, it doesn't work for me. First, I tested it using pfring 6.0.1 64bit, and then used pfring 5.6.1 32bit to make sure its not related to the version and the architecture. In both tests, I didn't see that the hardware filters work. The command line I used: ./pfcount_82599 -i eth2 -v -m The NIC details: 06:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) I used the following files: 1. https://drive.google.com/file/d/0B10Ms5GOXgCxYy1uWHZ2dXJrck0/view?usp=sharing The tester program pfcount_82599.c with slight changes: enabling 3 intel_82599_five_tuple_rule rules: - One that drops tcp packets - Second that drops packets with source 10.12.150.231 - Third that drops packets with dest 10.12.150.231 2. https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing The pcap file containing 344 packets, that should be filtered with the tester above. Note that all 3 rules should drop the packets in the attached tester. Any help to prove that hardware rules work, based on the above info, will be much appreciated. Thanks, Amir On Thu, Apr 16, 2015 at 12:54 AM, Alfredo Cardigliano cardigli...@ntop.org wrote: On 15 Apr 2015, at 19:10, Amir Kaduri akadur...@gmail.com wrote: Hi, I would like to update that the problem in the tester program was that I probably didn't add the vlan_id to rule. Once added, the hash rule began to work. Ok, that was definitely the problem, vlan-id is part of the hash input (vlan, protocol, src/dst ip, src/dst port). Also, I would like to refer to the argument There is no limit to the number of software hash rules: I assume that the meaning is that the algorithm doesn't have any limitation, though, since the rule_id field type is u_int16_t, there is a limit of 64K hash rules, de-facto. That’s right, we can consider changing the rule_id type if 64K rules is not enough in some use case. Alfredo Thanks, Amir On Tue, Apr 14, 2015 at 12:34 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir yes thank you for the files, I will look at them ASAP. Alfredo On 14 Apr 2015, at 11:17, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, I hope you saw what I've sent last week. Any chance that you will take a look at it soon? Thanks, Amir On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, Since the original pcap is huge, I sliced a smaller pcap from it, holding 344 packets. Link to the tester code (based on pfcount_82599.c): https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkwzVExNRHJXbmt3/view?usp=sharing Link to the pcap file (congtaining 344 packets): https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing Thanks a lot, Amir On Wed, Apr 8, 2015 at 4:07 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir could you provide your patched pfcount and a pcap with the packets matching your rule? Alfredo On 08 Apr 2015, at 13:36, Amir Kaduri akadur...@gmail.com wrote: I'm currently in a situation, that no rules work for me whatsoever (same configuration I described earlier), and they did work for me a few day ago. I'm using the pfcount_82599.c tester, and added a single hash rule. The only thing that does work is the default behavior, when using the API pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no packet arrive to the tester). This is the part of code: -- : rc = pfring_enable_ring(pd); if (rc0) printf(pfring_enable_ring() failed. rc=%d\n, rc
Re: [Ntop-misc] pfring filtering fails or not clear
Hi, I've made progress with adding hardware filtering rules, using pf_ring's API (pfring_add_hw_rule()). I managed to add 8190 (8K-2) perfect rules. I know that its possible to have ~32,000 (32K-2) hardware hash rules, but I fail to reach this limit. When loading the ixgbe kernel module, I use FdirPballoc=3,3,3,3. According to Intel's 82599 10GB controller datasheet http://www.intel.com/content/dam/www/public/us/en/documents/datasheets/82599-10-gbe-controller-datasheet.pdf page 289, I understand that its somehow related to rx packet buffer memory allocation, but I didn't manage to find out what exactly and how should I do the modification in order to be able to use 32K hardware hash (signature) rules. Any help will be appreciated. Regards, Amir On Wed, Apr 29, 2015 at 11:17 AM, Amir Kaduri akadur...@gmail.com wrote: Hi Pavel, Thanks for your response. I didn't try it yet, I'll consider trying it, but currently my goal is to evaluate pf_ring C/C++ package to see if it fits my needs, in comparison to other packages. This is since I need to develop a kind of business-logic application using it. I really want to be able to add/remove HW filtering rules using pf_ring's APIs. Thanks, Amir On Wed, Apr 29, 2015 at 10:57 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello, Amir! Have you tried manual configuration of hardware filters for 82599? You could try do it with ethtool: https://github.com/FastVPSEestiOu/fastnetmon/wiki/Traffic-filtration-using-NIC-capabilities-on-wire-speed-(10GE,-14Mpps) From my point of view ethtool interface is more flexible for hardware filter management. But I can't find any API for C/C++ for it. On Wed, Apr 29, 2015 at 10:50 AM, Amir Kaduri akadur...@gmail.com wrote: Hi, After testing software filtering rules, I've tried hardware filtering rules. Unfortunately, it doesn't work for me. First, I tested it using pfring 6.0.1 64bit, and then used pfring 5.6.1 32bit to make sure its not related to the version and the architecture. In both tests, I didn't see that the hardware filters work. The command line I used: ./pfcount_82599 -i eth2 -v -m The NIC details: 06:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01) I used the following files: 1. https://drive.google.com/file/d/0B10Ms5GOXgCxYy1uWHZ2dXJrck0/view?usp=sharing The tester program pfcount_82599.c with slight changes: enabling 3 intel_82599_five_tuple_rule rules: - One that drops tcp packets - Second that drops packets with source 10.12.150.231 - Third that drops packets with dest 10.12.150.231 2. https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing The pcap file containing 344 packets, that should be filtered with the tester above. Note that all 3 rules should drop the packets in the attached tester. Any help to prove that hardware rules work, based on the above info, will be much appreciated. Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Possible bug in SW hash rule API
Hi, I'm experiencing a possible bug in the API pfring_handle_hash_filtering_rule(), when adding a rule. The API return -1 (failure) although the hash rule has been added successfully. I experience it in versions 6.0.2 and 6.0.3. In version 6.0.1 it works fine. Can anyone please confirm that there is a bug? If yes, can I have a patch for it, or even a quick-and-dirty explanation on how to solve it? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Possible bug in SW hash rule API
After using latest code from git, the problem is indeed solved! Thanks a lot Alfredo! On Wed, May 13, 2015 at 1:43 AM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir the commit below should solve your issue, please pull latest code from git Alfredo https://github.com/ntop/PF_RING/commit/b67a6f46a06e68f2bb6cc53e9d452cc2cbe5f18f On 12 May 2015, at 12:15, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, The application itself is one of the example application provided by pf_ring itself called pfcount. Here is a decription of he steps done: Download pf_ring 6.0.3 and compile it 64bit (including drivers' PF_RING_aware subdirectory). (need to fix a small bug in pfcount.c by adding break; at line 799, so that it will be possible to use the option -u). On production env, suppose current working directory is PF_RING's source root. From this point, the commands are: rmmod ixgbe rmmod pf_ring insmod ./kernel/pf_ring.ko insmod ./drivers/PF_RING_aware/intel/ixgbe/ixgbe-3.22.3-zc/src/ixgbe.ko ./userland/examples/pfcount -i eth3 -w 1 -m -v 1 -u 1 You can replay the following pcap that contains 344 packets: https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing You should see that pfcount stops showing packets from this pcap (since the hash filter is applied), but on the other hand, it shows the following message (especially the first one): 13:02:28.616480105 [RX][if_index=29][00:30:05:4B:19:2C - 00:01:02:03:04:05] [vlan 12] [IPv4][10.12.150.231:2489 - 10.61.12.31:139] [l3_proto=TCP][hash=340372828][tos=0][tcp_seq_num=3256921511] [caplen=128][len=242][parsed_header_len=0][eth_offset=0][l3_offset=18][l4_offset=38][payload_offset=58] pfring_add_hash_filtering_rule(1) failed This error message indicates that pfring_add_hash_filtering_rule() returned failure in userland. Note that despite the returned error in userland, the hash rule has been added successfully in the kernel. Thanks, Amir On Mon, May 11, 2015 at 8:02 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir could you provide a sample application, or a code snippet, we can use to reproduce the issue? Thank you Alfredo On 11 May 2015, at 10:52, Amir Kaduri akadur...@gmail.com wrote: Hi, I'm experiencing a possible bug in the API pfring_handle_hash_filtering_rule(), when adding a rule. The API return -1 (failure) although the hash rule has been added successfully. I experience it in versions 6.0.2 and 6.0.3. In version 6.0.1 it works fine. Can anyone please confirm that there is a bug? If yes, can I have a patch for it, or even a quick-and-dirty explanation on how to solve it? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pfring filtering fails or not clear
Hi,
I would like to update that the problem in the tester program was that I
probably didn't add the vlan_id to rule. Once added, the hash rule began to
work.
Also, I would like to refer to the argument There is no limit to the
number of software hash rules:
I assume that the meaning is that the algorithm doesn't have any
limitation, though, since the rule_id field type is u_int16_t, there is a
limit of 64K hash rules, de-facto.
Thanks,
Amir
On Tue, Apr 14, 2015 at 12:34 PM, Alfredo Cardigliano cardigli...@ntop.org
wrote:
Hi Amir
yes thank you for the files, I will look at them ASAP.
Alfredo
On 14 Apr 2015, at 11:17, Amir Kaduri akadur...@gmail.com wrote:
Hi Alfredo,
I hope you saw what I've sent last week. Any chance that you will take a
look at it soon?
Thanks,
Amir
On Fri, Apr 10, 2015 at 11:08 AM, Amir Kaduri akadur...@gmail.com wrote:
Hi Alfredo,
Since the original pcap is huge, I sliced a smaller pcap from it,
holding 344 packets.
Link to the tester code (based on pfcount_82599.c):
https://drive.google.com/file/d/0B10Ms5GOXgCxdXhtRGRoQ1FTYVFDbkFGMkwzVExNRHJXbmt3/view?usp=sharing
Link to the pcap file (congtaining 344 packets):
https://drive.google.com/file/d/0B10Ms5GOXgCxS0dxR3lTZUoyRHZyUlpoemJfT0k2cS1QRGFr/view?usp=sharing
Thanks a lot,
Amir
On Wed, Apr 8, 2015 at 4:07 PM, Alfredo Cardigliano cardigli...@ntop.org
wrote:
Hi Amir
could you provide your patched pfcount and a pcap with the packets
matching your rule?
Alfredo
On 08 Apr 2015, at 13:36, Amir Kaduri akadur...@gmail.com wrote:
I'm currently in a situation, that no rules work for me whatsoever (same
configuration I described earlier), and they did work for me a few day ago.
I'm using the pfcount_82599.c tester, and added a single hash rule. The
only thing that does work is the default behavior, when using the API
pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive
normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no
packet arrive to the tester).
This is the part of code:
--
:
rc = pfring_enable_ring(pd);
if (rc0)
printf(pfring_enable_ring() failed. rc=%d\n, rc);
pfring_toggle_filtering_policy(pd, 1); /* Default to allow */
if (1) {
hash_filtering_rule rule;
u_int16_t rule_id = 0;
memset(rule, 0, sizeof(hash_filtering_rule));
rule.proto = 6;
rule.rule_id = rule_id++;
rule.rule_action = dont_forward_packet_and_stop_rule_evaluation;
rule.host4_peer_a = ntohl(inet_addr(10.12.150.231));
rule.port_peer_a = 2489;
rule.host4_peer_b = ntohl(inet_addr(10.61.12.31));
rule.port_peer_b = 139;
rc = pfring_handle_hash_filtering_rule(pd, rule, 1);
if(rc0)
printf(pfring_add_hash_filtering_rule(%d) failed [errno=%d: %s]\n,
rule.rule_id, errno, strerror(errno));
else
printf(pfring_add_hash_filtering_rule(%d) succeeded\n, rule.rule_id);
}
--
I don't get any errors when the following code run, but when I bombard
the machine
with a pcap containing more than 600,000 packets of the specified
session that I've expected to be filtered out,
packets of it still received at the tester.
I'm pretty sure that something goes wrong in pf_ring, but I can't tell
what.
What is the best way to get debug information, other than reading
/var/log/messages?
Thanks,
Amir
On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano
cardigli...@ntop.org wrote:
On 05 Apr 2015, at 16:07, Amir Kaduri akadur...@gmail.com wrote:
Hi,
I think I've made some progress: AFAIU, the packets that I see despite
the rule that supposed to filter them, are packets the receive during the
time interval from rule-set to rule-apply by pfring.
I'll appreciate getting some answers about the following:
1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way
to know which rules-ids are set and which are vacant?
This is because I have to follow the rules-ids when setting them,
but when I purge them, I don't know which of them were removed.
No, unfortunately this is not possible with the current API.
2. Does this API also purges HW rules?
No, It doesn’t.
3. According to the documentation, I know that HW rules have a limit of
32,000. What is the limit for hash rules? IS this limit includes the 32,000
of the HW, or additional to it?
There is no limit to the number of software hash rules.
4. I have a valid rule, but whenever I
call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
pfring_get_hash_filtering_rule_stats() should be used with sw rules to
get stats from kernel plugins (when used), otherwise there is no stats per
rule.
Br
Alfredo
- I've add the stats code to the pfcount_82599 tester
- In /var/log/messages I see the following message that is probably
originated from ring_setsockopt(): kernel: [PF_RING
Re: [Ntop-misc] pfring filtering fails or not clear
Hi, I think I've made some progress: AFAIU, the packets that I see despite the rule that supposed to filter them, are packets the receive during the time interval from rule-set to rule-apply by pfring. I'll appreciate getting some answers about the following: 1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way to know which rules-ids are set and which are vacant? This is because I have to follow the rules-ids when setting them, but when I purge them, I don't know which of them were removed. 2. Does this API also purges HW rules? 3. According to the documentation, I know that HW rules have a limit of 32,000. What is the limit for hash rules? IS this limit includes the 32,000 of the HW, or additional to it? 4. I have a valid rule, but whenever I call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why? - I've add the stats code to the pfcount_82599 tester - In /var/log/messages I see the following message that is probably originated from ring_setsockopt(): kernel: [PF_RING] Found rule but pluginId 0 is not registered Thanks, Amir On Thu, Apr 2, 2015 at 5:06 PM, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, Thanks for referring to my question. I hope the following answers: [root@CT10K10G]# cat /etc/pf_ring/pfring.conf min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1 enable_ip_defrag=1 [root@CT10K10G]# cat /proc/net/pf_ring/info PF_RING Version : 6.0.1 ($Revision: exported$) Total rings : 0 Standard (non DNA) Options Ring slots : 1024 Slot version : 15 Capture TX : Yes [RX+TX] IP Defragment: Yes Socket Mode : Standard Transparent mode : No [mode 2] Total plugins: 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 Thanks, Amir On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir how did you load pf_ring.ko? Can I see the command line? Please also try using latest code from svn, this helps us debugging the issue. Br Alfredo On 01 Apr 2015, at 18:22, Amir Kaduri akadur...@gmail.com wrote: Hello, I’m using PF_RING-6.0.1. I’m trying to develop an application that runs some algorithm consisting on rules. I made some tests using the “pfcount” tester, and unfortunately, I don’t understand the behavior: I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r –m” which AFAIU, adds a wildcard filter for each incoming packet. If I get it correctly, once a rule was added, I should not expect other packets of the same session to receive, and this is not what I’m getting. For example: --- [root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m Adding wildcard filtering rules Using PF_RING v.6.0.1 Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11] # Device RX channels: 6 # Polling threads:1 Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074 18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063] [caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 0 added successfully... 18:52:35.956301616 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596844523] [caplen=128][len=650][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 1 added successfully... 18:52:35.956303262 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596845111] [caplen=128][len=1086][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 2 added successfully... : --- How come, that once rule #0 was added for [10.61.10.9:52311 - 10.70.150.108:60189], I still see such packets in the next lines? Shouldn’t they be filtered by the rule that just as added? (BTW, when I use the command “./pfcount -i eth3 -u 1 -v 1 -r –m” (i.e. –u is 1 rather than 2), the tester uses hash filters, and in this case, I get errors: 18:53:19.052549112 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596847159] [caplen=128][len=1490][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] pfring_add_hash_filtering_rule(1) failed) Any help will be appreciated. Thanks, Amir
Re: [Ntop-misc] pfring filtering fails or not clear
I'm currently in a situation, that no rules work for me whatsoever (same
configuration I described earlier), and they did work for me a few day ago.
I'm using the pfcount_82599.c tester, and added a single hash rule. The
only thing that does work is the default behavior, when using the API
pfring_toggle_filtering_policy(pd, 1) (default to allow - packets arrive
normally) or pfring_toggle_filtering_policy(pd, 0); (default to drop, no
packet arrive to the tester).
This is the part of code:
--
:
rc = pfring_enable_ring(pd);
if (rc0)
printf(pfring_enable_ring() failed. rc=%d\n, rc);
pfring_toggle_filtering_policy(pd, 1); /* Default to allow */
if (1) {
hash_filtering_rule rule;
u_int16_t rule_id = 0;
memset(rule, 0, sizeof(hash_filtering_rule));
rule.proto = 6;
rule.rule_id = rule_id++;
rule.rule_action = dont_forward_packet_and_stop_rule_evaluation;
rule.host4_peer_a = ntohl(inet_addr(10.12.150.231));
rule.port_peer_a = 2489;
rule.host4_peer_b = ntohl(inet_addr(10.61.12.31));
rule.port_peer_b = 139;
rc = pfring_handle_hash_filtering_rule(pd, rule, 1);
if(rc0)
printf(pfring_add_hash_filtering_rule(%d) failed [errno=%d: %s]\n,
rule.rule_id, errno, strerror(errno));
else
printf(pfring_add_hash_filtering_rule(%d) succeeded\n, rule.rule_id);
}
--
I don't get any errors when the following code run, but when I bombard the
machine
with a pcap containing more than 600,000 packets of the specified session
that I've expected to be filtered out,
packets of it still received at the tester.
I'm pretty sure that something goes wrong in pf_ring, but I can't tell what.
What is the best way to get debug information, other than reading
/var/log/messages?
Thanks,
Amir
On Wed, Apr 8, 2015 at 1:08 PM, Alfredo Cardigliano cardigli...@ntop.org
wrote:
On 05 Apr 2015, at 16:07, Amir Kaduri akadur...@gmail.com wrote:
Hi,
I think I've made some progress: AFAIU, the packets that I see despite the
rule that supposed to filter them, are packets the receive during the time
interval from rule-set to rule-apply by pfring.
I'll appreciate getting some answers about the following:
1. If I use the pfring_purge_idle_hash_rules(..) API, is there any way to
know which rules-ids are set and which are vacant?
This is because I have to follow the rules-ids when setting them, but
when I purge them, I don't know which of them were removed.
No, unfortunately this is not possible with the current API.
2. Does this API also purges HW rules?
No, It doesn’t.
3. According to the documentation, I know that HW rules have a limit of
32,000. What is the limit for hash rules? IS this limit includes the 32,000
of the HW, or additional to it?
There is no limit to the number of software hash rules.
4. I have a valid rule, but whenever I
call pfring_get_hash_filtering_rule_stats(..), it fails.Any idea why?
pfring_get_hash_filtering_rule_stats() should be used with sw rules to get
stats from kernel plugins (when used), otherwise there is no stats per rule.
Br
Alfredo
- I've add the stats code to the pfcount_82599 tester
- In /var/log/messages I see the following message that is probably
originated from ring_setsockopt(): kernel: [PF_RING] Found rule but
pluginId 0 is not registered
Thanks,
Amir
On Thu, Apr 2, 2015 at 5:06 PM, Amir Kaduri akadur...@gmail.com wrote:
Hi Alfredo,
Thanks for referring to my question.
I hope the following answers:
[root@CT10K10G]# cat /etc/pf_ring/pfring.conf
min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1
enable_ip_defrag=1
[root@CT10K10G]# cat /proc/net/pf_ring/info
PF_RING Version : 6.0.1 ($Revision: exported$)
Total rings : 0
Standard (non DNA) Options
Ring slots : 1024
Slot version : 15
Capture TX : Yes [RX+TX]
IP Defragment: Yes
Socket Mode : Standard
Transparent mode : No [mode 2]
Total plugins: 0
Cluster Fragment Queue : 0
Cluster Fragment Discard : 0
Thanks,
Amir
On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano cardigli...@ntop.org
wrote:
Hi Amir
how did you load pf_ring.ko? Can I see the command line?
Please also try using latest code from svn, this helps us debugging the
issue.
Br
Alfredo
On 01 Apr 2015, at 18:22, Amir Kaduri akadur...@gmail.com wrote:
Hello,
I’m using PF_RING-6.0.1.
I’m trying to develop an application that runs some algorithm consisting
on rules.
I made some tests using the “pfcount” tester, and unfortunately, I don’t
understand the behavior:
I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r
–m” which AFAIU, adds a wildcard filter for each incoming packet.
If I get it correctly, once a rule was added, I should not expect other
packets of the same session
Re: [Ntop-misc] pfring filtering fails or not clear
Hi Alfredo, Thanks for referring to my question. I hope the following answers: [root@CT10K10G]# cat /etc/pf_ring/pfring.conf min_num_slots=1024 transparent_mode=2 enable_frag_coherence=1 enable_ip_defrag=1 [root@CT10K10G]# cat /proc/net/pf_ring/info PF_RING Version : 6.0.1 ($Revision: exported$) Total rings : 0 Standard (non DNA) Options Ring slots : 1024 Slot version : 15 Capture TX : Yes [RX+TX] IP Defragment: Yes Socket Mode : Standard Transparent mode : No [mode 2] Total plugins: 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 Thanks, Amir On Thu, Apr 2, 2015 at 4:10 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir how did you load pf_ring.ko? Can I see the command line? Please also try using latest code from svn, this helps us debugging the issue. Br Alfredo On 01 Apr 2015, at 18:22, Amir Kaduri akadur...@gmail.com wrote: Hello, I’m using PF_RING-6.0.1. I’m trying to develop an application that runs some algorithm consisting on rules. I made some tests using the “pfcount” tester, and unfortunately, I don’t understand the behavior: I’m running the following command line: “./pfcount -i eth3 -u 2 -v 1 -r –m” which AFAIU, adds a wildcard filter for each incoming packet. If I get it correctly, once a rule was added, I should not expect other packets of the same session to receive, and this is not what I’m getting. For example: --- [root@CT10K10G examples]# ./pfcount -i eth3 -u 2 -v 1 -r -m Adding wildcard filtering rules Using PF_RING v.6.0.1 Capturing from eth3 [00:E0:ED:FE:18:19][ifIndex: 11] # Device RX channels: 6 # Polling threads:1 Dumping statistics on /proc/net/pf_ring/stats/11993-eth3.1074 18:52:35.956295950 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596843063] [caplen=128][len=1522][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 0 added successfully... 18:52:35.956301616 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596844523] [caplen=128][len=650][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 1 added successfully... 18:52:35.956303262 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596845111] [caplen=128][len=1086][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] Rule 2 added successfully... : --- How come, that once rule #0 was added for [10.61.10.9:52311 - 10.70.150.108:60189], I still see such packets in the next lines? Shouldn’t they be filtered by the rule that just as added? (BTW, when I use the command “./pfcount -i eth3 -u 1 -v 1 -r –m” (i.e. –u is 1 rather than 2), the tester uses hash filters, and in this case, I get errors: 18:53:19.052549112 [RX][if_index=11][00:08:E3:FF:FC:C8 - 00:01:02:03:04:05] [vlan 70] [direction 1] [IPv4][10.61.10.9:52311 - 10.70.150.108:60189] [l3_proto=TCP][hash=344283189][tos=0][tcp_seq_num=596847159] [caplen=128][len=1490][parsed_header_len=0][eth_offset=-14][l3_offset=18][l4_offset=38][payload_offset=58] pfring_add_hash_filtering_rule(1) failed) Any help will be appreciated. Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Possible bug on rss rehash
I'll appreciate validating that the following is a real problem that I've found: *Problem description:* When I use the tester “pfcount_multichannel” with the –r argument (i.e. enable rss rehash), the application doesn’t receive the packets of some sessions. When I remove the “-r” argument from the command line, all packets received as expected. *Additional information:* 1. 1. I’m using pf_ring dev downloaded from github on 13/05/2015. 2. 2. The command-line I’m using is: pfcount_multichannel -i eth3 –r 3. 3. A sample pcap file of a single session, that I don’t receive its packets, can be found in the following link: https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazAauthuser=0 4. 4. Machine has 24 cores. 5. 5. Eth3, the receiving interface, is configured to use 6 cores. 6. 6. Machine details: Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 7. 7. Interface details: eth3 Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c] MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF 8. 8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6 9. 9. cat /proc/net/pf_ring/info PF_RING Version : 6.1.0 () Total rings : 6 Standard (non DNA/ZC) Options Ring slots : 4096 Slot version : 16 Capture TX : Yes [RX+TX] IP Defragment: No Socket Mode : Standard Total plugins: 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 *Additional remarks:* 1. 1. In function pfring_hash_pkt(..) in file pfring_utils.c, the fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a bug. 2. 2. By reviewing the code, it looks that the functions hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified. Regards, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Possible bug on rss rehash
Hi Alfredo, Any chance that you'll take a look at this issue? At least give an indication whether its a real problem or not? Thanks, Amir On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri akadur...@gmail.com wrote: I'll appreciate validating that the following is a real problem that I've found: *Problem description:* When I use the tester “pfcount_multichannel” with the –r argument (i.e. enable rss rehash), the application doesn’t receive the packets of some sessions. When I remove the “-r” argument from the command line, all packets received as expected. *Additional information:* 1. 1. I’m using pf_ring dev downloaded from github on 13/05/2015. 2. 2. The command-line I’m using is: pfcount_multichannel -i eth3 –r 3. 3. A sample pcap file of a single session, that I don’t receive its packets, can be found in the following link: https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazAauthuser=0 4. 4. Machine has 24 cores. 5. 5. Eth3, the receiving interface, is configured to use 6 cores. 6. 6. Machine details: Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 7. 7. Interface details: eth3 Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c] MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF 8. 8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6 9. 9. cat /proc/net/pf_ring/info PF_RING Version : 6.1.0 () Total rings : 6 Standard (non DNA/ZC) Options Ring slots : 4096 Slot version : 16 Capture TX : Yes [RX+TX] IP Defragment: No Socket Mode : Standard Total plugins: 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 *Additional remarks:* 1. 1. In function pfring_hash_pkt(..) in file pfring_utils.c, the fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a bug. 2. 2. By reviewing the code, it looks that the functions hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified. Regards, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Possible bug on rss rehash
Hi Alfredo, Per your suggestion, I've downloaded today the latest (dev) pf_ring code, compiled and repeated the test. Unfortunately, the problem repeats itself: without -r the pfcount_multichannel receives the packets, while with -r, it doesn't. Any thoughts will be much appreciated. Thanks, Amir On Wed, Jul 1, 2015 at 7:22 AM, Amir Kaduri akadur...@gmail.com wrote: Thank you Alfredo for your response. Soon I'll test again with latest code. Amir On Tue, Jun 30, 2015 at 7:54 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir sorry I didn’t have the testbed for testing this before, I ran some test with your traffic and your configuration (actually I used 8 RSS queues) and I am able to receive all traffic with pfcount_multichannel -r. Please try with latest code. Best Regards Alfredo On 02 Jun 2015, at 08:25, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, Any chance that you'll take a look at this issue? At least give an indication whether its a real problem or not? Thanks, Amir On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri akadur...@gmail.com wrote: I'll appreciate validating that the following is a real problem that I've found: *Problem description:* When I use the tester “pfcount_multichannel” with the –r argument (i.e. enable rss rehash), the application doesn’t receive the packets of some sessions. When I remove the “-r” argument from the command line, all packets received as expected. *Additional information:* 1. 1. I’m using pf_ring dev downloaded from github on 13/05/2015. 2. 2. The command-line I’m using is: pfcount_multichannel -i eth3 –r 3. 3. A sample pcap file of a single session, that I don’t receive its packets, can be found in the following link: https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazAauthuser=0 4. 4. Machine has 24 cores. 5. 5. Eth3, the receiving interface, is configured to use 6 cores. 6. 6. Machine details: Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 7. 7. Interface details: eth3 Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c] MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF 8. 8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6 9. 9. cat /proc/net/pf_ring/info PF_RING Version : 6.1.0 () Total rings : 6 Standard (non DNA/ZC) Options Ring slots : 4096 Slot version : 16 Capture TX : Yes [RX+TX] IP Defragment: No Socket Mode : Standard Total plugins: 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 *Additional remarks:* 1. 1. In function pfring_hash_pkt(..) in file pfring_utils.c, the fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a bug. 2. 2. By reviewing the code, it looks that the functions hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified. Regards, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Possible bug on rss rehash
Hi Alfredo, I would like to add that I repeated the test with RSS=8,8,8,8,8,8,8,8 like you, and now it works. I guess that If you'll do it like with (with RSS 6) you'll notice the problem as well. Thanks, Amir On Thu, Jul 2, 2015 at 6:06 PM, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, Per your suggestion, I've downloaded today the latest (dev) pf_ring code, compiled and repeated the test. Unfortunately, the problem repeats itself: without -r the pfcount_multichannel receives the packets, while with -r, it doesn't. Any thoughts will be much appreciated. Thanks, Amir On Wed, Jul 1, 2015 at 7:22 AM, Amir Kaduri akadur...@gmail.com wrote: Thank you Alfredo for your response. Soon I'll test again with latest code. Amir On Tue, Jun 30, 2015 at 7:54 PM, Alfredo Cardigliano cardigli...@ntop.org wrote: Hi Amir sorry I didn’t have the testbed for testing this before, I ran some test with your traffic and your configuration (actually I used 8 RSS queues) and I am able to receive all traffic with pfcount_multichannel -r. Please try with latest code. Best Regards Alfredo On 02 Jun 2015, at 08:25, Amir Kaduri akadur...@gmail.com wrote: Hi Alfredo, Any chance that you'll take a look at this issue? At least give an indication whether its a real problem or not? Thanks, Amir On Wed, May 27, 2015 at 4:59 PM, Amir Kaduri akadur...@gmail.com wrote: I'll appreciate validating that the following is a real problem that I've found: *Problem description:* When I use the tester “pfcount_multichannel” with the –r argument (i.e. enable rss rehash), the application doesn’t receive the packets of some sessions. When I remove the “-r” argument from the command line, all packets received as expected. *Additional information:* 1. 1. I’m using pf_ring dev downloaded from github on 13/05/2015. 2. 2. The command-line I’m using is: pfcount_multichannel -i eth3 –r 3. 3. A sample pcap file of a single session, that I don’t receive its packets, can be found in the following link: https://drive.google.com/open?id=0B10Ms5GOXgCxbnJSUHNrUXhLazAauthuser=0 4. 4. Machine has 24 cores. 5. 5. Eth3, the receiving interface, is configured to use 6 cores. 6. 6. Machine details: Linux 2.6.32-504.8.1.el6.x86_64 #1 SMP Wed Jan 28 21:11:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux 7. 7. Interface details: eth3 Ethernet controller [0200]: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01) Subsystem: Intel Corporation Ethernet Server Adapter X520-2 [8086:000c] MAC: 2, PHY: 12, SFP+: 6, PBA No: FF-0FF 8. 8. The ixgbe.ko has been loaded with RSS=6,6,6,6,6,6,6 9. 9. cat /proc/net/pf_ring/info PF_RING Version : 6.1.0 () Total rings : 6 Standard (non DNA/ZC) Options Ring slots : 4096 Slot version : 16 Capture TX : Yes [RX+TX] IP Defragment: No Socket Mode : Standard Total plugins: 0 Cluster Fragment Queue : 0 Cluster Fragment Discard : 0 *Additional remarks:* 1. 1. In function pfring_hash_pkt(..) in file pfring_utils.c, the fact that tunnel.tunneled_ip_src.v6.s6_addr32[0] is missing, looks like a bug. 2. 2. By reviewing the code, it looks that the functions hash_pkt_header(..) + hash_pkt(..) in file pf_ring.c, and function pfring_hash_pkt(..) in file pfring_utils.c, can somehow be unified. Regards, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Two following questions please: 1. fdir_miss (of ethtool -S) continues to increase even if my process doesn't apply the filter rules. Do you know what else causes it to increase if its not a result of a filter? I didn't find an answer for it in the following doc: https://www.kernel.org/doc/Documentation/networking/ixgbe.txt 2. Should I expect the mentioned feature request to be implemented in pf_ring soon, or its most probably be there with many other prioritized requests? Thanks, Amir On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri <akadur...@gmail.com> wrote: > Thanks for the answer. It looks to me very essential statistics for > user-space applications. > Github feature request: https://github.com/ntop/PF_RING/issues/52 > > Amir > > > On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano <cardigli...@ntop.org> > wrote: > >> Hi Amir >> there is no statistics by default (unless you code your own kernel plugin >> and read stats via pfring_get_hash_filtering_rule_stats, but this is >> overkilling for match/miss counters), >> please open a feature request on github. >> >> Alfredo >> >> On 09 Nov 2015, at 18:28, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Hi, >> >> In my application, I’m using pf_ring hash software filters (API: >> pfring_handle_hash_filtering_rule()). >> >> How can I get statistics info regarding the quantity of the filtered >> packets? >> >> I know that on ixgbe module, I can get the following parameters when >> using ethtool –s : >> >> fdir_match, fdir_miss, fdir_overflow. >> >> Are there any pf_ring user-space APIs to get these parameters? Any code >> examples? >> >> >> >> Thanks, >> >> Amir >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> >> >> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] How to get pf_ring packets filtering statistics
Hi, In my application, I’m using pf_ring hash software filters (API: pfring_handle_hash_filtering_rule()). How can I get statistics info regarding the quantity of the filtered packets? I know that on ixgbe module, I can get the following parameters when using ethtool –s : fdir_match, fdir_miss, fdir_overflow. Are there any pf_ring user-space APIs to get these parameters? Any code examples? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Hi Alfredo, Having it per interface under /proc is excellent. Thanks, Amir On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > > On 16 Nov 2015, at 10:53, Amir Kaduri <akadur...@gmail.com> wrote: > > Hi Alfredo, > > > I reviewed the implementation of the feature (great!) and I would like to > clarify please: > > 1. If I understood it correctly, the number of matched packets is per > given rule, while the number of missed packets is per ring (and not per the > given rule). Is it correct, or am I wrong? > > If I’m correct, can I provide a dummy rule and get the number of > missed packets on the given ring? > > Correct > > 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX | > grep fdir_match” you actually get the aggregation of missed/matched packets > on the interface (ethX). > > Is it possible to get the pf_ring statistics aggregated as well, per > interface, or at least per ring? > > Is it ok for you if we write this stats under /proc instead of adding > another API call? > > Alfredo > > > > Thanks, > > Amir > > On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <cardigli...@ntop.org > > wrote: > >> >> On 11 Nov 2015, at 18:02, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Two following questions please: >> 1. fdir_miss (of ethtool -S) continues to increase even if my process >> doesn't apply the filter rules. Do you know what else causes it to increase >> if its not a result of a filter? >> I didn't find an answer for it in the following doc: >> https://www.kernel.org/doc/Documentation/networking/ixgbe.txt >> >> >> Please have a look at the datasheet for all cases. >> >> 2. Should I expect the mentioned feature request to be implemented in >> pf_ring soon, or its most probably be there with many other prioritized >> requests? >> >> >> It is in queue with other feature requests, we handle them in best effort, >> however I think it will not require too much (1-2 weeks at most) >> >> Alfredo >> >> >> Thanks, >> Amir >> >> On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri <akadur...@gmail.com> wrote: >> >>> Thanks for the answer. It looks to me very essential statistics for >>> user-space applications. >>> Github feature request: https://github.com/ntop/PF_RING/issues/52 >>> >>> Amir >>> >>> >>> On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano < >>> cardigli...@ntop.org> wrote: >>> >>>> Hi Amir >>>> there is no statistics by default (unless you code your own kernel >>>> plugin and read stats via pfring_get_hash_filtering_rule_stats, but this is >>>> overkilling for match/miss counters), >>>> please open a feature request on github. >>>> >>>> Alfredo >>>> >>>> On 09 Nov 2015, at 18:28, Amir Kaduri <akadur...@gmail.com> wrote: >>>> >>>> Hi, >>>> >>>> In my application, I’m using pf_ring hash software filters (API: >>>> pfring_handle_hash_filtering_rule()). >>>> >>>> How can I get statistics info regarding the quantity of the filtered >>>> packets? >>>> >>>> I know that on ixgbe module, I can get the following parameters when >>>> using ethtool –s : >>>> >>>> fdir_match, fdir_miss, fdir_overflow. >>>> >>>> Are there any pf_ring user-space APIs to get these parameters? Any code >>>> examples? >>>> >>>> >>>> >>>> Thanks, >>>> >>>> Amir >>>> ___ >>>> Ntop-misc mailing list >>>> Ntop-misc@listgateway.unipi.it >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>> >>>> >>>> >>>> ___ >>>> Ntop-misc mailing list >>>> Ntop-misc@listgateway.unipi.it >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>> >>> >>> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> >> >> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Hi Alfredo, I reviewed the implementation of the feature (great!) and I would like to clarify please: 1. If I understood it correctly, the number of matched packets is per given rule, while the number of missed packets is per ring (and not per the given rule). Is it correct, or am I wrong? If I’m correct, can I provide a dummy rule and get the number of missed packets on the given ring? 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX | grep fdir_match” you actually get the aggregation of missed/matched packets on the interface (ethX). Is it possible to get the pf_ring statistics aggregated as well, per interface, or at least per ring? Thanks, Amir On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > > On 11 Nov 2015, at 18:02, Amir Kaduri <akadur...@gmail.com> wrote: > > Two following questions please: > 1. fdir_miss (of ethtool -S) continues to increase even if my process > doesn't apply the filter rules. Do you know what else causes it to increase > if its not a result of a filter? > I didn't find an answer for it in the following doc: > https://www.kernel.org/doc/Documentation/networking/ixgbe.txt > > > Please have a look at the datasheet for all cases. > > 2. Should I expect the mentioned feature request to be implemented in > pf_ring soon, or its most probably be there with many other prioritized > requests? > > > It is in queue with other feature requests, we handle them in best effort, > however I think it will not require too much (1-2 weeks at most) > > Alfredo > > > Thanks, > Amir > > On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri <akadur...@gmail.com> wrote: > >> Thanks for the answer. It looks to me very essential statistics for >> user-space applications. >> Github feature request: https://github.com/ntop/PF_RING/issues/52 >> >> Amir >> >> >> On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano <cardigli...@ntop.org >> > wrote: >> >>> Hi Amir >>> there is no statistics by default (unless you code your own kernel >>> plugin and read stats via pfring_get_hash_filtering_rule_stats, but this is >>> overkilling for match/miss counters), >>> please open a feature request on github. >>> >>> Alfredo >>> >>> On 09 Nov 2015, at 18:28, Amir Kaduri <akadur...@gmail.com> wrote: >>> >>> Hi, >>> >>> In my application, I’m using pf_ring hash software filters (API: >>> pfring_handle_hash_filtering_rule()). >>> >>> How can I get statistics info regarding the quantity of the filtered >>> packets? >>> >>> I know that on ixgbe module, I can get the following parameters when >>> using ethtool –s : >>> >>> fdir_match, fdir_miss, fdir_overflow. >>> >>> Are there any pf_ring user-space APIs to get these parameters? Any code >>> examples? >>> >>> >>> >>> Thanks, >>> >>> Amir >>> ___ >>> Ntop-misc mailing list >>> Ntop-misc@listgateway.unipi.it >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >>> >>> >>> ___ >>> Ntop-misc mailing list >>> Ntop-misc@listgateway.unipi.it >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >> >> > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Thanks for the answer. It looks to me very essential statistics for user-space applications. Github feature request: https://github.com/ntop/PF_RING/issues/52 Amir On Mon, Nov 9, 2015 at 7:34 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > there is no statistics by default (unless you code your own kernel plugin > and read stats via pfring_get_hash_filtering_rule_stats, but this is > overkilling for match/miss counters), > please open a feature request on github. > > Alfredo > > On 09 Nov 2015, at 18:28, Amir Kaduri <akadur...@gmail.com> wrote: > > Hi, > > In my application, I’m using pf_ring hash software filters (API: > pfring_handle_hash_filtering_rule()). > > How can I get statistics info regarding the quantity of the filtered > packets? > > I know that on ixgbe module, I can get the following parameters when using > ethtool –s : > > fdir_match, fdir_miss, fdir_overflow. > > Are there any pf_ring user-space APIs to get these parameters? Any code > examples? > > > > Thanks, > > Amir > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Hi Alfredo,
(1) Per your fix to update /proc/net/pf_ring with match+miss+rules
statistics of sw hash rules, I've noticed the following:
A received packet that doesn't match any rule, will cause an increase of
the counter ("Sw Filt Hash Miss") of all rings that have at least 1 rule,
rather than the relevant ring only.
I assume its not the intention, right?
(2) ethtool have also the "fdir_overflow" property. Is it possible to add
this statistics to /proc per ring, together with the other statistics
you've added?
(Note that I just want to know if pf_ring have enough info to do it
or not).
Thanks,
Amir
On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
>
> On 17 Nov 2015, at 15:54, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Hi Alfredo,
>
> Thanks for that.
> 1. stats per socket means stats per ring, correct?
>
>
> Yes
>
> 2. I would like to know the best way to read this info from /proc.
> Is there any dedicated API for that?
> I took a look at /proc on a machine running an application using
> pf_ring, and found many files containing
> the stats that appear in function proc_get_info(..). I would like to
> better understand the convention of files
> to read, in order to have calculate accurate aggregated stats.
>
>
> /proc/net/pf_ring/* contains one file per ring, the name contains the
> process pid and ring id.
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <cardigli...@ntop.org
> > wrote:
>
>> Added to git code, please note stats are per socket, not per interface.
>>
>> Alfredo
>>
>> On 16 Nov 2015, at 13:48, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> Having it per interface under /proc is excellent.
>>
>> Thanks,
>> Amir
>>
>> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>> On 16 Nov 2015, at 10:53, Amir Kaduri <akadur...@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>>
>>> I reviewed the implementation of the feature (great!) and I would like
>>> to clarify please:
>>>
>>> 1. If I understood it correctly, the number of matched packets is per
>>> given rule, while the number of missed packets is per ring (and not per the
>>> given rule). Is it correct, or am I wrong?
>>>
>>> If I’m correct, can I provide a dummy rule and get the number of
>>> missed packets on the given ring?
>>>
>>> Correct
>>>
>>> 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX |
>>> grep fdir_match” you actually get the aggregation of missed/matched packets
>>> on the interface (ethX).
>>>
>>> Is it possible to get the pf_ring statistics aggregated as well,
>>> per interface, or at least per ring?
>>>
>>> Is it ok for you if we write this stats under /proc instead of adding
>>> another API call?
>>>
>>> Alfredo
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Amir
>>>
>>> On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 11 Nov 2015, at 18:02, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>
>>>> Two following questions please:
>>>> 1. fdir_miss (of ethtool -S) continues to increase even if my process
>>>> doesn't apply the filter rules. Do you know what else causes it to increase
>>>> if its not a result of a filter?
>>>> I didn't find an answer for it in the following doc:
>>>> https://www.kernel.org/doc/Documentation/networking/ixgbe.txt
>>>>
>>>>
>>>> Please have a look at the datasheet for all cases.
>>>>
>>>> 2. Should I expect the mentioned feature request to be implemented in
>>>> pf_ring soon, or its most probably be there with many other prioritized
>>>> requests?
>>>>
>>>>
>>>> It is in queue with other feature requests, we handle them in best
>>>> effort,
>>>> however I think it will not require too much (1-2 weeks at most)
>>>>
>>>> Alfredo
>>>>
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Tue, Nov 10, 2015 at 3:32 PM, Amir Kaduri <akadur...@gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks for the answe
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Hi,
In regard of the problem that missed packets counted on all rings instead
of on the relevant ring only:
We use a quite simple configuration: We use PF_RING_Aware, splitting the
incoming traffic to multiple queues (just like the tester
pfcount_multichannel does),
currently we have 2 ixgbe interfaces, and the packets physically arrive to
only one of them.
Amir
On Thu, Dec 3, 2015 at 3:41 PM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
>
> On 03 Dec 2015, at 13:44, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Hi Alfredo,
>
> (1) Per your fix to update /proc/net/pf_ring with match+miss+rules
> statistics of sw hash rules, I've noticed the following:
> A received packet that doesn't match any rule, will cause an increase of
> the counter ("Sw Filt Hash Miss") of all rings that have at least 1 rule,
> rather than the relevant ring only.
> I assume its not the intention, right?
>
>
> Are all those ring bound to the same interface the packet is coming from?
> Please provide more details about the configuration.
>
> (2) ethtool have also the "fdir_overflow" property. Is it possible to add
> this statistics to /proc per ring, together with the other statistics
> you've added?
> (Note that I just want to know if pf_ring have enough info to do it
> or not).
>
>
> pf_ring does not usually use those info, they need to be retrieved
> somehow: do you really need the same stats in /proc?
>
> Alfredo
>
>
> Thanks,
> Amir
>
> On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano <cardigli...@ntop.org
> > wrote:
>
>>
>> On 17 Nov 2015, at 15:54, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> Thanks for that.
>> 1. stats per socket means stats per ring, correct?
>>
>>
>> Yes
>>
>> 2. I would like to know the best way to read this info from /proc.
>> Is there any dedicated API for that?
>> I took a look at /proc on a machine running an application using
>> pf_ring, and found many files containing
>> the stats that appear in function proc_get_info(..). I would like to
>> better understand the convention of files
>> to read, in order to have calculate accurate aggregated stats.
>>
>>
>> /proc/net/pf_ring/* contains one file per ring, the name contains the
>> process pid and ring id.
>>
>> Alfredo
>>
>>
>> Thanks,
>> Amir
>>
>> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Added to git code, please note stats are per socket, not per interface.
>>>
>>> Alfredo
>>>
>>> On 16 Nov 2015, at 13:48, Amir Kaduri <akadur...@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Having it per interface under /proc is excellent.
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 16 Nov 2015, at 10:53, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>>
>>>> I reviewed the implementation of the feature (great!) and I would like
>>>> to clarify please:
>>>>
>>>> 1. If I understood it correctly, the number of matched packets is per
>>>> given rule, while the number of missed packets is per ring (and not per the
>>>> given rule). Is it correct, or am I wrong?
>>>>
>>>> If I’m correct, can I provide a dummy rule and get the number of
>>>> missed packets on the given ring?
>>>>
>>>> Correct
>>>>
>>>> 2. When reading “ethtool –S ethX | grep fdir_miss” or “ethtool –S ethX
>>>> | grep fdir_match” you actually get the aggregation of missed/matched
>>>> packets on the interface (ethX).
>>>>
>>>> Is it possible to get the pf_ring statistics aggregated as well,
>>>> per interface, or at least per ring?
>>>>
>>>> Is it ok for you if we write this stats under /proc instead of adding
>>>> another API call?
>>>>
>>>> Alfredo
>>>>
>>>>
>>>>
>>>> Thanks,
>>>>
>>>> Amir
>>>>
>>>> On Wed, Nov 11, 2015 at 8:01 PM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>>
>>>>> On 11 Nov 2015, at 18:02, Amir Kaduri <akadur...@gmail.com> wro
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Hi,
Another problem in the counters (that were added to
function add_skb_to_ring()) is that it doesn't refer to rss-rehash:
If rss-rehash is disabled, and there are 2 rings involved in a session
(each for a direction), and each of them holds a sw-hash-rule,
then the sw_filtering_hash_match counter will be incremented in both of
them.
For example: Assume rss-rehash is disabled, and the machine receives 10,000
packets of a single session (5000 packets of each direction).
Each ring actually received 5000 packets, but each of them will report
10,000 packets filtered which is a total of 20,000 packets.
So to summarize the problems in the fix in add_skb_to_ring():
1. On miss, each ring that has at least 1 rule will increment the counter
("Sw Filt Hash Miss" in /proc)
2. On match, when rss is disabled, both rings of both directions will
increment their counter ("Sw Filt Hash Match" in /proc)
Thanks,
Amir
On Mon, Dec 7, 2015 at 2:10 PM, Amir Kaduri <akadur...@gmail.com> wrote:
> Hi,
>
> In regard of the problem that missed packets counted on all rings instead
> of on the relevant ring only:
> We use a quite simple configuration: We use PF_RING_Aware, splitting the
> incoming traffic to multiple queues (just like the tester
> pfcount_multichannel does),
> currently we have 2 ixgbe interfaces, and the packets physically arrive to
> only one of them.
>
> Amir
>
> On Thu, Dec 3, 2015 at 3:41 PM, Alfredo Cardigliano <cardigli...@ntop.org>
> wrote:
>
>>
>> On 03 Dec 2015, at 13:44, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> (1) Per your fix to update /proc/net/pf_ring with match+miss+rules
>> statistics of sw hash rules, I've noticed the following:
>> A received packet that doesn't match any rule, will cause an increase of
>> the counter ("Sw Filt Hash Miss") of all rings that have at least 1
>> rule, rather than the relevant ring only.
>> I assume its not the intention, right?
>>
>>
>> Are all those ring bound to the same interface the packet is coming from?
>> Please provide more details about the configuration.
>>
>> (2) ethtool have also the "fdir_overflow" property. Is it possible to add
>> this statistics to /proc per ring, together with the other statistics
>> you've added?
>> (Note that I just want to know if pf_ring have enough info to do it
>> or not).
>>
>>
>> pf_ring does not usually use those info, they need to be retrieved
>> somehow: do you really need the same stats in /proc?
>>
>> Alfredo
>>
>>
>> Thanks,
>> Amir
>>
>> On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>> On 17 Nov 2015, at 15:54, Amir Kaduri <akadur...@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> Thanks for that.
>>> 1. stats per socket means stats per ring, correct?
>>>
>>>
>>> Yes
>>>
>>> 2. I would like to know the best way to read this info from /proc.
>>> Is there any dedicated API for that?
>>> I took a look at /proc on a machine running an application using
>>> pf_ring, and found many files containing
>>> the stats that appear in function proc_get_info(..). I would like to
>>> better understand the convention of files
>>> to read, in order to have calculate accurate aggregated stats.
>>>
>>>
>>> /proc/net/pf_ring/* contains one file per ring, the name contains the
>>> process pid and ring id.
>>>
>>> Alfredo
>>>
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>> Added to git code, please note stats are per socket, not per interface.
>>>>
>>>> Alfredo
>>>>
>>>> On 16 Nov 2015, at 13:48, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Having it per interface under /proc is excellent.
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Mon, Nov 16, 2015 at 11:56 AM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>>
>>>>> On 16 Nov 2015, at 10:53, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>>
>>>>> Hi Alfredo,
>>>>>
>>>>>
>>>>> I reviewed the implementation of the feature (great!) and I would like
>
Re: [Ntop-misc] How to get pf_ring packets filtering statistics
Hi Alfredo,
Even if these two issues are not going to be fixed soon, I just want to
know that you are aware of them.
Thanks,
Amir
On Tue, Dec 8, 2015 at 9:53 AM, Amir Kaduri <akadur...@gmail.com> wrote:
> Hi,
>
> Another problem in the counters (that were added to
> function add_skb_to_ring()) is that it doesn't refer to rss-rehash:
> If rss-rehash is disabled, and there are 2 rings involved in a session
> (each for a direction), and each of them holds a sw-hash-rule,
> then the sw_filtering_hash_match counter will be incremented in both of
> them.
> For example: Assume rss-rehash is disabled, and the machine receives
> 10,000 packets of a single session (5000 packets of each direction).
> Each ring actually received 5000 packets, but each of them will report
> 10,000 packets filtered which is a total of 20,000 packets.
>
> So to summarize the problems in the fix in add_skb_to_ring():
> 1. On miss, each ring that has at least 1 rule will increment the counter
> ("Sw Filt Hash Miss" in /proc)
> 2. On match, when rss is disabled, both rings of both directions will
> increment their counter ("Sw Filt Hash Match" in /proc)
>
> Thanks,
> Amir
>
>
>
>
> On Mon, Dec 7, 2015 at 2:10 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>
>> Hi,
>>
>> In regard of the problem that missed packets counted on all rings instead
>> of on the relevant ring only:
>> We use a quite simple configuration: We use PF_RING_Aware, splitting the
>> incoming traffic to multiple queues (just like the tester
>> pfcount_multichannel does),
>> currently we have 2 ixgbe interfaces, and the packets physically arrive
>> to only one of them.
>>
>> Amir
>>
>> On Thu, Dec 3, 2015 at 3:41 PM, Alfredo Cardigliano <cardigli...@ntop.org
>> > wrote:
>>
>>>
>>> On 03 Dec 2015, at 13:44, Amir Kaduri <akadur...@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> (1) Per your fix to update /proc/net/pf_ring with match+miss+rules
>>> statistics of sw hash rules, I've noticed the following:
>>> A received packet that doesn't match any rule, will cause an increase of
>>> the counter ("Sw Filt Hash Miss") of all rings that have at least 1
>>> rule, rather than the relevant ring only.
>>> I assume its not the intention, right?
>>>
>>>
>>> Are all those ring bound to the same interface the packet is coming
>>> from? Please provide more details about the configuration.
>>>
>>> (2) ethtool have also the "fdir_overflow" property. Is it possible to
>>> add this statistics to /proc per ring, together with the other statistics
>>> you've added?
>>> (Note that I just want to know if pf_ring have enough info to do
>>> it or not).
>>>
>>>
>>> pf_ring does not usually use those info, they need to be retrieved
>>> somehow: do you really need the same stats in /proc?
>>>
>>> Alfredo
>>>
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Tue, Nov 17, 2015 at 5:04 PM, Alfredo Cardigliano <
>>> cardigli...@ntop.org> wrote:
>>>
>>>>
>>>> On 17 Nov 2015, at 15:54, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> Thanks for that.
>>>> 1. stats per socket means stats per ring, correct?
>>>>
>>>>
>>>> Yes
>>>>
>>>> 2. I would like to know the best way to read this info from /proc.
>>>> Is there any dedicated API for that?
>>>> I took a look at /proc on a machine running an application using
>>>> pf_ring, and found many files containing
>>>> the stats that appear in function proc_get_info(..). I would like
>>>> to better understand the convention of files
>>>> to read, in order to have calculate accurate aggregated stats.
>>>>
>>>>
>>>> /proc/net/pf_ring/* contains one file per ring, the name contains the
>>>> process pid and ring id.
>>>>
>>>> Alfredo
>>>>
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Mon, Nov 16, 2015 at 6:36 PM, Alfredo Cardigliano <
>>>> cardigli...@ntop.org> wrote:
>>>>
>>>>> Added to git code, please note stats are per socket, not per interface.
>>>>>
>>>>> Alfredo
>>>>>
>>>>> On 16 Nov 2015, at 13:48, Amir Kaduri <akadu
[Ntop-misc] Using SW hash filters without replacing the nic driver
Hi, When using SW hash filters only, without ZC or anything like that, is it a must to replace also the nic driver (igb or ixgbe) to a "pfringaware" one? I see that without replacing the nic driver, SW hash filters still work ok, and wonder whether there is anything else I should be aware of. Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Questions regarding pf_ring software hash filters
Hi, I'm using pf_ring software hash filters in my user-space process. 1. Is there a possibility that while the process is down (e.g. by "kill -9" or crash), the filters are still active in the pf_ring driver, so that when a new instance of the same process is up, traffic is still filtered by the rules of its own previous instance? 2. What is the easiest way or the right way to have a list of all active sw hash filters, not within the same process that put them. Preferably by using some command-line script. Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] ring_id is declared with 32 bit, but returned with 16 bit
Small correction about the example ring filename. It should be:
/proc/net/pf_ring/2851-eth8.251768
Amir
On Mon, Aug 15, 2016 at 2:20 PM, Amir Kaduri <akadur...@gmail.com> wrote:
> I found a bug in my code, that reveals a potential bug in pf_ring code:
>
> In user-mode, I’m reading ring statistics by parsing files under
> /proc/net/pf_ring/ in the format -..
>
> For example, /proc/5998/net/pf_ring/2851-eth8.251768
>
> The problem is that the ring_id in these files is generated in 32 bit,
> while the API that returns them uses 16 bit type.
>
> This causes the ring_id to wraparound 2^16, thus not to find the ring file
> in user-mode.
>
> See for example function void ring_proc_add(struct pf_ring_socket *pfr),
> that pointer pfr of type struct pf_ring_socket
>
> owns member ring_id of type u_int32_t, while API function
> pfring_get_ring_id(pfring *ring), that I use in user-mode
>
> to access the same file, returns type u_int16_t.
>
> It also seen very clearly in function pfring_mod_get_ring_id(), that
> declares u_int32_t id and returns u_int16_t:
>
> u_int16_t pfring_mod_get_ring_id(pfring *ring) {
>
> u_int32_t id;
>
> socklen_t len = sizeof(id);
>
> int rc = getsockopt(ring->fd, 0, SO_GET_RING_ID, , );
>
> return((rc == 0) ? id : -1);
>
> }
>
>
>
> The above code is similar to latest pf_ring code in github.
>
>
>
> Please confirm that this is indeed not the intention (i.e. bug).
>
>
> Thanks,
>
> Amir
>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] ring_id is declared with 32 bit, but returned with 16 bit
Excellent!
Thanks,
Amir
On Tue, Aug 16, 2016 at 11:00 AM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
> Hi Amir
> fixed in latest code on github, thank you for reporting.
>
> Best Regards
> Alfredo
>
> On 15 Aug 2016, at 13:20, Amir Kaduri <akadur...@gmail.com> wrote:
>
> I found a bug in my code, that reveals a potential bug in pf_ring code:
>
> In user-mode, I’m reading ring statistics by parsing files under
> /proc/net/pf_ring/ in the format -..
>
> For example, /proc/5998/net/pf_ring/2851-eth8.251768
>
> The problem is that the ring_id in these files is generated in 32 bit,
> while the API that returns them uses 16 bit type.
>
> This causes the ring_id to wraparound 2^16, thus not to find the ring file
> in user-mode.
>
> See for example function void ring_proc_add(struct pf_ring_socket *pfr),
> that pointer pfr of type struct pf_ring_socket
>
> owns member ring_id of type u_int32_t, while API function
> pfring_get_ring_id(pfring *ring), that I use in user-mode
>
> to access the same file, returns type u_int16_t.
>
> It also seen very clearly in function pfring_mod_get_ring_id(), that
> declares u_int32_t id and returns u_int16_t:
>
> u_int16_t pfring_mod_get_ring_id(pfring *ring) {
>
> u_int32_t id;
>
> socklen_t len = sizeof(id);
>
> int rc = getsockopt(ring->fd, 0, SO_GET_RING_ID, , );
>
> return((rc == 0) ? id : -1);
>
> }
>
>
> The above code is similar to latest pf_ring code in github.
>
>
> Please confirm that this is indeed not the intention (i.e. bug).
>
>
> Thanks,
>
> Amir
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Fail to rmmod pf_ring
Hi, I'm in a situation where I try to remove the pf_ring kernel module. "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use" "lsmod | grep pf_ring" gives "pf_ring 1234009 20" In this case, I try to find which process uses pf_ring, in order to stop/kill it. I run "ls -l /proc/net/pf_ring/" to hopefully get the list of process-ids of the relevant processes. The problem is that the list gives only "dev" (subdir) "info" (file) and "stats" (subdir). There are no files like (for example) "/proc/net/pf_ring/12345-eth0.23" that can give me an indication who uses the pf_ring kernel module. Any advice on how can I force removing it or at least know who uses it? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Fail to rmmod pf_ring
OS CentOS 7 pf_ring version: 6.6.0 On Mon, Jul 10, 2017 at 11:57 AM, Amir Kaduri <akadur...@gmail.com> wrote: > Hi, > > I'm in a situation where I try to remove the pf_ring kernel module. > "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use" > "lsmod | grep pf_ring" gives "pf_ring 1234009 20" > In this case, I try to find which process uses pf_ring, in order to > stop/kill it. > I run "ls -l /proc/net/pf_ring/" to hopefully get the list of process-ids > of the relevant processes. The problem is that the list gives only "dev" > (subdir) "info" (file) and "stats" (subdir). There are no files like (for > example) "/proc/net/pf_ring/12345-eth0.23" that can give me an indication > who uses the pf_ring kernel module. > Any advice on how can I force removing it or at least know who uses it? > > Thanks, > Amir > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Fail to rmmod pf_ring
No, not using ZC. But if it could be a reason and maybe I'm doing something wrong during load, how can I check about zc? Anyway, the background for this issue is that I'm trying to replace the igb.ko during machine installation and I'm trying to overcome the special situation that its used by the management interface. To debug the situation, I fiddle with the machine so I'm not sure what exactly brought me to the situation described above, so I probably interested in a tweak (just to remove the pf_ring module) rather than a proper solution. Amir On Mon, Jul 10, 2017 at 12:42 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > did you load a ZC driver perhaps? > > Alfredo > > 2017-07-10 11:00 GMT+02:00 Amir Kaduri <akadur...@gmail.com>: > >> OS CentOS 7 >> pf_ring version: 6.6.0 >> >> On Mon, Jul 10, 2017 at 11:57 AM, Amir Kaduri <akadur...@gmail.com> >> wrote: >> >>> Hi, >>> >>> I'm in a situation where I try to remove the pf_ring kernel module. >>> "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use" >>> "lsmod | grep pf_ring" gives "pf_ring 1234009 20" >>> In this case, I try to find which process uses pf_ring, in order to >>> stop/kill it. >>> I run "ls -l /proc/net/pf_ring/" to hopefully get the list of >>> process-ids of the relevant processes. The problem is that the list gives >>> only "dev" (subdir) "info" (file) and "stats" (subdir). There are no files >>> like (for example) "/proc/net/pf_ring/12345-eth0.23" that can give me >>> an indication who uses the pf_ring kernel module. >>> Any advice on how can I force removing it or at least know who uses it? >>> >>> Thanks, >>> Amir >>> >> >> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Fail to rmmod pf_ring
I've managed to rmmov the "used" pfring.ko by removing first the ixgbe.ko and igb.ko drivers. Not sure how its related (since no process really uses pf_ring) but currently its good enough for me. Thanks On Mon, Jul 10, 2017 at 4:22 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > you are "trying to replace the igb.ko" with? > Please send us the output from dmesg. > > Alfredo > > 2017-07-10 13:22 GMT+02:00 Amir Kaduri <akadur...@gmail.com>: > >> No, not using ZC. But if it could be a reason and maybe I'm doing >> something wrong during load, how can I check about zc? >> Anyway, the background for this issue is that I'm trying to replace the >> igb.ko during machine installation and I'm trying to overcome the special >> situation that its used by the management interface. >> To debug the situation, I fiddle with the machine so I'm not sure what >> exactly brought me to the situation described above, so I probably >> interested in a tweak (just to remove the pf_ring module) rather than a >> proper solution. >> >> Amir >> >> On Mon, Jul 10, 2017 at 12:42 PM, Alfredo Cardigliano < >> cardigli...@ntop.org> wrote: >> >>> Hi Amir >>> did you load a ZC driver perhaps? >>> >>> Alfredo >>> >>> 2017-07-10 11:00 GMT+02:00 Amir Kaduri <akadur...@gmail.com>: >>> >>>> OS CentOS 7 >>>> pf_ring version: 6.6.0 >>>> >>>> On Mon, Jul 10, 2017 at 11:57 AM, Amir Kaduri <akadur...@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I'm in a situation where I try to remove the pf_ring kernel module. >>>>> "rmmod pf_ring" gives "rmmod: ERROR: Module pf_ring is in use" >>>>> "lsmod | grep pf_ring" gives "pf_ring 1234009 20" >>>>> In this case, I try to find which process uses pf_ring, in order to >>>>> stop/kill it. >>>>> I run "ls -l /proc/net/pf_ring/" to hopefully get the list of >>>>> process-ids of the relevant processes. The problem is that the list gives >>>>> only "dev" (subdir) "info" (file) and "stats" (subdir). There are no files >>>>> like (for example) "/proc/net/pf_ring/12345-eth0.23" that can give me >>>>> an indication who uses the pf_ring kernel module. >>>>> Any advice on how can I force removing it or at least know who uses it? >>>>> >>>>> Thanks, >>>>> Amir >>>>> >>>> >>>> >>>> ___ >>>> Ntop-misc mailing list >>>> Ntop-misc@listgateway.unipi.it >>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>>> >>> >>> >>> ___ >>> Ntop-misc mailing list >>> Ntop-misc@listgateway.unipi.it >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >>> >> >> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Using pf_ring on VMware (installed with e1000 drivers)
Hi, I'm using pf_ring 6.6.0 (for SW hash filtering only - no ZC) on CentOS 7 on VMware. All drivers are e1000. The pf_ring filtering seems to work. Is there any benefit to remove the e1000 driver and install the e1000e drivers that comes with pf_ring? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Using pf_ring on VMware (installed with e1000 drivers)
Thanks. Under the same details/conditions described in my question (i.e. pf_ring SW hash filters only), is there any specific need to replace the standard igb.ko and ixgbe.ko drivers with the ones that are provided by pf_ring? On Wed, Aug 9, 2017 at 11:08 AM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > e1000 and e1000e support different adapters (the latter is for pcie > adapters in essence). > > Alfredo > > > On 9 Aug 2017, at 08:03, Amir Kaduri <akadur...@gmail.com> wrote: > > > > Hi, > > > > I'm using pf_ring 6.6.0 (for SW hash filtering only - no ZC) on CentOS 7 > on VMware. > > All drivers are e1000. The pf_ring filtering seems to work. > > Is there any benefit to remove the e1000 driver and install the e1000e > drivers that comes with pf_ring? > > > > Thanks, > > Amir > > ___ > > Ntop-misc mailing list > > Ntop-misc@listgateway.unipi.it > > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Deprecetaed bundle API
Small correction: Jul-15, 2016. Sorry. On Thu, Aug 17, 2017 at 5:04 PM, Amir Kaduri <akadur...@gmail.com> wrote: > I have an application written in the past, that heavily consists on bundle > API in general, and on pfcount_bundle example application specifically. > These APIs were deprecated on Jul-15, 2017 and don't exist anymore in > pf_ring 6.6.0. > What is the easiest way to adjust the existing code in order to be able to > compile and run based on pf_ring 6.6.0? > > Thanks > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Deprecetaed bundle API
I have an application written in the past, that heavily consists on bundle API in general, and on pfcount_bundle example application specifically. These APIs were deprecated on Jul-15, 2017 and don't exist anymore in pf_ring 6.6.0. What is the easiest way to adjust the existing code in order to be able to compile and run based on pf_ring 6.6.0? Thanks ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)
Thanks for the prompt reply. Amir On Tue, May 16, 2017 at 12:09 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > >> On 16 May 2017, at 10:44, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Hello, >> >> I assume that I miss a minor detail, but I'll appreciate explaining >> how the following files are deployed in the right place on the machine >> when the pf_ring installation rpm is installed: >> 1. /usr/local/pfring/kernel/pf_ring.ko: >>In source file ../package/rpm/pfring.spec.in line 37, the copy >> command is commented out. > > DKMS build and install the module when installing the package, I added some > comments to the spec. > This is where the module should be installed by dkms: > > /usr/lib/modules/`uname -r`/weak-updates/pf_ring.ko > >> 2. /etc/systemd/system/pf_ring.service: >>The file itself appears in the source code under >> ../package/etc/systemd/system/pf_ring.service, >>but it doesn't appear in source file ../package/rpm/pfring.spec.in > > Added to the spec (it was installed for ubuntu only) > > Alfredo > >> >> Thanks, >> Amir >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)
Hello, I assume that I miss a minor detail, but I'll appreciate explaining how the following files are deployed in the right place on the machine when the pf_ring installation rpm is installed: 1. /usr/local/pfring/kernel/pf_ring.ko: In source file ../package/rpm/pfring.spec.in line 37, the copy command is commented out. 2. /etc/systemd/system/pf_ring.service: The file itself appears in the source code under ../package/etc/systemd/system/pf_ring.service, but it doesn't appear in source file ../package/rpm/pfring.spec.in Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Question about pf_ring installation rpm (pf_ring 6.6.0)
Apparently, systemd doesn't like that the pf_ring.service and cluster.service files have executable bit set and keeps complaining in /var/log/messages. This is just a warning but it spams the file. Example: var/log/messages-20170521:May 21 07:28:18 localhost systemd: Configuration file /etc/systemd/system/pf_ring.service is marked executable. Please remove executable permission bits. Proceeding anyway. The executable bit is set probably due to my source-control system, and probably not a problem that many will suffer from. Still, you can consider changing command "cp" to "install -m 644" as follows: In file package/rpm/pfring.spec.in: From: cp $HOME/PF_RING/package/etc/systemd/system/pf_ring.service $RPM_BUILD_ROOT/etc/systemd/system cp $HOME/PF_RING/package/etc/systemd/system/cluster.service $RPM_BUILD_ROOT/etc/systemd/system To: install -m 644 $HOME/PF_RING/package/etc/systemd/system/pf_ring.service $RPM_BUILD_ROOT/etc/systemd/system install -m 644 $HOME/PF_RING/package/etc/systemd/system/cluster.service $RPM_BUILD_ROOT/etc/systemd/system Thanks, Amir On Wed, May 17, 2017 at 2:20 PM, Amir Kaduri <akadur...@gmail.com> wrote: > Thanks for the prompt reply. > Amir > > On Tue, May 16, 2017 at 12:09 PM, Alfredo Cardigliano > <cardigli...@ntop.org> wrote: >> Hi Amir >> >>> On 16 May 2017, at 10:44, Amir Kaduri <akadur...@gmail.com> wrote: >>> >>> Hello, >>> >>> I assume that I miss a minor detail, but I'll appreciate explaining >>> how the following files are deployed in the right place on the machine >>> when the pf_ring installation rpm is installed: >>> 1. /usr/local/pfring/kernel/pf_ring.ko: >>>In source file ../package/rpm/pfring.spec.in line 37, the copy >>> command is commented out. >> >> DKMS build and install the module when installing the package, I added some >> comments to the spec. >> This is where the module should be installed by dkms: >> >> /usr/lib/modules/`uname -r`/weak-updates/pf_ring.ko >> >>> 2. /etc/systemd/system/pf_ring.service: >>>The file itself appears in the source code under >>> ../package/etc/systemd/system/pf_ring.service, >>>but it doesn't appear in source file ../package/rpm/pfring.spec.in >> >> Added to the spec (it was installed for ubuntu only) >> >> Alfredo >> >>> >>> Thanks, >>> Amir >>> ___ >>> Ntop-misc mailing list >>> Ntop-misc@listgateway.unipi.it >>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> >> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()
In function hash_pkt(), there is a if-else-if statement based on ip_version. If ip_version is 0, the hash won't include the ipaddress. Since the ip_version might come from the user input, I suggest adding an "else" and issue a warning in case ip_version wasn't set. ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Question regarding filtering using software hash filtering rule
Hello, I have a situation where not all of the received packets are counted as filtered, and I would like to better understand why. To better understand it, I've done a controlled experiment, where after the software hash filtering rule was added on a specific 5-tuple, I send exactly 5000 packets on the same 5-tuple. All packets received by the same ring. When I look at the ring info file, I see that "Sw Filt Hash Match" increases by 4955 exactly. (Its the same number whenever I repeat the experiment on the same filter). Which means 45 packets are not counted. No other statistics parameter can explain the missing 45 packets, not in the ring info file (e.g. "Sw Filt Hash Miss") and not by using "ethtool -S" on the interface (although by using ethtool -S I see that all 5000 packets are definitely received to the NIC). When looking deeply into the replayed pcap, I see a high correlation between the number of missing packets (i.e. 45) and the number of packets that are "TCP Segment of a reassembled PDU" (by wireshark). My rss rehash set to 1. Questions: 1. Any explanation for packets that are not counted by "Sw Filt Hash Match" (and not by any other parameter)? 2. Does the "TCP Segment of a reassembled PDU" could explain it somehow? 3. Could it be a behavioral change compared to previous pf_ring versions (e.g. 6.0.3)? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Question regarding filtering using software hash filtering rule
Hi, Apparently, it was an MTU issue. Thanks (and sorry) On Sun, Jun 4, 2017 at 12:29 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > could you provide a pcap and the commands (baed on our examples) to test what > you are doing? > > Thank you > Alfredo > >> On 4 Jun 2017, at 10:45, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Hello, >> >> I have a situation where not all of the received packets are counted >> as filtered, and I would like to better understand why. >> To better understand it, I've done a controlled experiment, where >> after the software hash filtering rule was added on a specific >> 5-tuple, I send exactly 5000 packets on the same 5-tuple. All packets >> received by the same ring. >> When I look at the ring info file, I see that "Sw Filt Hash Match" >> increases by 4955 exactly. (Its the same number whenever I repeat the >> experiment on the same filter). Which means 45 packets are not >> counted. No other statistics parameter can explain the missing 45 >> packets, not in the ring info file (e.g. "Sw Filt Hash Miss") and not >> by using "ethtool -S" on the interface (although by using ethtool -S I >> see that all 5000 packets are definitely received to the NIC). >> When looking deeply into the replayed pcap, I see a high correlation >> between the number of missing packets (i.e. 45) and the number of >> packets that are "TCP Segment of a reassembled PDU" (by wireshark). >> My rss rehash set to 1. >> >> Questions: >> 1. Any explanation for packets that are not counted by "Sw Filt Hash >> Match" (and not by any other parameter)? >> 2. Does the "TCP Segment of a reassembled PDU" could explain it somehow? >> 3. Could it be a behavioral change compared to previous pf_ring >> versions (e.g. 6.0.3)? >> >> Thanks, >> Amir >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()
Hi Alfredo, This is the exact location of the function: https://github.com/ntop/PF_RING/blob/dev/kernel/pf_ring.c#L1794 Thanks, Amir On Mon, May 29, 2017 at 7:28 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > what is the file location you are talking about? > > Alfredo > >> On 29 May 2017, at 18:23, Amir Kaduri <akadur...@gmail.com> wrote: >> >> In function hash_pkt(), there is a if-else-if statement based on >> ip_version. If ip_version is 0, the hash won't include the ipaddress. >> Since the ip_version might come from the user input, I suggest adding >> an "else" and issue a warning in case ip_version wasn't set. >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()
Hi Alfredo,
I'm aware that this function is called per packet, but if you think of
it deeper, then if everything is configured correctly by the user,
ip_version should give 4 or 6 only, thus it will never rich the "else"
I suggest to add and it will never actually add any redundant check
nor print per packet. The only time the warning will be printed is
only if ip_version will not be 4 or 6 and its great. It clearly warns
about a problem (whether its not set or set to anything else than 4 or
6).
Just to make sure you got me right, the code with the fix should look like that:
if (ip_version == 4)
hash += host_peer_a.v4 + host_peer_b.v4;
else if (ip_version == 6)
hash += (<... Omitted long condition >);
else
printk("Warning: ip_version (currently equals %d) should be 4 or 6
only\n", ip_version);
So again, if you take a moment to think of it, this warning should
never cost you anything if everything is configured well, but it warns
if anything went wrong.
Thanks,
Amir
On Thu, Jun 1, 2017 at 11:39 AM, Alfredo Cardigliano
<cardigli...@ntop.org> wrote:
> Hi Amir
> hash_pkt is called per-packet, we want to a void a printk per packet to avoid
> flooding dmesg and slowing doen packet processing,
> if ip_version comes from user input, sanity check should happen in userspace
> in my opinion.
>
> Regards
> Alfredo
>
>> On 1 Jun 2017, at 07:38, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> This is the exact location of the function:
>> https://github.com/ntop/PF_RING/blob/dev/kernel/pf_ring.c#L1794
>>
>> Thanks,
>> Amir
>>
>> On Mon, May 29, 2017 at 7:28 PM, Alfredo Cardigliano
>> <cardigli...@ntop.org> wrote:
>>> Hi Amir
>>> what is the file location you are talking about?
>>>
>>> Alfredo
>>>
>>>> On 29 May 2017, at 18:23, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>
>>>> In function hash_pkt(), there is a if-else-if statement based on
>>>> ip_version. If ip_version is 0, the hash won't include the ipaddress.
>>>> Since the ip_version might come from the user input, I suggest adding
>>>> an "else" and issue a warning in case ip_version wasn't set.
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Consider adding a warning in case ip_version is missing in function hash_pkt()
Ok, thanks.
On Thu, Jun 1, 2017 at 12:16 PM, Alfredo Cardigliano
<cardigli...@ntop.org> wrote:
> Hi Amir
> regardless of the user input, when a packet is received ah hash is computed
> calling parse_pkt -> parse_raw_pkt -> hash_pkt_header -> hash_pkt
> if the packet is not an IP packet (ARP or whatever), ip_version is 0 and your
> message is printed, for every packet.
>
> Alfredo
>
>> On 1 Jun 2017, at 11:00, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> I'm aware that this function is called per packet, but if you think of
>> it deeper, then if everything is configured correctly by the user,
>> ip_version should give 4 or 6 only, thus it will never rich the "else"
>> I suggest to add and it will never actually add any redundant check
>> nor print per packet. The only time the warning will be printed is
>> only if ip_version will not be 4 or 6 and its great. It clearly warns
>> about a problem (whether its not set or set to anything else than 4 or
>> 6).
>> Just to make sure you got me right, the code with the fix should look like
>> that:
>> if (ip_version == 4)
>>hash += host_peer_a.v4 + host_peer_b.v4;
>> else if (ip_version == 6)
>>hash += (<... Omitted long condition >);
>> else
>>printk("Warning: ip_version (currently equals %d) should be 4 or 6
>> only\n", ip_version);
>>
>> So again, if you take a moment to think of it, this warning should
>> never cost you anything if everything is configured well, but it warns
>> if anything went wrong.
>>
>> Thanks,
>> Amir
>>
>> On Thu, Jun 1, 2017 at 11:39 AM, Alfredo Cardigliano
>> <cardigli...@ntop.org> wrote:
>>> Hi Amir
>>> hash_pkt is called per-packet, we want to a void a printk per packet to
>>> avoid flooding dmesg and slowing doen packet processing,
>>> if ip_version comes from user input, sanity check should happen in
>>> userspace in my opinion.
>>>
>>> Regards
>>> Alfredo
>>>
>>>> On 1 Jun 2017, at 07:38, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>
>>>> Hi Alfredo,
>>>>
>>>> This is the exact location of the function:
>>>> https://github.com/ntop/PF_RING/blob/dev/kernel/pf_ring.c#L1794
>>>>
>>>> Thanks,
>>>> Amir
>>>>
>>>> On Mon, May 29, 2017 at 7:28 PM, Alfredo Cardigliano
>>>> <cardigli...@ntop.org> wrote:
>>>>> Hi Amir
>>>>> what is the file location you are talking about?
>>>>>
>>>>> Alfredo
>>>>>
>>>>>> On 29 May 2017, at 18:23, Amir Kaduri <akadur...@gmail.com> wrote:
>>>>>>
>>>>>> In function hash_pkt(), there is a if-else-if statement based on
>>>>>> ip_version. If ip_version is 0, the hash won't include the ipaddress.
>>>>>> Since the ip_version might come from the user input, I suggest adding
>>>>>> an "else" and issue a warning in case ip_version wasn't set.
>>>>>> ___
>>>>>> Ntop-misc mailing list
>>>>>> Ntop-misc@listgateway.unipi.it
>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>>>
>>>>>
>>>>> ___
>>>>> Ntop-misc mailing list
>>>>> Ntop-misc@listgateway.unipi.it
>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>> ___
>>>> Ntop-misc mailing list
>>>> Ntop-misc@listgateway.unipi.it
>>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>>
>>>
>>> ___
>>> Ntop-misc mailing list
>>> Ntop-misc@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Clarification regarding pf_ring filtering using software hash filtering rule and the purge-idle feature
Following further investigation of the subject, I guess that I found a bug
and attached a patch with a suggested fix.
Following the problematic design I've described in this thread, at least
for my application purposes, I’ve decided to improve one of the APIs to
return the inactivity time of a specified sw hash filtering rule.
That is, enhance the API pfring_get_hash_filtering_rule_stats() to return
this data as well. That way the userland API will have the chance to know
for how long a specified hash rule is idle.
During my implementation, I’m pretty sure that I found a bug in the way the
timings are updated. This makes me assume that the
pfring_purge_idle_hash_rules() API doesn’t work well (but didn’t test it).
Anyway, attached a patch, based on pfring 6.6.0, that includes both the
fix and the enhancement for pfring_get_hash_filtering_rule_stats().
Since both of them are very short, I included them in the same patch file,
rather than splitting them into 2 separate patches (fix and enhancement).
Thanks,
Amir
On Thu, Sep 14, 2017 at 9:07 AM, Amir Kaduri <akadur...@gmail.com> wrote:
> Another question related to the same issue:
>
> I wonder why the design is that the application responsible for
> managing/allocating the rule_id, and not the infrastructure.
> I would expect the API just to ask to add a rule, and the infrastructure
> will return 2 parameters:
> 1. Success/Failure
> 2. The allocated rule_id in case of success (that will be used for
> removing the rule)
>
> Thanks
>
> On Tue, Sep 12, 2017 at 8:34 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>
>> Hi,
>>
>>
>>
>> I would like to clarify a point regarding sw hash filtering rules and the
>> purge idle feature:
>>
>> In my application, I’m using sw hash filtering rules to block sessions.
>> For that purpose, I’m using the API pfring_handle_hash_filtering_rule()
>> to add and remove rules.
>>
>> When adding a rule with an ID of a rule that that has been already added,
>> this “add rule” action fails. This means that the application’s
>> responsibility to manage the rule IDs.
>>
>> In other words, the application’s responsibility to know which IDs are in
>> use and which are not, if it wants to avoid trial-and-error every time a
>> rule should be added.
>>
>> On the other hand, there is the API pfring_purge_idle_hash_rules() that
>> can assist in removing (purging) hash filtering rules that are inactive for
>> a while, automatically.
>>
>> So the situation is that pf_ring purges a filtering rule, and the
>> application is not aware of it.
>>
>>
>> For this reason:
>> 1. The application continues to keep the relevant rule-id,
>> assuming it’s still active.
>> 2. The application avoids using this rule-id for filtering other
>> sessions, since it assumes it’s still occupied.
>> 3. If the session suddenly revives after the purge period arrives,
>> the application is confused since it’s not clear to it how come packets are
>> forwarded while there is an active rule.
>>
>> The above happens since there is no notification mechanism to the
>> userspace application, that a rule has been purged (removed).
>>
>>
>> Is the description correct or maybe I miss something critical in the APIs
>> usage?
>>
>>
>>
>> Thanks,
>>
>> Amir
>>
>
>
diff -Naur PF_RING-6.6.0-clean/kernel/linux/pf_ring.h
PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h
--- PF_RING-6.6.0-clean/kernel/linux/pf_ring.h 2017-04-24 12:53:37.0
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h2017-09-19
13:51:42.709946000 +0300
@@ -604,6 +604,7 @@
typedef struct {
u_int64_t match;
u_int64_t miss;
+ u_int32_t rule_inactivity;
} hash_filtering_rule_stats;
/* * */
diff -Naur PF_RING-6.6.0-clean/kernel/pf_ring.c
PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c
--- PF_RING-6.6.0-clean/kernel/pf_ring.c2017-04-24 12:53:37.0
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c 2017-09-19
14:04:11.680328000 +0300
@@ -2372,7 +2372,6 @@
&& (hash_bucket->rule.host4_peer_b == rule->host4_peer_a)
&& (hash_bucket->rule.port_peer_a == rule->port_peer_b)
&& (hash_bucket->rule.port_peer_b == rule->port_peer_a {
-hash_bucket->rule.internals.jiffies_last_match = jiffies;
return(1);
} else
return(0);
@@ -3291,6 +3290,7 @@
while(hash_bucket != NULL) {
if(hash_bucket_match(hash_bucket, hdr, 0, 0)) {
hash_bucket->match++;
+ hash_bucket->rule.internals.jiffies_last_match = ji
Re: [Ntop-misc] Clarification regarding pf_ring filtering using software hash filtering rule and the purge-idle feature
Another question related to the same issue: I wonder why the design is that the application responsible for managing/allocating the rule_id, and not the infrastructure. I would expect the API just to ask to add a rule, and the infrastructure will return 2 parameters: 1. Success/Failure 2. The allocated rule_id in case of success (that will be used for removing the rule) Thanks On Tue, Sep 12, 2017 at 8:34 PM, Amir Kaduri <akadur...@gmail.com> wrote: > Hi, > > > > I would like to clarify a point regarding sw hash filtering rules and the > purge idle feature: > > In my application, I’m using sw hash filtering rules to block sessions. > For that purpose, I’m using the API pfring_handle_hash_filtering_rule() > to add and remove rules. > > When adding a rule with an ID of a rule that that has been already added, > this “add rule” action fails. This means that the application’s > responsibility to manage the rule IDs. > > In other words, the application’s responsibility to know which IDs are in > use and which are not, if it wants to avoid trial-and-error every time a > rule should be added. > > On the other hand, there is the API pfring_purge_idle_hash_rules() that > can assist in removing (purging) hash filtering rules that are inactive for > a while, automatically. > > So the situation is that pf_ring purges a filtering rule, and the > application is not aware of it. > > > For this reason: > 1. The application continues to keep the relevant rule-id, assuming > it’s still active. > 2. The application avoids using this rule-id for filtering other > sessions, since it assumes it’s still occupied. > 3. If the session suddenly revives after the purge period arrives, > the application is confused since it’s not clear to it how come packets are > forwarded while there is an active rule. > > The above happens since there is no notification mechanism to the > userspace application, that a rule has been purged (removed). > > > Is the description correct or maybe I miss something critical in the APIs > usage? > > > > Thanks, > > Amir > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] New suggested (and implemented) feature: pf_ring sw filters packets sampling
I’ve added a new feature that I would like to share.
The pf_ring feature allows packets sampling of packets that belong to a sw
filtered session.
In other words, if a sw filter is applied, you will be able not to block
100% of the packets (till filtering rule removal). Some packets will be
allowed to “pass” the filter.
The use case is keeping the advantage of improved performance by filtering
most (e.g. 90%) of the traffic, while being able to get some (e.g. 10%) of
the traffic for analysis purposes. This will allow the application not be
100% blind.
This feature is not good for applications that are sensitive to
out-of-sequence packets.
Technical code details:
1. New pf_ring param “filtering_sampling_segment_size” is set to 100
by default.
2. New pf_ring userspace API int
pfring_set_sw_filtering_sampling_rate(pfring *ring, u_int32_t rate /* 0 =
no sampling */) was added.
This API should be used right after adding a new SW hash filtering rule,
with a “rate” parameter.
A rate=10 means that the 10 first packets of every segment of 100 filtered
packets of the session, will be forwarded.
If the API is not called, then the default rate=0, thus no change in
current behavior (i.e. 100% of the packets are filtered, since 0 filtered
packets are forwarded).
3. I’ve had to add a new member variable to struct pf_ring_socket
which is ‘sw_filtering_hash_filtered’ that counts the actual filtered
packets.
Current ‘sw_filtering_hash_match’ is not good enough anymore for this
purpose since it counts the matches before excluding the forwarded packets,
but it has a great help
in calculating the sampled filtered packets.
Technical build/test details:
1. Functionality was tested based on pf_ring version 6.6.0 (on CentOS
7)
2. Attached patch was created based on latest pf_ring code-base
(Sep-7, 2017), and tested for compilation only. Not functionality.
If this feature will ever make it to the main code, it would be great.
Any comments are welcome.
Amir
diff -Naur PF_RING-dev.20170907/kernel/linux/pf_ring.h
PF_RING-dev.20170907.sampling/kernel/linux/pf_ring.h
--- PF_RING-dev.20170907/kernel/linux/pf_ring.h 2017-09-07 13:02:53.0
+0300
+++ PF_RING-dev.20170907.sampling/kernel/linux/pf_ring.h2017-09-07
13:35:43.508272000 +0300
@@ -65,6 +65,7 @@
#define SO_SET_POLL_WATERMARK117
#define SO_SET_VIRTUAL_FILTERING_DEVICE 118
#define SO_REHASH_RSS_PACKET 119
+#define SO_SET_SW_FILTERING_SAMPLING_RATE 120
#define SO_SHUTDOWN_RING 124
#define SO_PURGE_IDLE_RULES 125 /* inactivity (sec) */
#define SO_SET_SOCKET_MODE 126
@@ -1078,6 +1079,7 @@
sw_filtering_hash_bucket **sw_filtering_hash;
u_int64_t sw_filtering_hash_match;
u_int64_t sw_filtering_hash_miss;
+ u_int64_t sw_filtering_hash_filtered;
u_int32_t num_sw_filtering_hash;
/* Sw Filtering Rules - wildcard */
@@ -1087,6 +1089,8 @@
/* Hw Filtering Rules */
u_int16_t num_hw_filtering_rules;
struct list_head hw_filtering_rules;
+ /* Sw Filtering - sampling rate */
+ u_int32_t sw_filtering_sampling_rate;
/* Locks */
atomic_t num_ring_users;
diff -Naur PF_RING-dev.20170907/kernel/pf_ring.c
PF_RING-dev.20170907.sampling/kernel/pf_ring.c
--- PF_RING-dev.20170907/kernel/pf_ring.c 2017-09-07 13:02:53.0
+0300
+++ PF_RING-dev.20170907.sampling/kernel/pf_ring.c 2017-09-07
13:54:45.102938000 +0300
@@ -365,6 +365,7 @@
static unsigned int force_ring_lock = 0;
static unsigned int enable_debug = 0;
static unsigned int transparent_mode = 0;
+static unsigned int filtering_sampling_segment_size = 100;
static atomic_t ring_id_serial = ATOMIC_INIT(0);
#if defined(RHEL_RELEASE_CODE)
@@ -382,6 +383,7 @@
module_param(force_ring_lock, uint, 0644);
module_param(enable_debug, uint, 0644);
module_param(transparent_mode, uint, 0644);
+module_param(filtering_sampling_segment_size, uint, 0644);
MODULE_PARM_DESC(min_num_slots, "Min number of ring slots");
MODULE_PARM_DESC(perfect_rules_hash_size, "Perfect rules hash size");
@@ -397,6 +399,7 @@
MODULE_PARM_DESC(enable_debug, "Set to 1 to enable PF_RING debug tracing into
the syslog, 2 for more verbosity");
MODULE_PARM_DESC(transparent_mode,
"(deprecated)");
+MODULE_PARM_DESC(filtering_sampling_segment_size, "Size of filtered packets
segment for sampling (currently, for sw hash filters only)");
/* ** */
@@ -1360,20 +1363,21 @@
if(m->private == NULL) {
/* /proc/net/pf_ring/info */
-seq_printf(m, "PF_RING Version : %s (%s)\n", RING_VERSION,
GIT_REV);
-seq_printf(m, "Total rings : %d\n",
atomic_read(_table_size));
+seq_printf(m, "PF_RING Version : %s (%s)\n", RING_VERSION,
GIT_REV);
+seq_printf(m, "Total rings : %d\n",
atomic_read(_table_size));
seq_printf(m, "\nStandard (non ZC) Options\n");
-
[Ntop-misc] Clarification regarding pf_ring filtering using software hash filtering rule and the purge-idle feature
Hi, I would like to clarify a point regarding sw hash filtering rules and the purge idle feature: In my application, I’m using sw hash filtering rules to block sessions. For that purpose, I’m using the API pfring_handle_hash_filtering_rule() to add and remove rules. When adding a rule with an ID of a rule that that has been already added, this “add rule” action fails. This means that the application’s responsibility to manage the rule IDs. In other words, the application’s responsibility to know which IDs are in use and which are not, if it wants to avoid trial-and-error every time a rule should be added. On the other hand, there is the API pfring_purge_idle_hash_rules() that can assist in removing (purging) hash filtering rules that are inactive for a while, automatically. So the situation is that pf_ring purges a filtering rule, and the application is not aware of it. For this reason: 1. The application continues to keep the relevant rule-id, assuming it’s still active. 2. The application avoids using this rule-id for filtering other sessions, since it assumes it’s still occupied. 3. If the session suddenly revives after the purge period arrives, the application is confused since it’s not clear to it how come packets are forwarded while there is an active rule. The above happens since there is no notification mechanism to the userspace application, that a rule has been purged (removed). Is the description correct or maybe I miss something critical in the APIs usage? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Deprecetaed bundle API
Thanks
On Mon, Aug 21, 2017 at 2:00 PM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
> Hi Amir
> copying those function is definitely the simplest solution and it should
> work fine for you,
> as of extended_hdr.parsed_header_len please just remove the if branch and
> delete any
> occurrence as that was the length of metadata when using kernel plugins
> (deprecated).
>
> Regards
> Alfredo
>
> On 21 Aug 2017, at 12:31, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Since its about 6 functions, a possible solution could be just copying
> them to the application as static functions. Not a perfect solution, but
> possible. If there is a better one (that is simple), I'll be glad to hear.
> Anyway, another thing in the same context:
> The (deprecated) pfcount_bundle.c example application, had a check "if
> (h->extended_hdr.parsed_header_len > 0)". "parsed_header_len" has been
> also removed from struct pfring_extended_pkthdr{} since pf_ring 6.6.0
> (on Oct 19, 2016). I assume it was calculated during packet parsing. Could
> you please suggest a simple alternative to this check, based on pf_ring
> 6.6.0?
> Thanks
>
> On Thu, Aug 17, 2017 at 5:11 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>
>> Small correction: Jul-15, 2016. Sorry.
>>
>> On Thu, Aug 17, 2017 at 5:04 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>>> I have an application written in the past, that heavily consists on
>>> bundle API in general, and on pfcount_bundle example application
>>> specifically. These APIs were deprecated on Jul-15, 2017 and don't exist
>>> anymore in pf_ring 6.6.0.
>>> What is the easiest way to adjust the existing code in order to be able
>>> to compile and run based on pf_ring 6.6.0?
>>>
>>> Thanks
>>>
>>
>>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Deprecetaed bundle API
Since its about 6 functions, a possible solution could be just copying them
to the application as static functions. Not a perfect solution, but
possible. If there is a better one (that is simple), I'll be glad to hear.
Anyway, another thing in the same context:
The (deprecated) pfcount_bundle.c example application, had a check "if
(h->extended_hdr.parsed_header_len > 0)". "parsed_header_len" has been also
removed from struct pfring_extended_pkthdr{} since pf_ring 6.6.0
(on Oct 19, 2016). I assume it was calculated during packet parsing. Could
you please suggest a simple alternative to this check, based on pf_ring
6.6.0?
Thanks
On Thu, Aug 17, 2017 at 5:11 PM, Amir Kaduri <akadur...@gmail.com> wrote:
> Small correction: Jul-15, 2016. Sorry.
>
> On Thu, Aug 17, 2017 at 5:04 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>
>> I have an application written in the past, that heavily consists on
>> bundle API in general, and on pfcount_bundle example application
>> specifically. These APIs were deprecated on Jul-15, 2017 and don't exist
>> anymore in pf_ring 6.6.0.
>> What is the easiest way to adjust the existing code in order to be able
>> to compile and run based on pf_ring 6.6.0?
>>
>> Thanks
>>
>
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] pf_ring: bug fix (rule inactivity)
Hi Alfredo,
The attached patch includes a fix of a bug that relates to updating the
last time a hash rule was active. I assume that the
pfring_purge_idle_hash_rules()
API doesn't work for this reason.
I also added an enhancement of the API pfring_get_hash_filtering_rule_stats()
to return for how long a hash filtering rule is inactive.
The patch is based on pfring 6.6.0.
Since the fix and the enhancement are pretty short, I put them together in
the same patch.
I'll appreciate getting a feedback on this fix (I already sent it over a
month ago).
Thanks,
Amir
diff -Naur PF_RING-6.6.0-clean/kernel/linux/pf_ring.h
PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h
--- PF_RING-6.6.0-clean/kernel/linux/pf_ring.h 2017-04-24 12:53:37.0
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/linux/pf_ring.h2017-09-19
13:51:42.709946000 +0300
@@ -604,6 +604,7 @@
typedef struct {
u_int64_t match;
u_int64_t miss;
+ u_int32_t rule_inactivity;
} hash_filtering_rule_stats;
/* * */
diff -Naur PF_RING-6.6.0-clean/kernel/pf_ring.c
PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c
--- PF_RING-6.6.0-clean/kernel/pf_ring.c2017-04-24 12:53:37.0
+0300
+++ PF_RING-6.6.0-rule_inactivity/kernel/pf_ring.c 2017-09-19
14:04:11.680328000 +0300
@@ -2372,7 +2372,6 @@
&& (hash_bucket->rule.host4_peer_b == rule->host4_peer_a)
&& (hash_bucket->rule.port_peer_a == rule->port_peer_b)
&& (hash_bucket->rule.port_peer_b == rule->port_peer_a {
-hash_bucket->rule.internals.jiffies_last_match = jiffies;
return(1);
} else
return(0);
@@ -3291,6 +3290,7 @@
while(hash_bucket != NULL) {
if(hash_bucket_match(hash_bucket, hdr, 0, 0)) {
hash_bucket->match++;
+ hash_bucket->rule.internals.jiffies_last_match = jiffies;
hash_found = 1;
break;
} else
@@ -7146,6 +7146,7 @@
hash_filtering_rule_stats hfrs;
hfrs.match = bucket->match;
hfrs.miss = pfr->sw_filtering_hash_miss;
+ hfrs.rule_inactivity = (u_int32_t)(jiffies_to_msecs(jiffies -
bucket->rule.internals.jiffies_last_match) / 1000);
rc = sizeof(hash_filtering_rule_stats);
if(copy_to_user(optval, , rc)) {
printk("[PF_RING] copy_to_user() failure\n");
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Hi, I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe drivers). As far as I understand the relation between poll-watermark and poll-duration, packets will be queued untill one of comes first: or passing the poll-watermark packets threshold, or a poll-duration milliseconds has passed. I set poll-watermark to the maximum (4096) (using pfring_set_poll_watermark()) and set poll-duration to the minimum (1) (using pfring_set_poll_duration()). I've sent 400 packets to the socket. I see that they are received by the NIC, but they didn't pass to userland. Only when passing 500 packets, a chunk of them passed to userland. I don't quite understand the behavior: since poll-duration is 1 (millisecond I assume), I've expected all the packets to pass to userland immediately, even though poll-watermark is much higher. Can anyone shed some light on the above? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Using upgraded i40e from pf_ring 7.0 (2.2.4), on pf_ring 6.6.0
Hi, I'm using pf_ring 6.6.0 (not using ZC) on CentOS 7.0. I would like to upgrade the i40e driver (only) to 2.2.4 (i.e. take the code of i40e from pf_ring 7.0 and put it in pf_ring 6.6.0 and use them together). The question is how risky it is? How pf_ring is tight up with the i40e driver version? Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Basically, the functionality that I would like to have is even if less than poll-watermark-threshold (default: 128) packets arrives the socket, they will be forwarded to userland if 1 millisecond has passed. How can I gain this? Isn't it by using pfring_set_poll_duration()? Alfredo, could you please clarify? Thanks, Amir On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri <akadur...@gmail.com> wrote: > Hi, > > I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe > drivers). > As far as I understand the relation between poll-watermark and > poll-duration, packets will be queued untill one of comes first: or passing > the poll-watermark packets threshold, or a poll-duration milliseconds has > passed. > I set poll-watermark to the maximum (4096) (using pfring_set_poll_watermark()) > and set poll-duration to the minimum (1) (using pfring_set_poll_ > duration()). > I've sent 400 packets to the socket. I see that they are received by the > NIC, but they didn't pass to userland. Only when passing 500 packets, a > chunk of them passed to userland. > I don't quite understand the behavior: since poll-duration is 1 > (millisecond I assume), I've expected all the packets to pass to userland > immediately, even though poll-watermark is much higher. > > Can anyone shed some light on the above? > > Thanks, > Amir > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Hi Alfredo,
I'm trying to debug the issue, and I have a question about the code, to
make sure that there is no problem there:
Specifically, I'm referring to the function "pfring_mod_recv":
In order that the line that refers to poll_duration ("pfring_poll(ring,
ring->poll_duration)") will be reached, there are 2 conditions that should
occur:
1. pfring_there_is_pkt_available(ring) should return false (otherwise, the
function returns at the end of the condition).
2. wait_for_incoming_packet should be set to true.
Currently, I'm referring to the first one:
In order that the macro pfring_there_is_pkt_available(ring) will return
false, ring->slots_info->tot_insert should be equal to
ring->slots_info->tot_read.
What I see in my tests that they don't get equal. I always see that
tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
but it happens inside the condition, so the "pfring_mod_recv" returns with
1.
I remind that I set the watermark to be high, in order to see the
poll_duration takes effect.
Could you please approve that you don't see any problem in the code?
Thanks,
Amir
On Thu, Oct 26, 2017 at 12:22 PM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
> Hi Amir
> yes, that’s the way it should work, if this is not the case, some
> debugging is needed to identify the problem
>
> Alfredo
>
> On 26 Oct 2017, at 10:14, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Basically, the functionality that I would like to have is even if less
> than poll-watermark-threshold (default: 128) packets arrives the socket,
> they will be forwarded to userland if 1 millisecond has passed.
> How can I gain this? Isn't it by using pfring_set_poll_duration()?
>
> Alfredo, could you please clarify?
>
> Thanks,
> Amir
>
> On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>
>> Hi,
>>
>> I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
>> drivers).
>> As far as I understand the relation between poll-watermark and
>> poll-duration, packets will be queued untill one of comes first: or passing
>> the poll-watermark packets threshold, or a poll-duration milliseconds has
>> passed.
>> I set poll-watermark to the maximum (4096) (using
>> pfring_set_poll_watermark())
>> and set poll-duration to the minimum (1) (using pfring_set_poll_duratio
>> n()).
>> I've sent 400 packets to the socket. I see that they are received by the
>> NIC, but they didn't pass to userland. Only when passing 500 packets, a
>> chunk of them passed to userland.
>> I don't quite understand the behavior: since poll-duration is 1
>> (millisecond I assume), I've expected all the packets to pass to userland
>> immediately, even though poll-watermark is much higher.
>>
>> Can anyone shed some light on the above?
>>
>> Thanks,
>> Amir
>>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Thanks. tot_insert apparently works ok.
Regarding function copy_data_to_ring():
At the end of it there is the statement:
if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
wake_up_interruptible(>ring_slots_waitqueue);
Since watermark is set to 128, and I send <128 packets, this causes them to
wait in kernel queue.
But since poll_duration is set to 1 (1 millisecond I assume), I expect the
condition to check this also (meaning, there are packets in queue but 1
millisecond passed and they weren't read),
the wake_up_interruptible should also be called. No?
Thanks,
Amir
On Tue, Oct 31, 2017 at 10:20 AM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
>
>
> On 31 Oct 2017, at 08:42, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Hi Alfredo,
>
> I'm trying to debug the issue, and I have a question about the code, to
> make sure that there is no problem there:
> Specifically, I'm referring to the function "pfring_mod_recv":
> In order that the line that refers to poll_duration ("pfring_poll(ring,
> ring->poll_duration)") will be reached, there are 2 conditions that should
> occur:
> 1. pfring_there_is_pkt_available(ring) should return false (otherwise,
> the function returns at the end of the condition).
> 2. wait_for_incoming_packet should be set to true.
> Currently, I'm referring to the first one:
> In order that the macro pfring_there_is_pkt_available(ring) will return
> false, ring->slots_info->tot_insert should be equal to
> ring->slots_info->tot_read.
> What I see in my tests that they don't get equal. I always see that
> tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
> but it happens inside the condition, so the "pfring_mod_recv" returns with
> 1.
>
>
> It seems to be correct. The kernel module inserts packets into the ring
> increasing tot_insert, the userspace library reads packets from the ring
> increasing tot_read. This means that if tot_insert == tot_read there is no
> packet to read. If there is a bug, it should be in the kernel module that
> is somehow not adding packets to the ring (thus not updating tot_insert).
>
> Alfredo
>
> I remind that I set the watermark to be high, in order to see the
> poll_duration takes effect.
>
> Could you please approve that you don't see any problem in the code?
>
> Thanks,
> Amir
>
>
> On Thu, Oct 26, 2017 at 12:22 PM, Alfredo Cardigliano <
> cardigli...@ntop.org> wrote:
>
>> Hi Amir
>> yes, that’s the way it should work, if this is not the case, some
>> debugging is needed to identify the problem
>>
>> Alfredo
>>
>> On 26 Oct 2017, at 10:14, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Basically, the functionality that I would like to have is even if less
>> than poll-watermark-threshold (default: 128) packets arrives the socket,
>> they will be forwarded to userland if 1 millisecond has passed.
>> How can I gain this? Isn't it by using pfring_set_poll_duration()?
>>
>> Alfredo, could you please clarify?
>>
>> Thanks,
>> Amir
>>
>> On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I'm using pf_ring 6.6.0 (no ZC) on CentOS 7, on 10G interfaces (ixgbe
>>> drivers).
>>> As far as I understand the relation between poll-watermark and
>>> poll-duration, packets will be queued untill one of comes first: or passing
>>> the poll-watermark packets threshold, or a poll-duration milliseconds has
>>> passed.
>>> I set poll-watermark to the maximum (4096) (using
>>> pfring_set_poll_watermark())
>>> and set poll-duration to the minimum (1) (using pfring_set_poll_duratio
>>> n()).
>>> I've sent 400 packets to the socket. I see that they are received by the
>>> NIC, but they didn't pass to userland. Only when passing 500 packets, a
>>> chunk of them passed to userland.
>>> I don't quite understand the behavior: since poll-duration is 1
>>> (millisecond I assume), I've expected all the packets to pass to userland
>>> immediately, even though poll-watermark is much higher.
>>>
>>> Can anyone shed some light on the above?
>>>
>>> Thanks,
>>> Amir
>>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>>
>>
>> ___
>> Ntop-misc mailing list
>> Ntop-misc@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> ___
> Ntop-misc mailing list
> Ntop-misc@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
___
Ntop-misc mailing list
Ntop-misc@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Good. There is a pull-request waiting. I hope you'll find it beneficial: https://github.com/ntop/PF_RING/pull/340 Thanks, Amir On Tue, May 1, 2018 at 1:19 AM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > > > On 30 Apr 2018, at 17:52, Amir Kaduri <akadur...@gmail.com> wrote: > > Thanks for the answers. > > So the only way to make handlep->timeout>=0, is by setting the > file-descriptor to "blocking" (nonblock=0) according to the logic in > function pcap_setnonblock_mmap() and this is something that we would like > to avoid. > Therefore, we do the polling (non-blocking) in the application that uses > pcap/pf_ring. > The problem we have is with low-traffic network. According to the logic in > function copy_data_to_ring(), as long as the queue didn't reach the > "poll_num_pkts_watermark" threshold (in our case 128 packets), > the poll() (in userspace) won't be called (since wake_up_interruptible(..) > is not called), which means that we have packets that are stuck in the ring > till the queue reaches the watermark. > > I wonder if you see any rationale in improving the pf_ring kernel module > code, to call wake_up_interruptible() (in order to flush the queue) if > some "timeout" passed and the queue is not empty (but still didn't reach > the watermark). > > > I think that using the watermark in combination with a timeout is a good > idea. > > Alfredo > > Amir > > > On Thu, Apr 26, 2018 at 6:00 PM, Alfredo Cardigliano <cardigli...@ntop.org > > wrote: > >> >> >> On 26 Apr 2018, at 15:34, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Hi Alfredo, >> >> My code is based on libpcap, while pfring's userland examples use pfring >> APIs directly, therefore things are a bit harder for me. >> >> Short clarification about a related code-line: >> Please look at the following line: https://github.com/ntop/ >> PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875 >> >> (1) If I understand it correctly, if wait_for_incoming_packet is true, >> then pfring_poll() should be called. >> Don't you want wait_for_incoming_packet to be true in case >> pf_ring_active_poll is true? >> >> >> “active” means spinning, thus poll should not be used in that case. >> >> Currently, its the opposite (i.e. if pf_ring_active_poll is true, >> wait_for_incoming_packet will be false thus pfring_poll() won't be >> called). >> >> >> This seems to be correct >> >> >> (2) If the code is ok, then the only way for me to make >> wait_for_incoming_packet true (for pfring_poll() to be called) is by >> making handlep->timeout >= 0. >> Correct? >> >> >> Correct >> >> Alfredo >> >> >> Thanks, >> Amir >> >> On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano < >> cardigli...@ntop.org> wrote: >> >>> Hi Amir >>> if I understand correctly, pfcount_multichannel is working, while in >>> your application >>> it seems that poll does not honor the timeout, if this is the case it >>> seems the problem >>> is not in the kernel module, I think you should look for differences >>> between the two applications.. >>> >>> Alfredo >>> >>> On 9 Apr 2018, at 07:20, Amir Kaduri <akadur...@gmail.com> wrote: >>> >>> Hi Alfredo, >>> >>> I'm back to investigate/debug this issue in my environment, and maybe >>> you'll manage to save me some time: >>> >>> When I use the example program "pfcount_multichannel", poll-duration >>> works for me as expected: >>> For watermark=128, poll-duration=1000, even if less than 128 packets >>> received, I get them in pfcount_multichannel. >>> >>> On the other hand, in my other program (which is a complex one), the >>> userspace application gets the packets only after 128 packets >>> aggregated by the ring, regardless the polling rate (which is done >>> always using 50ms timeout). >>> >>> Maybe you can figure out what can "hold" the packets in the ring and >>> forward them to userspace only when the watermark threshold passes? >>> Maybe something is missing during initialization? >>> (for simplicity I'm not using rehash, and not using any filters). >>> >>> Thanks >>> >>> On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano < >>> cardigli...@ntop.org> wrote: >>> >>>> Hi Amir >>>> that's
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Same feature, new implementation in PR: https://github.com/ntop/PF_RING/pull/343 On Sun, May 6, 2018 at 7:18 PM, Amir Kaduri <akadur...@gmail.com> wrote: > Good. > There is a pull-request waiting. I hope you'll find it beneficial: > https://github.com/ntop/PF_RING/pull/340 > > Thanks, > Amir > > On Tue, May 1, 2018 at 1:19 AM, Alfredo Cardigliano <cardigli...@ntop.org> > wrote: > >> >> >> On 30 Apr 2018, at 17:52, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Thanks for the answers. >> >> So the only way to make handlep->timeout>=0, is by setting the >> file-descriptor to "blocking" (nonblock=0) according to the logic in >> function pcap_setnonblock_mmap() and this is something that we would >> like to avoid. >> Therefore, we do the polling (non-blocking) in the application that uses >> pcap/pf_ring. >> The problem we have is with low-traffic network. According to the logic >> in function copy_data_to_ring(), as long as the queue didn't reach the >> "poll_num_pkts_watermark" threshold (in our case 128 packets), >> the poll() (in userspace) won't be called (since wake_up_interruptible(..) >> is not called), which means that we have packets that are stuck in the ring >> till the queue reaches the watermark. >> >> I wonder if you see any rationale in improving the pf_ring kernel module >> code, to call wake_up_interruptible() (in order to flush the queue) if >> some "timeout" passed and the queue is not empty (but still didn't reach >> the watermark). >> >> >> I think that using the watermark in combination with a timeout is a good >> idea. >> >> Alfredo >> >> Amir >> >> >> On Thu, Apr 26, 2018 at 6:00 PM, Alfredo Cardigliano < >> cardigli...@ntop.org> wrote: >> >>> >>> >>> On 26 Apr 2018, at 15:34, Amir Kaduri <akadur...@gmail.com> wrote: >>> >>> Hi Alfredo, >>> >>> My code is based on libpcap, while pfring's userland examples use pfring >>> APIs directly, therefore things are a bit harder for me. >>> >>> Short clarification about a related code-line: >>> Please look at the following line: https://github.com/ntop/ >>> PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875 >>> >>> (1) If I understand it correctly, if wait_for_incoming_packet is true, >>> then pfring_poll() should be called. >>> Don't you want wait_for_incoming_packet to be true in case >>> pf_ring_active_poll is true? >>> >>> >>> “active” means spinning, thus poll should not be used in that case. >>> >>> Currently, its the opposite (i.e. if pf_ring_active_poll is true, >>> wait_for_incoming_packet will be false thus pfring_poll() won't be >>> called). >>> >>> >>> This seems to be correct >>> >>> >>> (2) If the code is ok, then the only way for me to make >>> wait_for_incoming_packet true (for pfring_poll() to be called) is by >>> making handlep->timeout >= 0. >>> Correct? >>> >>> >>> Correct >>> >>> Alfredo >>> >>> >>> Thanks, >>> Amir >>> >>> On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano < >>> cardigli...@ntop.org> wrote: >>> >>>> Hi Amir >>>> if I understand correctly, pfcount_multichannel is working, while in >>>> your application >>>> it seems that poll does not honor the timeout, if this is the case it >>>> seems the problem >>>> is not in the kernel module, I think you should look for differences >>>> between the two applications.. >>>> >>>> Alfredo >>>> >>>> On 9 Apr 2018, at 07:20, Amir Kaduri <akadur...@gmail.com> wrote: >>>> >>>> Hi Alfredo, >>>> >>>> I'm back to investigate/debug this issue in my environment, and maybe >>>> you'll manage to save me some time: >>>> >>>> When I use the example program "pfcount_multichannel", poll-duration >>>> works for me as expected: >>>> For watermark=128, poll-duration=1000, even if less than 128 packets >>>> received, I get them in pfcount_multichannel. >>>> >>>> On the other hand, in my other program (which is a complex one), the >>>> userspace application gets the packets only after 128 packets >>>> aggregated by th
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Thanks for the answers. So the only way to make handlep->timeout>=0, is by setting the file-descriptor to "blocking" (nonblock=0) according to the logic in function pcap_setnonblock_mmap() and this is something that we would like to avoid. Therefore, we do the polling (non-blocking) in the application that uses pcap/pf_ring. The problem we have is with low-traffic network. According to the logic in function copy_data_to_ring(), as long as the queue didn't reach the "poll_num_pkts_watermark" threshold (in our case 128 packets), the poll() (in userspace) won't be called (since wake_up_interruptible(..) is not called), which means that we have packets that are stuck in the ring till the queue reaches the watermark. I wonder if you see any rationale in improving the pf_ring kernel module code, to call wake_up_interruptible() (in order to flush the queue) if some "timeout" passed and the queue is not empty (but still didn't reach the watermark). Amir On Thu, Apr 26, 2018 at 6:00 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > > > On 26 Apr 2018, at 15:34, Amir Kaduri <akadur...@gmail.com> wrote: > > Hi Alfredo, > > My code is based on libpcap, while pfring's userland examples use pfring > APIs directly, therefore things are a bit harder for me. > > Short clarification about a related code-line: > Please look at the following line: https://github.com/ntop/ > PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875 > > (1) If I understand it correctly, if wait_for_incoming_packet is true, > then pfring_poll() should be called. > Don't you want wait_for_incoming_packet to be true in case > pf_ring_active_poll is true? > > > “active” means spinning, thus poll should not be used in that case. > > Currently, its the opposite (i.e. if pf_ring_active_poll is true, > wait_for_incoming_packet will be false thus pfring_poll() won't be > called). > > > This seems to be correct > > > (2) If the code is ok, then the only way for me to make > wait_for_incoming_packet true (for pfring_poll() to be called) is by > making handlep->timeout >= 0. > Correct? > > > Correct > > Alfredo > > > Thanks, > Amir > > On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano <cardigli...@ntop.org > > wrote: > >> Hi Amir >> if I understand correctly, pfcount_multichannel is working, while in your >> application >> it seems that poll does not honor the timeout, if this is the case it >> seems the problem >> is not in the kernel module, I think you should look for differences >> between the two applications.. >> >> Alfredo >> >> On 9 Apr 2018, at 07:20, Amir Kaduri <akadur...@gmail.com> wrote: >> >> Hi Alfredo, >> >> I'm back to investigate/debug this issue in my environment, and maybe >> you'll manage to save me some time: >> >> When I use the example program "pfcount_multichannel", poll-duration >> works for me as expected: >> For watermark=128, poll-duration=1000, even if less than 128 packets >> received, I get them in pfcount_multichannel. >> >> On the other hand, in my other program (which is a complex one), the >> userspace application gets the packets only after 128 packets >> aggregated by the ring, regardless the polling rate (which is done always >> using 50ms timeout). >> >> Maybe you can figure out what can "hold" the packets in the ring and >> forward them to userspace only when the watermark threshold passes? >> Maybe something is missing during initialization? >> (for simplicity I'm not using rehash, and not using any filters). >> >> Thanks >> >> On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano < >> cardigli...@ntop.org> wrote: >> >>> Hi Amir >>> that's correct, however for some reason it seems it is not the case in >>> your tests. >>> >>> Alfredo >>> >>> On 31 Oct 2017, at 12:08, Amir Kaduri <akadur...@gmail.com> wrote: >>> >>> Thanks. tot_insert apparently works ok. >>> >>> Regarding function copy_data_to_ring(): >>> At the end of it there is the statement: >>> if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark) >>> wake_up_interruptible(>ring_slots_waitqueue); >>> >>> Since watermark is set to 128, and I send <128 packets, this causes them >>> to wait in kernel queue. >>> But since poll_duration is set to 1 (1 millisecond I assume), I expect >>> the condition to check this also (meaning, there are packets in queue but 1 >>> m
Re: [Ntop-misc] Question regarding implementation of API function pfring_get_ring_id()
Hi Alfredo, No reason not to cache it inside the application, but why let any application to implement it, if its possible to implement it in pf_ring's userspace API (the way I mentioned)? BTW, if its sounds reasonable, I can push a suggested solution, and you can consider adopting it. Amir On Thu, May 3, 2018 at 4:11 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > ring_id should not change, thus it can be cached in userspace, > any reason you cannot cache it inside the application? > > Alfredo > > > On 3 May 2018, at 15:09, Amir Kaduri <akadur...@gmail.com> wrote: > > > > User-space API function pfring_get_ring_id() always bring the ring id > from the kernel. In some applications, this API call might be used within > debug messages, in order to be able to track a relevant ring. If the > application uses this API too often (for debugging purposes), there could > be too much calls for the kernel, for getting information that usually > doesn't change. > > So my questions are: > > 1. Once a ring is initialized with a ring_id (in function > ring_create()), will it ever be changed during the application life-time? > > 2. Assuming the answer is negative (i.e. once a ring id is determined, > it doesn't change), > > is there a room for caching this value in userspace and avoid future > calls to kernel? > > (e.g. by adding "ring_id" member to "struct __pfring" in pfring.h, and > let function pfring_mod_get_ring_id() to return it once initialized, > instead of the call to getsockopt(.. SO_GET_RING_ID ..)) > > > > Thanks, Amir > > > > > > ___ > > Ntop-misc mailing list > > Ntop-misc@listgateway.unipi.it > > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Question regarding implementation of API function pfring_get_ring_id()
Hi Alfredo, I didn't cache it since I didn't bare in mind that the implementation is like that. I discovered it by chance, when debugging something else. I can cache it of course quite easily and add a wrapping API in the application, but I asked myself why to do it in the application if the right thing is to do it in the original API. The use-case BTW, is when you want the application to periodically dump statistical information about the channels (rings). Its nice to have the ring_id dumped as well. Also, there is no API for reading the information of a socket (e.g. /proc/net/pf_ring/1234-eth0.567), so to create it, you need the ring_id, and here I thought that its better to get it directly from the API instead of a variable that I'll hold. Anyway, its not a big issue for me, I can cache it in my application of course. I just raised it as a point to think about. Thanks, Amir On Thu, May 3, 2018 at 4:44 PM, Alfredo Cardigliano <cardigli...@ntop.org> wrote: > Hi Amir > please note I am not against caching it in the library, I was just > wondering > why you are not caching it in the application as it was the easiest > solution > and I am not aware of other use cases / applications reading it so often. > > Alfredo > > On 3 May 2018, at 15:40, Amir Kaduri <akadur...@gmail.com> wrote: > > Hi Alfredo, > > No reason not to cache it inside the application, but why let any > application to implement it, if its possible to implement it in pf_ring's > userspace API (the way I mentioned)? > BTW, if its sounds reasonable, I can push a suggested solution, and you > can consider adopting it. > > Amir > > On Thu, May 3, 2018 at 4:11 PM, Alfredo Cardigliano <cardigli...@ntop.org> > wrote: > >> Hi Amir >> ring_id should not change, thus it can be cached in userspace, >> any reason you cannot cache it inside the application? >> >> Alfredo >> >> > On 3 May 2018, at 15:09, Amir Kaduri <akadur...@gmail.com> wrote: >> > >> > User-space API function pfring_get_ring_id() always bring the ring id >> from the kernel. In some applications, this API call might be used within >> debug messages, in order to be able to track a relevant ring. If the >> application uses this API too often (for debugging purposes), there could >> be too much calls for the kernel, for getting information that usually >> doesn't change. >> > So my questions are: >> > 1. Once a ring is initialized with a ring_id (in function >> ring_create()), will it ever be changed during the application life-time? >> > 2. Assuming the answer is negative (i.e. once a ring id is determined, >> it doesn't change), >> > is there a room for caching this value in userspace and avoid future >> calls to kernel? >> > (e.g. by adding "ring_id" member to "struct __pfring" in pfring.h, and >> let function pfring_mod_get_ring_id() to return it once initialized, >> instead of the call to getsockopt(.. SO_GET_RING_ID ..)) >> > >> > Thanks, Amir >> > >> > >> > ___ >> > Ntop-misc mailing list >> > Ntop-misc@listgateway.unipi.it >> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> >> >> ___ >> Ntop-misc mailing list >> Ntop-misc@listgateway.unipi.it >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Question regarding implementation of API function pfring_get_ring_id()
User-space API function pfring_get_ring_id() always bring the ring id from the kernel. In some applications, this API call might be used within debug messages, in order to be able to track a relevant ring. If the application uses this API too often (for debugging purposes), there could be too much calls for the kernel, for getting information that usually doesn't change. So my questions are: 1. Once a ring is initialized with a ring_id (in function ring_create()), will it ever be changed during the application life-time? 2. Assuming the answer is negative (i.e. once a ring id is determined, it doesn't change), is there a room for caching this value in userspace and avoid future calls to kernel? (e.g. by adding "ring_id" member to "struct __pfring" in pfring.h, and let function pfring_mod_get_ring_id() to return it once initialized, instead of the call to getsockopt(.. SO_GET_RING_ID ..)) Thanks, Amir ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
[Ntop-misc] Determining rx queues of an interface
Do the rx queues of an interface are determined only when loading the kernel driver, or is there another thing that affects that? My problem is that I get a number that differs than what I think should be (because maybe something during the machine boot doesn't work properly) For example: What I assume our load script do is "*insmod /usr/local/pfring/drivers/intel/i40e.ko RSS=12,12,12,12,12,12,12,12*" What I actually get when I run "*cat /proc/net/pf_ring/dev/eth4/info*" is "*RX Queues:14*" (from previous reboot). So maybe the "insmod" doesn't actually work as expected, or is there anything else that I'm not aware of? A'ii appreciate any help. Thanks ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] Determining rx queues of an interface
Thank you! On Wed, Jul 11, 2018 at 5:09 PM Alfredo Cardigliano wrote: > Hi Amir > the number of rx queues is determined loading the driver or at runtime > with ethtool ‘combined’ as you can see at: > > > https://github.com/ntop/PF_RING/blob/dev/drivers/intel/i40e/i40e-2.4.6-zc/src/load_driver.sh#L39 > > Please also make sure that you are actually unloading the old driver > before running insmod (maybe the new insmod is failing) > > Alfredo > > On 10 Jul 2018, at 15:46, Amir Kaduri wrote: > > Do the rx queues of an interface are determined only when loading the > kernel driver, or is there another thing that affects that? > > My problem is that I get a number that differs than what I think should be > (because maybe something during the machine boot doesn't work properly) > > For example: > What I assume our load script do is "*insmod > /usr/local/pfring/drivers/intel/i40e.ko RSS=12,12,12,12,12,12,12,12*" > What I actually get when I run "*cat /proc/net/pf_ring/dev/eth4/info*" > is "*RX Queues:14*" (from previous reboot). > > So maybe the "insmod" doesn't actually work as expected, or is there > anything else that I'm not aware of? > > A'ii appreciate any help. > > Thanks > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > > > ___ > Ntop-misc mailing list > Ntop-misc@listgateway.unipi.it > http://listgateway.unipi.it/mailman/listinfo/ntop-misc ___ Ntop-misc mailing list Ntop-misc@listgateway.unipi.it http://listgateway.unipi.it/mailman/listinfo/ntop-misc
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Hi Alfredo,
I'm back to investigate/debug this issue in my environment, and maybe
you'll manage to save me some time:
When I use the example program "pfcount_multichannel", poll-duration works
for me as expected:
For watermark=128, poll-duration=1000, even if less than 128 packets
received, I get them in pfcount_multichannel.
On the other hand, in my other program (which is a complex one), the
userspace application gets the packets only after 128 packets
aggregated by the ring, regardless the polling rate (which is done always
using 50ms timeout).
Maybe you can figure out what can "hold" the packets in the ring and
forward them to userspace only when the watermark threshold passes?
Maybe something is missing during initialization?
(for simplicity I'm not using rehash, and not using any filters).
Thanks
On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
> Hi Amir
> that's correct, however for some reason it seems it is not the case in
> your tests.
>
> Alfredo
>
> On 31 Oct 2017, at 12:08, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Thanks. tot_insert apparently works ok.
>
> Regarding function copy_data_to_ring():
> At the end of it there is the statement:
> if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
> wake_up_interruptible(>ring_slots_waitqueue);
>
> Since watermark is set to 128, and I send <128 packets, this causes them
> to wait in kernel queue.
> But since poll_duration is set to 1 (1 millisecond I assume), I expect the
> condition to check this also (meaning, there are packets in queue but 1
> millisecond passed and they weren't read),
> the wake_up_interruptible should also be called. No?
>
> Thanks,
> Amir
>
>
> On Tue, Oct 31, 2017 at 10:20 AM, Alfredo Cardigliano <
> cardigli...@ntop.org> wrote:
>
>>
>>
>> On 31 Oct 2017, at 08:42, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Hi Alfredo,
>>
>> I'm trying to debug the issue, and I have a question about the code, to
>> make sure that there is no problem there:
>> Specifically, I'm referring to the function "pfring_mod_recv":
>> In order that the line that refers to poll_duration ("pfring_poll(ring,
>> ring->poll_duration)") will be reached, there are 2 conditions that should
>> occur:
>> 1. pfring_there_is_pkt_available(ring) should return false (otherwise,
>> the function returns at the end of the condition).
>> 2. wait_for_incoming_packet should be set to true.
>> Currently, I'm referring to the first one:
>> In order that the macro pfring_there_is_pkt_available(ring) will return
>> false, ring->slots_info->tot_insert should be equal to
>> ring->slots_info->tot_read.
>> What I see in my tests that they don't get equal. I always see that
>> tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
>> but it happens inside the condition, so the "pfring_mod_recv" returns with
>> 1.
>>
>>
>> It seems to be correct. The kernel module inserts packets into the ring
>> increasing tot_insert, the userspace library reads packets from the ring
>> increasing tot_read. This means that if tot_insert == tot_read there is no
>> packet to read. If there is a bug, it should be in the kernel module that
>> is somehow not adding packets to the ring (thus not updating tot_insert).
>>
>> Alfredo
>>
>> I remind that I set the watermark to be high, in order to see the
>> poll_duration takes effect.
>>
>> Could you please approve that you don't see any problem in the code?
>>
>> Thanks,
>> Amir
>>
>>
>> On Thu, Oct 26, 2017 at 12:22 PM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>> Hi Amir
>>> yes, that’s the way it should work, if this is not the case, some
>>> debugging is needed to identify the problem
>>>
>>> Alfredo
>>>
>>> On 26 Oct 2017, at 10:14, Amir Kaduri <akadur...@gmail.com> wrote:
>>>
>>> Basically, the functionality that I would like to have is even if less
>>> than poll-watermark-threshold (default: 128) packets arrives the socket,
>>> they will be forwarded to userland if 1 millisecond has passed.
>>> How can I gain this? Isn't it by using pfring_set_poll_duration()?
>>>
>>> Alfredo, could you please clarify?
>>>
>>> Thanks,
>>> Amir
>>>
>>> On Wed, Oct 18, 2017 at 8:48 PM, Amir Kaduri <akadur...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm using pf_rin
Re: [Ntop-misc] pf_ring: Clarification regarding the relation between poll-watermark and poll-duration
Hi Alfredo,
My code is based on libpcap, while pfring's userland examples use pfring
APIs directly, therefore things are a bit harder for me.
Short clarification about a related code-line:
Please look at the following line:
https://github.com/ntop/PF_RING/blob/dev/userland/libpcap-1.8.1/pcap-linux.c#L1875
(1) If I understand it correctly, if wait_for_incoming_packet is true,
then pfring_poll() should be called.
Don't you want wait_for_incoming_packet to be true in case
pf_ring_active_poll is true?
Currently, its the opposite (i.e. if pf_ring_active_poll is true,
wait_for_incoming_packet will be false thus pfring_poll() won't be called).
(2) If the code is ok, then the only way for me to make
wait_for_incoming_packet true (for pfring_poll() to be called) is by making
handlep->timeout >= 0.
Correct?
Thanks,
Amir
On Mon, Apr 9, 2018 at 10:51 AM, Alfredo Cardigliano <cardigli...@ntop.org>
wrote:
> Hi Amir
> if I understand correctly, pfcount_multichannel is working, while in your
> application
> it seems that poll does not honor the timeout, if this is the case it
> seems the problem
> is not in the kernel module, I think you should look for differences
> between the two applications..
>
> Alfredo
>
> On 9 Apr 2018, at 07:20, Amir Kaduri <akadur...@gmail.com> wrote:
>
> Hi Alfredo,
>
> I'm back to investigate/debug this issue in my environment, and maybe
> you'll manage to save me some time:
>
> When I use the example program "pfcount_multichannel", poll-duration works
> for me as expected:
> For watermark=128, poll-duration=1000, even if less than 128 packets
> received, I get them in pfcount_multichannel.
>
> On the other hand, in my other program (which is a complex one), the
> userspace application gets the packets only after 128 packets
> aggregated by the ring, regardless the polling rate (which is done always
> using 50ms timeout).
>
> Maybe you can figure out what can "hold" the packets in the ring and
> forward them to userspace only when the watermark threshold passes?
> Maybe something is missing during initialization?
> (for simplicity I'm not using rehash, and not using any filters).
>
> Thanks
>
> On Tue, Oct 31, 2017 at 6:32 PM, Alfredo Cardigliano <cardigli...@ntop.org
> > wrote:
>
>> Hi Amir
>> that's correct, however for some reason it seems it is not the case in
>> your tests.
>>
>> Alfredo
>>
>> On 31 Oct 2017, at 12:08, Amir Kaduri <akadur...@gmail.com> wrote:
>>
>> Thanks. tot_insert apparently works ok.
>>
>> Regarding function copy_data_to_ring():
>> At the end of it there is the statement:
>> if(num_queued_pkts(pfr) >= pfr->poll_num_pkts_watermark)
>> wake_up_interruptible(>ring_slots_waitqueue);
>>
>> Since watermark is set to 128, and I send <128 packets, this causes them
>> to wait in kernel queue.
>> But since poll_duration is set to 1 (1 millisecond I assume), I expect
>> the condition to check this also (meaning, there are packets in queue but 1
>> millisecond passed and they weren't read),
>> the wake_up_interruptible should also be called. No?
>>
>> Thanks,
>> Amir
>>
>>
>> On Tue, Oct 31, 2017 at 10:20 AM, Alfredo Cardigliano <
>> cardigli...@ntop.org> wrote:
>>
>>>
>>>
>>> On 31 Oct 2017, at 08:42, Amir Kaduri <akadur...@gmail.com> wrote:
>>>
>>> Hi Alfredo,
>>>
>>> I'm trying to debug the issue, and I have a question about the code, to
>>> make sure that there is no problem there:
>>> Specifically, I'm referring to the function "pfring_mod_recv":
>>> In order that the line that refers to poll_duration ("pfring_poll(ring,
>>> ring->poll_duration)") will be reached, there are 2 conditions that should
>>> occur:
>>> 1. pfring_there_is_pkt_available(ring) should return false (otherwise,
>>> the function returns at the end of the condition).
>>> 2. wait_for_incoming_packet should be set to true.
>>> Currently, I'm referring to the first one:
>>> In order that the macro pfring_there_is_pkt_available(ring) will return
>>> false, ring->slots_info->tot_insert should be equal to
>>> ring->slots_info->tot_read.
>>> What I see in my tests that they don't get equal. I always see that
>>> tot_insert>tot_read, and sometimes they get eual when tot_read++ is called
>>> but it happens inside the condition, so the "pfring_mod_recv" returns with
>>> 1.
>>>
>>>
>>> It seems to be correct. The kernel module inserts packets i
