Emanuele,
Here is ntopng.conf
-G=/var/run/ntopng.pid
-i=enp2s0
-m=10.12.17.0/24 <http://10.12.17.0/24>
-S=local
I do see unidirectional flows in flows_stats.lua for DNS.
Incidentally, I do also see alerts w/ non-zero replies (though most
alerts are 0):
Host pihole has sent 211 DNS requests but received 7 DNS replies
I tried 2 different 30 minute PCAP files. In both cases, right at the
10 minute mark, I got alerts. How can I get these PCAP files to you?
Thanks,
Aaron
On Tue, May 12, 2020 at 4:13 AM Emanuele Faranda <[email protected]
<mailto:[email protected]>> wrote:
Hi Aaron,
Please see below.
On 5/11/20 9:29 PM, Aaron Scamehorn wrote:
Hi Emanuele,
Thank you again for the detailed responses.
From the interfaces page, I see these stats:
Total Traffic 91.6 GB [103,062,265 Pkts] Dropped Packets
0 Pkts
I don't see any dropped packets on the NIC either:
ethtool -S enp2s0
NIC statistics:
tx_packets: 0
rx_packets: 106581943
tx_errors: 0
rx_errors: 0
rx_missed: 0
align_errors: 0
tx_single_collisions: 0
tx_multi_collisions: 0
unicast: 105432876
broadcast: 350738
multicast: 1149060
tx_aborted: 0
tx_underrun: 0
As of right now, 2 of the hosts we are discussing are still in
alert, at the original Date/Time of 07:25:01, and Duration is now
"3 Days, 08:06:59".
Given that my replies vs requests ratio is still configured at
50%, this means that, at every 5 minute interval for the last 3
Days, 8 hours, said host is receiving < 50% DNS replies,
correct? I find this difficult to believe, and cannot find ANY
missing packets in my pcap file.
I have captured a 30 minute pcap file captured with this command:
tcpdump -i enp2s0 -G 1800 -w /tmp/enp2s0.%FT%T.pcap host edgemax
and port 53
This file contains DNS traffic to/from edgemax only.
I can count responses like this:
tshark -t a -r enp2s0.2020-05-11T13:00:02.pcap | grep -c
"Standard query response"
349
And queries like this:
tshark -t a -r enp2s0.2020-05-11T13:00:02.pcap | grep -c
"Standard query 0x"
349
In other words, no missing DNS responses in the 30 minutes
spanning 13:00:02 to 13:29:51.
I would think that the alert should "clear" because the threshold
is not exceeded within that 30 minute pcap file.
In any case, at 13:23, I manually click on the "Release" button
for that alert. 2 minutes later, at 13:25:00, I receive this alert:
Host edgemax has received 62 DNS requests but sent 0 DNS replies
[5 Minutes ratio: 0%]
As stated previously, no missing DNS responses in the 30 minutes
spanning 13:00:02 to 13:29:51. Why does ntopng think 62 replies
are missing?
Please report your ntopng.conf. If you look at the active ntopng
DNS flows, can you identify unidirectional flows? You can also try
to run ntopng on the PCAP file (--original-speed -i file.pcap). If
you can reproduce using the PCAP file, please send it to me
privately so that I can troubleshoot the problem.
I exported 10 minutes of PCAP from if_stats.lua. Using the filter
"(ip.dst_host == "10.12.17.1" or ip.src_host == "10.12.17.1") and
dns" I am not able to find any missing DNS responses in
wireshark. Interestingly, If I specify a BPF Filter ("port 53"),
the downloaded PCAP file seems to only have 1 side (ie. edgemax
is only a source, never a dest. Without a BPF Filter, the
download is fine.
This is probably a bug, please open an issue at
https://github.com/ntop/ntopng .
Regards,
Emanuele
On Mon, May 11, 2020 at 8:59 AM Emanuele Faranda
<[email protected] <mailto:[email protected]>> wrote:
Hi Aaron,
Please see below:
On 5/8/20 10:27 PM, Aaron Scamehorn wrote:
Thank you for your response. In the screenshot below, can
you please explain the significance of the "Date/Time" and
the "Duration" columns? What do they mean in this context?
Date/Time: the time when the alert was triggered. Ntopng
performs periodic checks in order to trigger alerts. In this
particular case, the check on the requests/reply ratio is
performed every 5 minutes. So this means that problem started
between 07:20 and 07:25 .
Duration: the total time in which the problem was active.
Again, the check is performed every 5 minutes for this alert
so 5 minutes is the granularity.
Do I understand correctly that all 3 hosts triggered the
alert at 07:25:01 (OR 07:30:01) this morning? And that all
three alerts are active for the past 07:28:53 hours? Does
this mean that there have been no new additional DNS
Reply/Request issues have been detected?
As explained above, the problem started between 07:20 and
07:25 . For 07:28:53 hours the problem was active on all the
three hosts (the requests/reply ratio threshold was exceeded
for 07:28:53 hours).
I notice in "Past Alerts" tab, that there are many
Reply/Request Alerts for the same host with very short
durations (screen shot #2). When/how does an alert move
from the "Engaged" to "Past" tab?
In this case, the engaged alert becomes "past" alert when,
after the check performed every 5 minutes, the requests/reply
ratio threshold is not exceed anymore. This can happen as
soon as the next check is performed (5 minutes).
So in the 2nd screenshot, fire-TV had an alert at 06:20:00
for 05:00 minutes where 18 requests received 0 replies.
Then another alert at 06:50:00 for 05:00 minutes. Were the
18 replies from the first alert ultimately received? And
they were received 5 minutes the alert occurred?
The check is performed on the DNS packet counters. A DNS
request cannot take 5 minutes to be replied. The fact that
the alert was closed after 5/10 minutes could be related to
one of these events:
- The host went idle
- The host did not send enough DNS requests
- The new DNS requests made by the host were successfully
replied.
Context here is that 99% of the traffic is Internet
traffic. Almost all of the pihole traffic is to
forwarders. BTW, the way pihole works (by default) is it
replies 0.0.0.0 for blocked hosts. It should respond to
every query.
I tried the live_pcap_download.html
<https://www.ntop.org/guides/ntopng/advanced_features/live_pcap_download.html>
lua, but couldn't figure out the bpf_filter:
curl --cookie "user=admin; password=xxxxx"
"http://10.12.17.25:3000/lua/live_traffic.lua?ifid=0&duration=600&bpf_filter=\
<http://10.12.17.25:3000/lua/live_traffic.lua?ifid=0&duration=600&bpf_filter=%5C>"port
53\""
I also tried the download pcap on the if_stats.lua page.
The downloaded pcap file seems to only contain incoming data
(see wireshark)?
This is consistent with the above alerts, please ensure that
ntopng is not dropping packets as this would explain this
behavior.
If I just do a tshark on the same interface that ntopng is
listening on, I see all of the expected DNS query &
replies. I am not able to correlate the alerts to any
missing packets.
See response above.
Regards,
Emanuele
On Fri, May 8, 2020 at 2:53 AM Emanuele Faranda
<[email protected] <mailto:[email protected]>> wrote:
Hi Aaron,
The alerts that you are reporting basically tell you
that such hosts receive DNS requests but do not send a
reply. In order to troubleshoot possible problems you
should augment such information with the knowledge of
your network.
The first question to answer is, are that hosts expected
to accept DNS requests? If not, are the requests
generated from the internet or from the LAN? In the
first case a firewall to block such DNS requests may be
a good idea . In the latter case some hosts in the LAN
may be misconfigured. In case of the pihole hosts, I
expect pihole to block some DNS requests for
advertisement sites so this could be a normal behaviour.
The following ntopng features may also help you:
https://www.ntop.org/guides/ntopng/advanced_features/live_pcap_download.html
https://www.ntop.org/guides/ntopng/using_with_other_tools/n2disk.html
https://www.ntop.org/guides/ntopng/historical_flows.html
Regards,
Emanuele
On 5/7/20 5:57 PM, Aaron Scamehorn wrote:
Hello,
I'm trying to understand how/why I am getting the
"Replies / Requests Ratio" warnings for DNS.
I am suspect of these alerts, and would like to know
how/why they are being generated. I am suspect for for
the following reasons: 1) If it really is as bad as
indicated, I should notice problems. 2) the "events'
occur immediately after I clear the alerts, and tend to
persist for hours.
In any case, I cleared the alerts last night, and this
is what they look like:
06/05/2020 22:15:00 12:31:28 Warning Replies
/
Requests Ratio Host edgemax.example.net
<http://xps-630i.scamlan.net:3000/lua/host_details.lua?ifid=2&host=10.12.17.1@1&page=historical&epoch_begin=1588864588&epoch_end=1588868188>
has received 54 DNS requests but sent 0 DNS replies [5
Minutes ratio: 0%]
06/05/2020 22:15:00 12:31:28 Warning Replies
/
Requests Ratio Host pihole.example.net
<http://xps-630i.scamlan.net:3000/lua/host_details.lua?ifid=2&host=10.12.17.3@1&page=historical&epoch_begin=1588864588&epoch_end=1588868188>
has sent 93 DNS requests but received 3 DNS replies [5
Minutes ratio: 3.2%]
06/05/2020 22:15:00 12:31:28 Warning Replies
/
Requests Ratio Host pihole-2.example.net
<http://xps-630i.scamlan.net:3000/lua/host_details.lua?ifid=2&host=10.12.17.4@1&page=historical&epoch_begin=1588864588&epoch_end=1588868188>
has sent 97 DNS requests but received 1 DNS reply [5
Minutes ratio: 1.0%]
_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected] <mailto:[email protected]>
http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop
