Re: [gentoo-user] Rationalizing log files
Hello, On Thu, 13 May 2021, Walter Dnes wrote: [..] > And maybe either stop logging Facebook, or else log iptables messages >to a separate file (how is that done?). The Facebook tracker messages >are generated by iptables rules... > >-A INPUT -s 31.13.24.0/21 -j FECESBOOK >-A INPUT -s 31.13.64.0/18 -j FECESBOOK [..] >-A OUTPUT -d 31.13.24.0/21 -j FECESBOOK >-A OUTPUT -d 31.13.64.0/18 -j FECESBOOK FWIW: For one: why not filter the iptables messages into a seperate logfile? E.g. for syslog-ng (you'll need to add the filter to other filters/log, having them in the filters you can use those more intuitively): filter f_iptables { facility(kern) and message("IN=") and message("OUT="); }; filter f_console{ ... and not filter(f_iptables); } filter f_messages { ... and not filter(f_iptables); }; filter f_warn { ... and not filter(f_iptables); }; [..] log { source(src); source(chroots); filter(f_messages); destination(messages); }; # Firewall (iptables) messages in one file: destination firewall { file("/var/log/firewall" suppress(30)); }; log { source(src); source(chroots); filter(f_iptables); destination(firewall); }; You might be logging more specifically, so you could add more specific filters. That's what those filters (and log-prefixes in iptables) are for after all :) Also add a matching logrotate entry: /var/log/firewall { delaycompress missingok notifempty size +4096k sharedscripts postrotate /etc/init.d/syslog-ng reload > /dev/null 2>&1 || true endscript } or some such as /etc/logrotate.d/firewall (or however you name you iptables-logfile. And second, how about setting up a local dnsmasq to send all fb-crap to NXDOMAIN on the DNS-level? dnsmasq.conf or e.g. /etc/dnsmasq.d/blocklist.conf [1] address=/fb.com/ address=/fb.me/ address=/facebook.net/ address=/facebook.de/ address=/facebook.fr/ address=/facebook.co.uk/ address=/facebook.com/ address=/fbcdn.net/ address=/instagram.com/ address=/instagram.de/ address=/whatsapp.de/ address=/whatsapp.com/ address=/whatsapp.net/ That has the effect that all (sub-)domains with those names give NXDOMAIN, i.e. are non-existant. Compare to: $ nslookup there.is.no.such.domain.invalid Depending on what sites you visit, you might add more domains like e.g. facebook.ca, facebook.mx, facebook.es or whatever fb-domains sites that you visit include... Just as ideas, -dnh [1] you'll need a matching conf-dir or conf-file directive, preferably at the end of the main /etc/dnsmasq.conf then, I use: conf-dir=/etc/dnsmasq.d,*.conf which includes all *.conf files from /etc/dnsmasq.d/ (and ignores other files there like *.conf~ or Makefile or whatnot, so you can be creative and e.g. generate your blocklist from a simple list of domains ;) E.g.: /etc/dnsmasq.d/Makefile all: blocklist.conf blocklist.conf: blocklist.conf.in sort -u $< | sed 's@\(.*\)@address=/&/@' > $@ You get the ideas ;) (and if not: ask!) -- Of course. Anything with more than 2 buttons is too complex. This includes things with 2 or less buttons. This may include clothing. -- Satya
Re: [gentoo-user] Rationalizing log files
On Thu, May 13, 2021 at 03:42:44AM -0500, Dale wrote > Basically, it's two files, that I can find anyway. One is to run it as > a cron and the other tells it what to rotate. If you duplicate that, it > should help. Of course, make sure whatever cron you are using is > running as well. > > Hope that helps. Strange. My files match yours. Manual rotation did not work. I inserted... maxsize 8M ...into syslog-ng and ran logrotate, which finally worked. -- Walter Dnes I don't run "desktop environments"; I run useful applications
Re: [gentoo-user] Rationalizing log files
On Thu, May 13, 2021 at 6:20 AM Mark Knecht wrote: > > > > On Thu, May 13, 2021 at 12:58 AM Walter Dnes wrote: > > > > > # no packages own wtmp and btmp -- we'll rotate them here. > > /var/log/wtmp { > > monthly > > create 0664 root utmp > > minsize 1M > > rotate 1 > > } > > /var/log/btmp { > > missingok > > monthly > > create 0600 root utmp > > rotate 1 > > } > > > As you reported 'roughly monthly' failures my guess would be the above two sections > One additional thought: If the above sections are involved and if it's a bug then you might find it faster changing the above to daily vs monthly. If you were to try this then do them one at a time and change the rotate number to 30 or 40 to keep the data for the month. (I think...) Good luck, Mark
Re: [gentoo-user] Rationalizing log files
On Thu, May 13, 2021 at 12:58 AM Walter Dnes wrote: > > # no packages own wtmp and btmp -- we'll rotate them here. > /var/log/wtmp { > monthly > create 0664 root utmp > minsize 1M > rotate 1 > } > /var/log/btmp { > missingok > monthly > create 0600 root utmp > rotate 1 > } As you reported 'roughly monthly' failures my guess would be the above two sections > And maybe either stop logging Facebook, or else log iptables messages > to a separate file (how is that done?). The Facebook tracker messages > are generated by iptables rules... Don't log what you're not interested in. If your disk is getting filled up with billions of Facebook issues then limit how much of that you track. HTH, Mark
Re: [gentoo-user] Rationalizing log files
Walter Dnes wrote: > On another thread, I had to dive into into /var/log/messages, and I > realized that it was not being rotated. It's 32 megabytes+, most of > which is iptables reject messages for Facebook trackers. What do I need > to do to get log rotation working? > > /etc/logrotate.conf > > > > # > # Default logrotate(8) configuration file for Gentoo Linux. > # See "man logrotate" for details. > > # rotate log files weekly. > weekly > #daily > > # keep 4 weeks worth of backlogs. > rotate 4 > > # create new (empty) log files after rotating old ones. > create > > # use date as a suffix of the rotated file. > dateext > > # compress rotated log files. > compress > > notifempty > nomail > noolddir > > # packages can drop log rotation information into this directory. > include /etc/logrotate.d > > # no packages own wtmp and btmp -- we'll rotate them here. > /var/log/wtmp { > monthly > create 0664 root utmp > minsize 1M > rotate 1 > } > /var/log/btmp { > missingok > monthly > create 0600 root utmp > rotate 1 > } > > # system-specific logs may be also be configured here. > > > > /etc/logrotate.d contains... > dcron elog-save-summary hibernate-script openrc rsyncd syslog-ng > > > > And maybe either stop logging Facebook, or else log iptables messages > to a separate file (how is that done?). The Facebook tracker messages > are generated by iptables rules... > > -A INPUT -s 31.13.24.0/21 -j FECESBOOK > -A INPUT -s 31.13.64.0/18 -j FECESBOOK > -A INPUT -s 66.220.144.0/20 -j FECESBOOK > -A INPUT -s 69.63.176.0/20 -j FECESBOOK > -A INPUT -s 69.171.224.0/19 -j FECESBOOK > -A INPUT -s 74.119.76.0/22 -j FECESBOOK > -A INPUT -s 103.4.96.0/22 -j FECESBOOK > -A INPUT -s 173.252.64.0/18 -j FECESBOOK > -A INPUT -s 204.15.20.0/22 -j FECESBOOK > > -A OUTPUT -d 31.13.24.0/21 -j FECESBOOK > -A OUTPUT -d 31.13.64.0/18 -j FECESBOOK > -A OUTPUT -d 66.220.144.0/20 -j FECESBOOK > -A OUTPUT -d 69.63.176.0/20 -j FECESBOOK > -A OUTPUT -d 69.171.224.0/19 -j FECESBOOK > -A OUTPUT -d 74.119.76.0/22 -j FECESBOOK > -A OUTPUT -d 103.4.96.0/22 -j FECESBOOK > -A OUTPUT -d 173.252.64.0/18 -j FECESBOOK > -A OUTPUT -d 204.15.20.0/22 -j FECESBOOK > > -A FECESBOOK -j LOG --log-prefix "FECESBOOK:" --log-level 6 > -A FECESBOOK -j REJECT --reject-with icmp-port-unreachable > I may be missing something but this is what I could find on my system. root@fireball / # cat /etc/cron.daily/logrotate #!/bin/sh /usr/bin/logrotate /etc/logrotate.conf EXITVALUE=$? if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]" fi exit $EXITVALUE root@fireball / # cat /etc/logrotate.d/syslog-ng # # Syslog-ng logrotate snippet for Gentoo Linux # contributed by Michael Sterrett # /var/log/messages { delaycompress missingok sharedscripts postrotate /etc/init.d/syslog-ng reload > /dev/null 2>&1 || true endscript } root@fireball / # Basically, it's two files, that I can find anyway. One is to run it as a cron and the other tells it what to rotate. If you duplicate that, it should help. Of course, make sure whatever cron you are using is running as well. Hope that helps. Dale :-) :-)
[gentoo-user] Rationalizing log files
On another thread, I had to dive into into /var/log/messages, and I realized that it was not being rotated. It's 32 megabytes+, most of which is iptables reject messages for Facebook trackers. What do I need to do to get log rotation working? /etc/logrotate.conf # # Default logrotate(8) configuration file for Gentoo Linux. # See "man logrotate" for details. # rotate log files weekly. weekly #daily # keep 4 weeks worth of backlogs. rotate 4 # create new (empty) log files after rotating old ones. create # use date as a suffix of the rotated file. dateext # compress rotated log files. compress notifempty nomail noolddir # packages can drop log rotation information into this directory. include /etc/logrotate.d # no packages own wtmp and btmp -- we'll rotate them here. /var/log/wtmp { monthly create 0664 root utmp minsize 1M rotate 1 } /var/log/btmp { missingok monthly create 0600 root utmp rotate 1 } # system-specific logs may be also be configured here. /etc/logrotate.d contains... dcron elog-save-summary hibernate-script openrc rsyncd syslog-ng And maybe either stop logging Facebook, or else log iptables messages to a separate file (how is that done?). The Facebook tracker messages are generated by iptables rules... -A INPUT -s 31.13.24.0/21 -j FECESBOOK -A INPUT -s 31.13.64.0/18 -j FECESBOOK -A INPUT -s 66.220.144.0/20 -j FECESBOOK -A INPUT -s 69.63.176.0/20 -j FECESBOOK -A INPUT -s 69.171.224.0/19 -j FECESBOOK -A INPUT -s 74.119.76.0/22 -j FECESBOOK -A INPUT -s 103.4.96.0/22 -j FECESBOOK -A INPUT -s 173.252.64.0/18 -j FECESBOOK -A INPUT -s 204.15.20.0/22 -j FECESBOOK -A OUTPUT -d 31.13.24.0/21 -j FECESBOOK -A OUTPUT -d 31.13.64.0/18 -j FECESBOOK -A OUTPUT -d 66.220.144.0/20 -j FECESBOOK -A OUTPUT -d 69.63.176.0/20 -j FECESBOOK -A OUTPUT -d 69.171.224.0/19 -j FECESBOOK -A OUTPUT -d 74.119.76.0/22 -j FECESBOOK -A OUTPUT -d 103.4.96.0/22 -j FECESBOOK -A OUTPUT -d 173.252.64.0/18 -j FECESBOOK -A OUTPUT -d 204.15.20.0/22 -j FECESBOOK -A FECESBOOK -j LOG --log-prefix "FECESBOOK:" --log-level 6 -A FECESBOOK -j REJECT --reject-with icmp-port-unreachable -- Walter Dnes I don't run "desktop environments"; I run useful applications