pf-badhost and unbound-adblock v0.5 released
Hey folks, Since I've been getting a lot of emails about this, I just thought I'd drop a line here and let you know that I've released pf-badhost and unbound-adblock version 0.5. You can see the release pages here: pf-badhost is a fast, in-kernel, bi-directional network filtering utility powered by the PF firewall. https://geoghegan.ca/pfbadhost.html unbound-adblock is a fast, flexible, easy to use DNS firewall utility. https://geoghegan.ca/unbound-adblock.html This will be the last time I advertise my wares on here as I have established a notification service -- just send an email to "annou...@geoghegan.ca" to subscribe. One new feature that I would like to highlight is that unbound-adblock now supports florian@'s excellent unwind(8) resolver as an optional backend. I've been running this on my laptop for about a month now and it's working quite nicely. Thank you to everyone in the the OpenBSD community who have on many occasions, kindly assisted with development in one way or the other -- I am truly grateful. Regards, Jordan Geoghegan
Re: Understanding memory statistics
On Sun, Jan 10, 2021 at 09:34:49PM +, Anindya Mukherjee wrote: > Hi, I'm trying to understand the various numbers reported for memory > usage from top, vmstat, and systat. I'm running OpenBSD 6.8 on a Dell > Optiplex 7040 with an i7 6700, 3.4 Ghz and 32 GB RAM. The GPU is an > Intel HD Graphics 530, integrated. Everything is running smoothly. For > my own edification, I have a few questions. I searched the mailing lists > for similar questions in the past, and found some, but they did not > fully satisfy my curiosity. > > dmesg reports: > real mem = 34201006080 (32616MB) > avail mem = 33149427712 (31613MB) > I think the difference is due to the GPU reserving some memory. That might be, I think it at least includes mem used by the kernel for its code and static data. > Q: Is there a way to view the total amount of video memory, the amount > currently being used, and the GPU usage? AFAIK not. Some bioses have settings for the video mem used (if you have it shared with main mem). > > When I run top, it reports the following memory usage: > Memory: Real: 1497M/4672M act/tot Free: 26G Cache: 2236M Swap: 0K/11G > If I sum up the RES numbers for all the processes, it is close to the > act number = 1497 M (this is mostly due to Firefox). I read that the > cache number is included in tot, but even if I subtract cache and act > from tot there is 939 MB left. > Q: What is this 939 MB being used for, assuming the above makes sense? inactive pages? > Q: What is the cache number indicating exactly? memoy used for file systemn caching. > > If I sum up tot + free * 1024 I get 31296 MB, which less than the 31613 > MB of available memory reported by dmesg. I initially assumed that the > difference might be kernel wired memory. However the uvm view of systat > shows 7514 wired pages = approx 30 MB which is very small. > Q: What is the remaining memory being used for? I think you are looking at dynamic allocations done by the kernel. > Q: What is in kernel wired memory? In particular, is the file system > cache in kernel wired memory or in the cache number? Kernel wired means data pages allocated by the kernel that will not be paged out. The file system mem will also not be paged out (when evecting those they are discarded if not dirty or written to the file if dirty) but the file system cache pages are not in the wired count. > In the man page for systat(1) the active memory is described as being > used by processes active in the last 20 seconds (recently), while the > total is for all processes. These are the same two numbers as act and > tot in top, and act = avm as reported by vmstat. This confused me > because adding up the RES sizes of all the processes I get nowhere near > to tot (even after subtracting cache). Accounting of shared pages is hard and ambiguous. To illustrate: if you switch on S in top, you'll see a bunch of kenel space processes al at SIZE 0 and the same RES size. They do share the same (kernel) memory. > > There is another thing that confused me in the top output. At first I > assumed that SIZE is the total virtual memory size of the process > (allocated), while RES is the resident size. For example, this is so on > Linux and hence in that case by definition SIZE should always be greater > than RES. However here in many cases SIZE < RES. I am unsure how that is caused. It is possibly a shared pages thing. > > I read in the man page for top that SIZE is actually text + data + stack > for the process. However this did not clear up my confusion or > misunderstanding. Perhaps something to do with shared memory not being > counted? > Q: How can SIZE be less that RES? An example showing how this could > happen would be really helpful. I guess doing some experimenting and code analysis and share your findings. > > Q: Finally, where can I find documentation about the classification for > memory pages (active, inactive, wired, etc.)? I suspect some digging > around in the source in order, but could use some pointers. The start would be man uvm_init. But the rest is code. > > I hope these make sense and are not too pedantic. Looking forward to > comments from the experts, thanks! > > Anindya Mukherjee > -Otto
Re: Suggestion for small improvement in acme-client.conf.5
On Sat, Jan 09, 2021 at 05:08:14PM +0100, Wolf wrote: > Hello, > > I have small suggestion for improving man page for acme-client.conf.5. > Basically just adding "comma separated" to clarify on the format of the > list for alternative names. I had to dig into the parser.y to figure > this out, so it would be nice to have it documented. > hi. a modified version of this diff now committed. thanks, jmc > diff --git a/acme-client.conf.5 b/acme-client.conf.5 > index 7971fb6..a47a8e2 100644 > --- a/acme-client.conf.5 > +++ b/acme-client.conf.5 > @@ -125,9 +125,9 @@ If not specified, the > .Ar handle > of the domain block will be used as common name. > .It Ic alternative names Brq ... > -Specify a list of alternative names for which the certificate will be valid. > -The common name is included automatically if this option is present, > -but there is no automatic conversion/inclusion between "www." and > +Specify a comma separated list of alternative names for which the certificate > +will be valid. The common name is included automatically if this option is > +present, but there is no automatic conversion/inclusion between "www." and > plain domain name forms. > .It Ic domain key Ar file Op Ar keytype > The private key file for which the certificate will be obtained. > > > Have a nice day, > W. > > -- > There are only two hard things in Computer Science: > cache invalidation, naming things and off-by-one errors.
httpd fail to serve page with default httpd.conf; it shows: 403 Forbidden.
Hello misc list:
I have had a Web Server at home for 20 years, and this time, i am not
able to discover the error! I am Agronomist, then my knowledge is in
other field.
I rented a server at vultr, with clean installation, because i lost my
Laptop and back ups.
I created rc.conf.local and added httpd_flags="" and without any change
i did a reboot.
After that, acme-client -v my_domain, then did the test with default
httpd.conf! it worked for 1 second and 403 Forbidden message appeared!
httpd -n says OK, permissions seem ok to me, i have not touche them.
Your help and OS are very much appreciated
The new httpd.conf using my_domain:
# $OpenBSD: httpd.conf,v 1.20 2018/06/13 15:08:24 reyk Exp $
server "my_domain" {
listen on * port 80
location "/.well-known/acme-challenge/*" {
root "/acme"
request strip 2
}
location * {
block return 302 "https://$HTTP_HOST$REQUEST_URI;
}
}
server "my_domain" {
listen on * tls port 443
tls {
certificate "/etc/ssl/fullchain.pem"
key "/etc/ssl/private/key"
}
location "/pub/*" {
directory auto index
}
location "/.well-known/acme-challenge/*" {
root "/acme"
request strip 2
}
}
Understanding memory statistics
Hi, I'm trying to understand the various numbers reported for memory usage from top, vmstat, and systat. I'm running OpenBSD 6.8 on a Dell Optiplex 7040 with an i7 6700, 3.4 Ghz and 32 GB RAM. The GPU is an Intel HD Graphics 530, integrated. Everything is running smoothly. For my own edification, I have a few questions. I searched the mailing lists for similar questions in the past, and found some, but they did not fully satisfy my curiosity. dmesg reports: real mem = 34201006080 (32616MB) avail mem = 33149427712 (31613MB) I think the difference is due to the GPU reserving some memory. Q: Is there a way to view the total amount of video memory, the amount currently being used, and the GPU usage? When I run top, it reports the following memory usage: Memory: Real: 1497M/4672M act/tot Free: 26G Cache: 2236M Swap: 0K/11G If I sum up the RES numbers for all the processes, it is close to the act number = 1497 M (this is mostly due to Firefox). I read that the cache number is included in tot, but even if I subtract cache and act from tot there is 939 MB left. Q: What is this 939 MB being used for, assuming the above makes sense? Q: What is the cache number indicating exactly? If I sum up tot + free * 1024 I get 31296 MB, which less than the 31613 MB of available memory reported by dmesg. I initially assumed that the difference might be kernel wired memory. However the uvm view of systat shows 7514 wired pages = approx 30 MB which is very small. Q: What is the remaining memory being used for? Q: What is in kernel wired memory? In particular, is the file system cache in kernel wired memory or in the cache number? In the man page for systat(1) the active memory is described as being used by processes active in the last 20 seconds (recently), while the total is for all processes. These are the same two numbers as act and tot in top, and act = avm as reported by vmstat. This confused me because adding up the RES sizes of all the processes I get nowhere near to tot (even after subtracting cache). There is another thing that confused me in the top output. At first I assumed that SIZE is the total virtual memory size of the process (allocated), while RES is the resident size. For example, this is so on Linux and hence in that case by definition SIZE should always be greater than RES. However here in many cases SIZE < RES. I read in the man page for top that SIZE is actually text + data + stack for the process. However this did not clear up my confusion or misunderstanding. Perhaps something to do with shared memory not being counted? Q: How can SIZE be less that RES? An example showing how this could happen would be really helpful. Q: Finally, where can I find documentation about the classification for memory pages (active, inactive, wired, etc.)? I suspect some digging around in the source in order, but could use some pointers. I hope these make sense and are not too pedantic. Looking forward to comments from the experts, thanks! Anindya Mukherjee
Re: pf: brute-force ssh defence no longer working in OpenBSD 6.8
I'd said: >> Checking the pf log, it's definitely the final (pass quick) rule which is letting them in. And yes, dumping the table does indeed show the IP address(es) in question. So the block doesn't appear to be doing anything. Am I being a dumbass? Have I missed some subtle change in pf behaviour which is breaking my filter? << Peter N. M. Hansteen replied: >> Taking a peek at what I run the main difference I see is that I do a block by default at the very beginning of my pf.conf << Well, that's embarrassing. I'm officially an idiot. I *always* have a default deny at the start of pf.conf. Except this time, I didn't, and didn't spot the omission depsite reviewing it, well, a lot. Oops. (I did say it'd been a while...) Thank you, Peter, for setting this old twit right. Steve -- -- Steve Fairhead fivetrees ltd - for the complete music service www: http://www.fivetrees.com --
Re: -current amd64 packages not updated? Impatient or broken?
> While at it, link /bin/ls to /bin/rm An Apple fanboy trying to look 1337 in a linux style on an OpenBSD mailing list. Impressive not.
Re: Buying a New Laptop
I used a Thinkpad 13 2nd Gen with OpenBSD. Everything worked as expected, all the function keys worked, no problems with power management. I'm willing to bet that most modern Thinkpads can handle OpenBSD. There might be models that don't support OpenBSD but they're few and far between. Ben Raskin.
Re: pf: brute-force ssh defence no longer working in OpenBSD 6.8
> 10. jan. 2021 kl. 14:47 skrev Steve Fairhead : > > Hi folks, > > I hope I'm just missing something stupid. It's been a while since I deployed > public OpenBSD servers, but I've done plenty. I always use a defence in > pf.conf against brute-force SSH attacks, which has served me well in the past. > > On a new machine running 6.8, this no longer appears to work. I've stripped > it back to: > > table persist file "/etc/scanners" > > block quick from > > pass quick proto tcp from any to any port ssh flags S/SA keep state \ > (max-src-conn 10, max-src-conn-rate 3/15, overload flush > global) > > (taken directly from https://home.nuug.no/~peter/pf/en/bruteforce.html ) Taking a peek at what I run the main difference I see is that I do a block by default at the very beginning of my pf.conf, and # pass proto tcp to port ssh pass in quick log (all) on egress proto tcp to port ssh flags S/SA keep state \ (max-src-conn 15, max-src-conn-rate 2/10, overload flush global, pflow) I also run a cron job twice an hour to stuff anything trying for obvious stupidity like logging in as root or admin get added to the table (some handwaving tips at https://bsdly.blogspot.com/2017/04/forcing-password-gropers-through.html - Peter — Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://bsdly.blogspot.com/ http://www.bsdly.net/ http://www.nuug.no/ "Remember to set the evil bit on all malicious network traffic" delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds. signature.asc Description: Message signed with OpenPGP
pf: brute-force ssh defence no longer working in OpenBSD 6.8
Hi folks, I hope I'm just missing something stupid. It's been a while since I deployed public OpenBSD servers, but I've done plenty. I always use a defence in pf.conf against brute-force SSH attacks, which has served me well in the past. On a new machine running 6.8, this no longer appears to work. I've stripped it back to: table persist file "/etc/scanners" block quick from pass quick proto tcp from any to any port ssh flags S/SA keep state \ (max-src-conn 10, max-src-conn-rate 3/15, overload flush global) (taken directly from https://home.nuug.no/~peter/pf/en/bruteforce.html ) But: am still seeing e.g. Jan 10 13:25:20 ns3 sshd[3233]: Failed password for invalid user admin from 67.1.238.105 port 47102 ssh2 Jan 10 13:25:21 ns3 last message repeated 5 times Jan 10 13:25:21 ns3 sshd[3233]: error: maximum authentication attempts exceeded for invalid user admin from 67.1.238.105 port 47102 ssh2 [preauth] Jan 10 13:25:21 ns3 sshd[3233]: Disconnecting invalid user admin 67.1.238.105 port 47102: Too many authentication failures [preauth] Jan 10 13:25:25 ns3 sshd[98147]: Invalid user admin from 67.1.238.105 port 47232 Jan 10 13:25:25 ns3 sshd[98147]: Failed password for invalid user admin from 67.1.238.105 port 47232 ssh2 Jan 10 13:25:26 ns3 last message repeated 5 times Jan 10 13:25:26 ns3 sshd[98147]: error: maximum authentication attempts exceeded for invalid user admin from 67.1.238.105 port 47232 ssh2 [preauth] Jan 10 13:25:26 ns3 sshd[98147]: Disconnecting invalid user admin 67.1.238.105 port 47232: Too many authentication failures [preauth] Jan 10 13:25:32 ns3 sshd[17711]: Invalid user admin from 67.1.238.105 port 47366 On an older server, searching for "repeated" in authlog shows a typical max of 2 times. Checking the pf log, it's definitely the final (pass quick) rule which is letting them in. And yes, dumping the table does indeed show the IP address(es) in question. So the block doesn't appear to be doing anything. Am I being a dumbass? Have I missed some subtle change in pf behaviour which is breaking my filter? Thanks, Steve -- -- Steve Fairhead fivetrees ltd - for the complete music service www: http://www.fivetrees.com --
Re: cmp(1) '-s' flag ignoring byte offset argument?
On 1/9/21 1:59 AM, Otto Moerbeek wrote:
> On Sat, Jan 09, 2021 at 12:05:31AM -0800, William Ahern wrote:
>
>> On Fri, Jan 08, 2021 at 07:09:01PM -0800, Jordan Geoghegan wrote:
>>> Hey folks,
>>>
>>> I've noticed some surprising behaviour from cmp(1) when using the '-s'
>>> flag.
>>>
>>> It appears that cmp -s is ignoring the byte offset arguments I'm giving
>>> it.
>>
>>> Not sure what to make of this, I noticed this same behaviour on
>>> DragonflyBSD and FreeBSD, so maybe I'm just missing something obvious.
>>> This certainly caused some frustration before I figured out what was going
>>> on.
>> The bug seems to be in the short-circuit optimization for regular files[1]:
>>
>> void
>> c_regular(int fd1, char *file1, off_t skip1, off_t len1,
>> int fd2, char *file2, off_t skip2, off_t len2)
>> {
>> u_char ch, *p1, *p2;
>> off_t byte, length, line;
>> int dfound;
>>
>> if (sflag && len1 != len2)
>> exit(1);
>>
>> if (skip1 > len1)
>> eofmsg(file1);
>> len1 -= skip1;
>> if (skip2 > len2)
>> eofmsg(file2);
>> len2 -= skip2;
>>
>> The short-circuit should probably be moved below the subsequent chunk of
>> code (i.e. below `len2 -= skip2`). The eofmsg function already obeys sflag,
>> so it'll be quiet.[2] Doing this works for me. See patch at end of message.
>>
>> Interestingly, DragonflyBSD and FreeBSD already do it this way[3][4], yet I
>> can confirm FreeBSD still has the problem. (DragonflyBSD has nearly
>> identical code.) But that implementation duplicates the short-circuit, along
>> with the bug of not accounting for skip1 and skip2, in cmp.c as part of
>> implementing the -z flag[5]:
>>
>> if (special)
>> c_special(fd1, file1, skip1, fd2, file2, skip2);
>> else {
>> if (zflag && sb1.st_size != sb2.st_size) {
>> if (!sflag)
>> (void) printf("%s %s differ: size\n",
>> file1, file2);
>> exit(DIFF_EXIT);
>> }
>> c_regular(fd1, file1, skip1, sb1.st_size,
>> fd2, file2, skip2, sb2.st_size);
>> }
>> exit(0);
>>
>> It appears that the June 20, 2000 fix to the short-circuit in regular.c
>> wasn't recognized during the July 14, 2000 -z feature addition.[6][7]
>>
>> [1] https://cvsweb.openbsd.org/src/usr.bin/cmp/regular.c?rev=1.12
>> [2] https://cvsweb.openbsd.org/src/usr.bin/cmp/misc.c?rev=1.7
>> [3]
>> https://gitweb.dragonflybsd.org/dragonfly.git/blob/4d4f84f:/usr.bin/cmp/regular.c
>> [4]
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/regular.c?revision=344551
>> [5]
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/cmp.c?revision=344551=markup#l193
>> [6]
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/regular.c?revision=61883=markup
>> [7]
>> https://svnweb.freebsd.org/base/head/usr.bin/cmp/cmp.c?view=markup=63157
>>
>> --- regular.c6 Feb 2015 23:21:59 - 1.12
>> +++ regular.c9 Jan 2021 07:51:13 -
>> @@ -51,15 +51,15 @@ c_regular(int fd1, char *file1, off_t sk
>> off_t byte, length, line;
>> int dfound;
>>
>> -if (sflag && len1 != len2)
>> -exit(1);
>> -
>> if (skip1 > len1)
>> eofmsg(file1);
>> len1 -= skip1;
>> if (skip2 > len2)
>> eofmsg(file2);
>> len2 -= skip2;
>> +
>> +if (sflag && len1 != len2)
>> +exit(1);
>>
>> length = MINIMUM(len1, len2);
>> if (length > SIZE_MAX) {
>>
> I came to the same diff independently. In the meantime it has been committed.
>
> -Otto
>
Hi Otto and William,
Thanks for confirming that this is a bug, and also for the fixes!
Regards,
Jordan
Re: adding user to a group
On 10.1.2021 12:06, Ville Valkonen wrote: Not true. It's opposite. -- Ville I tried that lately on one of installs. Using -G it removed me from already assigned groups and left only that new one provided (unless you specify ALL of them after -G switch) While using -S resulted in behavior which I can remember and is available on other systems when using -G I say nothing against it, I just say I did not notice this particular change somewhere in the past which is of course my fault. On Fri 8. Jan 2021 at 19.53, Bodie wrote: On 8.1.2021 16:21, Rudolf Sykora wrote: > Dear list, > > > I tried to add myself to the "dialer" group: > > #usermod -G dialer ruda > > But when I write > > $groups > > in a terminal I still do not see the new group. Not even if I open a > new login > shell (by writing "ksh -l"). However, when I log in in a text console > (ctrl-alt-1), I see the new group there. > > What is it that I have to do to have the membership updated, i.e., how > can I open e.g. a terminal in the running environment that would see my > new groups? > > > Thanks for comments > Ruda There seems to be some change in behavior in OpenBSD and to be honest do not know when it happened. This is your start https://man.openbsd.org/user which will get you to https://man.openbsd.org/usermod.8 BUT using -G resets your membership and you will be in only group you specified. If you want to add additional group you need to use -S instead
Re: adding user to a group
Not true. It's opposite. -- Ville On Fri 8. Jan 2021 at 19.53, Bodie wrote: > > > On 8.1.2021 16:21, Rudolf Sykora wrote: > > Dear list, > > > > > > I tried to add myself to the "dialer" group: > > > > #usermod -G dialer ruda > > > > But when I write > > > > $groups > > > > in a terminal I still do not see the new group. Not even if I open a > > new login > > shell (by writing "ksh -l"). However, when I log in in a text console > > (ctrl-alt-1), I see the new group there. > > > > What is it that I have to do to have the membership updated, i.e., how > > can I open e.g. a terminal in the running environment that would see my > > new groups? > > > > > > Thanks for comments > > Ruda > > There seems to be some change in behavior in OpenBSD and to be honest do > not > know when it happened. > > This is your start https://man.openbsd.org/user > > which will get you to https://man.openbsd.org/usermod.8 > > BUT using -G resets your membership and you will be in only group you > specified. > If you want to add additional group you need to use -S instead > >
