Re: malloc: 1st small step in long way to multiple pools
On Tue, Mar 15, 2016 at 10:22:23PM +0100, Norman Golisz wrote: > Hi Otto, > > On Wed Mar 9 2016 10:06, Otto Moerbeek wrote: > > Lightly tested by myself on amd64, you can help by reviewing and > > testing this. > > I also don't see regressions on my amd64 - running since 3 days on my > production system. Thanks to all testers and reviewers, the diff has been commited. -Otto
Re: Update `ticks' in the right place
This doesn't use atomic operation, because:
- hardclock() is the only writer
- Clock interrupt doesn't run simultaneously
- Reading int should be atomic on all architectures
On Wed, Mar 16, 2016 at 02:13:13PM +0900, Masao Uebayashi wrote:
> This clarifies that the single, global `ticks' is owned by kern_clock.c.
> timeout(9) is only one of users of `ticks', even though its handler,
> timeout_hardclock_update(), is called from hardclock() after update of
> `ticks' every time.
>
> Theoretically timecounter(9)'s tick, tc_ticktock(), is independent of
> the value of `ticks'. Then it must be safe to update `ticks' first then
> tick timecounter(9). I leave this order as is for now.
>
> Index: sys/kern/kern_clock.c
> ===
> RCS file: /cvs/src/sys/kern/kern_clock.c,v
> retrieving revision 1.88
> diff -u -p -r1.88 kern_clock.c
> --- sys/kern/kern_clock.c 11 Jun 2015 16:03:04 - 1.88
> +++ sys/kern/kern_clock.c 16 Mar 2016 04:30:42 -
> @@ -191,6 +191,7 @@ hardclock(struct clockframe *frame)
> return;
>
> tc_ticktock();
> + ticks++;
>
> /*
>* Update real-time timeout queue.
> Index: sys/kern/kern_timeout.c
> ===
> RCS file: /cvs/src/sys/kern/kern_timeout.c,v
> retrieving revision 1.43
> diff -u -p -r1.43 kern_timeout.c
> --- sys/kern/kern_timeout.c 20 Jul 2015 23:47:20 - 1.43
> +++ sys/kern/kern_timeout.c 16 Mar 2016 04:30:42 -
> @@ -304,8 +304,6 @@ timeout_hardclock_update(void)
>
> mtx_enter(&timeout_mutex);
>
> - ticks++;
> -
> MOVEBUCKET(0, ticks);
> if (MASKWHEEL(0, ticks) == 0) {
> MOVEBUCKET(1, ticks);
Update `ticks' in the right place
This clarifies that the single, global `ticks' is owned by kern_clock.c.
timeout(9) is only one of users of `ticks', even though its handler,
timeout_hardclock_update(), is called from hardclock() after update of
`ticks' every time.
Theoretically timecounter(9)'s tick, tc_ticktock(), is independent of
the value of `ticks'. Then it must be safe to update `ticks' first then
tick timecounter(9). I leave this order as is for now.
Index: sys/kern/kern_clock.c
===
RCS file: /cvs/src/sys/kern/kern_clock.c,v
retrieving revision 1.88
diff -u -p -r1.88 kern_clock.c
--- sys/kern/kern_clock.c 11 Jun 2015 16:03:04 - 1.88
+++ sys/kern/kern_clock.c 16 Mar 2016 04:30:42 -
@@ -191,6 +191,7 @@ hardclock(struct clockframe *frame)
return;
tc_ticktock();
+ ticks++;
/*
* Update real-time timeout queue.
Index: sys/kern/kern_timeout.c
===
RCS file: /cvs/src/sys/kern/kern_timeout.c,v
retrieving revision 1.43
diff -u -p -r1.43 kern_timeout.c
--- sys/kern/kern_timeout.c 20 Jul 2015 23:47:20 - 1.43
+++ sys/kern/kern_timeout.c 16 Mar 2016 04:30:42 -
@@ -304,8 +304,6 @@ timeout_hardclock_update(void)
mtx_enter(&timeout_mutex);
- ticks++;
-
MOVEBUCKET(0, ticks);
if (MASKWHEEL(0, ticks) == 0) {
MOVEBUCKET(1, ticks);
Re: spamd - blacklists
On 2016/03/16 10:53, Damien Miller wrote: > On Tue, 15 Mar 2016, li...@wrant.com wrote: > > > What's going on with the BGP as a transport then, is it available to > > the general public? Must be much better than the fubar DNS. Nackts > > thing and we'd be attempting carping on tunnelled over DNS syndrome. > > Years ago I added the pftable keyword to bgpd.conf for this very > reason. Assuming it hasn't bitrotted, it's trivial to use bgpd > to fill a PF table that can be used to block or tarpit spammers. The default setup with spamd is in greylisting mode, this mode doesn't use a big PF table for blacklist but instead the blacklist-or-greylist decision is done inside spamd itself, the only PF table involved is the (usually much smaller) one containing hosts that have previously made it through greylisting. It is possible to use BGP to distribute addresses for spamd but the usual setup for this involves 'bgpctl sh rib' to write out a file for spamdb to read so it's not realtime. There is another way to use spamd, in blacklist-only mode (-b flag), which does use a PF table to hold the blacklist. In that case I think the table could be populated directly from BGP but I don't know if that combines well with also loading a blacklist via spamd-setup. Additionally for large blacklists this can end up using a bunch of kvm, some people did run into problems with this in the past before spamd changed to using the greylisting mode by default.
Re: spamd - blacklists
On Tue, 15 Mar 2016, li...@wrant.com wrote: > What's going on with the BGP as a transport then, is it available to > the general public? Must be much better than the fubar DNS. Nackts > thing and we'd be attempting carping on tunnelled over DNS syndrome. Years ago I added the pftable keyword to bgpd.conf for this very reason. Assuming it hasn't bitrotted, it's trivial to use bgpd to fill a PF table that can be used to block or tarpit spammers. -d
Re: spamd - blacklists
Tue, 15 Mar 2016 12:52:06 -0400 Michael McConville > Stuart Henderson wrote: > > On 2016/03/15 12:55, Craig Skinner wrote: > > > There are a few more paid rsync lists here: > > > http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists > > > > Ah that is a useful page. Maybe we could list it, e.g. > > > > Index: spamd.conf > > === > > RCS file: /cvs/src/etc/mail/spamd.conf,v > > retrieving revision 1.5 > > diff -u -p -r1.5 spamd.conf > > --- spamd.conf 14 Mar 2016 21:36:52 - 1.5 > > +++ spamd.conf 15 Mar 2016 13:27:04 - > > @@ -13,8 +13,10 @@ > > # Lists specified with the :white: capability apply to the previous > > # list with a :black: capability. > > # > > -# As of November 2004, a place to search for blacklists is > > -# http://spamlinks.net/filter-bl.htm > > +# As of March 2016, a place to search for blacklists is > > +# http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists > > +# - most of these are DNS-based only and cannot be used with spamd(8), > > +# but some of the lists also provide access to text files via rsync. > > > > all:\ > > :uatraps:nixspam: > > ok mmcc@ > > > > Generally, everything has changed from file feeds to DNS. > > > > Yep, because for the more actively maintained ones 1) new entries show > > up more quickly than any sane rsync interval, this is quite important > > for good blocking these days 2) DNS is less resource intensive and more > > easily distributed than rsync, and 3) importantly for the rbl providers, > > it gives additional input to them about new mail sources (if an rbl > > suddenly starts seeing queries from all over the world for a previously > > unseen address, it's probably worth investigation - I am sure this is > > why some of the commercial antispam operators provide free DNS-based > > lookups for smaller orgs). > > > > A more flexible approach would be to skip the PF table integration > > completely and do DNS lookups in spamd (or, uh, relayd, or something > > new) and based on that it could choose whether to tarpit, greylist or > > transparent-forward the connection to the real mail server. This > > would also give a way to use dnswl.org's whitelist to avoid greylisting > > for those hosts where it just doesn't work well (gmail, office365 etc). > > Interesting, I didn't even know that rsync blacklists existed. That was > the cause for confusion about Spamhaus's price earlier. A list is a flat data file, which is text, word / line oriented. The DNS transport for this is incomprehensible even on hard attempt at it. Records, routes, filters, zones, regions, are text, should be carried by all means using text interfaces. No matter word, line or region at a time. What's going on with the BGP as a transport then, is it available to the general public? Must be much better than the fubar DNS. Nackts thing and we'd be attempting carping on tunnelled over DNS syndrome. Constructively, it's all about money extortion when it comes to email malpractice, nobody wants to stop spam except a handful of idealists, so resource lists are another financial abstraction to allow only the good much more worthier spam you actually need to reach your mailbox. > Would it make sense to enable a blacklist or two by default in spamd? > They seem to be an effectively necessary part of a sane mail server > configuration these days. Commercial interests, are best avoided by peer review. So, why not enable OpenBSD own block list then? It would just take a few mythic man years to maintain. Hm, never thought of it this way, scratch it.
newfs_ext2fs image permissions
I don't see how execute permissions on a file system image would be useful. Ok? Index: newfs_ext2fs.c === RCS file: /cvs/src/sbin/newfs_ext2fs/newfs_ext2fs.c,v retrieving revision 1.24 diff -u -p -r1.24 newfs_ext2fs.c --- newfs_ext2fs.c 14 Mar 2016 20:30:34 - 1.24 +++ newfs_ext2fs.c 15 Mar 2016 21:31:17 - @@ -241,7 +241,7 @@ main(int argc, char *argv[]) /* creating image in a regular file */ if (!Nflag && fssize > 0) fl |= O_CREAT; - fd = open(special, fl, 0777); + fd = open(special, fl, 0666); if (fd == -1) err(EXIT_FAILURE, "can't open file %s", special); if (fstat(fd, &sb) == -1) natano
Re: malloc: 1st small step in long way to multiple pools
Hi Otto, On Wed Mar 9 2016 10:06, Otto Moerbeek wrote: > Lightly tested by myself on amd64, you can help by reviewing and > testing this. I also don't see regressions on my amd64 - running since 3 days on my production system.
Re: spamd - blacklists
Absolutely not. do not enable a blacklist by default On Tue, Mar 15, 2016 at 10:52 AM, Michael McConville wrote: > Stuart Henderson wrote: >> On 2016/03/15 12:55, Craig Skinner wrote: >> > There are a few more paid rsync lists here: >> > http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists >> >> Ah that is a useful page. Maybe we could list it, e.g. >> >> Index: spamd.conf >> === >> RCS file: /cvs/src/etc/mail/spamd.conf,v >> retrieving revision 1.5 >> diff -u -p -r1.5 spamd.conf >> --- spamd.conf14 Mar 2016 21:36:52 - 1.5 >> +++ spamd.conf15 Mar 2016 13:27:04 - >> @@ -13,8 +13,10 @@ >> # Lists specified with the :white: capability apply to the previous >> # list with a :black: capability. >> # >> -# As of November 2004, a place to search for blacklists is >> -# http://spamlinks.net/filter-bl.htm >> +# As of March 2016, a place to search for blacklists is >> +# http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists >> +# - most of these are DNS-based only and cannot be used with spamd(8), >> +# but some of the lists also provide access to text files via rsync. >> >> all:\ >> :uatraps:nixspam: > > ok mmcc@ > >> > Generally, everything has changed from file feeds to DNS. >> >> Yep, because for the more actively maintained ones 1) new entries show >> up more quickly than any sane rsync interval, this is quite important >> for good blocking these days 2) DNS is less resource intensive and more >> easily distributed than rsync, and 3) importantly for the rbl providers, >> it gives additional input to them about new mail sources (if an rbl >> suddenly starts seeing queries from all over the world for a previously >> unseen address, it's probably worth investigation - I am sure this is >> why some of the commercial antispam operators provide free DNS-based >> lookups for smaller orgs). >> >> A more flexible approach would be to skip the PF table integration >> completely and do DNS lookups in spamd (or, uh, relayd, or something >> new) and based on that it could choose whether to tarpit, greylist or >> transparent-forward the connection to the real mail server. This >> would also give a way to use dnswl.org's whitelist to avoid greylisting >> for those hosts where it just doesn't work well (gmail, office365 etc). > > Interesting, I didn't even know that rsync blacklists existed. That was > the cause for confusion about Spamhaus's price earlier. > > Would it make sense to enable a blacklist or two by default in spamd? > They seem to be an effectively necessary part of a sane mail server > configuration these days. >
Re: opendev(3) tweak
> On Mon, Mar 14, 2016 at 10:19:53PM +0100, Theo Buehler wrote:
> > On Thu, Mar 10, 2016 at 12:52:35PM +0100, Marc Espie wrote:
> > > Already shown to a few people, but since pledge(2) aborts on non-dev,
> > > let's
> > > check upfront that we're of the right type.
> > >
> > > I don't think this requires a bump. It doesn't really change the
> > > interface,
> > > just makes it stricter.
> > >
> >
> > If we are going to fix this in userland, I think espie@'s patch is the
> > way to go. I like the idea that opendev(3) validates the file descriptor
> > it returns.
> >
> From the hints I've got from Theo, looks like he wants to do something else
> entirely... don't understand what exactly... but then that's not my area.
I am simply saying that pledge before opendev() makes no sense,
because opendev() does not gaurantee the type of descriptor it is
opening.
===
RCS file: /cvs/src/sbin/fdisk/fdisk.c,v
retrieving revision 1.99
diff -u -p -u -r1.99 fdisk.c
--- fdisk.c 12 Dec 2015 04:14:36 - 1.99
+++ fdisk.c 14 Mar 2016 23:37:39 -
@@ -85,10 +85,6 @@ main(int argc, char *argv[])
struct dos_mbr dos_mbr;
struct mbr mbr;
- /* "proc exec" for man page display */
- if (pledge("stdio rpath wpath disklabel proc exec", NULL) == -1)
- err(1, "pledge");
-
while ((ch = getopt(argc, argv, "iegpuvf:c:h:s:l:b:y")) != -1) {
const char *errstr;
@@ -169,6 +165,10 @@ main(int argc, char *argv[])
disk.name = argv[0];
DISK_open(i_flag || u_flag || e_flag);
+
+ /* "proc exec" for man page display */
+ if (pledge("stdio rpath wpath disklabel proc exec", NULL) == -1)
+ err(1, "pledge");
error = MBR_read(0, &dos_mbr);
if (error)
Re: [PATCH] Fix size of object being freed in atascsi.c
Jeremie Courreges-Anglas writes: > Nick Permyakov writes: > >> Fixed a boot panic on my machine. >> >> Index: atascsi.c >> === >> RCS file: /cvs/src/sys/dev/ata/atascsi.c,v >> retrieving revision 1.127 >> diff -u -p -r1.127 atascsi.c >> --- atascsi.c3 Jan 2016 21:07:46 -1.127 >> +++ atascsi.c14 Mar 2016 11:46:21 - >> @@ -491,7 +491,7 @@ atascsi_free(struct scsi_link *link) >> * free ahp itself. this relies on the order luns are >> * detached in scsi_detach_target(). >> */ >> -free(ahp, M_DEVBUF, sizeof(*ap)); >> +free(ahp, M_DEVBUF, sizeof(*ahp)); >> as->as_host_ports[port] = NULL; >> } >> } > > Looks fine to me. Committed, thanks. Nit: the patch had tabs converted to spaces. :) -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
Re: [PATCH] Fix size of object being freed in atascsi.c
Nick Permyakov writes: > Fixed a boot panic on my machine. > > Index: atascsi.c > === > RCS file: /cvs/src/sys/dev/ata/atascsi.c,v > retrieving revision 1.127 > diff -u -p -r1.127 atascsi.c > --- atascsi.c3 Jan 2016 21:07:46 -1.127 > +++ atascsi.c14 Mar 2016 11:46:21 - > @@ -491,7 +491,7 @@ atascsi_free(struct scsi_link *link) > * free ahp itself. this relies on the order luns are > * detached in scsi_detach_target(). > */ > -free(ahp, M_DEVBUF, sizeof(*ap)); > +free(ahp, M_DEVBUF, sizeof(*ahp)); > as->as_host_ports[port] = NULL; > } > } Looks fine to me. -- jca | PGP : 0x1524E7EE / 5135 92C1 AD36 5293 2BDF DDCC 0DFA 74AE 1524 E7EE
Guard a macro arg in less(1)
ok?
Index: ch.c
===
RCS file: /cvs/src/usr.bin/less/ch.c,v
retrieving revision 1.16
diff -u -p -r1.16 ch.c
--- ch.c27 Dec 2015 17:51:19 - 1.16
+++ ch.c15 Mar 2016 17:18:56 -
@@ -103,8 +103,8 @@ struct filestate {
* Macros to manipulate the list of buffers in thisfile->hashtbl[n].
*/
#defineFOR_BUFS_IN_CHAIN(h, bn) \
- for (bn = thisfile->hashtbl[h].hnext; \
- bn != END_OF_HCHAIN(h); bn = bn->hnext)
+ for ((bn) = thisfile->hashtbl[h].hnext; \
+ (bn) != END_OF_HCHAIN(h); (bn) = (bn)->hnext)
#defineBUF_HASH_RM(bn) \
(bn)->hnext->hprev = (bn)->hprev; \
scp(1), sftp(1): unsafe terminal output string handling
Hi, in various situations, including directory listings, progress and diagnostic messages, the sftp(1) and scp(1) programs print untrusted strings to the local terminal, often strings that were received from a remote system over the wire, often containing whatever random filenames on the remote system may contain, or in general whatever the remote server sends. That can screw up terminal state, for example hang the local terminal, change key bindings, change fonts, encodings, windows titles and many other terminal properties. With xterm(1), i did not find any vulnerabilities that could execute arbitrary code injected remotely. It is unclear what other terminals and terminal emulators may do. Here are a few examples of file names you can use for testing: touch `printf "bel\a."` touch `printf "esc\033[4munder"` touch `printf "wt\033]0;rogue_title\a."` Try listing the directory containing such files in sftp(1) interactive mode remotely or downloading them with sftp(1) or scp(1), in particular using wildcards. Typically, filename display silently lacks some of the bytes contained in the filename, but the terminal executes control codes instead (like beeping, changing fonts, changing titles, or whatever the controls do). The following patch sanitizes such strings before sending them to the terminal. I'm posting it here for wider review because deraadt@ suggested that public feedback is needed, in particular from people who know a bit about multibyte character handling. Note that the problems are not related to the following areas: - Portable OpenSSH. They also affect OpenBSD-current to OpenBSD-current connections. - Line editing with libedit / editline(3). They also affect non-interactive mode. - Multibyte characters. They also occur when all relevant locales are set to C/POSIX and only seven-bit bytes are sent. - Character display widths. The patch below does not attempt to fix output columnation but only output corruption involving terminal control codes. While some of the new functions are already instrumented for width calculation (it doesn't complicate the code since wcwidth(3) validates printability of a character just like iswprint(3) does and isn't any harder to use), the calling code doesn't use that information yet, except in the progressmeter, where is it easily used without complicating the code. Some general remarks: * The function names in utf8.c are composed from "mbs" for "multibyte string" and standard printf(3) function names, and they follow the spirit of the standard functions as closely as they can while still providing the additional sanitation required. See the comments in utf8.c for details. The general style of the functions resembles the style we developed for our ls(1) and similar programs, but as we already experienced, not two such programs needed exactly the same helper functions so far, but all have subtly different needs. * For a few of the strings printed with the new sanitizing functions, it may not be possible that they contain remote data. But even local data may pose risks, for example when root copies around user home directories for backup or migration purposes. So i'd prefer to err to the side of caution. * At first, i intended to do this in two steps: First implement safe handling of US-ASCII, then in a second step add multibyte character handling. But unfortunately, that doesn't work. US-ASCII sanitation would break display of filenames containing multibyte characters, and even though there isn't much explicit code in the programs to handle such characters yet, they partially work because most of the code just didn't care in the past what bytes in file names represented and just passed everything through verbatim, which is exactly where the problems come from, but OpenSSH developers pointed out to me that some users may rely on that, so they don't want to break the existing partial multibyte support, not even to improve ASCII terminal sanitation. So, unfortunately, introducing the first bits of explicit multibyte support and introducing the first bits of terminal sanitation must be done in a single, rather big step. * Even this patch isn't perfect yet. It still allows printing '\r' and '\n' to the terminal at too many places, for example when remote filenames contain these characters. But the patch is already big, so fixing that should probably be kept seperate. * As noted in my recent undeadly posting, protecting the terminal is only possible if either (a) the terminal is UTF-8 *and* the remote side is either UTF-8 or US-ASCII, or (b) the *user* makes sure that the local locale(1) agrees with the remote one. If the remote side uses a non-UTF-8 and non-ASCII locale *or* you don't set a matching locale(1) before starting sftp(1) or scp(1), these programs can still screw up your terminal even wit
Re: hang with processes in fltamap: how can I identify running out of RAM?
Tobias Ulmer wrote: > Just wanted to note this diff in combination with your other uvm diff > does really well on sparc, building ports. Cuts down amap "INUSE" by > about a factor of 20. > Will report if anything bad happens. Cool, thanks for testing as well! So I plan to commit both diffs if the ports build succeeds and if there are no objections otherwise.
Re: spamd - blacklists
> > Generally, everything has changed from file feeds to DNS. > > Yep, because for the more actively maintained ones 1) new entries show > up more quickly than any sane rsync interval, this is quite important > for good blocking these days 2) DNS is less resource intensive and more > easily distributed than rsync, and 3) importantly for the rbl providers, > it gives additional input to them about new mail sources (if an rbl > suddenly starts seeing queries from all over the world for a previously > unseen address, it's probably worth investigation - I am sure this is > why some of the commercial antispam operators provide free DNS-based > lookups for smaller orgs). > > A more flexible approach would be to skip the PF table integration > completely and do DNS lookups in spamd (or, uh, relayd, or something > new) and based on that it could choose whether to tarpit, greylist or > transparent-forward the connection to the real mail server. This > would also give a way to use dnswl.org's whitelist to avoid greylisting > for those hosts where it just doesn't work well (gmail, office365 etc). The argument does not make sense. There are numerous tools that can do DNS lookup in userland. spamd was the only one which could do it via radix tables on the kernel edge, which even allows you to do management in a bridge. pf integration is the basis of spamd, that is how it steps out of the accepted address list, so that userland can make the decision. Your proposal can be summed up as: delete spamd.
Re: spamd - blacklists
Stuart Henderson wrote: > On 2016/03/15 12:55, Craig Skinner wrote: > > There are a few more paid rsync lists here: > > http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists > > Ah that is a useful page. Maybe we could list it, e.g. > > Index: spamd.conf > === > RCS file: /cvs/src/etc/mail/spamd.conf,v > retrieving revision 1.5 > diff -u -p -r1.5 spamd.conf > --- spamd.conf14 Mar 2016 21:36:52 - 1.5 > +++ spamd.conf15 Mar 2016 13:27:04 - > @@ -13,8 +13,10 @@ > # Lists specified with the :white: capability apply to the previous > # list with a :black: capability. > # > -# As of November 2004, a place to search for blacklists is > -# http://spamlinks.net/filter-bl.htm > +# As of March 2016, a place to search for blacklists is > +# http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists > +# - most of these are DNS-based only and cannot be used with spamd(8), > +# but some of the lists also provide access to text files via rsync. > > all:\ > :uatraps:nixspam: ok mmcc@ > > Generally, everything has changed from file feeds to DNS. > > Yep, because for the more actively maintained ones 1) new entries show > up more quickly than any sane rsync interval, this is quite important > for good blocking these days 2) DNS is less resource intensive and more > easily distributed than rsync, and 3) importantly for the rbl providers, > it gives additional input to them about new mail sources (if an rbl > suddenly starts seeing queries from all over the world for a previously > unseen address, it's probably worth investigation - I am sure this is > why some of the commercial antispam operators provide free DNS-based > lookups for smaller orgs). > > A more flexible approach would be to skip the PF table integration > completely and do DNS lookups in spamd (or, uh, relayd, or something > new) and based on that it could choose whether to tarpit, greylist or > transparent-forward the connection to the real mail server. This > would also give a way to use dnswl.org's whitelist to avoid greylisting > for those hosts where it just doesn't work well (gmail, office365 etc). Interesting, I didn't even know that rsync blacklists existed. That was the cause for confusion about Spamhaus's price earlier. Would it make sense to enable a blacklist or two by default in spamd? They seem to be an effectively necessary part of a sane mail server configuration these days.
Re: some more vlan protocol defines
On 2016/03/15 23:34, David Gwynne wrote:
> Can you find any doco that states that? Or any other thing that does it?
IEEE 802.1Q-2014 (https://standards.ieee.org/about/get/802/802.1.html)
section 9.6 (on p160):
0 The null VID. Indicates that the tag header contains only priority
information; no VID is present in the frame. This VID value shall not be
configured as a PVID or a member of a VID Set, or configured in any FDB
entry, or used in any Management
> On 15 Mar 2016 02:21, "Stuart Henderson" wrote:
>
> > On 2016/03/14 21:13, David Gwynne wrote:
> > > this adds macros to describe the min and max valid vlan ids.
> > >
> > > this will be used in upcoming checks of the configured values from
> > > userland.
> > >
> > > ok?
> >
> > Hmm - I think there may be something about using tag 0 to allow
> > sending packets on an "untagged" vlan with .1q priorities ..
> >
> > > Index: if_ether.h
> > > ===
> > > RCS file: /cvs/src/sys/netinet/if_ether.h,v
> > > retrieving revision 1.68
> > > diff -u -p -r1.68 if_ether.h
> > > --- if_ether.h14 Mar 2016 03:48:47 - 1.68
> > > +++ if_ether.h14 Mar 2016 11:12:24 -
> > > @@ -87,7 +87,11 @@ struct ether_vlan_header {
> > > u_int16_t evl_proto;
> > > };
> > >
> > > -#define EVL_VLID_MASK 0x0FFF
> > > +#define EVL_VLID_MASK0xFFF
> > > +#define EVL_VLID_NONE0x000
> > > +/* 0x000 and 0xfff are reserved */
> > > +#define EVL_VLID_MIN 0x001
> > > +#define EVL_VLID_MAX 0xFFE
> > > #define EVL_VLANOFTAG(tag) ((tag) & EVL_VLID_MASK)
> > >
> > > #define EVL_PRIO_MAX7
> > >
> >
Re: New scheduler for OpenBSD
On 14:57:40, 15.03.16, Alexandre Ratchov wrote: > On Sat, Mar 12, 2016 at 05:36:21PM +0100, Michal Mazurek wrote: > > > > p_usrpri and p_priority will go away, so userland utilities like 'ps' > > will need to be changed. > > > > AFAIU, this would hurt interactive programs (audio, players, games, > etc). Currently i/o bound processes wake up with increased > priority and steal the cpu from cpu-bound processes. Removing > p_priority would break this mechanism. No, with the new scheduler a process gets assigned a 'deadline' - the value of an arbitrary monotonically increasing timer. If it runs out of ticks it gets a new deadline. The process with the smallest deadline gets to run. If a process goes to sleep, and then after a while wakes up, it will have a smaller deadline than the processes that were allwed to keep running (and thus kept getting new deadlines). It will therefore preempt other tasks. -- Michal Mazurek
Re: New scheduler for OpenBSD
On 16:28:33, 14.03.16, Martin Pieuchot wrote:
> > The number of calls to yield() dropped to 4,576.
>
> This is really similar to what I observed with Firefox and Chrome.
>
> > This is where I get stuck, I don't know how to replace the call to
> > sched_yield(), or whether it is a good idea to do so. Any advice?
>
> I'd assume the problem is not in the _spinlock() implementation itself
> but rather on the code calling this lock. Do you know which code is
> triggering this?
>
> See r1.13 of lib/librthread/rthread_libc.c, this is an example where a
> the use of a spinlock created similar symptoms.
I don't know how to fix it, but in the meanwhile here is a workaround so
I can continue working on the scheduler. In yield():
+ tmp = p->p_rrticks;
+ sched_deadline(p);
+ p->p_rrticks = tmp;
So penalise a process calling yield() by giving it the worst deadline,
i.e. make it go to the end of the run queue.
Other than this, no changes.
Chrome still isn't smooth.
Please test, and let me know if the performance of something else
degrades.
Index: kern/init_main.c
===
RCS file: /cvs/src/sys/kern/init_main.c,v
retrieving revision 1.248
diff -u -p -r1.248 init_main.c
--- kern/init_main.c3 Jan 2016 00:15:59 - 1.248
+++ kern/init_main.c15 Mar 2016 13:56:58 -
@@ -265,6 +265,8 @@ main(void *framep)
*/
pr = &process0;
process_initialize(pr, p);
+ p->p_deadline = 0;
+ p->p_rrticks = 10;
LIST_INSERT_HEAD(&allprocess, pr, ps_list);
atomic_setbits_int(&pr->ps_flags, PS_SYSTEM);
@@ -554,6 +556,7 @@ main(void *framep)
/*
* proc0: nothing to do, back to sleep
*/
+ printf("*** modified scheduler ***\n");
while (1)
tsleep(&proc0, PVM, "scheduler", 0);
/* NOTREACHED */
Index: kern/kern_clock.c
===
RCS file: /cvs/src/sys/kern/kern_clock.c,v
retrieving revision 1.88
diff -u -p -r1.88 kern_clock.c
--- kern/kern_clock.c 11 Jun 2015 16:03:04 - 1.88
+++ kern/kern_clock.c 15 Mar 2016 13:56:58 -
@@ -180,7 +180,7 @@ hardclock(struct clockframe *frame)
if (stathz == 0)
statclock(frame);
- if (--ci->ci_schedstate.spc_rrticks <= 0)
+ if (p && (--(p->p_rrticks) <= 0))
roundrobin(ci);
/*
@@ -398,15 +398,7 @@ statclock(struct clockframe *frame)
if (p != NULL) {
p->p_cpticks++;
- /*
-* If no schedclock is provided, call it here at ~~12-25 Hz;
-* ~~16 Hz is best
-*/
- if (schedhz == 0) {
- if ((++curcpu()->ci_schedstate.spc_schedticks & 3) ==
- 0)
- schedclock(p);
- }
+ ++curcpu()->ci_schedstate.spc_schedticks;
}
}
Index: kern/kern_fork.c
===
RCS file: /cvs/src/sys/kern/kern_fork.c,v
retrieving revision 1.185
diff -u -p -r1.185 kern_fork.c
--- kern/kern_fork.c11 Mar 2016 19:10:14 - 1.185
+++ kern/kern_fork.c15 Mar 2016 13:56:58 -
@@ -498,6 +498,7 @@ fork1(struct proc *curp, int flags, void
/*
* Make child runnable and add to run queue.
*/
+ sched_deadline(p);
if ((flags & FORK_IDLE) == 0) {
SCHED_LOCK(s);
p->p_stat = SRUN;
Index: kern/kern_resource.c
===
RCS file: /cvs/src/sys/kern/kern_resource.c,v
retrieving revision 1.55
diff -u -p -r1.55 kern_resource.c
--- kern/kern_resource.c5 Dec 2015 10:11:53 - 1.55
+++ kern/kern_resource.c15 Mar 2016 13:56:58 -
@@ -178,8 +178,6 @@ int
donice(struct proc *curp, struct process *chgpr, int n)
{
struct ucred *ucred = curp->p_ucred;
- struct proc *p;
- int s;
if (ucred->cr_uid != 0 && ucred->cr_ruid != 0 &&
ucred->cr_uid != chgpr->ps_ucred->cr_uid &&
@@ -193,10 +191,13 @@ donice(struct proc *curp, struct process
if (n < chgpr->ps_nice && suser(curp, 0))
return (EACCES);
chgpr->ps_nice = n;
+ /* XXXNICE */
+ /*
SCHED_LOCK(s);
TAILQ_FOREACH(p, &chgpr->ps_threads, p_thr_link)
(void)resetpriority(p);
SCHED_UNLOCK(s);
+ */
return (0);
}
Index: kern/kern_sched.c
===
RCS file: /cvs/src/sys/kern/kern_sched.c,v
retrieving revision 1.41
diff -u -p -r1.41 kern_sched.c
--- kern/kern_sched.c 23 Dec 2015 14:51:17 - 1.41
+++ kern/kern_sched.c 15 Mar 2016 13:56:58 -
@@ -33,6 +33,21 @@ void sched_kthreads_create(void *);
int sched_proc_to_cpu_cost(struct cpu_info *ci, stru
Re: New scheduler for OpenBSD
On Sat, Mar 12, 2016 at 05:36:21PM +0100, Michal Mazurek wrote: > > p_usrpri and p_priority will go away, so userland utilities like 'ps' > will need to be changed. > AFAIU, this would hurt interactive programs (audio, players, games, etc). Currently i/o bound processes wake up with increased priority and steal the cpu from cpu-bound processes. Removing p_priority would break this mechanism.
Re: some more vlan protocol defines
Can you find any doco that states that? Or any other thing that does it?
On 15 Mar 2016 02:21, "Stuart Henderson" wrote:
> On 2016/03/14 21:13, David Gwynne wrote:
> > this adds macros to describe the min and max valid vlan ids.
> >
> > this will be used in upcoming checks of the configured values from
> > userland.
> >
> > ok?
>
> Hmm - I think there may be something about using tag 0 to allow
> sending packets on an "untagged" vlan with .1q priorities ..
>
> > Index: if_ether.h
> > ===
> > RCS file: /cvs/src/sys/netinet/if_ether.h,v
> > retrieving revision 1.68
> > diff -u -p -r1.68 if_ether.h
> > --- if_ether.h14 Mar 2016 03:48:47 - 1.68
> > +++ if_ether.h14 Mar 2016 11:12:24 -
> > @@ -87,7 +87,11 @@ struct ether_vlan_header {
> > u_int16_t evl_proto;
> > };
> >
> > -#define EVL_VLID_MASK 0x0FFF
> > +#define EVL_VLID_MASK0xFFF
> > +#define EVL_VLID_NONE0x000
> > +/* 0x000 and 0xfff are reserved */
> > +#define EVL_VLID_MIN 0x001
> > +#define EVL_VLID_MAX 0xFFE
> > #define EVL_VLANOFTAG(tag) ((tag) & EVL_VLID_MASK)
> >
> > #define EVL_PRIO_MAX7
> >
>
Re: spamd - blacklists
On 2016/03/15 12:55, Craig Skinner wrote: > Hi Stuart, > > On 2016-03-14 Mon 16:27 PM |, Stuart Henderson wrote: > > > > There aren't many who provide their whole dataset to anyone other > > than paying customers - e.g. Spamhaus' rsync feeds are for > > organisations with >5000 users and cost US$1700+/year. > > > > I've found these free rsync feeds useful: > > The Passive Spam Block List (collates IPs sending to spam traps): > http://psbl.org/howto/ > CBL (SpamHaus) writes: "The PSBL is a solid and reliable DNSBL. > Amazingly effective for such a modest effort. Generally recommended" > http://www.abuseat.org/faq.html > > UCE Protect (IPs sending to spam traps, and more aggresive options): > http://www.uceprotect.net/en/index.php?m=6&s=10 > > The Composite Blocking List (CBL - a big part of SpamHaus DNSRBLs) can > be rsync'd after rego (free, execpt for spam filter service operators): > http://www.abuseat.org/faq.html > > There are a few more paid rsync lists here: > http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists Ah that is a useful page. Maybe we could list it, e.g. Index: spamd.conf === RCS file: /cvs/src/etc/mail/spamd.conf,v retrieving revision 1.5 diff -u -p -r1.5 spamd.conf --- spamd.conf 14 Mar 2016 21:36:52 - 1.5 +++ spamd.conf 15 Mar 2016 13:27:04 - @@ -13,8 +13,10 @@ # Lists specified with the :white: capability apply to the previous # list with a :black: capability. # -# As of November 2004, a place to search for blacklists is -# http://spamlinks.net/filter-bl.htm +# As of March 2016, a place to search for blacklists is +# http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists +# - most of these are DNS-based only and cannot be used with spamd(8), +# but some of the lists also provide access to text files via rsync. all:\ :uatraps:nixspam: > Generally, everything has changed from file feeds to DNS. Yep, because for the more actively maintained ones 1) new entries show up more quickly than any sane rsync interval, this is quite important for good blocking these days 2) DNS is less resource intensive and more easily distributed than rsync, and 3) importantly for the rbl providers, it gives additional input to them about new mail sources (if an rbl suddenly starts seeing queries from all over the world for a previously unseen address, it's probably worth investigation - I am sure this is why some of the commercial antispam operators provide free DNS-based lookups for smaller orgs). A more flexible approach would be to skip the PF table integration completely and do DNS lookups in spamd (or, uh, relayd, or something new) and based on that it could choose whether to tarpit, greylist or transparent-forward the connection to the real mail server. This would also give a way to use dnswl.org's whitelist to avoid greylisting for those hosts where it just doesn't work well (gmail, office365 etc).
Re: spamd - blacklists
Hi Stuart, On 2016-03-14 Mon 16:27 PM |, Stuart Henderson wrote: > > There aren't many who provide their whole dataset to anyone other > than paying customers - e.g. Spamhaus' rsync feeds are for > organisations with >5000 users and cost US$1700+/year. > I've found these free rsync feeds useful: The Passive Spam Block List (collates IPs sending to spam traps): http://psbl.org/howto/ CBL (SpamHaus) writes: "The PSBL is a solid and reliable DNSBL. Amazingly effective for such a modest effort. Generally recommended" http://www.abuseat.org/faq.html UCE Protect (IPs sending to spam traps, and more aggresive options): http://www.uceprotect.net/en/index.php?m=6&s=10 The Composite Blocking List (CBL - a big part of SpamHaus DNSRBLs) can be rsync'd after rego (free, execpt for spam filter service operators): http://www.abuseat.org/faq.html There are a few more paid rsync lists here: http://en.wikipedia.org/wiki/Comparison_of_DNS_blacklists Generally, everything has changed from file feeds to DNS.
Re: Remove handrolled function from spamd(8) for privdrop
Hi Ricardo, On 2016-03-14 Mon 16:15 PM |, Ricardo Mestre wrote: > > Comments? > http://marc.info/?l=openbsd-tech&m=145047019223179
Re: hang with processes in fltamap: how can I identify running out of RAM?
Just wanted to note this diff in combination with your other uvm diff does really well on sparc, building ports. Cuts down amap "INUSE" by about a factor of 20. Will report if anything bad happens.
Re: opendev(3) tweak
I'm aware I'm kicking an old horse here, but...
On Thu, Mar 10, 2016 at 12:52:35PM +0100, Marc Espie wrote:
> @@ -106,5 +108,17 @@ opendev(const char *path, int oflags, in
> if (realpath)
> *realpath = namebuf;
If anything like this goes in (or did it already?) the *realpath
assignment should probably happen iff passing the new chunk.
/Alexander
>
> + if (fd != -1) {
> + if (fstat(fd, &st) == -1) {
> + close(fd);
> + return (-1);
> + }
> + if ((dflags & OPENDEV_BLCK) ? !S_ISBLK(st.st_mode) :
> + !S_ISCHR(st.st_mode)) {
> + close(fd);
> + errno = EFTYPE;
> + return (-1);
> + }
> + }
> return (fd);
> }
>
Re: COLUMNS handling
On Mon, Mar 14, 2016 at 02:02:47PM -0600, Abel Abraham Camarillo Ojeda wrote: > On Mon, Mar 14, 2016 at 5:32 AM, Martin Natano wrote: > > On Mon, Mar 14, 2016 at 10:57:36AM +0100, Marc Espie wrote: > >> > >> So, does it make sense to put COLUMNS and SIZE forward ? > >> I think this is the first important question to ask... > >> > >> (I remember having COLUMNS and LINES hardcoded in my old, old .profile > >> around SunOS4... we can probably assume this is gone now) > >> > > > > This is still useful when working over a serial line. I sometimes use > > COLUMNS/LINES for vi over the qemu serial console. > > > > calling resize(1), ignoring its output, isnt enough over serial line? > > over serial I just do $ resize; after resizing my xterm and everything works > - and LINES and COLUMNS never get set in env Yeah, resize does work. I didn't even know it exists ;) Now, that I have a much more convenient replacement for COLUMNS/LINES (for my usecase that is), I don't really care whether they stay or go away. natano
pledge: wl_paths evaluation order error + memory leak
Hi,
The following diff corrects an evaluation order error and a memory leak
in error code path.
Comments or OK ?
--
Sebastien Marie
Index: sys/kern/kern_pledge.c
===
--- sys/kern/kern_pledge.c.orig 2016-03-15 08:54:33.500610285 +0100
+++ sys/kern/kern_pledge.c 2016-03-15 09:19:18.679200485 +0100
@@ -503,8 +503,10 @@ sys_pledge(struct proc *p, void *v, regi
/* resolved is allocated only if !error */
break;
- if (maxargs += resolvedlen > ARG_MAX) {
+ maxargs += resolvedlen;
+ if (maxargs > ARG_MAX) {
error = E2BIG;
+ free(resolved, M_TEMP, resolvedlen);
break;
}
wl->wl_paths[i].name = resolved;
