Re: bgpd, announce to ibgp from 2 routers, prefixes only show up from 1

[Adam Thompson]

[apologies in advance for top-posting]

bgpd(8) is excellent in many ways, and I am SO very grateful it exists.  (Thank 
you Henning, Claudio, Peter and everyone else who has contributed to it over 
the years!  It has straight-up saved my bacon a couple of times.)

But one feature it does not yet AFAIK have is Additional Paths 
("additional-path", or just "add-path") [1].  This is where a BGP speaker 
advertises not only its "best" routes, but *all* its routes, to its peers.  The 
FIB remains unchanged, but the RIB can grow very large in a well-connected AS.  
Since each router now knows all the available paths through every other router, 
convergence is - at least in theory - sped up quite dramatically, and you 
mostly avoid the "hole" described.  It's not a perfect solution but it works 
very well.

If you're brave enough, at least some versions/forks of Bird/Quagga/Zebra have 
support for add-path.  I wouldn't recommend running these on OpenBSD, generally 
speaking - bgpd(8) is more appropriate 99.999% of the time - but you might find 
it worthwhile, depending on your needs.  Be particularly careful of any routing 
software that lacks the ability to affect kernel routes - unless you're just 
running a route reflector, that will change your design *significantly*.

Or, as Stuart said, running a "proper" IGP like OSPF could bridge some of the 
gaps you might see.  YMMV.
 
-Adam 

P.S. From what I heard a few years ago, OpenBSD isn't completely ignoring 
add-path, it's just 
complex/difficult/time-consuming/unfunded/low-priority/pick-your-favourite-reason.

[1] https://datatracker.ietf.org/doc/html/rfc7911


-Original Message-
From: owner-m...@openbsd.org  On Behalf Of Sebastian 
Benoit
Sent: Monday, November 29, 2021 3:38 PM
To: misc@openbsd.org
Subject: Re: bgpd, announce to ibgp from 2 routers, prefixes only show up from 1

Stuart Henderson(s...@spacehopper.org) on 2021.11.13 00:11:08 +:
> I have a pair of -current routers running bgpd (let's call them rtr-a 
> and rtr-b) on a subnet which also has some vpn gateways and firewalls.
> 
> These routers provide a carp address which the vpn gateways are using 
> as default route. There are some networks behind the vpn gateways (a
> /32 to accept incoming vpn connections and some other prefixes that 
> vpn clients are numbered from).
> 
> rtr-a and rtr-b have static routes to those networks, and they have 
> network statements in bgpd.conf to announce them to their ibgp peers 
> ("network 172.24.232.0/21 set nexthop XXX" etc) so the paths are 
> reachable from the rest of the network. (This is replacing an existing 
> setup using ospf, trying to remove routing protocols from machines 
> that don't really need them).
> 
> It is working but something seems a little odd - the paths are 
> announced from both routers briefly and show up on the rest of the 
> network from both rtr-a and rtr-b. But after a few seconds, rtr-b 
> receives these paths from rtr-a, and then rtr-b stops announcing them 
> itself. (they stop showing in "bgpctl sh rib out" on rtr-b; "bgpctl sh 
> nex" does correctly identify the associated nexthops as connected/UP).
> 
> Is this expected/correct behaviour?

It is expected: once rtr-b receives the route from rtr-a, it will run the route 
decision process on it. IF both routers are configured identically except for 
the router-id, one of the routes will be prefered at either the "oldest path" 
or the "lowest bgp id" criteria.

As only one route is a best route, that one will be annouced to the neighbors. 
However this is IBGP. In a set of IBGP connected routers, a router will not 
announce a route to other IBGP peers that it received from on a IBGP session. 
Thus, rtr-b will stop announcing that route.

When rtr-a goes down, the session is shut down or the prefix is filtered, bgpd 
wont see the "better" route anymore and announce its own instead.

> I'd prefer to have them announced from both rtr-a and rtr-b, so 
> there's no blackhole period if rtr-a is restarted while rtr-b figures 
> out that it should start announcing them, etc. (No need for tracking 
> carp state in this case, I'm not using stateful pf rules on the traffic 
> involved).

This is a place where ospf might give you faster failover, especiall y with the 
redistribute ... depend on ... syntax.
 
> If rtr-b stops seeing the prefixes from rtr-a (either by taking down 
> the ibgp session, or by filtering) I see the announcements from both 
> rtr-a and rtr-b again. So the obvious workaround is to filter, but I 
> thought I'd ask first in case it's something that is better handled by 
> code changes rather than config.

Re: USB devices power control

[Adam Thompson]

The simplest could be something like these, 
https://www.amazon.ca/Powered-USB-Hub/s?k=Powered+USB+Hub.
 11 (of the first 12) products are USB 3 hubs with individual port power 
controls.
I have seen single-port USB cables with power switches, too, although I 
don't remember where.

Your idea could be better, but these already exist.

Only some (although, *most* AFAIK) USB hub chipsets support turning 
power on and off for individual ports.  Under Linux you can use 
https://github.com/mvp/uhubctl to control it.  Nothing exists (that I 
know of) under OpenBSD today.


You might also use a "smart" hub like those seen at 
https://electronics.stackexchange.com/questions/393468/efficient-way-to-selectively-unpower-usb-ports 
and port the necessary software to OpenBSD.  (The ugen device driver 
would probably be adequate, but it might be more of a rewrite than a 
port.  No idea how painful that would wind up being, I've never 
programmed anything using ugen.)


Options exist, but it's possible none of them are *exactly* what you 
want.


-Adam


On 2021-10-07 11:57, jeanfrancois wrote:

Ok thank both,

I might develop such device then, if other people interested I'd share
the product.

I'll be used to have backup / spare drives online for the work time 
only;


Jean-François

Le 06/10/2021 à 16:36, m...@josuah.net a écrit :

If nothing can be found software-side, a dedicated hardware
could possibly do it.

If it exists driver side, some tool like this could give a
hint for finding it on other operating system, and then comparing
with OpenBSD as well as getting the actual standard names for
that feature: https://github.com/mvp/uhubctl

Not really a solution, but rather a way to get a little
closer.

Re: How to unlock a serial port

[Adam Thompson]

On 2021-01-19 19:15, Nick Holland wrote:

On 1/19/21 4:35 PM, Adam Thompson wrote:
I ran into this exact problem last year.  It'll be in the list 
archives.
According to Theo (if I understood him correctly) it's partly due to 
the

way BSD serial ports have always worked, i.e. in a rather
under-specified manner.
Apparently the core tty(4) code around this particular symptom hasn't
really changed at all since OpenBSD forked.


I'm curious what you think "this exact problem" is.


Based on later messages in the thread, it's clear I did *not* have the 
exact same problem, after all.
I was experiencing ttys that remained busy after the process that opened 
them had closed.


So, for example, if I used "screen /dev/cua0 9600", after exiting 
screen(1), confirming with ps(1) that all screen processes were gone, 
and confirming with lsof/fstat that nothing still had either cua00 or 
tty00 open, if I re-ran "screen /dev/cua00 9600" - or ANY other 
application, including cu(1) - the new process would simply block.
ktrace(8) confirmed that the open(2) call was simply blocking... 
forever.  I could reproduce it with cu(1) as well as screen.  Don't 
think I tried any other apps.


I recall that someone suggested at the time that it might be signal line 
related, but it didn't matter whether I had full RS-232 CTS/RTS and/or 
DCD/DSR signalling, basic 3-wire signalling only, or even if I unplugged 
the cable entirely, the blocking behaviour stuck around until I 
rebooted.  Granted, rebooting on a single-port terminal server isn't the 
end of the world, but it was a PITA that I got tired of.  (Oh, if I used 
a USB serial adapter, unplugging and replugging the USB dongle also 
cleared the problem.)




I'm not going to say serial support in OpenBSD is perfect, but it works
Darned Well,


Mostly, I agree.  I still don't know what was different about this one 
use case.  Pretty sure it wasn't hardware - I reproduced it on three 
vastly different amd64 systems, and with both traditional UARTs and USB 
dongles.  Pretty sure it wasn't (Exclusively) a PEBCAK issue.  But still 
not sure what it was.


I'll try again at some point, and I'll post if/when it happens to me 
again.


-Adam

Re: How to unlock a serial port

[Adam Thompson]

[Replying directly as well, as I believe my MTA is still blacklisted by 
the OpenBSD mail server. Guess we'll find out! -Adam]


On 2021-01-17 20:09, Tilo Stritzky wrote:

On 14/01/21 17:38  Andrew Grillet wrote:

Hi

I am running OpenBSD on a T2000 (Sparc64).
I was trying to use the serial port from the primary domain, connected 
via

ssh, and my network lost the connection.
My tty00 is now locked:
jay# stty -f /dev/tty00
stty: /dev/tty00: Device busy
I do not want to reboot the primary, as the guests are running various 
live

services. I cannot find evidence of a lock file in /dev/spool/lock.
Is there a way out of this predicament?


fstat(1) is your friend here.
Note that each tty has a corresponding cua device, they're both under 
the

same lock.

tilo


I ran into this exact problem last year.  It'll be in the list archives.
According to Theo (if I understood him correctly) it's partly due to the 
way BSD serial ports have always worked, i.e. in a rather 
under-specified manner.
Apparently the core tty(4) code around this particular symptom hasn't 
really changed at all since OpenBSD forked.


My solution was to install Linux, sorry - I never did find a way around 
the problem on OpenBSD.


-Adam

Re: RCS file ownership?

[Adam Thompson]

Being neither a C programmer nor a Texinfo fan, checking GNU RCS is a 
bit painful, and my conclusions aren't guaranteed.



AFAICT, GNU RCS (v5.9.4, ca. 2015, examined) creates a temp file, 
unlinks the target file, then renames the temp file.  I beleve this 
guarantees(-ish, modulo "special" filesystems including NFS and 
FreeBSD's directory-SUID behaviour) that resulting file ownership = 
euid.


The GNU docs mention the repo owner in passing a few times but do not 
have a section describing multi-user operation.


The Tichy docs also don't mention file ownership.  I'm trying to review 
the O'Donovan book, too, but it's been a long time since I had to tool 
up to handle raw PS... not quite there yet.


Purdue RCS appears to be the direct ancestor of GNU RCS.

I'm not sure which other implementations you'd be worried about - I 
thought OpenBSD's RCS was the direct descendant of NetBSD's and shares 
common lineage with the other *BSDs?


All in all, it looks like RCS and its docs were written in the era when 
UNIX machines were - more or less by universally - used by multiple 
people, and you just had an innate sense of how multi-user file 
ownership would work.  Most of my UNIX machines now resemble appliances, 
and exactly zero of them are multi-user in the classical sense.


-Adam


On 2020-04-29 21:53, Theo de Raadt wrote:

Sorry, but my mail goes further.

It says it should be correct.  For some definition of correct.  It
should either behave somehow for a logical reason, or it should behave
in the historical fashion.  Or once the historical behaviour is looked
at, if there is an argument that is wrong, then it should be changed
with logic about "this is an improvement" backing the argument.

I think it is wrong to document how *this* rcs implimentation behaves,
without comparing it against others.

My guess is 50% that the others don't unlink, but rewrite the file.

And the changes it might require to be compatible are not substantial.
At most a 20 line diff, to a few programs in the family.

athom...@athompso.net wrote:


Thank you for that detail!

Addressing this one corner case would require substantial changes, I 
think.  Not worth

it, in my opinion.

I think it would be worthwhile describing the multi-user mode of 
operation of RCS in
the manual, as it's currently completely absent/omitted. Patch coming 
soon, maybe

tomorrow if I can make time.

-Adam

On Apr. 29, 2020 21:28, Theo de Raadt  wrote:

 athom...@athompso.net wrote:

 > Heh, good point.  Didn't even occur to me because as it happens, I 
am

 > running as root and would like to not change the ownership.-Adam
 > On Apr. 29, 2020 13:32, Anders Andersson  
wrote:

 >
 >   On Wed, Apr 29, 2020 at 7:46 PM Adam Thompson 


 >   wrote:
 >   >
 >   > When I use co(1) with "-l" to check out a file (and/or "ci -l") 
is

 >   there
 >   > any way to preserve file ownership and *not* have it reset to 
the

 >   user
 >   > running co(1) or ci(1)?
 >   > I don't see anything in rcs(1), co(1) or ci(1) that even 
mentions

 >   the
 >   > fact that the file will wind up owned by the user running the
 >   command.
 >   > Ideas?  Pointers to documentation?
 >
 >   How could it possibly do anything else unless you always run co 
as

 >   root?

 Our rcs tools do:

 52628 co   RET   kbind 0
 52628 co   CALL  unlink(0x7f7ed1f3)
 52628 co   NAMI  "ls.c"
 52628 co   RET   unlink -1 errno 2 No such file or directory
 52628 co   CALL  open
 (0x7f7ed1f3,0x601,0100444)
 52628 co   NAMI  "ls.c"
 52628 co   RET   open 4
 52628 co   CALL  kbind(0x7f7ec908,24,0x847da2a816b5d817)

 Which appears to be this:

 else {
 (void)unlink(dst);

 if ((fd = open(dst, O_WRONLY|O_CREAT|O_TRUNC, mode)) 
== -1)

 err(1, "%s", dst);

 I don't know what older or gnu rcs do.  I guess whichever way this is 
done
 it must balance concerns between atomicity of concurrent reads 
performed
 by earlier processes, versus replacement of a potentially active 
file.


 If the latter is chosen, it would unlink(), perform the open more
 carefully, check that it is in the right place with fstat, but then
 it needs some work for ftruncate (to get rid of extra file contents
 at the end).  If the open() failed, it might try an unlink followed 
by

 open()?

 Other rcs implimentations need to be checked.  It is better if they 
work

 the same.

RCS file ownership?

[Adam Thompson]

When I use co(1) with "-l" to check out a file (and/or "ci -l") is there 
any way to preserve file ownership and *not* have it reset to the user 
running co(1) or ci(1)?
I don't see anything in rcs(1), co(1) or ci(1) that even mentions the 
fact that the file will wind up owned by the user running the command.

Ideas?  Pointers to documentation?

Thanks,
-Adam

Re: mount dir over another dir

[Adam Thompson]

On 2020-04-16 02:13, Ono Caritofilaxy wrote:

Hello.

I want to mount /usr/local/srcdir /usr/local/dstdir/subdir

answer was "no" 3 years ago
https://marc.info/?l=openbsd-misc=149743861203607=2

Can I do this now?
If not - why? Is it dangerous?


You should be able to do this as an NFS mount.  With all the nastiness 
that NFS mounts come with, but it's an option.  (I'm doing it in 
production on 6.6-STABLE.)

-Adam

door handles

[Adam Thompson]

None of the Taymor levers are quite right.  So I went looking, and I 
found some of what I'm looking for.


Short list:

(top pick)
1. Omnia 762, plus privacy bolt.  I love it but holy shit that's 
expensive @ ~US$180ea!

https://www.omniaindustries.com/product/762/

2. Rocky Mountain Hardware's "BELLA" lever, included in door set, price 
unknown, distributor/retailer unknown.

https://www.rockymountainhardware.com/products/door-hardware/handles/levers/door-levers/bella-lever-l150


3. Emtek (an Assa Abloy company)'s "Spencer" Lever.  Price unknown.  
Lots of local retailers.

https://emtek.com/passage-privacy-levers/spencer-brass-lever

4. Schlage/Dexter J-series "Dover" lever.  Price: cheap.  Local 
availability unknown.

https://www.schlagecanada.com/en/home/products/J40DOVFFF.html#

5. Schlage/Dexter "Jazz" level.  Price: cheapish.  Local availability 
unknown.

https://www.schlagecanada.com/en/home/products/F40JAZFFF.html#

(bottom pick)
6. Weslock "Bristol" or "Bristol UL"
http://weslock.com/product/bristol-2/ or 
http://weslock.com/product/bristol-ul-2/

Re: syspatch(8) return values?

[Adam Thompson]

On 2020-02-08 06:03, Antoine Jacoutot wrote:

On Fri, Jan 31, 2020 at 09:03:59AM -0600, Adam Thompson wrote:

There's no mention of what syspatch(8) returns, in the manpage.

I can prove quickly enough that it exits(0) when there's nothing to 
do, but
I'm more interested in knowing (for automation purposes) what the 
return
values are in other circumstances, and all my systems are already up 
to

date.  Before standing up yet another system, I figured I'd ask here.

I can think of four scenarios syspatch(8) perhaps ought to 
distinguish, at

least I'm interested in these 4 outcomes:
  1. nothing to do
  2. patches waiting, but didn't do anthing
  3. patches applied
  4. something went wrong

Can I reliably tell based on $? or do I have to parse the output?


Most likely parse, yes.

"patches waiting, but didn't do anything" might be interesting (i.e 
patches are

available); dunno...


Equally if not more interesting would be "I tried to apply patches but 
failed somehow".  Which happened to me, and which is why I asked in the 
first place.
*sigh*  I'll try to propose some bad diffs [if I'm writing them, they'll 
be bad] someday soon.

-Adam

Re: Dell Latitude e6400 OpenBSD Drive Issue

[Adam Thompson]

On 2020-02-10 09:36, Michael G Workman wrote:

Ok, thanks for the info.


For your E6400, see this guide: 
https://www.parts-people.com/blog/2012/10/16/dell-latitude-e6420-cmos-battery-removal-and-installation/


I found E6400 CMOS batteries from multiple vendors on the first page of 
Google results, most around US$10.  The only recommendation I can give 
is that I've used Parts-People.com in the past, no complaints with them.


Make sure you get the right replacement - check if your battery is 
2-wire or 3-wire *before* ordering a replacement - many of the listings 
don't worry about such minor details, g.


The older the Latitude, the harder it is to open, but even an E6400 is 
pretty easy, even if you've never opened up a laptop before.


Good luck,
-Adam

Re: Dell Latitude e6400 OpenBSD Drive Issue

[Adam Thompson]

On 2020-02-09 06:58, Michael G Workman wrote:

Hello,

Shout out to the OpenBSD developers for making a great OS!

I was able to install OpenBSD 6.6 on a Dell Latitude e6400 laptop, with 
a

USB Install. Sent the dmesg in already.

The installer would not recognize the hard drive, a brand new SSD 
drive.
The solution to that, from stack exchange, was to change the SATA 
settings

in BIOS from IRRTL to AHCI, that fixed the problem.

However if my laptop is powered off for a while, the SATA setting 
changes
back to IRRTL instead of AHCI, very annoying, not sure why the BIOS 
would

not make my changes persistent. I think it may be a hardware issue, but
just wanted to know if anyone else has encountered this before?

Thanks.

*Michael G. Workman*
(321) 432-9295
michael.g.work...@gmail.com


I have run several laptops from that series with OpenBSD.  The other 
replies are correct, your BIOS battery is dead.  Unfortunately, on many 
of the Latitudes, the BIOS battery is of the variety that's embedded in 
the RTC chip, and is not separately replaceable.
Some, however, including - the 6430 for example - have a regular coin 
cell, albeit wrapped in a proprietary cover with a non-standard 
connector, but at least is *is* replaceable without insane amounts of 
work.
I have the owner's manuals for many of the 6400 series, email me 
directly if you can't find the guide to replacing parts for your 
particular model.

-Adam

Re: suggestions for USB printer (maybe even with scanner)?

[Adam Thompson]

On 2020-02-05 13:56, Claus Assmann wrote:

I need to buy a printer to connect to one of my OpenBSD machines
and I prefer a USB connection (as I don't control the network at
my current place).  Can I just buy any USB printer or are there
printers which do not work with OpenBSD? If so, what do I need
to check / avoid?

Any suggestion for something "cheap" (to print just a few documents
as needed)? I never had to buy a printer before, so I'm not familiar
with this area -- if possible I would like to get a printer/scanner
but I have no idea what I can buy locally :-(
A HP laserjet (which was a gift but broke today) worked only with
one of my OpenBSD machines which seemingly was related to the USB
HW, using a printcap entry like this:
usb:lp=/dev/ulpt0:sd=/var/spool/output/usb:sf:sh:tr=^D:


I don't know what you need in a printer, and I don't know what you mean 
by cheap, so... YMMV.


However, I've found Brother **LASER** printers to be very good, and most 
of them support PCL6 and/or PS3.
For example, the HL-L2370DW can only connect via USB, and supports PCL6, 
and currently sells for ~C$150-160.


Just don't try to use their MFC-* line of color printers under UNIX 
(except MacOS).  FWIW, if you're in a situation where you have a spare 
Mac, the Mac can bridge from CUPS/PDF format to Brother proprietary 
format... bit pf a pain but it works.


-Adam

syspatch(8) return values?

[Adam Thompson]

There's no mention of what syspatch(8) returns, in the manpage.

I can prove quickly enough that it exits(0) when there's nothing to do, 
but I'm more interested in knowing (for automation purposes) what the 
return values are in other circumstances, and all my systems are already 
up to date.  Before standing up yet another system, I figured I'd ask 
here.


I can think of four scenarios syspatch(8) perhaps ought to distinguish, 
at least I'm interested in these 4 outcomes:

  1. nothing to do
  2. patches waiting, but didn't do anthing
  3. patches applied
  4. something went wrong

Can I reliably tell based on $? or do I have to parse the output?

Thanks,
-Adam

password-less user (without bothering security(8))?

[Adam Thompson]

Hi,
On 6.6-STABLE, I'm looking at security(8) and it's not immediately 
obvious to me how I can have an SSH key-only user who does not have a 
password, that also does not trigger daily security warnings.


The goal is to have a user that can never log in on the console, or via 
password any other way (FTP, SMTP auth, POP, etc., etc.), but only via 
the RSA key provided.


Is there a way to placate security(8) that I'm just not seeing?  Or is 
my goal fundamentally misguided for some reason I'm not seeing?  The 
user in this case is semi-trusted (e.g. yes, we'll let you login using 
an unprivileged account to run bgpctl in pipelines) but not 
organizationally-trusted (i.e. but that's ALL we want you to do on this 
system).


Thanks,
-Adam

Re: Is there an easier way to browse ports?

[Adam Thompson]

Ah, there's a good answer to the question I just asked Marc, thanks!-Adam

Re: Is there an easier way to browse ports?

[Adam Thompson]

Oh, ok... Do you recall an example offhand?  (I haven't noticed systemic
problems with either, but then I'm hardly a ports expert!)Thanks,-Adam
On Nov. 7, 2019 07:18, Marc Espie  wrote:

  On Wed, Nov 06, 2019 at 04:44:48PM -0600, Adam Thompson wrote:
  > Also http://openports.se/ and http://ports.su/ .

  Don't use those, they don't know how the openbsd ports are named.

Re: Is there an easier way to browse ports?

[Adam Thompson]

On 2019-11-01 06:12, Mischa wrote:

On 1 Nov 2019, at 12:08, Alfred Morgan  wrote:

My current workflow looks something like this:

$ cd /usr/ports
$ make print-index | less
I search and scroll through and find something interesting such as
opensonic.
I read the Info: game based on the Sonic the Hedgehog universe
^Z
$ cat games/opensonic/pkg/DESCR # I can't get make describe to work
I read more about it.
I google opensonic for screenshots.
$ pkg_add opensonic
$ opensonic
$ fg

Ideally I would like a graphical ports browser with name, screenshots, 
and
description that I can scroll and search through. Curation would be 
nice:

ports suggestions, popular ports, dev team ports picks, etc.

-alfred


Have a look at: https://openports.pl 

I think it ticks some of your boxes. :)

Mischa


Also http://openports.se/ and http://ports.su/ .

-Adam

Re: Tools for writers

[Adam Thompson]

On 2019-11-02 11:14, Peter Nicolai Mathias Hansteen wrote:

2. nov. 2019 kl. 16:00 skrev Oliver Leaver-Smith :

What tools do people find useful for writing on OpenBSD? By writing I 
mean long form such as novels and technical books, including plot and 
character development, outlining, and formatting for publishing (not 
all the same application necessarily)


I have found a number which boast Linux support, but not really 
anything that stands out which supports OpenBSD (aside from the 
obvious LaTeX et al.)


I really can’t speak to plot and character development, but all three
editions of The Book of PF were written using OpenOffice and later
LibreOffice write on OpenBSD snapshots.

Earlier versions of that manuscript were developed using DocBook SGML
(editing with emacs), but the publisher (fortunately) did not want any
truck with that.

For any new projects I would likely look half-heartedly for something
markdown based but would probably end up going the LibreOffice route
again.



FWIW, Brian Kernighan's new book was written using groff(1), with final 
rasterization done by gs(1), obviously there's a number of other tools 
involved.


On the other hand, unless you name is Brian Kernighan (or possibly 
Kristaps, Ingo, or jmc@) I doubt that toolset will satisfy you :-).


A few people around here have used TeX, LaTeX, and LyX (a LaTeX 
front-end) all of which are very much capable of large projects split 
into sections/chapters/etc.
 AFAIK OpenBSD's LaTeX / TeX packages are all more than adequate to the 
task, and all of the necessary tools are in ports.


-Adam

bgpctl(8) community question

[Adam Thompson]

[OpenBSD 6.5-STABLE, up to date]

When using bgpctl(8), I'm able to do almost everything I need, but I'm 
having trouble figuring out how to do one thing:


How do I show routes that do NOT have a community (or ext-community, or 
large-community) attribute?


The best I can come up with so far is a fairly ugly AWK script that 
buffers the detailed route output, then emits it if it doesn't see a 
Communities: line.  Am I missing a better way?


Thanks,
-Adam

N.B. manually looking through N sets of DFZ route tables isn't going to 
happen, I need a mostly-automatic solution.

Re: help understanding cua/tty EBUSY behaviour?

[Adam Thompson]

On 2019-08-03 18:14, Theo de Raadt wrote:

Adam Thompson  wrote:


Summary:  I open cua0 with cu(1), quit cu(1), try to re-open with
cu(1) but now it immediately fails with EBUSY.  *Usually* doesn't
happen with USB-to-serial (cuaU[0-9]) but have still seen it once or
twice.

[...]

You are observing the forcing-down of DTR and RTS for a long enough
period that the other side of the link notices the event.

Therefore it is not suprising you are finding this behaviour very
similar in many drivers and operating systems.


Thanks!
I feel as though I'm seeing something that's slightly off (see below), 
but I was focused on the open() end of things not the close() end - I 
now need to do more reading.



FWIW, things that don't *feel* right about this:
* "long enough" doesn't typically describe multiple days, in DTE/DCE 
signalling.  Far-end device is the same when testing using cua00 vs. 
cuaU0.
* isn't cua(4) supposed to be the device we use to ignore line signals?  
Why would closing it fiddle with line status?  Maybe I'm reading too 
much into hupcl in stty(1) manpage...

* and now I'm wondering if cu(1) fiddles with [-]clocal and/or [-]hupcl

As I said, more reading now you've given me a direction and I'll 
probably come back with at least one or two of my own answers.


-Adam

help understanding cua/tty EBUSY behaviour?

[Adam Thompson]

Summary:  I open cua0 with cu(1), quit cu(1), try to re-open with cu(1) 
but now it immediately fails with EBUSY.  *Usually* doesn't happen with 
USB-to-serial (cuaU[0-9]) but have still seen it once or twice.


I've seen this behaviour on OpenBSD 6.4, OpenBSD 6.5, and FreeBSD 11.2, 
and on 3 radically different systems (hardware-wise) so I don't think 
it's a version-specific or even hardware-specific bug, more likely 
something I'm failing to understand.


I'm using OpenBSD as a remote serial console server for up to 3 switches 
at a time (OOB access to a few switches in my lab).  This works, 
mostly... but occasionally, a serial port, almost always the onboard 
hardware cua0/tty0 port, somehow wedges and requires me to reboot the 
OBSD system to regain access to it.  The symptom is that when attempting 
to open(2) the device, I get EBUSY... for no obvious reason.  fuser(1) 
shows no other processes having a filehandle on /dev/cua00.


I don't understand why this happens inconsistently - about ~75% of the 
time on /dev/cua00, but only ~10% of the time on /dev/cuaU[0-1].  Of the 
~75% of the time it happens on /dev/cua00, about 1/3 of those times, if 
I wait a minute or ten, I can then re-open the device (again using 
cu(1)), and 2/3 of those times it persists until a reboot.


On the USB devices, I can - with 100% success - eliminate the problem by 
walking over, unplugging the serial adapter, and re-inserting it.


I haven't tried using e.g. screen(1) from ports, I've only tested using 
cu(1) in base so far.  I can try something else if there's a reason 
to... on the OpenBSD box anyway, it'll be a bit harder on the FreeBSD 
appliance.


As I said, I've even seen this on FreeBSD, so I expect I just need 
someone to provide an explanation of what nuance of tty(4) usage I'm 
missing.


Help?

Thanks,
-Adam

Re: SCM

[Adam Thompson]

On 2019-07-23 12:43, Stuart Henderson wrote:

On 2019-07-22, Stefan Sperling  wrote:


If your university class prefers using git, I'd recommend the 
repository at

https://github.com/openbsd/src.


However, it doesn't include branches/tags, because we haven't found
anything that
is able to successfully convert the OpenBSD CVS repository to git
unless it ignores
these.


I haven't tried this with the OpenBSD code base, but in a previous life 
I did a CVS to Git conversion starting with a repo that resembled 
OpenBSD's in many ways.
The "solution" was to get to git by way of SVN.  SVN was able to 
preserve branches/tags/etc. from CVS into SVN, and was then able to in 
turn be converted to git through SVN's git-compatibility layer (IIRC).

Whether this helps anyone out there... *shrug*
-Adam

Re: ipv6 nmap breakage under 6.5-STABLE ?

[Adam Thompson]

On 2019-07-22 09:51, Adam Thompson wrote:

Hi,
[Cross-posted to misc & ports as I'm not sure if there's a bug in
software or in wetware.]

I'm trying to run nmap (from ports) on 6.5-STABLE but am getting an
ungoogle-able error message every time:



Forgot to mention - this occurs under OpenBSD, but not Linux (CentOS 
7.6.1810 & nmap 6.40).  Obviously that's not an apples-to-apples 
comparison, but it's what I've got access to from here right now.

-Adam

ipv6 nmap breakage under 6.5-STABLE ?

[Adam Thompson]

Hi,
[Cross-posted to misc & ports as I'm not sure if there's a bug in 
software or in wetware.]


I'm trying to run nmap (from ports) on 6.5-STABLE but am getting an 
ungoogle-able error message every time:


  root@bgpmirror:~# nmap -Pn -A -n --top-ports=100 -6 
2620:132:300e:700::113

  Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-22 09:43 CDT
  sendmsg: Message too long
  sendmsg: Message too long
  [...continues like this for a really long time...]

Interspersed semi-randomly (not always at the same point) I also see 
mixed in there a single occurrence:


  sendmsg: Message too long
  sendmsg: Bad address
  Unable to send packet in probe_transmission_handler: Bad address (14)
  sendmsg: Invalid argument
  Unable to send packet in probe_transmission_handler: Invalid argument 
(22)

  sendmsg: Message too long
  Unable to send packet in probe_transmission_handler: Message too long 
(40)

  sendmsg: Message too long

Performing a similar scan on an IPv4 address works as expected.  This 
appears to be a v6-specific problem.


Known issue?
Workarounds?

Thanks,
-Adam

Re: Postscript printer recommendations

[Adam Thompson]

On 2019-07-14 15:40, Stuart Henderson wrote:

If you don't want trackable prints, don't buy a colour laser printer
of any brand, it is very common. Unsure about mono and inkjet printers,
I would tend to assume that they're common on at least most hi-res
colour printers.


Nearly every printer sold today (including cheap inkjets AND b 
printers) have these tracking dots.  It's unclear why/how this came to 
be; I've heard multiple stories and am unsure which to believe, if any.  
Regardless of the why/how, consensus is clear: those dots are real and 
can tie output back to your printer.




 I read that Postscript printers produce superior graphics (from
Xerox website):


In 2019, that remains true only in a very small number of edge cases.  
The absolute highest resolution graphics today are mostly printed using 
ESC/P (if Epson), PCL3 (if HP), or whatever Canon uses (if Canon).  
Mostly people "prefer" Postscript output over PCL because the default 
rendering gammas(sic) are slightly different and Postscript is usually 
perceptually more pleasing.  Adjust gamma on your PCL printer from 1.1 
to 1.2 and *poof* just as good as Postscript.  YMMV, even 
Postscript-compatible printers are all different nowadays.



If I was looking to spend money on a nice printer I'd get one which can
accept postscript and PDF directly over lpr. (I'd be very tempted by
the hp "pagewide" printers. Also Kyocera seem good especially for
high volume stuff).


I haven't tried Kyocera recently, but I have tried HP recently in that 
line.  Stay as far away as you can get.  All the decent PageWide 
printers seem to be discontinued already, and the replacement 11x17" 
color OfficeJet Pros are severely lobotomized and need to talk to the 
"cloud" for almost all functions except basic print-to-paper... and even 
that doesn't work very reliably, even with the Windows/MacOS drivers.




If I was looking for something cheap and cheerful then I would worry
less about postscript but look for something where I can see signs
of support in the ports tree. The HPLIP ports are nice and easy to use
and work with a wide range of HP and some other printers, easiest to
use with CUPS (it really isn't difficult), but if you want to avoid it
I believe the HPIJS part of HPLIP can be used without CUPS. I've heard
good things about brlaser for the non-postscript Brother printers too,
though most of them do have ps anyway. Even if keeping things cheap
I would definitely want something with an ethernet port (it doesn't
add much if anything to the cost) for flexibility of positioning the
printer, easier sharing between machines, and not having to mess
around with permissions on device nodes etc..


Agreed - most decent business printers nowadays come with both ethernet 
and wifi, so you get the best of both worlds.


Brother's mono lasers are generally *excellent* value for the money.  
Color lasers unknown.  Stay away from their color inkjets, they are 
effectively Win/Mac-only.  (But if you are on Win/Mac, those provide 
excellent $/page.)


Epson's color inkjets are pretty good, and at the *really* high end beat 
color lasers in all ways, and include Postscript.  (Those high-end units 
also cost many 10s of $Ks.  But, wow, nice specs.)  Lower-end units 
support ESC/P and mostly work with CUPS.  Buy the business models if you 
want a decent cost-per-page.


Canon printers can be decent, but are not usually good value for 
money... YMMV, just make sure you can return it if you can't make it 
work.


HP used to be very good, and their mono LaserJets are still acceptable.  
Color LaserJets... less good, but maybe still acceptable.  Inkjets?  Who 
knows, there's a new model every week, each incompatible with the last 
one.


Your best bet will likely be to purchase a new "business-grade" printer 
from a shop that will let you return it if you can't get it to work.


Unfortunately, YMMV depending on what part of the world you're in, any 
local promotions/sales, the phase of the moon, whether Jupiter is in 
alignment with Saturn, etc., etc.


Good luck,
-Adam

Re: How does OpenBSD probe for I/O devices?

[Adam Thompson]

On 2019-06-12 13:12, ¯\__/¯ ¯\__/¯ wrote:

I've search for the answer to this question, but I can't find it.
I also read the source code, but I still don't get how it works.
Help pl0x


Not sure exactly what you're looking for...

On modern architectures, most OSes (including OpenBSD) "walk the 
hardware device tree".  The possible topologies and nodes of the device 
tree are controlled by the kernel source code.  OpenBSD does 99% of it 
at boot time, with a few notable exclusions (PCMCIA, PC CARD, USB, can't 
remember what else).


Look under /usr/src/sys/arch/* for functions with "_attach_" in their 
names, which should give you a very rough idea of where to start 
looking.


For both historical and somewhat-current documentation on how this 
works, check out 
https://en.wikipedia.org/wiki/Marshall_Kirk_McKusick#Bibliography .
(I'm unaware of any OpenBSD-specific publications covering that sort of 
thing, but OpenBSD *is* derived from the same BSD UNIX that Kirk wrote 
about.  Lessons learned about one BSD can, usually, have their concepts 
applied to their cousins - although the implementation details have 
diverged quite a bit by now.)


-Adam

Re: The su manual doesn't mention use root account by default

[Adam Thompson]

On 2019-06-12 03:55, Ingo Schwarze wrote:

Even though su(1) can still be used today to relinquish privilege
when you are already root, no more development is done on it and people
rarely look at the manual page.  The last time new functionality was
added to the su(1) manual page was almost a decade ago, and the
last time before that 17 years ago.


Well, su(8) also is used to obtain root privileges in the first place.

FWIW, I regularly use "su" on OpenBSD because it's a relatively 
consistent cross-platform way to have root run a command as someone 
else.  I recall a good number of ports using su(8) internally in, e.g. 
process-control scripts - but that was years ago, not sure if it's still 
true or not.


doas simply isn't available anywhere else (yet).  (IMHO, I don't think a 
portable version of doas has a lot of potential - it's not complicated 
enough! )


During initial system installation & deployment, before doas is 
configured, and assuming you haven't [yet] added your SSH keys to 
~root/.ssh/allowed_keys, it's quite impossible to avoid using su.  
(AFAIK.  If there's another way, let me know!)


I hope you're just saying that su(8) is a mature, stable utility that 
needs no further work right now.  It kind of sounds like you might be 
saying that su(8) could be on the chopping block, much like sudo(8)... 
have I misread that?


-Adam

openup failing?

[Adam Thompson]

I've seen a large number failures recently from m:tier's openup tool, 
complaining of:


ftp: connect: Host is down
!!! Cannot retrieve https://stable.mtier.org/openup
!!! Please verify your Internet connection, proxy settings and 
firewall.


I'm seeing this from two different networks/providers/companies, so I'm 
assuming it's not me, but am posting this to validate that assumption.


Assuming it's not just me, does anyone know what's going on with them?  
The relevant routes appear in the DFZ, but none of the *.mtier.org IPs I 
know of respond.


-Adam

"Invalid argument" when exec'ing and/or ktrace'ing a file?

[Adam Thompson]

I have a binary - built on this 6.5-STABLE amd64 system by an automatic 
build process as part of a CPAN module installation, that will not 
execute:



rt@rt$ /var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf
ksh: /var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf: 
Invalid argument


or even allow itself to be ktrace'd:

rt@rt$ ktrace 
/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf
ktrace: exec of 
'/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf' 
failed: Invalid argument


At least ktrace(8) told me it was an exec(3) problem.  So off I go to 
exec(3) which leads me to execve(2), wherein I see:



[EINVAL] argv did not contain at least one element.


errno(2) confirms the english expansion of "Illegal argument"

file(1) gives me a clue, but I don't know what to do with it:


rt@rt$ file `locate bin/wkhtmltopdf`
/usr/local/bin/wkhtmltopdf:   ELF 
64-bit LSB shared object, x86-64, version 1
/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf: ELF 
64-bit LSB executable, x86-64, version 1


where the version in /usr/local/bin is from ports.

the RT::Extension::TicketPDF version is significantly larger, ~4x, which 
led me to:



rt@rt$ ldd `locate bin/wkhtmltopdf`
/usr/local/bin/wkhtmltopdf:
StartEnd  Type  Open Ref GrpRef Name
041737f61000 04173ac77000 exe   20   0  
/usr/local/bin/wkhtmltopdf
0419ee50 0419ee599000 rlib  01   0  
/usr/local/lib/libjpeg.so.70.0

[...elided...]
/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf:
not a dynamic executable


So... how do I figure out why a static binary (whose build process is 
pretty opaque) won't/can't run?


Clueless at this level, my detailed knowledge of how exec worked under 
OpenBSD ended in the a.out era.


(oh, the reason I don't use the version from ports?  It's too new for 
the TicketPDF code. *sigh*)


Thanks for any pointers,
-Adam

Re: OpenBSD on VMware ESXi

[Adam Thompson]

On 2019-05-22 09:25, mxb wrote:
I think FreeBSD or any Linux template will work just fine and add 
vmxnet3.

However, last I checked (1year ago) vmxnet3 been less stable than
e1000 under pressure.


Don't use the Linux templates.  I would recommend against using the 
FreeBSD templates, and go with "Other (64-bit)" instead.  YMMV on using 
FreeBSD vs Other, I haven't seen consistent results here yet... just 
don't pick Linux, or DOS, or Windows - in some situations, that allows 
VMware to take certain shortcuts that are based on assumptions about the 
Linux/Win/etc. kernel & device drivers that (probably) aren't valid 
under OpenBSD.


Various people have reported different problems with vmxnet3; I'm aware 
of at least 4 or 5 different environment-specific issues (i.e. can't be 
reproduced on any other vSphere/ESXi system).  I have some of those 
problems, and I cannot reproduce them outside my production environment, 
but they don't prevent me from running OpenBSD.


Workarounds:
* use vmxnet2
* use e1000

If vmxnet3 and pvscsi work for you (you'll know pretty darn fast!), use 
them.  When they work, which is usually (in my experience), they're 
generally very stable and high-performing compared to the emulated h/w 
(e1000, lsisas, lsiscsi, buslogic).


I also experience timer issues, and I've had to specify 
kern.timecounter.hardware=i8254 in sysctl.conf.  This is likely a VMware 
problem, not an OpenBSD problem, but it's non-trivial to diagnose.  
(Even i8254 doesn't work perfectly: e.g. usleep() isn't very accurate in 
my VMs!)  I'm also running these VMs on very heavily-loaded hosts, which 
is probably a factor.


My disk write throughput is horrible, but that's an interaction between 
how OpenBSD does writes, how VMware handles writes into thin-provisioned 
disks, and how my NFS storage handles writes on thin-provisioned 
volumes; it's not an OpenBSD problem, strictly speaking, although that's 
the only place it's normally visible.


Overall, OpenBSD works well under ESXi, but there are semi-random 
problems that do have workarounds.


Several years ago, Theo noted (approximately, I'm going from memory here 
AND paraphrasing) that it was hard enough for OpenBSD to handle broken 
hardware implementations, it's exponentially harder to handle an 
incorrect software emulation of hardware that was incorrect in the first 
place.  This has proven accurate, and VMware doesn't really care much 
about OpenBSD, since I doubt it even registers on their radar so they're 
not terribly interested in fixing VMware bugs that are only visible 
under OpenBSD.  (If you have a support contract, please submit bug 
reports to VMware.  If enough of us do so, they might start fixing some 
of the problems.)


-Adam

Re: relayd without pf?

[Adam Thompson]

FWIW, I also encountered some slightly different error messages, I'll see if I 
can reproduce those.
-Adam

On May 14, 2019 4:48:29 p.m. CDT, Reyk Floeter  wrote:
>
>> Am 14.05.2019 um 23:06 schrieb Adam Thompson :
>> 
>>> On 2019-05-14 15:42, Adam Thompson wrote:
>>> OK, I'm pretty sure this is a dumb question, but...
>>> Does relayd work properly, or at all with pf disabled?  (in
>6.5-RELEASE)
>> 
>> 
>> I have partially answered my own question.  That last message was
>posted prematurely, in more than one way, sorry!
>> 
>> 1. the relayd.conf in the previous message was copied-and-pasted from
>the wrong window, in mid-edit.
>> 
>> 2. relayd(8) does not work with pf(4) disabled.  I'm unclear if this
>is a bug, or by design.  With pf disabled, it outputs:
>> root@rt:~# relayd -dv
>> startup
>> relayd: pfe: pf is disabled
>> parent: proc_open: imsg_flush: Broken pipe
>> ca exiting, pid 37187
>> ca exiting, pid 79962
>> ca exiting, pid 95113
>> root@rt:~# hce exiting, pid 91576
>> relay exiting, pid 26432
>> relay exiting, pid 6966
>> relay exiting, pid 50166
>> 
>> The message "pfe: pf is disabled" looks like an informational message
>to me, I'm not using any pf features, so it shouldn't matter... but it
>very much does matter, and relayd exits shortly after starting if pf is
>disabled.
>> 
>> Pinging @reyk - is this a bug or deliberate?
>> 
>
>It’s a historical reason because redirects existed first. And most
>OpenBSD systems keep pf enabled by default.
>
>But you’re right; it should be easy to fix.
>
>Reyk

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: relayd without pf?

[Adam Thompson]

On 2019-05-14 15:42, Adam Thompson wrote:

OK, I'm pretty sure this is a dumb question, but...

Does relayd work properly, or at all with pf disabled?  (in 
6.5-RELEASE)



I have partially answered my own question.  That last message was posted 
prematurely, in more than one way, sorry!


1. the relayd.conf in the previous message was copied-and-pasted from 
the wrong window, in mid-edit.


2. relayd(8) does not work with pf(4) disabled.  I'm unclear if this is 
a bug, or by design.  With pf disabled, it outputs:

root@rt:~# relayd -dv
startup
relayd: pfe: pf is disabled
parent: proc_open: imsg_flush: Broken pipe
ca exiting, pid 37187
ca exiting, pid 79962
ca exiting, pid 95113
root@rt:~# hce exiting, pid 91576
relay exiting, pid 26432
relay exiting, pid 6966
relay exiting, pid 50166

The message "pfe: pf is disabled" looks like an informational message to 
me, I'm not using any pf features, so it shouldn't matter... but it very 
much does matter, and relayd exits shortly after starting if pf is 
disabled.


Pinging @reyk - is this a bug or deliberate?

-Adam

relayd without pf?

[Adam Thompson]

OK, I'm pretty sure this is a dumb question, but...

Does relayd work properly, or at all with pf disabled?  (in 6.5-RELEASE)

It looks like it should as long as I use "relay" instead of "redirect", 
but I'm having trouble, and don't want to keep banging my head against a 
wall if it's something this simple.


Thanks,
-Adam


--begin relayd.conf--
http protocol rtproxy {
pass quick
}
relay rt4 {
listen on 0.0.0.0 port 80
protocol rtproxy
forward to 127.0.0.1 port 8080
}
relay rt6 {
listen on :: port 80
protocol rtproxy
forward to ::1 port 8080
}
--end relayd.conf--

Re: post-6.5-upgrade bgpd(8) problem

[Adam Thompson]

On 2019-05-09 13:53, Sebastian Benoit wrote:

bgpctl sh rib neigh  out
for all neighbors.


All empty.



Also look at
bgpctl sh rib best


Completely empty.



if any routes are actually selected - maybe the "nexthop qualify via
default" isnt working.


I see two things...
1) when run as "bgpd -dv" I get repeated notifications about the same 
next-hops, dunno if that's normal or not.

And 2) from bgpd.conf(5):

route-collector (yes|no)
 If set to yes, the route selection process is turned 
off.  The

 default is no.

and I have it set to yes, since this is supposed to behave like a route 
collector.  Granted, with a single source of routes at the moment, I 
could turn that off... let's see what happens when I do:


Yup.  That behaviour has definitely changed, somehow - taking a 
not-so-wild guess, it's likely _somehow_ related to claudio@'s 
introduction of Adj-Rib-Out but I'm not enough of a C programmer (or 
kernel routing expert) to say for sure.


For now, commenting out the "route-collector yes" line in bgpd.conf will 
work, but there's a (for me) minor regression.  I have no idea which way 
the code is supposed to behave, so can't tell if this is a bug or a 
bugfix!


Based on CVS logs, it's probably a change introduced in rde.c after 
v1.442.


-Adam

post-6.5-upgrade bgpd(8) problem

[Adam Thompson]

I've upgraded my looking glass from 6.4 to 6.5, and an experiencing an 
unexpected problem - routes learned from one (iBGP) peer are not being 
automatically exported to other (eBGP) peers.


I did not change /etc/bgpd.conf, but behaviour seems to have changed 
nonetheless.  The upgrade from 6.4 to 6.5 appeared smooth otherwise, 
nothing to suggest subtle breakage under the hood.


As you can see below, this bgpd.conf is almost so simple it *can't* have 
problems.  Apparently "almost" being the operative word.


Under 6.4, this behaved as though "export none" only applied to the 
first group.  Under 6.5, it behaves as though "export none" is a global 
setting.


Under 6.5, bgpctl show produces:
root@bgpmirror:~# bgpctl sh
Neighbor   ASMsgRcvdMsgSent  OutQ 
Up/Down  State/PrfRcvd
Hermes IPv4 16796 128773 85 0 
00:41:40 753748
Hermes IPv6 16796  29727 85 0 
00:41:40  68228
MBNOG IPv4  65204 97 85 0 
00:41:40  0
MBNOG IPv6  65204 97 85 0 
00:41:40  0
BGPMon.io IPv4   6447  0 21 0 Never  
  Active
isolario.it IPv465517 86 85 0 
00:41:39  0
isolario.it IPv665517 86 85 0 
00:41:39  0
and the operator of the MBNOG route collector confirms that I stopped 
sending him a full routing table at the same time I did the OS upgrade.


Any ideas?  What other information would help diagnose this problem?

Thanks,
-Adam



Dmesg & bgpd.conf:  
https://gist.github.com/athompso/e334d8621ce458925e25bb44b8068341



bgpd.conf, duplicated here for convenience:

===BOF===
route-collector yes
socket "/var/www/run/bgpd.rsock" restricted # for bgplg

# settings
nexthop qualify via default
fib-update no
dump table-v2 "/var/www/htdocs/bgplg/mrt/rib-dump.mrt" 3600
dump updates in "/var/www/htdocs/bgplg/mrt/updates-in-%H%M" 300
dump all in "/var/www/htdocs/bgplg/mrt/all-in-%H%M" 300

# myself
AS X
router-id X.X.X.X

# neighbors

group hermes {
enforce local-as no
enforce neighbor-as no
export none

neighbor X.X.X.X {
remote-as X
descr "Hermes IPv4"
}
neighbor X:X:X:X::X {
remote-as X
descr "Hermes IPv6"
}
}

group bgpresearch {
multihop 32
enforce local-as no
enforce neighbor-as no

neighbor 192.160.102.196 {
remote-as 65204
descr "MBNOG IPv4"
}
neighbor 2620:132:3003:300::196 {
remote-as 65204
descr "MBNOG IPv6"
}
neighbor 129.82.138.6 {
remote-as 6447
descr "BGPMon.io IPv4"
}
neighbor 146.48.78.12 {
remote-as 65517
descr "isolario.it IPv4"
}
neighbor 2a00:1620:c0:4e:146:48:78:12 {
remote-as 65517
descr "isolario.it IPv6"
}
}

# policies
allow quick from group hermes
allow quick to group bgpresearch
===EOF===

(if you want to see the unredacted version of bgpd.conf, ask and I'll 
email it directly to you, I just don't want internal addresses in the 
public archive.)

Re: How to restrict ip to access a directory in OpenBSD's httpd

[Adam Thompson]

On 2019-04-03 11:30, Stuart Henderson wrote:

On 2019-04-03, =?utf-8?B?RnVuZw==?=  wrote:

apache support somthing like

Order Allow,Deny
Allow from all
Deny from 1.2.3.4


How to achieve in OpenBSD's httpd?
We are using OpenBSD 6.4.




There is no built-in simple way.

It can be done by having httpd listen on two different ports,
one allowing access to this directory, the other denying access,
and using a PF rdr-to rule to send traffic to the "allow access"
port if it has the correct source IP address. But this is a bit
of a mess.


I vaguely recall hearing someone (possibly Reyk, several years ago?) 
mention that relayd can  handle access control for httpd, if httpd is 
listening only on loopback.
This seems like overkill, but does fit the "UNIX philosophy" of doing 
one thing well.


I'm not at all sure it was Reyk, and I'm sure not 100% confident of this 
solution, but a quick glance at the man pages suggests it's not totally 
insane, either.


-Adam

Re: security - preferred way to make check_access_file happy?

[Adam Thompson]

On 2019-02-25 11:14, Stuart Henderson wrote:

On 2019/02/25 09:13, Adam Thompson wrote:

> Use vipw to put 13 * in the password field
>
> From passwd(5)
> [...]
>  authentication, conventionally have 13 asterisks in the password field.

Thank you!  Now that I know what I'm looking for, I can see the 
relevant

code in security(8), too.

I wonder if there's a way for ports to do that for me while calling 
useradd?

Another rabbit hole to go down.

Thanks again,
-Adam



It normally does already. Do you have an unusual "password" line in
/etc/usermgmt.conf?


Nope.
Ugh, although I said ports does this often in my experience, the only 
system/port I have right now where it's doing it is the RANCID package 
on that dedicated VM.  I have two other VMs where I have manually 
created single-purpose userids without passwords that also complain (one 
for RT [not from ports], one for webhosting).
So it could just be that package; at least I can't demonstrate any 
others right now.

-Adam

Re: security - preferred way to make check_access_file happy?

[Adam Thompson]

Use vipw to put 13 * in the password field

From passwd(5)
[...]
 authentication, conventionally have 13 asterisks in the password 
field.


Thank you!  Now that I know what I'm looking for, I can see the relevant 
code in security(8), too.


I wonder if there's a way for ports to do that for me while calling 
useradd?  Another rabbit hole to go down.


Thanks again,
-Adam

Re: security - preferred way to make check_access_file happy?

[Adam Thompson]

Whoops... I'm getting the messages from 3 systems, all running 
6.4-STABLE, with no local modifications, under both VMware and 
Openstack, using openup to keep systems updated.  Dmesg available if 
anyone thinks it's relevant.

-Adam


On 2019-02-25 08:50, Adam Thompson wrote:

Hi,
I'm getting daily insecurity (i.e. security(8)) nags about userids
that are off but still have a valid shell and access files.
(Specifically, I'm getting the nag from check_access_files() in
/usr/libexec/security.)

Since ports (at least in my experience) regularly creates userids that
will trigger this warning, what's the "best" way to disable the
warning?  I'm reluctant to mess with permissions on directories
created by packages, but maybe that's the best way?

Otherwise, it looks like I can disable the warning by setting a
password on the userid in question.

However, that leads to the question: what if I don't *want* a password
on the account, because it's supposed to be a SFTP-only,
public-key-authentication-only account, but still needs to be readable
and needs a valid shell for various cron jobs to be happy?  If I'm
following the logic correctly, one of the warnings I'm getting is for
~/.ssh being readable on a userid with no password - exactly the
scenario I just mentioned.  But AFAIK they can't login if I take away
S_IRUSR on ~/.ssh?

The most distasteful option is to hack /usr/libexec/security to ignore
certain userids, but ... it's there for a reason.

The cleanest example I have right now from ports is _rancid, created
by the rancid package, and triggered by the existence of ~_rancid/.ssh
with S_IRUSR (u+r) permissions.

Suggestions / advice?

Thanks,
-Adam

security - preferred way to make check_access_file happy?

[Adam Thompson]

Hi,
I'm getting daily insecurity (i.e. security(8)) nags about userids that 
are off but still have a valid shell and access files.  (Specifically, 
I'm getting the nag from check_access_files() in /usr/libexec/security.)


Since ports (at least in my experience) regularly creates userids that 
will trigger this warning, what's the "best" way to disable the warning? 
 I'm reluctant to mess with permissions on directories created by 
packages, but maybe that's the best way?


Otherwise, it looks like I can disable the warning by setting a password 
on the userid in question.


However, that leads to the question: what if I don't *want* a password 
on the account, because it's supposed to be a SFTP-only, 
public-key-authentication-only account, but still needs to be readable 
and needs a valid shell for various cron jobs to be happy?  If I'm 
following the logic correctly, one of the warnings I'm getting is for 
~/.ssh being readable on a userid with no password - exactly the 
scenario I just mentioned.  But AFAIK they can't login if I take away 
S_IRUSR on ~/.ssh?


The most distasteful option is to hack /usr/libexec/security to ignore 
certain userids, but ... it's there for a reason.


The cleanest example I have right now from ports is _rancid, created by 
the rancid package, and triggered by the existence of ~_rancid/.ssh with 
S_IRUSR (u+r) permissions.


Suggestions / advice?

Thanks,
-Adam

cvsweb.openbsd.org - same as cvsweb in ports?

[Adam Thompson]

I know this has been asked before, but my google-fu cannot unearth any 
trace of it, so I have to ask again - sorry!


What version of cvsweb does cvsweb.openbsd.org run?  And where is that 
software available?  It appears to not quite be the same as cvsweb in 
ports, so... ?


Thanks,
-Adam

Re: keeping track of MAC addresses

[Adam Thompson]

On 2019-02-14 02:01, mailingli...@dotbit.ro wrote:

I would like to keep tabs on the MAC/IP addresses in my secure net.
I do know how to do this, but keeping track of ethernet MAC addresses 
seems
quite cumbersome in OpenBSD, not that it is more convenient in any 
other
general purpose operating system but many interfaces for ex. routers 
make it

easy to manage, especially MAC filtering.


Perhaps look at the "arpwatch" package in ports, which may be 
applicable.


But... you know that both ARP and MAC addresses can be trivially 
spoofed, right?  Just using /etc/ethers instead of ARP does *not* make 
your network secure.


Some "intelligent" switches do ARP sniffing to populate their internal 
hardware FIBs.  (Yes, that's a dumb idea.  Switch vendors still do it.)  
Disabling ARP on your hosts is... not generally a good idea.


PS: after running ifconfig em0 -arp my Allied Telesis AT-GS950-16 
managed
switch took the link down and refuses to bring it back up on the same 
port

without a reset. Other ports work fine.


I won't say this is impossible, but it seems unlikely.  I think it's 
more likely the lack of ARP traffic on the port caused the switch to do 
something "interesting" with IP traffic destined for this host.  Or 
maybe something else triggered storm-prevention features in the switch?  
Running an ifconfig(8) command should not be able to persistently shut 
down a switch port in any network environment.  Did you observe the link 
lights on the NIC and switch actually turn off and stay off?


As I have already mentioned I can manage by myself, but it seems to me 
that

this is something that a lot of people would want.


Not so much, AFAIK.  Disabling core IP protocols usually generates more 
problems than it solves.  Let us know how disabling/blocking ICMPv6 
works out for you... ;-)  [Hint: that's a trick question.  You can't run 
IPv6 without ICMPv6.]


You could filter on MAC addresses instead of restricting ARP:  
https://www.openbsd.org/faq/pf/tagging.html#ethernet   That requires 
using bridge(4) which apparently is on its way out, and I don't know if 
the replacement (switch(4)) supports filtering packets based on MAC 
address or not - it's OpenFlow-compliant, so there has to be a way, but 
it may or may not be easily accessible from inside OpenBSD.


You may also want to assign new MAC addresses to your hosts, both to 
eliminate the need to gather the MACs, and to simplify maintenance (e.g. 
the labour involved in replacing a NIC on a server or a motherboard is 
O(n^2) with hardware-bound MAC addresses in your setup, instead of 
O(1)).  There are special LAAs (Locally-Assigned Addresses) that you can 
use for this.  OpenBSD supports setting a locally-assigned MAC address 
with ifconfig(8) "lladdr" option.


Good luck on your strange quest,
-Adam

Re: purpose of bgpd.conf dump "timeout" parameter?

[Adam Thompson]

Yes, that clarifies it, thank you.  I will attempt to explain that better and 
provide patch for bgpd.conf.5 next week.
May I suggest a future function for bgpctl/bgpd - creating an MRT dump on 
demand?  Even if only able to run in the context of bgpd(8), I think it could 
be helpful.  (Certainly it would be helpful to me.  Which is probably obvious 
since I'm suggesting it...)  Something else to tack onto the to-do list, I 
guess.
Thanks,
-Adam

On February 8, 2019 5:23:24 PM CST, Claudio Jeker  
wrote:
>On Fri, Feb 08, 2019 at 03:56:12PM -0600, Adam Thompson wrote:
>> In bgpd.conf(5), for the "dump" directive there is an optional
>"timeout"
>> parameter.  What is its purpose?  I assume from the examples that
>it's
>> denominated in seconds...
>
>Yes it is.
> 
>> my first guess was to time out on attempting to write to the dump
>file, but
>> that doesn't seem realistic.  It looks like it's a cycle, i.e. the
>dump file
>> will be recreated every X seconds, but if so, why is it called
>"timeout" and
>> not "interval" ?
>
>Because naming is hard :)
> 
>> I see in the source that MRT_MAX_TIMEOUT is set to 7200 - does this
>mean
>> that if I leave the parameter unset, the MRT file will be re-dumped
>every 2
>> hrs?
>
>No, it just limits the poll timeout but it will only fire once the time
>really ran out.
> 
>> Yes, I'll produce a patch for the manpage if someone can explain what
>the
>> parameter is supposed to do / how it works.
>
>After timeout seconds the file is reopened (or maybe a new file is
>opened
>depending on the strftime expand of the filename). For all and update
>dumps this just puts new messages into a new file. For table dumps it
>will
>issue a new dump. e.g.
>   dump table "/tmp/rib-dump-%H%M" 300
>will create a new table dump every 5 minutes.
>
>Hope that helps.
>-- 
>:wq Claudio

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

purpose of bgpd.conf dump "timeout" parameter?

[Adam Thompson]

In bgpd.conf(5), for the "dump" directive there is an optional "timeout" 
parameter.  What is its purpose?  I assume from the examples that it's 
denominated in seconds...


my first guess was to time out on attempting to write to the dump file, 
but that doesn't seem realistic.  It looks like it's a cycle, i.e. the 
dump file will be recreated every X seconds, but if so, why is it called 
"timeout" and not "interval" ?


I see in the source that MRT_MAX_TIMEOUT is set to 7200 - does this mean 
that if I leave the parameter unset, the MRT file will be re-dumped 
every 2 hrs?


Yes, I'll produce a patch for the manpage if someone can explain what 
the parameter is supposed to do / how it works.


Thanks,
-Adam

Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]

[Adam Thompson]

On 2019-01-21 04:08, Gilles Chehade wrote:

In this test case, my translations map had:

What is a translation map ?
There is no such thing in OpenSMTPD (as of today).


A virtual map that happened to be called .



You're feeding the virtual table with invalid values.


Apparently, yes.


Also, this is a recipient translation mechanism, similar to aliases, 
and

not a sender rewriting mechanism which we do not have at this point.
[...]
virtual _now_ only works on recipients, not senders ?
the virtual code hasn't changed, it works the way it always did.

there is no way it could ever do what you're describing or attempting 
to
do given that it doesn't operate at all anywhere near the message. 
there

is no way it has ever parsed:


This is all very surprising to hear.  The existing system works 
(somehow).  So I am apparently misunderstanding what is happening, 
because with the configuration as shown, telling the various broken 
email senders to use that box as their mailhost _somehow_ fixes the 
bogus From: headers and envelopes.


Oh, this just occurred to me as I'm writing:  I really hope I didn't 
switch to a different MTA on that system years ago, and then just forgot 
to check which MTA was actually running.  If that's the case, I'm not 
going to bother posting an update, because I'll be busy banging my head 
on the wall and then hiding in shame.



I'm not convinced the new smtpd.conf grammar improves anything at all, 
but I assume it must help someone or it wouldn't have changed... but I 
believe my use case got thrown out with the bathwater, so to speak.  
Oh, well.  :-(

This is bullshit.
The grammar doesn't reduce the functional scope, it can only expand it.


I'm taking your word for it - you will know far better than I do!



What you are describing has never existed in smtpd, there's never been
code to translate sender addresses and there's a good reason for that:


Good reasons aside, I still need to accommodate other vendor's broken 
mail implementations, because I can't fix them.  I know of multiple 
reasons source rewriting is a bad idea, in general, but I get paid to 
make stuff work, not just say that it's broken.




it not considered doable before the grammar change...
But sure, blame it on the grammar.


I believed that the grammar change had rendered my use case impossible 
because  was now limited to local delivery methods.  Clearly I 
was wrong... and not even in the way I thought I might be wrong.



I may sound a bit harsh, but starting a thread with "this is my last 
try

or I'll switch" (as if it actually matters)


My apologies - that was meant to sound more like "I have a plan B so if 
this isn't possible, that's OK but I've wasted so much time on this I'm 
kinda running out of time, please tell me if I should just stop now and 
switch".  I know *exactly* how much OpenBSD devs care if I use their 
code or not!  I do not want to be "that asshole", although it seems I've 
succeeded again - sorry.


Thank you for taking the time to reply.  Now I'm going to go check that 
mail server a 7,000,000th time, this time to see what MTA is actually 
*running*, not just *configured*.  I'm not sure whether I want it to be 
such a blatant mistake on my part or not... if yes, this all makes sense 
but I'm an idiot, whereas if no, then WTF, how is it working at all?


FWIW: I am much happier with OpenSMTPd than with other MTAs because of 
its forward-declarative configuration syntax.  Thank you for your work 
on bringing a modern, lean, secure(-er) MTA into existence.


-Adam

Re: smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

I found the "-T" (trace) flag to smtpd(8), and it gives me this, which AFAICT 
confirms my suspicions:
[...]
rule #2 matched: match from src allowed-hosts for any => translate
lookup: lookup "athom...@athompso.net" as ALIAS in table 
static:translations -> 0
lookup: lookup "athompso" as ALIAS in table static:translations -> 0
lookup: lookup "@athompso.net" as ALIAS in table static:translations -> 0
lookup: lookup "@" as ALIAS in table static:translations -> 0
expand: lka_expand: no aliases for virtual
mproc: lka -> pony : 53 IMSG_SMTP_EXPAND_RCPT
expand: 0x154201b89018: clearing expand tree
imsg: pony <- lka: IMSG_SMTP_EXPAND_RCPT (len=53)
smtp: 0x1127a72e6000: >>> 524 5.2.4 Mailing list expansion problem
[...]

complete output attached below, I've changed to @old.athompso.net and 
@new.athompso.net during testing since the last email.

On the sending side, from another host (listed in ), I'm running:
swaks --to athom...@athompso.net --from athom...@old.athompso.net 
--server 
which faithfully reports the 5.2.4 error.

I'm slightly disappointed, I still like OpenSMTPd's concise configuration 
syntax.  Postfix could still rewrite source addresses last time I checked, I 
hope it's still there - I do NOT want to run sendmail, thank you very much.

-Adam



Smtpd(8) trace output including invocation:
===from here to end of message===
bhs# smtpd -d -T all -v -d
debug: init ssl-tree
debug: init ca-tree
debug: init ssl-tree
debug: using "fs" queue backend
debug: using "ramqueue" scheduler backend
debug: using "ram" stat backend
info: OpenSMTPD 6.4.0 starting
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ca-tree
debug: init ca-tree
debug: init ca-tree
debug: init ca-tree
debug: init ca-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ca-tree
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: init ssl-tree
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "fs" queue backend
debug: using "ram" stat backend
setup_peer: klondike -> control[63932] fd=4
setup_peer: klondike -> pony express[40666] fd=5
setup_done: ca[35575] done
debug: using "ram" stat backend
setup_peer: control -> klondike[35575] fd=4
setup_peer: control -> lookup[49698] fd=5
setup_peer: control -> pony express[40666] fd=6
setup_peer: control -> queue[21502] fd=7
setup_peer: control -> scheduler[14152] fd=8
setup_done: control[63932] done
debug: using "ram" stat backend
setup_peer: lookup -> control[63932] fd=4
setup_peer: lookup -> pony express[40666] fd=5
setup_peer: lookup -> queue[21502] fd=6
setup_done: lka[49698] done
debug: using "ram" stat backend
setup_peer: pony express -> control[63932] fd=4
setup_peer: pony express -> klondike[35575] fd=5
setup_peer: pony express -> lookup[49698] fd=6
setup_peer: pony express -> queue[21502] fd=7
setup_done: pony[40666] done
debug: using "ram" stat backend
setup_peer: queue -> control[63932] fd=4
setup_peer: queue -> pony express[40666] fd=5
setup_peer: queue -> lookup[49698] fd=6
setup_peer: queue -> scheduler[14152] fd=7
setup_done: queue[21502] done
debug: using "ramqueue" scheduler backend
debug: using "ram" stat backend
setup_peer: scheduler -> control[63932] fd=4
setup_peer: scheduler -> queue[21502] fd=5
setup_done: scheduler[14152] done
smtpd: setup done
mproc: parent -> control: enabled
mproc: parent -> lka: enabled
mproc: parent -> queue: enabled
mproc: parent -> ca: enabled
mproc: parent -> pony: enabled
debug: parent_send_config_ruleset: reloading
mproc: parent -> lka : 0 IMSG_CONF_START
mproc: parent -> lka : 0 IMSG_CONF_END
debug: parent_send_config: configuring pony process
mproc: parent -> pony : 0 IMSG_CONF_START
mproc: parent -> pony : 0 IMSG_CONF_END
debug: parent_send_config: configuring ca process
mproc: parent -> ca : 0 IMSG_CONF_START
mproc: parent -> ca : 0 IMSG_CONF_END
setup_proc: klondike done
setup_proc: control done
setup_proc: lookup done
setup_proc: pony express done
setup_proc: queue done
setup_proc: scheduler done
mproc: ca -> control: enabled
debug: bounce warning after 4h
mproc: ca -> parent: enabled
mproc: ca -> pony: enabled
mproc: ca -> pony: disabled
mproc: pony -> parent: enabled
mproc: scheduler -> control: enabled
imsg: ca <- parent: IMSG_CONF_START (len=0)
mproc: lka -> parent: enabled
mproc: pony -> queue: enabled
mproc: scheduler -> queue: enabled
scheduler: getting batch: mask=0x3f, count=10
debug: /--- ramqueue: scheduler_ram_batch()
debug: \---
scheduler: got r=0, delay=-1, count=10
scheduler: sleeping
imsg: 

Re: smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

As it turns out, no, that doesn't work.
Trying to fix up broken sender mail domain-parts only simply gets me a "5.2.4 
Mailing list expansion problem" error, with no debug output to suggest why.

In this test case, my translations map had:

@bad.athompso.net @good.athompso.net

in it.  Obviously, this is a test setup :).
Smtpd.conf itself consisted of:

listen on all received-auth
smtp max-message-size 100M
table translations file:/etc/mail/translations  # ORIG->NEW 
mappings
table allowed-hosts file:/etc/mail/allowed-hosts# Who can 
connect?  (bare IP addresses or CIDR subnets)
action translate lmtp "/var/run/lmtp.sock" virtual
# 1st pass on allowed rewrite mail
action forward forward-only 
# and now it's not our problem anymore
match for any from local action forward # 2nd pass for 
reinjected mail, this time just forward it
match for any from src  action translate # inbound mail 
- hand it to LMTP, translating as we go

A cursory glance at the source code (yikes, it's been a long time since I was a 
programmer) suggests that virtual now only works on recipients, not senders.  
Which is too bad for me, as that means I'll have to switch at least one box to 
use Postfix.

I'm not convinced the new smtpd.conf grammar improves anything at all, but I 
assume it must help someone or it wouldn't have changed... but I believe my use 
case got thrown out with the bathwater, so to speak.  Oh, well.  :-(

(If anyone cares, the bad sender addresses are mostly alerts coming from older 
Sun ALOMs and at least one Lexmark printer that also sends email with broken 
From addresses.)

-Adam

 
-Original Message-
From: owner-m...@openbsd.org  On Behalf Of Adam Thompson
Sent: Wednesday, January 16, 2019 8:26 AM
To: 'Edgar Pettijohn' ; misc@openbsd.org
Subject: Re: smtpd - help needed tranlsating to new virtual map syntax

As I said, I haven't tried anything yet as I don't want to break a working 
system, and I don't have a good way to test this in parallel right now. 

The manpage says "The local delivery methods support additional options: [...] 
virtual" without specifying which delivery methods are "local".  My assumption 
was that only "mbox" and "mda" were local, as lmtp can, and often does, point 
to another server.

Some brief experiments with a VM only got me syntax errors, so I didn't pursue 
that very thoroughly before asking for clarification.

-Adam

-Original Message-
From: Edgar Pettijohn 
Sent: Wednesday, January 16, 2019 8:12 AM
To: Adam Thompson ; misc@openbsd.org
Subject: Re: smtpd - help needed tranlsating to new virtual map syntax

It would be helpful if you show what you have tried.

Should be as simple as:

action "relay-01" lmtp /var/run/lmtp.sock virtual 

match from src  action "relay-01"

Edgar
On Jan 16, 2019 7:37 AM, Adam Thompson  wrote:
>
> [Cross-posting here before I give up and switch to Postfix  -Adam]
>
>
> I have an old instance that uses smtpd's virtual  to rewrite *sender* 
> addresses.
> Reading the 6.4-STABLE version of the smtpd.conf(5) manpage, I can't see how 
> to accomplish my goal any more - it looks impossible.
>
> I don't want to upgrade a working mail relay server to something that might 
> be broken, so I'm seeking assistance first.
>
> The purpose of this system is purely to relay mail from internal, 
> semi-broken-ish systems out to our Office365 tenant, but I need to clean up 
> bogus MAIL FROM / "From:" headers first, lest they be flagged as spam.
>
> In general, I think I'm asking how to use virtual  with the "relay" 
> action in the new syntax - the manual tells me this is impossible!?
>
> Thanks,
> -Adam
>
>
> Old smtpd.conf:
>
> ===start===
> listen on 0.0.0.0
> listen on ::
> table aliases db:/etc/opensmtpd/aliases.db table vmap 
> db:/etc/opensmtpd/vmap.db table localnets { 192.168.10.0/24, 
> 192.168.100.0/24, 192.168.157.0/24, 192.168.158.0/24,
> 192.168.101.0/24, 10.158.0.0/16 } accept from local 
> for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca 
> accept from source 192.168.158.63 for domain 192.168.158.63 virtual 
>  deliver to lmtp localhost:25 accept from source 192.168.100.63 
> for domain 192.168.100.63 virtual  deliver to lmtp localhost:25
> accept from source 192.168.158.63 for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
> remote.XXX.ca accept from source 192.168.100.63 for any
> relay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca 
> hostname remote.XXX.ca accept from source for any  

Re: smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

As I said, I haven't tried anything yet as I don't want to break a working 
system, and I don't have a good way to test this in parallel right now. 

The manpage says "The local delivery methods support additional options: [...] 
virtual" without specifying which delivery methods are "local".  My assumption 
was that only "mbox" and "mda" were local, as lmtp can, and often does, point 
to another server.

Some brief experiments with a VM only got me syntax errors, so I didn't pursue 
that very thoroughly before asking for clarification.

-Adam

-Original Message-
From: Edgar Pettijohn  
Sent: Wednesday, January 16, 2019 8:12 AM
To: Adam Thompson ; misc@openbsd.org
Subject: Re: smtpd - help needed tranlsating to new virtual map syntax

It would be helpful if you show what you have tried.

Should be as simple as:

action "relay-01" lmtp /var/run/lmtp.sock virtual 

match from src  action "relay-01"

Edgar
On Jan 16, 2019 7:37 AM, Adam Thompson  wrote:
>
> [Cross-posting here before I give up and switch to Postfix  -Adam]
>
>
> I have an old instance that uses smtpd's virtual  to rewrite *sender* 
> addresses.
> Reading the 6.4-STABLE version of the smtpd.conf(5) manpage, I can't see how 
> to accomplish my goal any more - it looks impossible.
>
> I don't want to upgrade a working mail relay server to something that might 
> be broken, so I'm seeking assistance first.
>
> The purpose of this system is purely to relay mail from internal, 
> semi-broken-ish systems out to our Office365 tenant, but I need to clean up 
> bogus MAIL FROM / "From:" headers first, lest they be flagged as spam.
>
> In general, I think I'm asking how to use virtual  with the "relay" 
> action in the new syntax - the manual tells me this is impossible!?
>
> Thanks,
> -Adam
>
>
> Old smtpd.conf:
>
> ===start===
> listen on 0.0.0.0
> listen on ::
> table aliases db:/etc/opensmtpd/aliases.db table vmap 
> db:/etc/opensmtpd/vmap.db table localnets { 192.168.10.0/24, 
> 192.168.100.0/24, 192.168.157.0/24, 192.168.158.0/24, 
> 192.168.101.0/24, 10.158.0.0/16 } accept from local 
> for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca 
> accept from source 192.168.158.63 for domain 192.168.158.63 virtual 
>  deliver to lmtp localhost:25 accept from source 192.168.100.63 
> for domain 192.168.100.63 virtual  deliver to lmtp localhost:25 
> accept from source 192.168.158.63 for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
> remote.XXX.ca accept from source 192.168.100.63 for any
> relay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca 
> hostname remote.XXX.ca accept from source for any  
>   
> relay via smtp://XXX-ca.mail.protection.outlook.com hostname 
> remote.XXX.ca ===end===
>
> old vmap:
>
> ===start===
> ilom-alert@192.168.100.63:  sys...@xxx.ca
> sys...@xxx.ca:   sys...@xxx.ca
> sys...@ad.xxx.ca: sys...@xxx.ca
> root@XXX.local: sys...@xxx.ca
> ===end===
>
>

smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

[Cross-posting here before I give up and switch to Postfix  -Adam]


I have an old instance that uses smtpd's virtual  to rewrite *sender* 
addresses.
Reading the 6.4-STABLE version of the smtpd.conf(5) manpage, I can't see how to 
accomplish my goal any more - it looks impossible.

I don't want to upgrade a working mail relay server to something that might be 
broken, so I'm seeking assistance first.

The purpose of this system is purely to relay mail from internal, semi-broken-ish systems 
out to our Office365 tenant, but I need to clean up bogus MAIL FROM / "From:" 
headers first, lest they be flagged as spam.

In general, I think I'm asking how to use virtual  with the "relay" action 
in the new syntax - the manual tells me this is impossible!?

Thanks,
-Adam


Old smtpd.conf:

===start===
listen on 0.0.0.0
listen on ::
table aliases db:/etc/opensmtpd/aliases.db
table vmap db:/etc/opensmtpd/vmap.db
table localnets { 192.168.10.0/24, 192.168.100.0/24, 192.168.157.0/24, 
192.168.158.0/24, 192.168.101.0/24, 10.158.0.0/16 }
accept from local for anyrelay via 
smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca
accept from source 192.168.158.63 for domain 192.168.158.63 virtual  
deliver to lmtp localhost:25
accept from source 192.168.100.63 for domain 192.168.100.63 virtual  
deliver to lmtp localhost:25
accept from source 192.168.158.63 for anyrelay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
remote.XXX.ca
accept from source 192.168.100.63 for anyrelay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
remote.XXX.ca

accept from source for anyrelay via 
smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca
===end===

old vmap:

===start===
ilom-alert@192.168.100.63:  sys...@xxx.ca
sys...@xxx.ca:   sys...@xxx.ca
sys...@ad.xxx.ca: sys...@xxx.ca
root@XXX.local: sys...@xxx.ca
===end===

bgplg doesn't work with wildcard httpd servers

[Adam Thompson]

Running 6.4 (-stable, via openup/mtier).
I have bgpd(8) talking to my border router, acting as a route collector. 
 That part seems fine.
I now have httpd(8) configured trivially to run bgplg(8) (per the 
bgplg(8) manpage) but it's not working, and I can't tell why.  **EDIT: 
yes, I can, see below**


httpd.conf:
===start===
server "*" {
listen on * port 80
location "/cgi-bin/*" {
fastcgi
root ""
}
}
===end===

On the client end, I get:

  bgpmirror# wget -v http://localhost/cgi-bin/bgplg
  --2019-01-11 10:12:05--  http://localhost/cgi-bin/bgplg
  Resolving localhost (localhost)... 127.0.0.1, ::1
  Connecting to localhost (localhost)|127.0.0.1|:80... connected.
  HTTP request sent, awaiting response... 200 No headers, assuming 
HTTP/0.9

  Length: unspecified
  Saving to: 'bgplg'
(it never completes until I kill it)

Ktrace'ing slowcgi and httpd in -d mode reveals that bgplg execve's 
properly, loads, spits out "invalid character in input" and dies.  
Slowcgi and/or httpd do not handle this... well, at all, really.  That 
error message also does not get logged anywhere nor is visible anywhere 
except ktrace logs.


Looking at the bgplg source code, this means there's something funky in 
its environment that it doesn't like.  Ah.  It looks like it's the "*" 
in server_name, as passed in by slowcgi:

  slowcgi: env[18], SERVER_NAME=*

Yup.  That's the problem, all right: /usr/src/usr.bin/bgplg/bgplg.c:115 
excludes '*'.  But I want my looking glass to be accessible from at 
least two different hostnames, and I really would prefer to not have to 
define them all manually in httpd.conf(5).


The naive local fix is trivial (adding '*' to the strchr call in line 
115), but what else might I be breaking or letting in?  Clearly this is 
supposed to ensure the environment is sanitized before continuing, but 
is "*" forbidden because it's unsafe, or simply because it never 
occurred to anyone?


Thoughts / suggestions ?

Thanks,
-Adam

Re: 6.4-release tset(1) really slow, what have I missed?

[Adam Thompson]

On 2018-12-02 22:12, Adam Thompson wrote:

> I'm unsure if my test is valid, but I switched to i8254 (confirmed successful 
> via sysctl), and tset(1) continues to pause for an unnaturally long time.  
> But then I rebooted and re-tested the same sysctl vaules, and this time 
> tset(1) took 1sec, as expected.  WTF... 
> 
> Well, putting that into sysctl.conf seems to be a reasonable workaround for 
> now.  I also enabled timestepwarnings, and they are occurring, although I 
> don't yet know how often.

I understand why I got inconsistent results... I had two different hosts
open in the two windows I was using.  Thank goodness they were both just
personal systems! 

I'll re-test tomorrow morning when I'm more awake! 

-Adam

Re: 6.4-release tset(1) really slow, what have I missed?

[Adam Thompson]

On 2018-12-02 20:50, Philip Guenther wrote:

> On Sun, Dec 2, 2018 at 2:15 PM Adam Thompson  wrote: 
> 
>> I've successfully installed OpenBSD 6.4-RELEASE at OVH, but I'm noticing 
>> one thing there that's different from everywhere else I've used 6.4.
>> 
>> tset(1) takes approximately 12-15 seconds to execute, (almost) every 
>> time.
>> 
>> On a DigitalOcean VPS running 6.3-STABLE (via openup) tset sensibly 
>> takes about 1 or 2 seconds:
>> athom...@mail.athompso.net:~$ time tset -s
>> TERM=xterm;
>> 0m01.01s real 0m00.00s user 0m00.01s system
>> athom...@mail.athompso.net:~$ uname -r
>> 6.3
>> 
>> On the OVH VPS running 6.4-STABLE (via openup), the same command takes 
>> 15 seconds:
>> athom...@mail2.athompso.net:~$ time tset -s
>> TERM=xterm;
>> 0m15.19s real 0m00.00s user 0m00.01s system
>> athom...@mail2.athompso.net:~$ uname -r
>> 6.4
>> 
>> That's from two SSH sessions from the same client with the same 
>> parameters.
>> 
>> I've captured ktrace(1) output, which shows tset(1) doing, well, 
>> nothing:
>> ...
>> 57429/443422  tset 0.035908 CALL  
>> kbind(0x7f7f7678,24,0xecf2201fc1aab9ca)
>> 57429/443422  tset 0.035933 RET   kbind 0
>> 57429/443422  tset 0.035950 CALL  
>> nanosleep(0x7f7f7760,0x7f7f7750)
>> 57429/443422  tset 0.035967 STRU  struct timespec { 1 }
>> 57429/443422  tset 15.809238 STRU  struct timespec { 0 }
>> 57429/443422  tset 15.809272 RET   nanosleep 0
>> 57429/443422  tset 15.809303 CALL  
>> kbind(0x7f7f76c8,24,0xecf2201fc1aab9ca)
>> 57429/443422  tset 15.809380 RET   kbind 0
>> ...
>> 
>> I don't think this is a bug in 6.4, it's clearly environment-specific... 
>> but I have no idea what on earth could be causing it.
> 
> It requested a sleep of 1 second and 15 seconds passed.  That's a kernel 
> timetracking issue, so the output of "sysctl kern.timecounter" would be a 
> good place to start.  Is this is an MP kernel using the CPU TSC, but on a VM 
> where the virtual CPU's TSCs aren't in sync? 
> 
> Philip Guenther

Thanks for the pointer!  I noticed, once, that the system clock was way
off, but I assumed that was one of the boots where I didn't have
networking at startup so ntpd(8) -s failed to operate as expected.  Bad
kernel timekeeping would also produce that result, though. 

Anyway: 

athom...@mail2.athompso.net:~$ sysctl kern.timecounter
kern.timecounter.tick=1
kern.timecounter.timestepwarnings=0
kern.timecounter.hardware=acpitimer
kern.timecounter.choice=i8254(0) acpitimer0(1000) dummy(-100) 

and it's an SP kernel running on one vCPU.  No way of knowing what's
happening under the hood, other than that it's OpenStack Nova, which
IIRC means KVM virtualization.  Note the lack of advertised TSC support.


I'm unsure if my test is valid, but I switched to i8254 (confirmed
successful via sysctl), and tset(1) continues to pause for an
unnaturally long time.  But then I rebooted and re-tested the same
sysctl vaules, and this time tset(1) took 1sec, as expected.  WTF... 

Well, putting that into sysctl.conf seems to be a reasonable workaround
for now.  I also enabled timestepwarnings, and they are occurring,
although I don't yet know how often. 

Is this likely to be a big enough problem that I should ditch this VPS
platform for now? 

dmesg output, to verify SP kernel with no TSC: 

OpenBSD 6.4 (GENERIC) #1: Mon Nov 26 09:32:17 CET 2018
r...@syspatch-64-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 4177379328 (3983MB)
avail mem = 4041621504 (3854MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf6890 (10 entries)
bios0: vendor SeaBIOS version "2:1.10.2-58953eb7" date 04/01/2014
bios0: OpenStack Foundation OpenStack Nova
acpi0 at bios0: rev 0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP SSDT APIC
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel Core Processor (Haswell, no TSX, IBRS), 2394.81 MHz,
06-3c-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,VMX,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,ARAT,XSAVEOPT,MELTDOWN
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache
cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C1(@1 halt!)
...etc... 

Thanks, 

-Adam

6.4-release tset(1) really slow, what have I missed?

[Adam Thompson]

I've successfully installed OpenBSD 6.4-RELEASE at OVH, but I'm noticing 
one thing there that's different from everywhere else I've used 6.4.


tset(1) takes approximately 12-15 seconds to execute, (almost) every 
time.


On a DigitalOcean VPS running 6.3-STABLE (via openup) tset sensibly 
takes about 1 or 2 seconds:

  athom...@mail.athompso.net:~$ time tset -s
  TERM=xterm;
  0m01.01s real 0m00.00s user 0m00.01s system
  athom...@mail.athompso.net:~$ uname -r
  6.3

On the OVH VPS running 6.4-STABLE (via openup), the same command takes 
15 seconds:

  athom...@mail2.athompso.net:~$ time tset -s
  TERM=xterm;
  0m15.19s real 0m00.00s user 0m00.01s system
  athom...@mail2.athompso.net:~$ uname -r
  6.4


That's from two SSH sessions from the same client with the same 
parameters.


I've captured ktrace(1) output, which shows tset(1) doing, well, 
nothing:

...
 57429/443422  tset 0.035908 CALL  
kbind(0x7f7f7678,24,0xecf2201fc1aab9ca)

 57429/443422  tset 0.035933 RET   kbind 0
 57429/443422  tset 0.035950 CALL  
nanosleep(0x7f7f7760,0x7f7f7750)

 57429/443422  tset 0.035967 STRU  struct timespec { 1 }
 57429/443422  tset 15.809238 STRU  struct timespec { 0 }
 57429/443422  tset 15.809272 RET   nanosleep 0
 57429/443422  tset 15.809303 CALL  
kbind(0x7f7f76c8,24,0xecf2201fc1aab9ca)

 57429/443422  tset 15.809380 RET   kbind 0
...

I don't think this is a bug in 6.4, it's clearly environment-specific... 
but I have no idea what on earth could be causing it.


(dmesg, etc. omitted in this first message, since it's so ridiculously 
specific.)


Oh, and to make it even weirder, it doesn't ALWAYS happen.  It ran 
quickly twice earlier today, but never again.


Normally I'd say "it's DNS", and I thought it was due to the slow login 
times, but ktrace(1) says otherwise.


Any ideas?

Thanks,
-Adam

Qsynth midi latency not low enough... what to do?

[Adam Thompson]

PROBLEM STATEMENT: driving FluidSynth from a MIDI controller produces ~1/4sec 
delay between keypress and sound.

NARRATIVE:

I finally got Qsynth working under Xfce (it freezes X under twm!) so I can 
control fluidsynth in a reasonably-obvious way... but I am now experiencing 
substantial latency.
The good news is that it feels just like playing an old pneumatic or tracker 
organ, where there’s a ~0.25sec delay between keypress and sound.
The bad news is that it feels just like playing an old pneumatic or tracker 
organ...

I’m not a good enough musician to handle the roughly quarter-second delay when 
playing live.  I know from many musicians that near-zero-latency from MIDI 
softsynths (even when using soundfonts) is possible... although no-one I know 
of uses OpenBSD.

Is sndio(4) suitable for real-time(-ish) performance?  Or do I need a (OS) 
platform that does ASIO or JACK?  (I mostly play by ear so I'm targeting 
<<0.1sec latency.)

If sndio core or umidi(4) support isn’t the problem, the only obvious thing to 
blame is FluidSynth... but the CPU in this laptop should be more than up to the 
task, and – again – I know this particular piece of software handles 
low-latency live performance in other configurations (i.e. on Linux, using 
JACK).
I don't suspect Qsynth at the moment, as it only controls how fluidsynth 
launches, it doesn't put itself in the data path.

Unsurprisingly, I’ve been unable to find any useful information on running this 
kind of setup under OpenBSD.  But as mentioned before, I’m trying to avoid the 
(to me) insanity that is JACK.

Dmesg follows, just in case anyone spots anything useful in there…

Hardware setup, broadly:
* Dell Latitude E6430 laptop
  - booting in EFI mode to work around a weird bootloader bug
* onboard azalia(4) audio (for now) using onboard speakers (for now)
* Roland A500PRO MIDI controller, connected via USB
* M-audio Uno USB-MIDI, nothing connected to it yet
* No-name USB 5.1ch Audio DAC from Amazon, nothing connected to it yet
  - (leaving the M-audio umidi and the "ABC" uaudio devices disconnected makes 
no difference)


Advice / pointers gratefully accepted, including pointers to documentation or 
threads I may have missed.

Thanks,
-Adam


Dmesg << __EOF__ (so to speak)
 OpenBSD 6.4 (GENERIC.MP) #1: Mon Nov 26 10:18:14 CET 2018

r...@syspatch-64-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8471482368 (8079MB)
avail mem = 8205459456 (7825MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xebf40 (101 entries)
bios0: vendor Dell Inc. version "A21" date 05/08/2017
bios0: Dell Inc. Latitude E6430
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG SSDT HPET SSDT SSDT SSDT ASF! SLIC BGRT 
SSDT
acpi0: wakeup devices P0P1(S4) USB1(S3) USB2(S3) USB3(S3) USB5(S3) USB6(S3) 
USB7(S3) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP05(S4) PXSX(S4) 
RP06(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2193.29 MHz, 06-3a-09
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2192.89 MHz, 06-3a-09
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2192.89 MHz, 06-3a-09
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2192.89 MHz, 

recommended h/w for fanless audio-out?

[Adam Thompson]

Hello,

I’d like to use OpenBSD to build a MIDI synthesizer using SoundFonts, as the 
OpenBSD MIDI and audio subsystems are remarkably understandable and sane, 
compared to everything else out there today.

 �

However, I’m having difficulty finding a combination of hardware that is known 
to be supported.  (I tried this a few years ago, but was unhappy with the 
results due to the quality of the audio-out ports on the laptop I used.)

 �

I’ve heard a fair bit here about USB audio not working very well, or at all, in 
-stable right now.  I’m unsure if this only applies to XHCI ports or not?

 �

Therefore, ideally, I’d like a fanless amd64 architecture system, that boots 
from SATA (not eMMC) with either onboard stereo++ audio (up to and including 
7.1ch) *or* a PCI slot for a sound card.

A laptop with really good audio-out would work, too, but I don’t know of any.

If connecting USB audio to a RPi or BeagleBone Black works well, then I could 
try that, too.

 �

I intend to use USB MIDI interfaces, as they appear to be supported and working 
fine (IIRC).

 �

The problem is that, as I browse the list of embedded, fanless systems I can 
get from DigiKey, Mouser, et al., all the boards with audio have HD Audio 
Codecs that I cannot confirm work with OpenBSD, and I don’t know enough about 
how azalia(4) [et a.] works with random unsupported DACs to drop $250-$500 on 
something that may or may not work (i.e. the codec isn’t explicitly supported).

 �

Can anyone make any recommendations for specific hardware that’s fanless and 
has working audio-out ports?

 �

And/or if I’ve gone off completely in the wrong direction about USB audio, 
please tell me so.  (Working multi-channel USB audio would made this a lot 
simpler/easier/cheaper.)

 �

Thanks in advance,

-Adam

 �

Re: Best way to serve files to Windows?

[Adam Thompson]

On 2018-07-18 09:35, Tom Smyth wrote:

Hi John,
You would need microsoft services for unix (SFU) for NFS connectivity


FYI - so no-one goes haring off in the wrong direction.

SFU is the server-side component, equivalent to running nfsd(8).

On the client side, only certain editions of Windows can speak NFS:
- Windows 10 *Enterprise* can mount remote NFS shares.
- Windows 7 *Ultimate* can mount remote NFS shares.
(No idea about Win8, sorry.)

Win10Ent, at least, has flexible authentication options, but IIRC 
defaults to uid=0/gid=0 (gee, thanks).  It prefers to use Kerberos 
security, which won't work with OpenBSD's NFS server.  It's possible to 
make this work reasonably well, but it takes a fair bit of time.


So, as everyone else said, you're better off running Samba on your 
OpenBSD system.  Have fun.

-Adam

Re: supported Audio card with SPDIF input

[Adam Thompson]

On 2018-07-24 17:54, Diana Eichert wrote:

ok, answered my own question by grep'ng within /usr/share/man/man4,
looks like azalia(4) systems.  Was hoping for something usb attached
but no such luck.

On Tue, 24 Jul 2018, Diana Eichert wrote:

I'm trying to connect to an audio system that only has SPDIF output.
I looked at man pages but nothing obvious regarding supported audio
devices with SPDIF input support.

Anyone have recommendations?  Or is it supported?


Very broadly speaking, as long as the USB device conforms to the USB 
Audio Class spec, it doesn't matter whether it's got S/PDIF or Coax or 
2xRCA or XLR - audio waveforms should still get transferred from PC to 
output.
What I think you might lose is any sort of decent mixer control, 
although with S/PDIF I expect you might not really care about that?


Caveat: I have not personally tried a USB-to-S/PDIF audio device, but I 
*have* read the specs and the datasheets.  In theory, theory is the same 
as reality...


Best-case, a USB SPDIF output device would a) conform to the USB Audio 
Device Class interface, and b) report that it has no adjustable mixers, 
so that on the software side, the mixer doesn't get confused.


-Adam

Re: Viewport for man.openbsd.org -- readability on phones

[Adam Thompson]

On 2018-05-19 02:59, justina colmena wrote:

https://man.openbsd.org/mandoc.css
That's the css. You style it how you like it. That's the whole point
of it. And I agree. It's very readable on my phone.
 Original message From: Mihai Popescu
 Date: 5/18/18  11:04 PM  (GMT-09:00) To:
misc@openbsd.org Subject: Re: Viewport for man.openbsd.org --
readability on phones

I don't understand what you are trying to say.


I took and iPhone with iOS and Safari ( i think!) on it and pointed
the browser to the current link of man pages [1]. All i can say is the
layout is displayed on full display, not stretched.
Text is fine, paragraphs are scaled ok, not even a simple problem. Font 
is fine.


[1] https://man.openbsd.org/



Just to confirm, it also works and looks fine on my phone (is that 
"works phine"? ) which is running Chrome under Android 8.1.


I can see why it would not look fine on an extremely low-res display, 
though: the margins could overwhelm the page, and the resultant wrapping 
would be very not-pretty.


OTOH, *every* website will look bad on a phone that old.

-Adam

Re: Virtualbox vs latest snapshot

[Adam Thompson]

On 2018-04-12 20:02, Nick Holland wrote:

On 04/12/18 09:47, Consus wrote:

On 08:28 Thu 12 Apr, Nick Holland wrote:

Another "failure mode" of VirtualBox people should be aware of:
I understand through good sources, Oracle monitors the IP addresses 
that
it's downloaded from, and if they can trace it back to a commercial 
IP
(i.e., not a home address), and if they see you download (or update) 
the

"not for unrestricted free use" parts, their lawyers will contact you
and send you a bill...and they really don't care about "for work" or
"not for work related" uses.

I'd really recommend removing this product from your computers.


This won't stand in court. You sources are so high on crack it's not
even funny.


Think about it a moment,
Using my real name, and a public, trackable identity, I just accused a
very big company with lots of lawyers (and they know how to use them!)
of something.  If my facts are not in order, I could be in big trouble.
My facts are in order.

It's not about court.  It's about threatening lots of companies and
hoping a few pay up to avoid the cost of going to court -- which is
considerable, win or lose.

What you believe changes nothing.  Their licenses are complicated, easy
to use wrong, and they seem to care.  I recommend against using their
products for that reason.

Nick.



My tale of Oracle woe:

My company got spanked by Oracle a couple of years ago because one of 
our developers was downloading multiple versions of the RDBMS, trying to 
find a version that would be happy with a binary database file we got 
from a client.
Their License Enforcement dept. (it's part of the Sales division, which 
tells you something...) undertook a sneaky campaign of phoning all our 
staff asking them how they were satisfied with their Oracle products.
They finally audited us, and - not for the products that dev was 
downloading, but for something else entirely - it turned out we were 
accidentally in violation of one of our licenses, as one sentence had 
changed somewhere in the ~20yrs we'd been using it to invalidate our use 
case.
Since it was not intentional, and we were cooperating with them, and we 
are very small (~10 persons, at the time) they ONLY required a top-up 
payment roughly equal to 1 developer's annual salary.
The threat of being sued out of existence, if we didn't cooperate, was 
made explicitly.  We were, in my opinion, deliberately maneuvered into a 
non-compliant position over a period of many years, then subject to a 
"sting" operation, and finally bullied into compliance.


And we also use VirtualBox :-(.  For now.

It's notable that at least in Canada, BSA (MS,Adobe,Sybase,etc.,etc.) 
audits are legal - and are often accompanied by provincial Sheriffs.  
(Not the same thing as a U.S Sheriff, but still a law enforcement 
officer.)  The threat of legal action is NOT just an empty threat.


-Adam

Re: Dell Latitude E6540 OpenBSD 6.2 amd64 freezes when adjusting refresh rate using xrandr

[Adam Thompson]

On 2018-03-20 15:18, Xianwen Chen wrote:

Dear Mihai,
Although your tone in your email was not pleasant,


You are posting to OpenBSD-misc.  Objectionable tone is very common, 
particularly for users who *appear* to be complaining about 
immeasurably-small problems that aren't actually significant in the real 
world.  If you wish to remain an OpenBSD user, get used to people being 
rude.  (I am not going to speculate whether this is good or bad, but it 
is the case.)



If you are right that humans are not able to see the difference
between 60 Hz and 59.95 Hz, then something is wrong with xrandr that
the actual refresh rate is quite below 60, not as much as 59.95 as
reported by xrandr, because I can clearly see the flickering. I do not
think that this is a minimal thing, because the flickering screen
makes my head dizzy.


Then my suggestion would be to replace the lights in your room, not try 
to fix a 0.05Hz deviation.  The vast majority of systems I own report 
that the LCDs actually run at 59.95Hz; I've only seen one or two that 
ever reported 60Hz.  This is normal.


In a worst-case scenario, room lighting that is at a very slightly 
different frequency can cause odd effects, this is known as 
interference, and can produce a "beat frequency".  If the difference is 
small it's possible you could experience this as flickering.  (I believe 
the flickering is actually an neuro-optical illusion, but you might 
still be experiencing it.)



I do not think that there is problem with the connection cable, as
there was no such problem when the same external monitor is connected
to a ThinkPad R52 using the same VGA cable a couple of days ago. I can
check the cable connection again tomorrow.


That is irrelevant.  You have measured one VGA driver against a 
completely different VGA driver.  Different laptops = different 
electronics = different results.



I am sceptical whether there is any other source of distortion. I
don't know where to start if there should be distortion.


Fluorescent lights, or cheap LED lights - anything that needs a ballast 
or rectifier.  Any of these things can cause not only the beat frequency 
optical interference mentioned above, but ALSO can cause electromagnetic 
interference in the cable.  Oh, and cheap USB chargers are another 
common source of really bad EM interference.


And guess what's almost immune to this type of EM interference?  HDMI, 
DVI-D or DP cables.  Guess what's REALLY susceptible? VGA cables.  Hmm.



The exact frequency of a monitor, regardless of type, is almost always 
irrelevant to human eyes.  Whether it's 30Hz, 47.8Hz, 59.95Hz or 60.0Hz


As you are apparently experiencing real problems, I would check, in 
order:
1. your cable - switch to HDMI (or HDMI->DVI or HDMI->DP) and get rid of 
VGA **immediately**.
2. your lights - try it with all the lights EXCEPT regular incandescent 
/ halogen lights turned off.
3. your eyes - get a thorough examination by a doctor; there are some 
rare conditions that could cause odd things like this.
4. your brain - make sure you don't have a brain tumour (yes, I'm 
serious, it can cause things like this!)



As to the VGA cable - this is such an important point that I agree with 
Nick - please go away and don't complain further until you have switched 
to a digital connection of some sort.  I recall that my Dell E6430 was 
quite capable of producing so-called "120Hz" digital signals, and yours 
is a generation newer than that.  I am 99% certain that your problem 
will go away with a different cable.  (If you want DVI + DP connectors 
instead of HDMI, buy a Dell E-series dock on eBay for $50.)


-Adam

Re: OpenBSD SPARC T4-1 softraid boot issues

[Adam Thompson]

I'm afraid I don't know the answer to that, sorry.  Not sure if anyone does... 
I think the SCSI HBA driver is the same on SPARC64 as on AMD64, but then Sun 
had a habit of "improving" control interfaces for Solaris so who knows.  I do 
know under Solaris there is a bioctl-like tool to manage it, for what that's 
worth.
-Adam

On December 28, 2017 1:32:40 PM CST, Jordan Geoghegan <jgeoghega...@gmail.com> 
wrote:
>Yes I had considered using the onboard hardware raid, but I don't 
>particularly trust it. I also need the ability to rebuild my arrays 
>while the machine is online and was hoping to be able to do the 3 disk 
>RAID1 offered by OpenBSD softraid. Do you know if bioctl(8) is capable 
>of controlling the onboard raid controller, or will I need to do all 
>raid rebuilds via the hardware raid bios on the T4?
>
>
>On 12/28/17 08:58, Adam Thompson wrote:
>> On 2017-12-26 14:56, Jordan wrote:
>>> I've recently gotten my hands on a couple shiny new SPARC T4-1 and
>>> T3-1 servers and I was looking to install OpenBSD with a softraid
>>> mirror on them for production use. The problem is, is that I end up
>>> with this upon following the install instructions and rebooting:
>>
>> FWIW...
>> AFAIK every single T4-1 (probably T3-1s but not certain) is capable
>of 
>> hardware RAID-0/1/10 on the SCSI controller, and that's how they're 
>> shipped from Sun/Oracle even when running Solaris. I'm not absolutely
>
>> certain that it's supported by OpenBSD, but I do recall seeing that
>it 
>> could be used as-is when I was investigating running OpenBSD on my 
>> T-series gear.  There's Oracle documentation on how to access the 
>> on-card firmware to set up and manage the RAID set in a pre-OS
>situation.
>>
>> YMMV.  Unfortunately, both of mine are still running Solaris so I 
>> can't confirm right now.
>> -Adam

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: OpenBSD SPARC T4-1 softraid boot issues

[Adam Thompson]

On 2017-12-26 14:56, Jordan wrote:

I've recently gotten my hands on a couple shiny new SPARC T4-1 and
T3-1 servers and I was looking to install OpenBSD with a softraid
mirror on them for production use. The problem is, is that I end up
with this upon following the install instructions and rebooting:


FWIW...
AFAIK every single T4-1 (probably T3-1s but not certain) is capable of 
hardware RAID-0/1/10 on the SCSI controller, and that's how they're 
shipped from Sun/Oracle even when running Solaris.  I'm not absolutely 
certain that it's supported by OpenBSD, but I do recall seeing that it 
could be used as-is when I was investigating running OpenBSD on my 
T-series gear.  There's Oracle documentation on how to access the 
on-card firmware to set up and manage the RAID set in a pre-OS 
situation.


YMMV.  Unfortunately, both of mine are still running Solaris so I can't 
confirm right now.

-Adam

Re: Doubts about the successors of OpenBSD leadership and development

[Adam Thompson]

> From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On Behalf Of 
> Stefan Sperling
> Sent: July 10, 2017 16:17
> Subject: Re: Doubts about the successors of OpenBSD leadership and development
>
> Obviously, Theo de Raadt will succeed Theo de Raadt in the leadership
> and development of OpenBSD: 
> http://marc.info/?l=openbsd-misc=137609553004700=2

I had forgotten about that old message :-).  Thank you for reminding us!

FYI, Anatolian Shepherd dogs (the Turkish national dog, IIRC) are extremely 
beautiful dogs (IMHO), and very nice, gentle and friendly, as a rule, unless 
they think you're threatening whatever they're protecting.  In which case 
there's no need for an entire group of them to kill Theo, one dog will likely 
be more than sufficient, regardless of how fit Theo may or may not be at that 
point in time.

Since we must, of course, take all statements from Theo literally (otherwise 
what kind of a cult are we?!?!), this does beg the question of what happens if 
he gets attacked by a group of dogs somewhere *other* than central Turkey.

If the project were based out of the US, the solution would be obvious: have 
Theo go armed at all times.  But since it's Canadian... hmm.  If we say he can 
only leave the country protected by one of our submarines, he'll - effectively 
- be protected, since none of them can actually go anywhere most of the time.



-Adam

OpenBSD 6.1 is now supported on Microsoft Azure

[Adam Thompson]

Maybe I missed the email here, but in case it actually doesn't exist:

OpenBSD 6.1 is now supported on Microsoft Azure courtesy of reyk@ and 
the team over at Esdenera® Networks, with assistance from Microsoft.  At 
least that's what I got out of the BSDCan announcement.


I'll let Reyk blow his own horn on the products & marketing.  But I 
think it's great that MS sanctions running OpenBSD in Azure now.  For 
me, this is a significant milestone.


Thanks, Reyk (et al.)

https://azuremarketplace.microsoft.com/en-us/marketplace/apps/esdenera.esdenera-firewall-3
The link to the Sponsor BOF doesn't have any info so I'm not going 
providing it here.
https://azure.microsoft.com/en-au/blog/running-openbsd-on-azure/ 
(apparently this page is about to get updated Real Soon Now)


-Adam

inet6 nat-to (group) address selection algorithm question

[Adam Thompson]

So I’ve discovered that, when trying to do NAT66 (for a ULA network), a line 
like:
 "match out on egress inet6 from !(egress:network) to any nat-to (egress:0)"
doesn’t work.  (Yes, the network in this case is ridiculously simple.)

I believe it doesn’t work because :0 indicates that aliases on the if should be 
ignored, and the first IPv6 address on an if is typically going to be the LLA… 
which of course doesn’t work very well for the NAT use case!

However, to my surprise, simply using "nat-to (egress)" does work, instead.

What I’d like to know is, what magic in pf(4) allows it to auto-select a 
"better" (in my case, at least) address when performing the translation?  
There’s nothing in the man page that I can see talking about this.  The man 
page talks about what happens when I *do* specify :0, but not when I *don’t*.  
(And neither does the FAQ, fyi.)

My concern is that the algorithm (or blind luck, depending) that allows "nat-to 
(egress)" to work for me will stop working in some slightly different scenario 
or at some random point in the future when something seemingly-unrelated 
changes.

Thoughts/hints/explanations?  (Other than "read the source", please - there's a 
reason I'm not a kernel programmer!)

Thanks,
-Adam

Re: Does pf support NPT (RFC6296) ?

[Adam Thompson]

> > I know I can do NAT66, but I don't think it's feasible to emulate NPT
> > using NAT66 rules.
> 
> No, NPT is different and can't be emulated by anything that OpenBSD's
> PF currently does.

Shoot.  I was really hoping pfSense managed it through some feature that 
predated FreeBSD's pf(4) import, but that I had merely overlooked.  That sucks, 
right now.
 
> The closest it can get is NAT with bitmask and "static-port", but
> 1) that's stateful, and 2) it doesn't do the "checksum neutral"
> modification that NPT uses (NPT doesn't replace just the network prefix;
> it also adjusts the host part of the address in a complementary manner
> so that the IPv6 checksum doesn't change).

Ah, thank you for that explanation - I wasn't clear on what the manipulations 
were supposed to accomplish.

In my unfortunate scenario, NAT66 would probably work just as well, assuming my 
intuition about how IPv4 NAT/SNAT/PNAT works in pf(4) extends to the IPv6 
world.  An HTTP proxy would also work, I suppose, but would require more 
configuration on the inner hosts.

All I need is a way to give ULA-addressed hosts a way *out* to reach, e.g. DNS, 
NTP, mirrors, probably various CDNs - all the maintenance traffic a modern 
(non-OpenBSD) host generates by itself.  As I write this, I'm starting to 
wonder if NAT66 isn't the better solution anyway since it's (kind-of) 
inherently unidirectional.

Oh, and in case anyone's wondering - this is all because a) VMware NSX 6.0 
supports IPv6, but neglects to include any form of NAT or NPT or outbound 
proxy; and b) OVH, even in their private cloud offering (which is where the 
VMware NSX 6.0 comes in!), will not route public IP address space to a VLAN 
behind my firewall... which works for IPv4 ("just use NAT!"), but not so well 
for IPv6.  And I need IPv6 on the protected hosts.  *sigh*  If anyone reading 
this thinks they can see a better way around this pair of problems, please let 
me know.

-Adam 

Does pf support NPT (RFC6296) ?

[Adam Thompson]

I still haven't found this answer anywhere...

Does OpenBSD (more specifically, pf(4), I guess) support RFC 6296, 
IPv6-to-IPv6 Network Prefix Translation?  Looks like FreeBSD can do it, 
but I can't tell if that's something they added to their own pf fork, or 
if I'm just missing something in the OpenBSD docs.


I know I can do NAT66, but I don't think it's feasible to emulate NPT 
using NAT66 rules.


Thanks,
-Adam

openup vs. syspatch vs. pkg_add -u ?

[Adam Thompson]

Now that we’re in the wonderful world of syspatch(8) – which works well for me 
so far (thanks for the hard work, everyone!), I’m trying to figure out if 
there’s still any point to using m:tier’s openup tool.

>From what I can tell, running “syspatch ; pkg_add -u” is pretty much 
>equivalent to what openup does… but am I missing anything?

 

Thanks,

-Adam

 

Re: acme-client(1) and http_proxy

[Adam Thompson]

On 2017-04-25 05:27, Stuart Henderson wrote:

On 2017-04-25, Adam Thompson <athom...@athompso.net> wrote:

By definition, you will (probably) not be able to use the ACME
protocol - it only works (normally) when your system is connected
directly to the public internet with a static IP address.

Simply because you say "behind a corporate firewall", I already know
(or at least assume) that ACME will not work for you, ever.

ACME, and LetsEncrypt, only handles public websites. There are ways
around this, but they are painful and likely not worthwhile - it
*will* be cheaper to just buy a regular SSL certificate than to get a
LetsEncrypt certificate for an internal server.


Fake news :)


Ha!  That made me laugh!
I was deliberately omitting all the details of the other challenge 
protocols, because (see below).  But yes, I deliberately sacrificed 
correctness for utility in my response.



Firstly, with dns-01 challenge you can get a certificate for a server
which doesn't allow external access at all (the request and challenge
can be done with completely separate machines than the certificate
is for).

Secondly, some environments permit inbound connections but require
a proxy for outbound access from a DMZ. In a hosting environment,
restricting outbound access is often more important than inbound.


While it's possible that this was the case, the fact the OP was even 
asking the question in the first place strongly suggests that this is 
not his situation.


I stand by my statement that just buying a cheap SSL cert will, for 
anything other than the simple case of an online, directly-connected, 
webserver, be cheaper than the labour required to obtain a LetsEncrypt 
certificate.


From what I've read so far, you'd have to be *really* committed to 
LetsEncrypt to go to the bother of using any of the alternate challenge 
protocols.  In all the situations where one person could complete the 
process themselves, that person is highly likely to simply be directly 
connected anyway - so why bother?


Once the entire CA industry moves towards ACME (if that happens) then I 
can see a number of situations where those alternate challenge protocols 
will be useful and/or required, but for a LetsEncrypt certificate?  It 
just doesn't seem worthwhile.  Especially when the cost of a 
single-hostname SSL cert (which meets the needs of many users) is now 
somewhere below US$5/year!


And neither of us addressed the fact that for a server that's "behind a 
corporate firewall", there's a good chance that it's not even using a 
legit gTLD/ccTLD, which means getting an external domain-validated SSL 
cert for it will be (or should be!) impossible in the first place.


-Adam

Re: acme-client(1) and http_proxy

[Adam Thompson]

By definition, you will (probably) not be able to use the ACME protocol - it 
only works (normally) when your system is connected directly to the public 
internet with a static IP address.

Simply because you say "behind a corporate firewall", I already know (or at 
least assume) that ACME will not work for you, ever.

ACME, and LetsEncrypt, only handles public websites.  There are ways around 
this, but they are painful and likely not worthwhile - it *will* be cheaper to 
just buy a regular SSL certificate than to get a LetsEncrypt certificate for an 
internal server.

-Adam

> -Original Message-
> From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On
> Behalf Of Manuel Giraud
> Sent: April 21, 2017 07:37
> To: misc@openbsd.org
> Subject: acme-client(1) and http_proxy
> 
> Hi,
> 
> I'm trying to use the new acme-client on a server behind a corporate
> proxy (i.e. I have to set a http_proxy to get out). It seems (from reading
> the code) that acme-client(1) does not honor http_proxy.
> 
> Is this on purpose? If so, can someone point me to another acme client
> that does this?
> --
> Manuel Giraud

Re: Topics for revised PF and networking tutorial

[Adam Thompson]

> -Original Message-
> From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On
> Behalf Of bytevolc...@safe-mail.net
> Sent: April 10, 2017 19:31
>
> > Plus, this year it appears that Peter is co-delivering the seminar
> > with Massimiliano Stucchi from RIPE, so it will presumably cover a lot
> > of IPv6 topics as well, which are poorly represented in existing
> > materials and yet increasingly relevant.
>
> And for those of us who cannot attend, hopefully it will be on video.

I can't say with 100% certainty, but it's unlikely.  The tutorials are not
typically recorded.  (Among other things, AFAIK the people who do the
recording are only present for the conference itself.)  There's also the
matter of the tutorials not  necessarily being covered by the same broadcast
license (hmm, I wonder if Henning will consent this year?).  I don't have
anything to do with any of those parts of the conference, so I can't speak to
the details.

The slides and material are sometimes - not always - made available afterward,
and that depends on the individual presenters.  Max is working for RIPE -
which makes large amounts of their material available for free - and Peter
historically makes his material available online for free, so I therefore have
at least moderate hopes that they'll be able to find a way to sort out the
copyright issues and get the slides put up somewhere.

-Adam

Re: Topics for revised PF and networking tutorial

[Adam Thompson]

On 2017-04-07 16:41, Mihai Popescu wrote:

I don;t want to offend you folks, but I'm curious and I will ask: is
this BSDCon so useful? Does it pay the efforts?

If someone has time and knowledge to do a PF tutorial he/she can do it
and post. Do you need the Con?

I'm asking this having in my mind Google Summer of (no)Code thread from 
misc@.

Again, i'm asking, i've never been to a Con to sense the feeling.

Thanks.


I'll take a stab at this...

* BSDCan (not Con) is cheap.  Stupidly cheap, in fact: $195/person if 
you're paying your own way.
* The PF tutorial is not free - there is an additional cost ($75) to 
attend the tutorial.
* Peter clearly has the time and knowledge to do it, he has huge amounts 
of raw material on his website, including what amounts to last year's 
tutorial slides, for free - but also chooses to deliver this tutorial.  
Based on the fee, the number of attendees, and the number of presenters, 
no-one's getting rich off this.
* The tutorial is a focused, half-day session where you get to interact 
with the top PF trainer in the world, and ask the questions specific to 
your network.
* Peter keeps his tutorial up to date, unlike most if not all of the 
resources you'll find online, some of which predate the change in syntax 
from several years ago.


You've asked almost the same question as "why does anyone need 
tutorials? just read the man pages!" just at the next level up.  The 
answer is because the man pages aren't adequate to cover every scenario, 
and not everyone can read man pages effectively.  People have different 
learning styles, if nothing else.  I learn best by seeing examples and 
asking questions.  (In fact, the lack of good examples is a pet peeve of 
mine with the OpenBSD man pages, but that's another story.)


I've attended Peter's seminar two?, maybe three? times now, and got 
something new out of it each time - some nuance that wasn't obvious just 
from reading pf.conf(5).  Sometimes it was something Peter said, 
sometimes it was something another attendee said.  That's the value of 
attending any training class or seminar, not just this one for PF.


The tutorial is aimed not at people who would go and produce another 
tutorial, but at ordinary system administrators who don't have time to 
pore over the entire manpage, who want the most relevant information to 
them distilled and delivered efficiently.


Plus, this year it appears that Peter is co-delivering the seminar with 
Massimiliano Stucchi from RIPE, so it will presumably cover a lot of 
IPv6 topics as well, which are poorly represented in existing materials 
and yet increasingly relevant.


Disclaimer: I now help organize (one small) part of BSDCan & PgCon, so 
I'm not *entirely* unbiased, but this is pretty much what I would have 
said the first two years I attended, anyway.


-Adam

Re: Can't install -current on a Dell precision t3500

[Adam Thompson]

On 2017-02-13 07:11, STeve Andre' wrote:

I'm puzzled and am asking for help.  I'm attempting to install
the -current snapshot (feb 12) on a Dell precision t3500.  The
install formats a 6T disk very quickly, like in 25 seconds.  Hmm.

   After installing the tar files, installboot fails with a
"Bad magic number in superblock".  If I mount the a partition I
see real data.  Changing to a 160G disk everything works & boots,
but not with the 6T disk.

   The t3500 is a sata 2 machine, as is the 160G disk. The 6T disk
is sata 3, but since I see the OS written to the 6T disk it's been
written out OK so thats not it.  I'm missing something with regards
the size of the disk?   Probably I'm forgetting to include something
relevant but I've been dealing with this last night and am tired.
Clues?


You're missing the fact that a T3500 cannot boot off a disk larger than 
2TB, because it supports legacy BIOS boot only, not UEFI.  It is 
possible you might occasionally get lucky but AFAIK the disk geometry 
messes up the BIOS bootloader pretty much 100% of the time.


The Precision line didn't gain UEFI (i.e. >2TB boot disk support) until 
the Tx6xx series, AFAIK.
If your T3500 has a UEFI boot option, then I'm wrong, and you should be 
able to solve your problem by switching to UEFI.


In theory, creating a 2TB partition would be sufficient, but in practice 
it usually isn't.


My solution to these problems is to leave a small USB stick permanently 
installed and boot off that; the easiest way to manage that in OpenBSD 
that I know of is to mount it at /boot and replace /bsd with a symlink 
to /boot/bsd (after moving /bsd, of course).  This will require some 
manual work before rebooting off the ramdisk, naturally, and works IMHO 
best if you use diskids instead of device paths.


-Adam

Re: security(8) question - how to skip a single file?

[Adam Thompson]

Thanks, Vijay. That’s exactly what I couldn’t find in the documentation.
(Now that I know what to look for, I see the line in security(8)’s manpage
that I overlooked.)

CC’ing list to help the next person with this question…

-Aadm



From: Vijay Sankar [mailto:vsan...@foretell.ca]
Sent: October 6, 2016 10:20
To: Adam Thompson <athom...@athompso.net>
Subject: Re: security(8) question - how to skip a single file?



Hi Adam,

Not replying to list in case I did not understand the question.

I have the following towards the end of /etc/changelist

.
.
.
/var/nsd/etc/nsd.conf
# /var/unbound/etc/root.key
/var/unbound/etc/unbound.conf
/var/yp/Makefile.main
/var/yp/Makefile.yp

Is that what you are looking for?

Vijay

Quoting Adam Thompson <athom...@athompso.net <mailto:athom...@athompso.net>
>:

I have RTFMed and googled, but I still can’t figure out how to do one
simple
thing: make security(8) ignore a single file that changes on a daily basis,
where that file is otherwise monitored due to /etc/mtree/4.4BSD.dist.



The file in question is /var/unbound/db/root.key, which I have auto-updating.



Yes, I understand why this file is important, but on this particular system,
being lulled into complacency by a daily false-positive security(8) report is
more of a danger than someone managing to hack the root DNS key.



Suggestions or pointers or interpretation of the docs appreciated.



Thanks,
-Adam



Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
vsan...@foretell.ca
<https://server3.foretell.ca/post/imp/dynamic.php?page=mailbox>

security(8) question - how to skip a single file?

[Adam Thompson]

I have RTFMed and googled, but I still can’t figure out how to do one simple
thing: make security(8) ignore a single file that changes on a daily basis,
where that file is otherwise monitored due to /etc/mtree/4.4BSD.dist.



The file in question is /var/unbound/db/root.key, which I have auto-updating.



Yes, I understand why this file is important, but on this particular system,
being lulled into complacency by a daily false-positive security(8) report is
more of a danger than someone managing to hack the root DNS key.



Suggestions or pointers or interpretation of the docs appreciated.



Thanks,

-Adam

Re: OT: Any experience connecting OpenBSD via ONT ?

[Adam Thompson]

On 16-04-26 05:29 PM, Jeremy wrote:

Yeah, that's half the problem. My ISP isn't telling me much. Their
helpdesk is handled out of the Philippines and it seems they're reading
off a script. They don't mention PPPoE but from what I've tried so far,
this looks like it will be necessary.
  
Jeremy


If all else fails, run "ifconfig em2 up", and then "tcpdump -i em2 - 
-l -n" and see what, if any, traffic is coming from the ONT on the raw 
ethernet port (this will include the VLAN 10 packets, too). If you're 
lucky, something it emits will give you a clue.

-Adam

Re: openbsd vs freebsd NAT performance

[Adam Thompson]

On 16-04-16 11:55 AM, Mihai Popescu wrote:

Hi,
beside OpenBSD 5.8 i installed FreeBSD 10.3 on my router-pc. For routing i
use pf.
I noticed that the routing/NAT-performance is in FreeBSD noticeable higher
than in OpenBSD.
I think that is due to the SMP-support of pf in FreeBSD.


I would point you to George Neville-Neil's and Jim Thompson's work, 
titled "Measure Twice, Code Once".  They've presented at multiple 
conferences over the last two years, and you can find transcripts, 
slides and recordings via google.


Google "george neville neil jim thompson measure twice code once" and 
you'll pretty much get all of their results on the front page.


As mentioned by others, it's an active area of research and/or 
development (take your pick) in the OpenBSD kernel.
FreeBSD traditionally focuses more on performance, whereas OpenBSD 
traditionally focuses more on correctness, security, and to some extent, 
elegance.


In other words, your results are entirely normal.
-Adam

Re: OT: True hardware UNIX terminal

[Adam Thompson]

On 2016-04-01 11:07, ropers wrote:
And if anyone has ever operated the OpenBSD installer via a 
teleprinter, I want to hear that story.


I think there's still a first-generation TI Silent 700 somewhere in my 
parents' basement.  If, when they either die and/or move out to a 
seniors' residence prior to that certain event, I should run across it, 
and I can find a compatible telephone (acoustic handset coupler, 
remember!), and can find a compatible 300bps modem to dial into, and can 
find an honest-to-god POTS phone line (I expect this to be the hardest 
part) and can find a compatible system with a serial console that can be 
stepped down to 300bps, and the thermal paper is still viable, I'll do a 
fresh install just so I can mail you the ~3-4m of thermal paper I 
suspect that would generate.  Would that be close enough for you?  :-)


(Actually, it just occurred to me that I don't need the phone line as 
long as I can also find the old PENRIL modem that can start training on 
a front-panel button-press instead of a -90v ring signal.  Or maybe the 
local museum will have a 300bps acoustic-coupler modem I can borrow?)


Many of the readers here have absolutely installed OpenBSD (and other 
*NIXes) via serial port, sometimes using a local terminal, sometimes 
using a modem, sometimes even vastly more esoteric combinations than 
anyone could reasonably expect.

I vaguely recall once doing an OpenBSD install where the "console" path was:
Local VT220 -> multiplexer -> modem -> DATAPAC 3101 (Canadian X.25 
service) PAD -> remote PAD -> remote dial-out service -> another modem 
-> another multiplexer -> serial line into, IIRC, ttyA on a Sun system I 
was helping someone repurpose.  The entire install completed 
successfully off a network boot in about an hour at 2400bps (*and* 
simultaneously 2400baud, all you pedants out there...).
So while that wasn't quite an actual teletype, the whole purpose of 
serial ports and serial terminals as a "standard" is that crazy shit 
like that can actually happen!


-Adam

Re: OT: True hardware UNIX terminal

[Adam Thompson]

On 16-03-30 03:07 AM, Sean Kamath wrote:

Still using a Wyse (50?) on my Ultrasparc 80.

In college, we had these weird DEC PC’s that we used as VT100 compatible
terminals.
That would either have been a DEC Rainbow, which was a 
hybrid-dual-processor 8088/Z80 machine that ran MS/DOS, CP/M *and* had a 
full-blown VT220 emulator in ROM, or a VT180 "Robin" which was basically 
a (Z80-based) VT102/VT103 with enough memory (i.e. 64k) to run CP/M off 
the attached floppy drives.


I had a Rainbow, which is in *many* ways an architecturally fascinating 
machine, during the late '80s/early '90s as my primary PC.  I also had a 
Northern Telecom Displayphone, and then later a DisplayPhone II, for 
those of you with a perverse bent for terminal history.


Of course, I'm also the author of at least five termcap(5)/terminfo(5) 
entries, some of which have not yet been superseded by better 
definitions in the ncurses master list... so naturally I had some really 
f*ing weird terminals at various points in my life.


I wish I could remember what the name was of the "portable" terminal I 
once had - off-white (of course), looked like a Buck Rogers spaceship 
(pointy cylinder) in profile, and the entire front two inches of it 
unsnapped to become the keyboard kind of like an Osborne...


-Adam

firefox and ssh(1) -D behaviour vs. localhost

[Adam Thompson]

When using "ssh -D" to establish a SOCKS-type proxy, I can specify the 
bind_address for the local end of the connection, but how do I control 
the bind address on the far end?


I'm accustomed to using -D to remotely administer various web services 
that are behind a firewall/bastion-host instead of using commercial VPN 
software, but I ran into a situation today that doesn't seem to permit 
it: accessing "localhost".


The remote server has a web-based management service that only binds to 
0.0.0.0, but only accepts connections *from* 127.0.0.1 and [::1].


First, I can't seem to convince Firefox to connect to "localhost" or 
"127.0.0.1" using a SOCKS proxy.


Second, I can't figure out a way to get sshd(8) on the remote side to 
use 127.0.0.1 as a source address when hitting the public IP address.  
(Yes, the web service rejects connections from its own public IP 
addresses, too.)


I can accomplish the task with -L instead, which works well, but that 
approach doesn't scale nearly as easily when I'm connecting to a wide 
variety of systems in quick succession, and it fails utterly when the 
remote app insists on constantly rewriting its own URL to a canonical 
value (because of the wrong port#).


Is there any way to do what I want with -D instead of -L ?

And is the second problem (source IP) just an artifact of Firefox 
refusing to even send the request over the SOCKS tunnel in the first place?


Thanks,
-Adam

New, compatible laptops with warranty?

[Adam Thompson]

I know this topic has been hashed out a few times, but since it's a 
moving target, I need to ask again.


My trusty Thinkpad X201t has finally been consigned to the parts 
bin[1].  I'm now looking for recommendations for a new or recent laptop 
that is *known* to work well with OpenBSD.


ThinkPads or Dell Latitudes preferred.
No larger than 15", preferably 14"-class.  (The X201 was 12"-class!)

Shoulds/wants:
- Should have a "real" (not just USB) docking station.
- Should be available with at least a 3-year on-site warranty.
- Should be rugged enough to last ~5 years (including warranty service).
- I'm willing to go with off-lease/refurb if I get find it from a 
highly-reputable source that will offer at least a 1yr replacement warranty.


Suggestions/recommendations?

Please feel free to reply *OFF-LIST*, particularly if you feel this has 
already been discussed recently enough that you need to flame me for it.


Thanks,
-Adam

[1] The damage it suffered coming back from BSDCan 2015 has finally 
proved too much for me to repair.  Since I bought it new, and I've used 
the more-expensive-than-the-laptop warranty to have the entire thing 
rebuilt at least twice in the last 6 years, it doesn't really owe me 
anything.  So I'm not devastated, but still not looking forward to 
buying a new desktop-replacement-class laptop.


P.S. If any of you need ThinkPad X2xx-generation parts, feel free to let 
me know :-(


--
-Adam Thompson
 athom...@athompso.net

Re: Can I accelerate my magnet HDD using a SSD in any way?? E.g. softraid patch/ARC, dedicated hardware e.g. Intel RCS25ZB040LX="Nytro MegaRAID", anything

[Adam Thompson]

On 16-02-01 12:19 PM, Tinker wrote:

My purpose with asking for SSD-accelerated HDD was DOUBLE:

 1) I need some SSD storage but don't like that it could break 
together - I mean, a bug in your system will feed your SSD at full 
bandwidth for ~7h-7 days, it's completely fried - that's not OK, so 
putting a "redundance layer" in the from of an underlying magnet 
storage layer is really justified.


...so what?  I don't understand what your concern is here.  If you buy a 
batch of HDDs from the same lot, they'll probably all die around the 
same time, too.  I don't think I understand what you're trying to say.


If you're worried about SSD write endurance, I'd say... don't.  Or buy 
better-quality SSDs ;-).


See http://ssdendurancetest.com/  for real-world stress-test data. 
Especially 
http://ssdendurancetest.com/ssd-endurance-test-report/Intel-SSD-520-60.


The Intel DC P3700 series should last about 1.5yrs in an absolutely 
worst-case scenario.  Even the worst-case DC series I could find should 
still last about a month and a half under murderous stress-test conditions.


http://estimator.intel.com/ssdendurance/

This report 
(http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead) 
confirms that even if you completely fill an SSD, that's no big deal.


If your application has a "bug" it's unlikely the sort of bug that would 
fill the entire SSD, erase it all, fill it again, erase it all again, 
etc... and most SSDs can handle ~3000 full-device writes nowadays.


Also remember to factor in the fact that UFS/UFS2 doesn't (usually) 
saturate a block device with I/O write requests.  Maybe if it was just 
writing to a single enormous file it might, but could that actually 
happen in your situation?



2) I need some bulk storage, and I want the terabytes to be really 
cheap so that i NEVER will run out of archival space. An 8TB magnet 
HDD costs in the range USD 500.


Here, I like it to be stored "symmetrically" with how I store the 
other stuff, that is having separate disks, directories, mount points 
etc. for the two doesn't really appeal to me in this particular case -


Simply knowing that the less frequently accessed data will be 
taken from magnet and the more frequently accessed data from SSD seems 
both convenient and practical for my usecase, and, I'll try to have 
SSD volume to cover for *MORE THAN ALL* of my frequently stored data.


Perhaps knowing the prioritization algorithm as to be able to 
calculate more closely what's in the SSD and what's on the HDD only 
would make some sense, BUT, training it by telling it what's to load 
fast using "cat `find the-relevant-data/` > /dev/null" single-shot and 
perhaps via crontab, should deliver really well.


Cheap storage is a valid argument.  HDD vs SSD is still easily an order 
of magnitude difference in price.


So go run FreeNAS (or TrueNAS, or Solaris, or FreeBSD, or even 
Linux...), run ZFS with both a L2ARC and ZIL device on fast SSD, and 
remote-mount the directory.  You even have a choice of protocols :-).  
(NFS or iSCSI.  I suspect you'd want to use NFS, but YMMV.)


Short answer: I don't think any part of OpenBSD does what you're asking, 
natively.  Some supported hardware devices (like the MegaRAID Cachecade, 
et al.) do this for you.  But at that point, what's the difference 
between having the disks hooked up to some speciality hardware in an 
OpenBSD box, versus having the disks hooked up to generic, not-special 
hardware on a non-OpenBSD box? (Particularly given that UFS/UFS2 on 
OpenBSD isn't exactly known for its award-winning performance in the 
first place?)


FYI, even if Intel SRT was available on a Xeon board, you'd still be out 
of luck, because it requires device driver support  - it relies on the 
Intel fake-raid technology which, IIRC, is not supported at all under 
OpenBSD.


I'd also like to see something like this available in OpenBSD's VFS 
layer, but I'm not holding my breath.


IIRC there was some interest in integrating Hammer/Hammer2 into OpenBSD 
instead of ZFS due to licensing issues, but even Hammer2 doesn't appear 
to have tiered storage in its feature list.


Some more random thoughts:
- the Dell H800 supports up to 1GB NVRAM, which is just a CacheCade 
implementation.
- CacheVault is the newer version of CacheCade (AFAICT), and is 
supported on newer MegaRAID cards.  See 
http://www.avagotech.com/products/server-storage/raid-controllers/cache-protection#overview 
for some pretty skimpy info.
- moving your data from HDD to SSD isn't going to dramatically speed up 
READ operations.  Carefully examine the specsheets on both the HDDs and 
SSDs and compare read bandwidths.  Access time (latency) is where SSDs 
are useful, along with acting as write-back caches.
- Areca raid controllers support up to 8GB of cache and a sh*tload of 
SAS ports; this might be good enough for your needs - have a look.



Either iSCSI or NFS against a ZFS-based NAS is probably still your best 
bets 

Re: Willing to help

[Adam Thompson]

On 16-01-26 10:32 AM, Peter Hessler wrote:

On 2016 Jan 26 (Tue) at 08:13:22 -0600 (-0600), Edgar Pettijohn wrote:
:> * adduser(8)/useradd(8):
:>   Needs to be unified into one single

One binary, with symlinks.  Both methods should still work, however.


$0.02:

s/sym/hard /g

might satisfy a larger percentage of people...

;-)

-Adam

Re: VAX - are we dropping support in 5.9?

[Adam Thompson]

On 16-01-23 08:34 PM, Ted Unangst wrote:
I will add that one of the reasons we have support for all these 
museum pieces is that people can build their very own museum and run 
something interesting on it. But running on emulators doesn't really 
satisfy that goal. If there are, in fact, no museum pieces left in the 
world, we no longer need to supply an OS to run on them. 


Huh.  Previous discussions had led me to believe that the OpenBSD 
project's rationale for supporting all these various architectures was 
that it ultimately resulted in much-higher-quality code because 
platforms like VAX and SPARC64 acted as canaries for suboptimal coding 
practices?  (Endian issues, stack issues, framing issues, alignment 
issues, etc., etc., etc.)


Besides, I thought the run-on-everything-and-anything (including the 
verging-on-absurd) was NetBSD's thing, not OpenBSD's?  See 
http://netbsd.org/ports/, make your own opinions on which platforms 
verge on the absurd...


-Adam

[PATCH INCLUDED] Re: Problem with CARP interfaces not responding until VHID is changed.

[Adam Thompson]

On 16-01-25 03:43 PM, rizz2pro . wrote:
> Ok we've figured it out.
>
> We have a couple identical environments all attached to one switch and
> they are all advertising the same VHIDs to each other and it looks to
> be causing some arp problems. (Environment A was getting CARP
> advertisements from Environment B and vice versa)
>
> After specifying a "carppeer" on each CARP interface attached to that
> switch in all 3 environments the issue went away. All 3 environments
> are using the same VHID.
>
> Thanks for the help

Umm... per http://www.openbsd.org/faq/pf/carp.html

> /vhid/
> The Virtual Host ID. This is a unique number that is used to
> identify the redundancy group to other nodes in the group, and to
> distinguish between groups on the same network. Acceptable values
> are from 1 to 255. This must be the same on all members of the group. 

Note the word "unique".  Basically, by having multiple clusters with the 
same VHID, you deliberately broke carp(4).  VRRP would have broken in 
exactly the same way, and I think HSRP would also have broken in same 
way, too.

However, I see that neither carp(4) nor ifconfig (8) indicates that VHID 
should be unique and not shared, except by very weak implication.
If specifying a common VHID is not only contraindicated, but causes 
actual breakage on the network (as seen here), then the manpage(s) 
should say so, IMO.  (IIRC, the manpages are canonical, not the FAQ...?)

Drat, I don't have a copy of -current on this system... following is a 
proposed diff against 5.8-RELEASE.  Sorry for not doing a proper diff 
against -current but if I wait, I'll forget.  At least this way I might 
remember later.

I know in VRRP, the VHID is used to generate the MAC address, but I 
don't recall if carp(4) works the same way.  If it does, then the 
language I suggest for carp(4) below may be too permissive, in that use 
of carppeer will stop each system from complaining, but external clients 
will still encounter difficulties with ARP.

-Adam

--- carp.4.dist 2016-01-25 18:04:39.152065865 -0600
+++ carp.4  2016-01-25 18:08:14.326975564 -0600
@@ -58,6 +58,10 @@
  a common virtual host ID (VHID) and
  virtual host IP address on each machine which is to take part in the virtual
  group.
+The VHID uniquely identifies a cluster locally within a broadcast domain
+(network segment), but may be reused on other networks.  The
+.Cm carppeer
+option may also be used avoid conflicting VHID multicasts.
  Additional parameters can also be set on a per-interface basis:
  .Cm advbase
  and

--- ifconfig.8.dist 2016-01-25 18:15:07.089080002 -0600
+++ ifconfig.8  2016-01-25 18:14:35.745168361 -0600
@@ -849,6 +849,7 @@
  Set the virtual host ID to
  .Ar n .
  Acceptable values are 1 to 255.
+Clusters on the same network should use unique IDs.
  .El
  .Pp
  Taken together, the

Re: Problem with CARP interfaces not responding until VHID is changed.

[Adam Thompson]

On 16-01-21 04:02 PM, rizz2pro . wrote:

I know the CARP interface's MAC address is generated by the VHID so I am
sort of leaning towards it be an ARP issue and possibly not an issue with
the OBSD system. But I am hoping for some hints or ideas from you guys.
I have a suspicion... what kind of switches are the OpenBSD hosts in 
question connected to?  If they are managed switches, have you looked at 
the logs on the switch?


Some switches interpret CARP (and VRRP, for that matter) as either an 
attack or a misconfigured system, and will temporarily block traffic to 
one of the two ports.  Essentially, they think it's "MAC flapping"... 
which it sort of is, in fact.


-Adam

Re: pfctl -f /etc/pf.conf fails on boot when DNS-resolved symbolic names are used

[Adam Thompson]

On 15-11-10 01:45 PM, Giancarlo Razzolini wrote:
As a general rule you should avoid using dns names on anything that 
might cause the boot process to fail. Even more, you should really 
avoid using names on hostname.if files.

Anybody run into this before?  - is the fix to add all the symbolic
names to /etc/hosts?

Well, if the hosts have fixed addresses, you'd be better using macros on
pf.conf that translate to their IP address. This way you won't run into
boot issues (or reload issues, in case your resolver is down). This has
the added inconvenience that you need to update your pf.conf file
manually every time one address changes.

Now, if you really, really need to use fqdn's on pf.conf, my suggestion
is that you use ifstated to detected if your link is up and your
resolver working, and them load the rules into an anchor afterwards.
Also, you can update the anchor to reflect any uplink unavailability. Or
you can use unbound with local-zones or a unbound + nsd combo, if you
also need authoritative. I think you'll need to hack your /etc/rc file
to load them before your pf.conf is loaded.



FWIW, yes, putting the entries into /etc/hosts *will* work, and it 
avoids the need to use pf.conf macros, ifstated, etc.


However, it now means that you have to ensure /etc/hosts remains 100% 
accurate... although I shudder to think of using ifstated and anchors to 
do this, it does avoid the /etc/hosts maintenance problem.  And make no 
mistake: you *will* eventually forget to update /etc/hosts.  Absolutely, 
100% guaranteed.


-Adam

Re: What hardware spec would I need to push 20 gigabit of network traffic on an OpenBSD server?

[Adam Thompson]

Thank you for the reply. I see now that my request was wildly 
unrealistic.

Not "wildly", just unrealistic unless you have a massive budget.

Basically I'm trying to write a business plan and am trying to plan 
for the worst case scenario so I don't fall over if traffic somehow 
spikes to such levels. My expected level of traffic is probably in the 
300 megabits a second range but the incoming links from my upstream 
provider are 10GbE so I need to have some plan just in case I get a 
spike of 10 gigabits (as unlikely as that may be I still need to plan 
for it).
I would suggest simply limiting your incoming traffic... and paying for 
less bandwidth.  Most providers are quite happy to sell you, e.g. 4Gbps 
capacity on a 10Gbps physical line.  That moves the problem back a layer 
to the "someone else's problem" domain.


I haven't ruled out the possibility of using Cisco / Juniper for some 
of my requirements but obviously would like to use OpenBSD if possible 
because I've used it in the past and it includes everything that I 
need and best of all the documentation is excellent.
I think you might be able to do 10Gbps of L2TP traffic on Linux or 
FreeBSD (on commodity hardware) if you use one of the highly-optimized 
networking stacks that are available (e.g. DPDK). But even they won't 
handle it gracefully.
I'm not even certain that Cisco or Juniper gear will decapsulate L2tP 
traffic at that rate.


I can drop the logging requirement pretty easily. That isn't really 
important at all.


I wonder what other people do when they are disaster planning for 
their new services? At the same time I wonder how the internet 
backbone is handled. As far as I am aware they handle speeds of about 
100 gigabits a second so the hardware / software must be available for 
handling such speeds. I just guess they are ridiculously expensive to 
buy and maintain (well outside my budget anyway).

You build-to-budget.
The 100Gbps backbones/routers you hear about are a) few and far between, 
b) insanely expensive (multi-millions of dollars), and c) only shuttling 
packets as-is from one port to another.
Everyone else just buys what they can afford, and then they take steps 
(like throttling) to ensure their servers don't fall over.  Of course, a 
lot of people just let their servers fall over, too...


Also: load-balancers.

-Adam

Re: What hardware spec would I need to push 20 gigabit of network traffic on an OpenBSD server?

[Adam Thompson]

On 15-10-25 03:46 AM, Some Developer wrote:
I'm just wondering what hardware spec I'd need push 20 gigabits of 
network traffic on an OpenBSD server?


Short answer:  It's not generally possible today, at least for your use 
case.


Medium answer:  Contact Esdenera Networks to find out.  They manage to 
do it somehow.  I'm sure they'll be happy to make it happen for you in 
exchange for suitable amounts of money...



Longer answer:

Network performance research numbers have presented by gnn at various 
conferences over the last year or so, and they consistently showed that 
OpenBSD, while performing well for a single-threaded stack, fell badly 
behind in multi-core, and wasn't able to keep up to 10Gbps.  The OpenBSD 
team is (currently, AFAIK) working on making the network stack 
multi-threaded, or at least not giant-locked, which should (eventually) 
dramatically increase performance scalability.


On top of that, there are substantial optimizations possible; research 
in the FreeBSD camp (and experience under OpenBSD as well) has shown 
that seemingly-similar hardware can perform radically differently.  
Drivers make a big difference.


You talk about storing the data - *writing* data to disk at 10Gbps 
(sustained) is currently in the realm of high-energy physics, with 
multi-million-dollar budgets for the storage arrays.  A 7200rpm disk can 
charitably be said to write at up to 100MBytes/sec, but that's not 
necessarily sustained speed, so minimum 10-unit array assuming 100% 
ideal throughput, which doesn't actually exist in the real world.  More 
likely you'd have to buy a large HDS array to get that kind of 
throughput.  Plus, that's about 2.5PB (yes, PETAbytes) of data every 
month.  Are you building this for the NSA?!?


You do realize that this means you're now trying to push *30* Gbits/sec 
on a single server, right?  (10 in, 10 out, 10 logged) Even Netflix, who 
spend a ridiculous amount of time doing optimization, have only recently 
gotten FreeBSD servers with tons of custom code and tweaks to pass the 
65Gbps-per-socket mark.


Lastly, Gbits/sec isn't the bottleneck.  The bottleneck is 
packets-per-second.  If you're pushing 10Gbps worth of 1500-byte 
packets, then this is possible today. (Not sure about 30Gbps.)  If 
you're trying to push 10Gbps worth of 64-byte packets on commodity 
hardware, forget about this pipe dream for another few years until the 
fully-MP network stack is finished and optimized.


Good luck... but you might want to consider doing this on a Juniper MX 
series or Cisco ASR instead - those platforms can at least maybe do the 
tunnelling part for around $250k, then feed the output into a 10GE 
switch with port mirroring (~$10k), then a Network Flight Recorder or 
similar to actually capture that much data (~$150k).


-Adam

[OT] Re: What hardware spec would I need to push 20 gigabit of network traffic on an OpenBSD server?

[Adam Thompson]

On 15-10-27 02:53 PM, Martin Schröder wrote:

And then there are SSDs. PCIE SSDs do up to 3000 MB/s write throughput.
https://www-ssl.intel.com/content/www/us/en/solid-state-drives/solid-state-drives-dc-p3608-series.html

And I'm sure there are tape libraries that can write that, too. :-)

I disregarded that part, based on the unsustainable assumptions elsewhere.
Plus, how much would 2.5PB of PCIe SSD cost?!?!?

As to tape libraries... interesting idea.  An LTO-6 drive can write at 
~160MBytes/sec, and that's pretty much the top end of 
generally-available tape drives today, so you'd still need an 8-drive 
RAIT-0 setup.  After taking into account library changer latency, I'd 
guess you'd need a custom buffering solution, maybe one system to shard 
the data, then ~10 systems to buffer the data to ~10 tape drives with a 
big enough buffer to sustain the ~2-4min it can take to eject and load a 
new tape in a big library.  Perhaps that's where the PCIe SSDs come in? :-)
(10Gbits/sec ~= 1 Gbyte/sec ~= 60 Gbytes/min... you could probably get 
away with a 256MB SSD)


A 12-drive StorageTek LTO6 library with ~1000 tapes costs around $2M 
last I heard, but at least the tape cost is reasonable once you get 
going.  Of course, you also need the multi-path FC networking gear for 
that, and I think OpenBSD just dropped FC support.


An alternative might be a z-series mainframe, I hear they have massive 
I/O capabilities.  But they don't (AFAIK) run OpenBSD.


-Adam

Re: Recommended miniPCI express wireless module for PC Engines' APU system board?

[Adam Thompson]

On 15-09-23 05:01 PM, Mike Bregg wrote:
I'm using an APU as a firewall/router and it works very well.  
However, after experimenting with some different wireless cards, I 
actually opted to install a separate EnGenius EAP600 Access Point on 
the main floor of my house, using PoE to run to the router/switch.


[OT, sorry...]

One word of warning: don't *ever* put an EnGenius AP outside the 
firewall... it has an open DNS resolver running on it that you can't 
disable.  Found that out the hard way when I used an EAP600 to bridge a 
cable modem connection to a router in another room :-(.


-Adam

Re: update/upgrade

[Adam Thompson]

On 09/20/2015 10:26 PM, Quartz wrote:
It looks like the M:tier thing is pretty close, my only concern is how 
long it'll last before the maintainers lose interest and the project 
gets abandoned.


Handling updates/upgrades in OpenBSD has always been one of the more 
difficult parts for ordinary users.

Having said that, Antoine  have developed a great service.

As to "lose interest", I think you're missing the fact that m:Tier is a 
company, not just another open-source project.  They've been around for 
over seven (7) years already.  If they were going to simply "lose 
interest", I think they'd have done so long ago.  They do have a regular 
website, at www.mtier.org, that fills in all the gaps you were talking 
about in a previous post.


You can also *pay* for a subscription, which I would assume - barring 
utter insanity on the part of every employee there - would go a long way 
towards ensuring they stick around.  (Per a previous conversation with 
them, you don't have to buy a subscription for every single machine 
you're updating - but confirm that with them before basing any plans on 
that.)


-Adam

Re: rdomain with BGP dynamic route

[Adam Thompson]

Then you should have been at BSDCan last month!

Olivier Cochard-Labbé of Orange(?) is deploying something like this but 
using FreeBSD (http://www.bsdcan.org/2015/schedule/events/555.en.html), 
and Peter Hessler talked at length about rdomains in OpenBSD 
(http://www.bsdcan.org/2015/schedule/events/555.en.html).
Both presentations were recorded, but obviously you can't do interactive 
QA with a YouTube video :-).


-Adam


On 07/31/2015 10:48 AM, XU, YANG (YANG) wrote:

Adam,

That's good suggestion, thanks and I will give it a try.  I plan to use OpenBSD 
as a PE from a service provider point of view, I guess that's not a popular use 
case.

Regards,
-Yang

-Original Message-
From: Adam Thompson [mailto:athom...@athompso.net]
Sent: Thursday, July 30, 2015 6:04 PM
To: XU, YANG (YANG) y...@research.att.com
Cc: misc@openbsd.org
Subject: Re: rdomain with BGP dynamic route

On 07/30/2015 10:26 AM, XU, YANG (YANG) wrote:

Adam,

Your  comments and links are very helpful, they made some concepts clear for 
me. Many thanks!

What I need essentially is VRF function which converts IPv4 prefix to
VPNv4 prefix dynamically. I hope experts can help on this. After
spending so much time on searching for the answer, sending beer is an
easier thing to do. Just give me a working example, and beer will be
on the way :)

Regards,
-Yang

After that clarification, I understand even less than I did before of what you 
want to accomplish.  Regardless, at this point I'd say your options for 
continuing with OpenBSD might be limited to:
a) go to one of the BSD conferences, locate one of the developers who work on 
the network stack [this isn't very hard], and ask them; or
b) pay someone (e.g. maybe reyk@ et al. over at Esdenera Networks, not sure who 
else would be a candidate) to figure it out for you and, if necessary, fix it.

-Adam

Re: rdomain with BGP dynamic route

[Adam Thompson]

On 07/30/2015 10:26 AM, XU, YANG (YANG) wrote:

Adam,

Your  comments and links are very helpful, they made some concepts clear for 
me. Many thanks!

What I need essentially is VRF function which converts IPv4 prefix to VPNv4 
prefix dynamically. I hope experts can help on this. After spending so much 
time on searching for the answer, sending beer is an easier thing to do. Just 
give me a working example, and beer will be on the way :)

Regards,
-Yang


After that clarification, I understand even less than I did before of 
what you want to accomplish.  Regardless, at this point I'd say your 
options for continuing with OpenBSD might be limited to:
a) go to one of the BSD conferences, locate one of the developers who 
work on the network stack [this isn't very hard], and ask them; or
b) pay someone (e.g. maybe reyk@ et al. over at Esdenera Networks, not 
sure who else would be a candidate) to figure it out for you and, if 
necessary, fix it.


-Adam

Re: rdomain with BGP dynamic route

[Adam Thompson]

I see what you mean.

This, I think, is close to what you're looking for, but I'm not 100% 
certain of how to accomplish exactly what you want:


  nexthop qualify via bgp
  listen on A.B.C.D  ## vrf member address of, say, em1
  rtable 2  ## put vrf interface into rdomain 2 via ifconfig?
  rdomain 2 {
rd 123:456
depend on em1
  }
  neighbor VPNCLIENT1 {
depend on em1
  }

...I don't know how to avoid running a separate instance of bgpd(8) per 
VRF, however.


Given that OpenBSD has, apparently, a working MPLS + LDP implementation, 
this is obviously yet another case where the manual pages contain enough 
information


Based on jeker's MPLS paper back at EuroBSDCon 2011, it may be the case 
that BGP + VRF really only works work mpe(4).


Ah!  I found someone else's documentation that explains it... you do 
still need to use rdomain0, but you MUST tag the routes.  rdomain0 
contains everything, including the [possibly overlapping] routes, but 
with prefixes to keep them separate.


See http://firstyear.id.au/entry/21. Also see jeker's explanation 
(perhaps dated) here: 
http://openbsd-archive.7691.n7.nabble.com/Using-RDomain-setup-with-pf-4-and-bgpd-8-td42066.html.


(Also check out http://lmgtfy.com/?q=openbsd+bgp+vrf. Admittedly, the 
fourth or fifth result _is_ this thread :-/.)


Failing that, as I can't tell for certain if it does what you want, wait 
for Claudio, Henning, or whoever else is working on it now to figure out 
how to make it work better!  I hear sending them beer sometimes helps...


-Adam




On 07/24/2015 08:07 PM, XU, YANG (YANG) wrote:

Adam,

I really appreciate your reply. I read bgpd.conf and see rdomain can only 
define network as explicit, static or connected. In my case, I need to import 
dynamic prefix from BGP session. Right now all prefix learned from BGP goes to 
rdomain 0.  I want to put prefix learned from BGP into the rdomain I specify.

Thanks,
-Yang

From: Adam Thompson [athom...@athompso.net]
Sent: 24 July 2015 20:33
To: XU, YANG  (YANG)
Subject: Re: rdomain with BGP dynamic route

On 2015-07-24 06:47 AM, XU, YANG (YANG) wrote:

Let me describe it in another way. Can I create a new rdomain as a VRF and use 
the rdomain to import/export customer's prefix through BGP?

I will greatly appreciate it if you can provide any information. I have seen 
some information online, but prefix is either from static configuration or 
connected network. In my case, I need to support dynamic routes from BGP in VRF.

Short answer: yes.

See man bgpd.conf(5):

ROUTING DOMAIN CONFIGURATION
bgpd(8) supports the setup and distribution of Virtual Private Networks.
It is possible to import and export prefixes between routing domains.
Each routing domain is specified by an rdomain section, which allows
properties to be set specifically for that rdomain:
rdomain 1 {
  descr a rdomain
  rd 65002:1
  import-target rt 65002:42
  export-target rt 65002:42
  network 192.168.1/24
  depend on mpe0
}

--
-Adam Thompson
   athom...@athompso.net

Re: which netbook not to buy?

[Adam Thompson]

On 2015-07-12 04:08 PM, Theo de Raadt wrote:

I would suggest x201s for Many reasons. You can find it without hdd on ebay
for 100usd... + ssd any... but I am not sure about openbsd... with debian
it just works.

The x201 works great.

I used one for about 3 years, and the beatings continued until
everything worked perfectly.

Mark Kettenis applied the same pressure to the x220.

And Mike Larkin to the x230.

But the general rule is that the upper-class Thinkpads work the best,
first of all because their BIOS are written by Team-A at Lenovo,
secondly because we fixed things for ourselves.



One notable exception: none of the pre-x220 touchscreens work, they use 
the old Wacom serial protocol that lacks a working driver in modern 
OpenBSD.  And of course there's the various power/heat/battery issues 
that are gradually getting fixed.
But overall, my x201t works great under OpenBSD.  If you want small + 
light, don't get the tablet model, and IMHO get at least the x220 or newer.

Of course, none of us are actually answering your original question :-/.

--
-Adam Thompson
 athom...@athompso.net

5.7 bootloader hang on Toshiba Portege M400

[Adam Thompson]

Trying to install OpenBSD 5.7-RELEASE amd64 onto a Toshiba Portege M400, 
with no success - the bootloader hangs right after printing out fd0 
hd0+.  A (very) quick perusal of boot-related manpages doesn't suggest 
anything I can do to debug this, so I'm looking for suggestions.  Do I 
need to recompile the bootloader with some debug flag set?  Did I just 
zone out while reading the relevant part of a manpage?


(FWIW, WinXP, Ubuntu 15.04 and current Sysresccd all boot OK, so I'm 
pretty sure the hardware is fine.)


--
-Adam Thompson
 athom...@athompso.net

Re: OpenBSD hosting in Hong Kong?

[Adam Thompson]

For the usual reasons - most VPS providers do not allow you to install 
from arbitrary ISOs, and even fewer are willing to give you any 
assistance at all with unsupported OSes.  (You're only getting 1kbps? 
Let's see... oh, you're running OpenBSD.  Have a nice day, bye.)

-Adam


On 04/17/2015 08:00 AM, Paul S. wrote:

Just buy a box with KVM/IPMI from any provider?

Why does it have to be OpenBSD centric?

On 4/17/2015 午前 10:41, OpenBSD user wrote:

Hello,

I was wondering if there are any hosting companies
in Hong Kong offering OpenBSD dedicated servers?

Would be truly grateful for any replies!

Re: Route for a special IP

[Adam Thompson]

On 2015-03-11 10:58 PM, Zhi-Qiang Lei wrote:
 It was just a router which does NAT for local devices in 
 192.168.1.0/24. The external interface, of cause, was pppoe0. Now for 
 some reason, I want one of the device with IP 192.168.1.200 
 communicate with outside through the tunnel interface tun0 created by 
 OpenVPN. Normally I should setup OpenVPN client on that device, but it 
 has a low frequency CPU.
Ah, you want the opposite of what I said then.  The only way I know to 
do that is to use pf, and the route-to directive, which I think someone 
else already covered.
-Adam

Very-small fully-functional systems?

[Adam Thompson]

I'm looking for a very (physically) small (embedded) platform that can 
run OpenBSD properly, including at least:

USB 2.0,
ethernet,
MIDI (presumably via USB),
OK-to-good-quality analog audio out (can be USB),
some kind of decent storage (i.e. not USB!),
and enough CPU horsepower to run a soundfont-based softsynth like 
timitidy or fluidsynth (so ~500MHz+).


All preferably for no more than ~C$100 per board.

Locally, I can get RasPis, BeagleBone Blacks, and MinnowBoards cheaply.
The RasPi isn't supported at all, the BeagleBone Black doesn't have USB 
yet, and the MinnowBoards don't have ethernet yet.


Looking at the OpenBSD platform list, I see these options
Wandboard, SABRE Lite, Nitrogen6X, Pandaboard (armv7): all too 
expensive, although the wandboard comes close on price (~C$110 to $125 + 
SH).

SH-4 (landisk): not fast enough
RB600A (socppc): not available anymore.

That leaves i386 and amd64 platforms.
There are a few Intel J1800 boards available for C$100 (not including 
RAM), but I have no idea how well they run OpenBSD, and most of them 
seem to have UEFI boot only.


If you know of a small, cheap, fanless, x86-compatible board available 
in Canada *that runs OpenBSD reasonably well*, please let me know.  
Otherwise I'll probably have to give the Wandboard a shot despite the 
slightly-too-high price.


(In case it's not blazingly obvious, I want to try creating a 
replacement for a hardware MIDI synthesizer, but that can choose between 
soundfonts.)


Thanks,
-Adam

--
-Adam Thompson
 athom...@athompso.net