openup vs. syspatch vs. pkg_add -u ?

[Adam Thompson]

Now that we’re in the wonderful world of syspatch(8) – which works well for me 
so far (thanks for the hard work, everyone!), I’m trying to figure out if 
there’s still any point to using m:tier’s openup tool.

>From what I can tell, running “syspatch ; pkg_add -u” is pretty much 
>equivalent to what openup does… but am I missing anything?

 

Thanks,

-Adam

 

Does pf support NPT (RFC6296) ?

[Adam Thompson]

I still haven't found this answer anywhere...

Does OpenBSD (more specifically, pf(4), I guess) support RFC 6296, 
IPv6-to-IPv6 Network Prefix Translation?  Looks like FreeBSD can do it, 
but I can't tell if that's something they added to their own pf fork, or 
if I'm just missing something in the OpenBSD docs.


I know I can do NAT66, but I don't think it's feasible to emulate NPT 
using NAT66 rules.


Thanks,
-Adam

Re: Does pf support NPT (RFC6296) ?

[Adam Thompson]

> > I know I can do NAT66, but I don't think it's feasible to emulate NPT
> > using NAT66 rules.
> 
> No, NPT is different and can't be emulated by anything that OpenBSD's
> PF currently does.

Shoot.  I was really hoping pfSense managed it through some feature that 
predated FreeBSD's pf(4) import, but that I had merely overlooked.  That sucks, 
right now.
 
> The closest it can get is NAT with bitmask and "static-port", but
> 1) that's stateful, and 2) it doesn't do the "checksum neutral"
> modification that NPT uses (NPT doesn't replace just the network prefix;
> it also adjusts the host part of the address in a complementary manner
> so that the IPv6 checksum doesn't change).

Ah, thank you for that explanation - I wasn't clear on what the manipulations 
were supposed to accomplish.

In my unfortunate scenario, NAT66 would probably work just as well, assuming my 
intuition about how IPv4 NAT/SNAT/PNAT works in pf(4) extends to the IPv6 
world.  An HTTP proxy would also work, I suppose, but would require more 
configuration on the inner hosts.

All I need is a way to give ULA-addressed hosts a way *out* to reach, e.g. DNS, 
NTP, mirrors, probably various CDNs - all the maintenance traffic a modern 
(non-OpenBSD) host generates by itself.  As I write this, I'm starting to 
wonder if NAT66 isn't the better solution anyway since it's (kind-of) 
inherently unidirectional.

Oh, and in case anyone's wondering - this is all because a) VMware NSX 6.0 
supports IPv6, but neglects to include any form of NAT or NPT or outbound 
proxy; and b) OVH, even in their private cloud offering (which is where the 
VMware NSX 6.0 comes in!), will not route public IP address space to a VLAN 
behind my firewall... which works for IPv4 ("just use NAT!"), but not so well 
for IPv6.  And I need IPv6 on the protected hosts.  *sigh*  If anyone reading 
this thinks they can see a better way around this pair of problems, please let 
me know.

-Adam 

inet6 nat-to (group) address selection algorithm question

[Adam Thompson]

So I’ve discovered that, when trying to do NAT66 (for a ULA network), a line 
like:
 "match out on egress inet6 from !(egress:network) to any nat-to (egress:0)"
doesn’t work.  (Yes, the network in this case is ridiculously simple.)

I believe it doesn’t work because :0 indicates that aliases on the if should be 
ignored, and the first IPv6 address on an if is typically going to be the LLA… 
which of course doesn’t work very well for the NAT use case!

However, to my surprise, simply using "nat-to (egress)" does work, instead.

What I’d like to know is, what magic in pf(4) allows it to auto-select a 
"better" (in my case, at least) address when performing the translation?  
There’s nothing in the man page that I can see talking about this.  The man 
page talks about what happens when I *do* specify :0, but not when I *don’t*.  
(And neither does the FAQ, fyi.)

My concern is that the algorithm (or blind luck, depending) that allows "nat-to 
(egress)" to work for me will stop working in some slightly different scenario 
or at some random point in the future when something seemingly-unrelated 
changes.

Thoughts/hints/explanations?  (Other than "read the source", please - there's a 
reason I'm not a kernel programmer!)

Thanks,
-Adam

OpenBSD 6.1 is now supported on Microsoft Azure

[Adam Thompson]

Maybe I missed the email here, but in case it actually doesn't exist:

OpenBSD 6.1 is now supported on Microsoft Azure courtesy of reyk@ and 
the team over at Esdenera® Networks, with assistance from Microsoft.  At 
least that's what I got out of the BSDCan announcement.


I'll let Reyk blow his own horn on the products & marketing.  But I 
think it's great that MS sanctions running OpenBSD in Azure now.  For 
me, this is a significant milestone.


Thanks, Reyk (et al.)

https://azuremarketplace.microsoft.com/en-us/marketplace/apps/esdenera.esdenera-firewall-3
The link to the Sponsor BOF doesn't have any info so I'm not going 
providing it here.
https://azure.microsoft.com/en-au/blog/running-openbsd-on-azure/ 
(apparently this page is about to get updated Real Soon Now)


-Adam

Re: Doubts about the successors of OpenBSD leadership and development

[Adam Thompson]

> From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On Behalf Of 
> Stefan Sperling
> Sent: July 10, 2017 16:17
> Subject: Re: Doubts about the successors of OpenBSD leadership and development
>
> Obviously, Theo de Raadt will succeed Theo de Raadt in the leadership
> and development of OpenBSD: 
> http://marc.info/?l=openbsd-misc&m=137609553004700&w=2

I had forgotten about that old message :-).  Thank you for reminding us!

FYI, Anatolian Shepherd dogs (the Turkish national dog, IIRC) are extremely 
beautiful dogs (IMHO), and very nice, gentle and friendly, as a rule, unless 
they think you're threatening whatever they're protecting.  In which case 
there's no need for an entire group of them to kill Theo, one dog will likely 
be more than sufficient, regardless of how fit Theo may or may not be at that 
point in time.

Since we must, of course, take all statements from Theo literally (otherwise 
what kind of a cult are we?!?!), this does beg the question of what happens if 
he gets attacked by a group of dogs somewhere *other* than central Turkey.

If the project were based out of the US, the solution would be obvious: have 
Theo go armed at all times.  But since it's Canadian... hmm.  If we say he can 
only leave the country protected by one of our submarines, he'll - effectively 
- be protected, since none of them can actually go anywhere most of the time.



-Adam

Re: Virtualbox vs latest snapshot

[Adam Thompson]

On 2018-04-12 20:02, Nick Holland wrote:

On 04/12/18 09:47, Consus wrote:

On 08:28 Thu 12 Apr, Nick Holland wrote:

Another "failure mode" of VirtualBox people should be aware of:
I understand through good sources, Oracle monitors the IP addresses 
that
it's downloaded from, and if they can trace it back to a commercial 
IP
(i.e., not a home address), and if they see you download (or update) 
the

"not for unrestricted free use" parts, their lawyers will contact you
and send you a bill...and they really don't care about "for work" or
"not for work related" uses.

I'd really recommend removing this product from your computers.


This won't stand in court. You sources are so high on crack it's not
even funny.


Think about it a moment,
Using my real name, and a public, trackable identity, I just accused a
very big company with lots of lawyers (and they know how to use them!)
of something.  If my facts are not in order, I could be in big trouble.
My facts are in order.

It's not about court.  It's about threatening lots of companies and
hoping a few pay up to avoid the cost of going to court -- which is
considerable, win or lose.

What you believe changes nothing.  Their licenses are complicated, easy
to use wrong, and they seem to care.  I recommend against using their
products for that reason.

Nick.



My tale of Oracle woe:

My company got spanked by Oracle a couple of years ago because one of 
our developers was downloading multiple versions of the RDBMS, trying to 
find a version that would be happy with a binary database file we got 
from a client.
Their License Enforcement dept. (it's part of the Sales division, which 
tells you something...) undertook a sneaky campaign of phoning all our 
staff asking them how they were satisfied with their Oracle products.
They finally audited us, and - not for the products that dev was 
downloading, but for something else entirely - it turned out we were 
accidentally in violation of one of our licenses, as one sentence had 
changed somewhere in the ~20yrs we'd been using it to invalidate our use 
case.
Since it was not intentional, and we were cooperating with them, and we 
are very small (~10 persons, at the time) they ONLY required a top-up 
payment roughly equal to 1 developer's annual salary.
The threat of being sued out of existence, if we didn't cooperate, was 
made explicitly.  We were, in my opinion, deliberately maneuvered into a 
non-compliant position over a period of many years, then subject to a 
"sting" operation, and finally bullied into compliance.


And we also use VirtualBox :-(.  For now.

It's notable that at least in Canada, BSA (MS,Adobe,Sybase,etc.,etc.) 
audits are legal - and are often accompanied by provincial Sheriffs.  
(Not the same thing as a U.S Sheriff, but still a law enforcement 
officer.)  The threat of legal action is NOT just an empty threat.


-Adam

Re: Viewport for man.openbsd.org -- readability on phones

[Adam Thompson]

On 2018-05-19 02:59, justina colmena wrote:

https://man.openbsd.org/mandoc.css
That's the css. You style it how you like it. That's the whole point
of it. And I agree. It's very readable on my phone.
 Original message From: Mihai Popescu
 Date: 5/18/18  11:04 PM  (GMT-09:00) To:
misc@openbsd.org Subject: Re: Viewport for man.openbsd.org --
readability on phones

I don't understand what you are trying to say.


I took and iPhone with iOS and Safari ( i think!) on it and pointed
the browser to the current link of man pages [1]. All i can say is the
layout is displayed on full display, not stretched.
Text is fine, paragraphs are scaled ok, not even a simple problem. Font 
is fine.


[1] https://man.openbsd.org/



Just to confirm, it also works and looks fine on my phone (is that 
"works phine"? ) which is running Chrome under Android 8.1.


I can see why it would not look fine on an extremely low-res display, 
though: the margins could overwhelm the page, and the resultant wrapping 
would be very not-pretty.


OTOH, *every* website will look bad on a phone that old.

-Adam

Re: supported Audio card with SPDIF input

[Adam Thompson]

On 2018-07-24 17:54, Diana Eichert wrote:

ok, answered my own question by grep'ng within /usr/share/man/man4,
looks like azalia(4) systems.  Was hoping for something usb attached
but no such luck.

On Tue, 24 Jul 2018, Diana Eichert wrote:

I'm trying to connect to an audio system that only has SPDIF output.
I looked at man pages but nothing obvious regarding supported audio
devices with SPDIF input support.

Anyone have recommendations?  Or is it supported?


Very broadly speaking, as long as the USB device conforms to the USB 
Audio Class spec, it doesn't matter whether it's got S/PDIF or Coax or 
2xRCA or XLR - audio waveforms should still get transferred from PC to 
output.
What I think you might lose is any sort of decent mixer control, 
although with S/PDIF I expect you might not really care about that?


Caveat: I have not personally tried a USB-to-S/PDIF audio device, but I 
*have* read the specs and the datasheets.  In theory, theory is the same 
as reality...


Best-case, a USB SPDIF output device would a) conform to the USB Audio 
Device Class interface, and b) report that it has no adjustable mixers, 
so that on the software side, the mixer doesn't get confused.


-Adam

Re: Best way to serve files to Windows?

[Adam Thompson]

On 2018-07-18 09:35, Tom Smyth wrote:

Hi John,
You would need microsoft services for unix (SFU) for NFS connectivity


FYI - so no-one goes haring off in the wrong direction.

SFU is the server-side component, equivalent to running nfsd(8).

On the client side, only certain editions of Windows can speak NFS:
- Windows 10 *Enterprise* can mount remote NFS shares.
- Windows 7 *Ultimate* can mount remote NFS shares.
(No idea about Win8, sorry.)

Win10Ent, at least, has flexible authentication options, but IIRC 
defaults to uid=0/gid=0 (gee, thanks).  It prefers to use Kerberos 
security, which won't work with OpenBSD's NFS server.  It's possible to 
make this work reasonably well, but it takes a fair bit of time.


So, as everyone else said, you're better off running Samba on your 
OpenBSD system.  Have fun.

-Adam

recommended h/w for fanless audio-out?

[Adam Thompson]

Hello,

I’d like to use OpenBSD to build a MIDI synthesizer using SoundFonts, as the 
OpenBSD MIDI and audio subsystems are remarkably understandable and sane, 
compared to everything else out there today.

 �

However, I’m having difficulty finding a combination of hardware that is known 
to be supported.  (I tried this a few years ago, but was unhappy with the 
results due to the quality of the audio-out ports on the laptop I used.)

 �

I’ve heard a fair bit here about USB audio not working very well, or at all, in 
-stable right now.  I’m unsure if this only applies to XHCI ports or not?

 �

Therefore, ideally, I’d like a fanless amd64 architecture system, that boots 
from SATA (not eMMC) with either onboard stereo++ audio (up to and including 
7.1ch) *or* a PCI slot for a sound card.

A laptop with really good audio-out would work, too, but I don’t know of any.

If connecting USB audio to a RPi or BeagleBone Black works well, then I could 
try that, too.

 �

I intend to use USB MIDI interfaces, as they appear to be supported and working 
fine (IIRC).

 �

The problem is that, as I browse the list of embedded, fanless systems I can 
get from DigiKey, Mouser, et al., all the boards with audio have HD Audio 
Codecs that I cannot confirm work with OpenBSD, and I don’t know enough about 
how azalia(4) [et a.] works with random unsupported DACs to drop $250-$500 on 
something that may or may not work (i.e. the codec isn’t explicitly supported).

 �

Can anyone make any recommendations for specific hardware that’s fanless and 
has working audio-out ports?

 �

And/or if I’ve gone off completely in the wrong direction about USB audio, 
please tell me so.  (Working multi-channel USB audio would made this a lot 
simpler/easier/cheaper.)

 �

Thanks in advance,

-Adam

 �

Re: mount dir over another dir

[Adam Thompson]

On 2020-04-16 02:13, Ono Caritofilaxy wrote:

Hello.

I want to mount /usr/local/srcdir /usr/local/dstdir/subdir

answer was "no" 3 years ago
https://marc.info/?l=openbsd-misc&m=149743861203607&w=2

Can I do this now?
If not - why? Is it dangerous?


You should be able to do this as an NFS mount.  With all the nastiness 
that NFS mounts come with, but it's an option.  (I'm doing it in 
production on 6.6-STABLE.)

-Adam

RCS file ownership?

[Adam Thompson]

When I use co(1) with "-l" to check out a file (and/or "ci -l") is there 
any way to preserve file ownership and *not* have it reset to the user 
running co(1) or ci(1)?
I don't see anything in rcs(1), co(1) or ci(1) that even mentions the 
fact that the file will wind up owned by the user running the command.

Ideas?  Pointers to documentation?

Thanks,
-Adam

Re: RCS file ownership?

[Adam Thompson]

Being neither a C programmer nor a Texinfo fan, checking GNU RCS is a 
bit painful, and my conclusions aren't guaranteed.



AFAICT, GNU RCS (v5.9.4, ca. 2015, examined) creates a temp file, 
unlinks the target file, then renames the temp file.  I beleve this 
guarantees(-ish, modulo "special" filesystems including NFS and 
FreeBSD's directory-SUID behaviour) that resulting file ownership = 
euid.


The GNU docs mention the repo owner in passing a few times but do not 
have a section describing multi-user operation.


The Tichy docs also don't mention file ownership.  I'm trying to review 
the O'Donovan book, too, but it's been a long time since I had to tool 
up to handle raw PS... not quite there yet.


Purdue RCS appears to be the direct ancestor of GNU RCS.

I'm not sure which other implementations you'd be worried about - I 
thought OpenBSD's RCS was the direct descendant of NetBSD's and shares 
common lineage with the other *BSDs?


All in all, it looks like RCS and its docs were written in the era when 
UNIX machines were - more or less by universally - used by multiple 
people, and you just had an innate sense of how multi-user file 
ownership would work.  Most of my UNIX machines now resemble appliances, 
and exactly zero of them are multi-user in the classical sense.


-Adam


On 2020-04-29 21:53, Theo de Raadt wrote:

Sorry, but my mail goes further.

It says it should be correct.  For some definition of correct.  It
should either behave somehow for a logical reason, or it should behave
in the historical fashion.  Or once the historical behaviour is looked
at, if there is an argument that is wrong, then it should be changed
with logic about "this is an improvement" backing the argument.

I think it is wrong to document how *this* rcs implimentation behaves,
without comparing it against others.

My guess is 50% that the others don't unlink, but rewrite the file.

And the changes it might require to be compatible are not substantial.
At most a 20 line diff, to a few programs in the family.

athom...@athompso.net wrote:


Thank you for that detail!

Addressing this one corner case would require substantial changes, I 
think.  Not worth

it, in my opinion.

I think it would be worthwhile describing the multi-user mode of 
operation of RCS in
the manual, as it's currently completely absent/omitted. Patch coming 
soon, maybe

tomorrow if I can make time.

-Adam

On Apr. 29, 2020 21:28, Theo de Raadt  wrote:

 athom...@athompso.net wrote:

 > Heh, good point.  Didn't even occur to me because as it happens, I 
am

 > running as root and would like to not change the ownership.-Adam
 > On Apr. 29, 2020 13:32, Anders Andersson  
wrote:

 >
 >   On Wed, Apr 29, 2020 at 7:46 PM Adam Thompson 


 >   wrote:
 >   >
 >   > When I use co(1) with "-l" to check out a file (and/or "ci -l") 
is

 >   there
 >   > any way to preserve file ownership and *not* have it reset to 
the

 >   user
 >   > running co(1) or ci(1)?
 >   > I don't see anything in rcs(1), co(1) or ci(1) that even 
mentions

 >   the
 >   > fact that the file will wind up owned by the user running the
 >   command.
 >   > Ideas?  Pointers to documentation?
 >
 >   How could it possibly do anything else unless you always run co 
as

 >   root?

 Our rcs tools do:

 52628 co   RET   kbind 0
 52628 co   CALL  unlink(0x7f7ed1f3)
 52628 co   NAMI  "ls.c"
 52628 co   RET   unlink -1 errno 2 No such file or directory
 52628 co   CALL  open
 (0x7f7ed1f3,0x601,0100444)
 52628 co   NAMI  "ls.c"
 52628 co   RET   open 4
 52628 co   CALL  kbind(0x7f7ec908,24,0x847da2a816b5d817)

 Which appears to be this:

 else {
 (void)unlink(dst);

 if ((fd = open(dst, O_WRONLY|O_CREAT|O_TRUNC, mode)) 
== -1)

 err(1, "%s", dst);

 I don't know what older or gnu rcs do.  I guess whichever way this is 
done
 it must balance concerns between atomicity of concurrent reads 
performed
 by earlier processes, versus replacement of a potentially active 
file.


 If the latter is chosen, it would unlink(), perform the open more
 carefully, check that it is in the right place with fstat, but then
 it needs some work for ftruncate (to get rid of extra file contents
 at the end).  If the open() failed, it might try an unlink followed 
by

 open()?

 Other rcs implimentations need to be checked.  It is better if they 
work

 the same.

Re: Postscript printer recommendations

[Adam Thompson]

On 2019-07-14 15:40, Stuart Henderson wrote:

If you don't want trackable prints, don't buy a colour laser printer
of any brand, it is very common. Unsure about mono and inkjet printers,
I would tend to assume that they're common on at least most hi-res
colour printers.


Nearly every printer sold today (including cheap inkjets AND b&w 
printers) have these tracking dots.  It's unclear why/how this came to 
be; I've heard multiple stories and am unsure which to believe, if any.  
Regardless of the why/how, consensus is clear: those dots are real and 
can tie output back to your printer.




 I read that Postscript printers produce superior graphics (from
Xerox website):


In 2019, that remains true only in a very small number of edge cases.  
The absolute highest resolution graphics today are mostly printed using 
ESC/P (if Epson), PCL3 (if HP), or whatever Canon uses (if Canon).  
Mostly people "prefer" Postscript output over PCL because the default 
rendering gammas(sic) are slightly different and Postscript is usually 
perceptually more pleasing.  Adjust gamma on your PCL printer from 1.1 
to 1.2 and *poof* just as good as Postscript.  YMMV, even 
Postscript-compatible printers are all different nowadays.



If I was looking to spend money on a nice printer I'd get one which can
accept postscript and PDF directly over lpr. (I'd be very tempted by
the hp "pagewide" printers. Also Kyocera seem good especially for
high volume stuff).


I haven't tried Kyocera recently, but I have tried HP recently in that 
line.  Stay as far away as you can get.  All the decent PageWide 
printers seem to be discontinued already, and the replacement 11x17" 
color OfficeJet Pros are severely lobotomized and need to talk to the 
"cloud" for almost all functions except basic print-to-paper... and even 
that doesn't work very reliably, even with the Windows/MacOS drivers.




If I was looking for something cheap and cheerful then I would worry
less about postscript but look for something where I can see signs
of support in the ports tree. The HPLIP ports are nice and easy to use
and work with a wide range of HP and some other printers, easiest to
use with CUPS (it really isn't difficult), but if you want to avoid it
I believe the HPIJS part of HPLIP can be used without CUPS. I've heard
good things about brlaser for the non-postscript Brother printers too,
though most of them do have ps anyway. Even if keeping things cheap
I would definitely want something with an ethernet port (it doesn't
add much if anything to the cost) for flexibility of positioning the
printer, easier sharing between machines, and not having to mess
around with permissions on device nodes etc..


Agreed - most decent business printers nowadays come with both ethernet 
and wifi, so you get the best of both worlds.


Brother's mono lasers are generally *excellent* value for the money.  
Color lasers unknown.  Stay away from their color inkjets, they are 
effectively Win/Mac-only.  (But if you are on Win/Mac, those provide 
excellent $/page.)


Epson's color inkjets are pretty good, and at the *really* high end beat 
color lasers in all ways, and include Postscript.  (Those high-end units 
also cost many 10s of $Ks.  But, wow, nice specs.)  Lower-end units 
support ESC/P and mostly work with CUPS.  Buy the business models if you 
want a decent cost-per-page.


Canon printers can be decent, but are not usually good value for 
money... YMMV, just make sure you can return it if you can't make it 
work.


HP used to be very good, and their mono LaserJets are still acceptable.  
Color LaserJets... less good, but maybe still acceptable.  Inkjets?  Who 
knows, there's a new model every week, each incompatible with the last 
one.


Your best bet will likely be to purchase a new "business-grade" printer 
from a shop that will let you return it if you can't get it to work.


Unfortunately, YMMV depending on what part of the world you're in, any 
local promotions/sales, the phase of the moon, whether Jupiter is in 
alignment with Saturn, etc., etc.


Good luck,
-Adam

ipv6 nmap breakage under 6.5-STABLE ?

[Adam Thompson]

Hi,
[Cross-posted to misc & ports as I'm not sure if there's a bug in 
software or in wetware.]


I'm trying to run nmap (from ports) on 6.5-STABLE but am getting an 
ungoogle-able error message every time:


  root@bgpmirror:~# nmap -Pn -A -n --top-ports=100 -6 
2620:132:300e:700::113

  Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-22 09:43 CDT
  sendmsg: Message too long
  sendmsg: Message too long
  [...continues like this for a really long time...]

Interspersed semi-randomly (not always at the same point) I also see 
mixed in there a single occurrence:


  sendmsg: Message too long
  sendmsg: Bad address
  Unable to send packet in probe_transmission_handler: Bad address (14)
  sendmsg: Invalid argument
  Unable to send packet in probe_transmission_handler: Invalid argument 
(22)

  sendmsg: Message too long
  Unable to send packet in probe_transmission_handler: Message too long 
(40)

  sendmsg: Message too long

Performing a similar scan on an IPv4 address works as expected.  This 
appears to be a v6-specific problem.


Known issue?
Workarounds?

Thanks,
-Adam

Re: ipv6 nmap breakage under 6.5-STABLE ?

[Adam Thompson]

On 2019-07-22 09:51, Adam Thompson wrote:

Hi,
[Cross-posted to misc & ports as I'm not sure if there's a bug in
software or in wetware.]

I'm trying to run nmap (from ports) on 6.5-STABLE but am getting an
ungoogle-able error message every time:



Forgot to mention - this occurs under OpenBSD, but not Linux (CentOS 
7.6.1810 & nmap 6.40).  Obviously that's not an apples-to-apples 
comparison, but it's what I've got access to from here right now.

-Adam

Re: SCM

[Adam Thompson]

On 2019-07-23 12:43, Stuart Henderson wrote:

On 2019-07-22, Stefan Sperling  wrote:


If your university class prefers using git, I'd recommend the 
repository at

https://github.com/openbsd/src.


However, it doesn't include branches/tags, because we haven't found
anything that
is able to successfully convert the OpenBSD CVS repository to git
unless it ignores
these.


I haven't tried this with the OpenBSD code base, but in a previous life 
I did a CVS to Git conversion starting with a repo that resembled 
OpenBSD's in many ways.
The "solution" was to get to git by way of SVN.  SVN was able to 
preserve branches/tags/etc. from CVS into SVN, and was then able to in 
turn be converted to git through SVN's git-compatibility layer (IIRC).

Whether this helps anyone out there... *shrug*
-Adam

help understanding cua/tty EBUSY behaviour?

[Adam Thompson]

Summary:  I open cua0 with cu(1), quit cu(1), try to re-open with cu(1) 
but now it immediately fails with EBUSY.  *Usually* doesn't happen with 
USB-to-serial (cuaU[0-9]) but have still seen it once or twice.


I've seen this behaviour on OpenBSD 6.4, OpenBSD 6.5, and FreeBSD 11.2, 
and on 3 radically different systems (hardware-wise) so I don't think 
it's a version-specific or even hardware-specific bug, more likely 
something I'm failing to understand.


I'm using OpenBSD as a remote serial console server for up to 3 switches 
at a time (OOB access to a few switches in my lab).  This works, 
mostly... but occasionally, a serial port, almost always the onboard 
hardware cua0/tty0 port, somehow wedges and requires me to reboot the 
OBSD system to regain access to it.  The symptom is that when attempting 
to open(2) the device, I get EBUSY... for no obvious reason.  fuser(1) 
shows no other processes having a filehandle on /dev/cua00.


I don't understand why this happens inconsistently - about ~75% of the 
time on /dev/cua00, but only ~10% of the time on /dev/cuaU[0-1].  Of the 
~75% of the time it happens on /dev/cua00, about 1/3 of those times, if 
I wait a minute or ten, I can then re-open the device (again using 
cu(1)), and 2/3 of those times it persists until a reboot.


On the USB devices, I can - with 100% success - eliminate the problem by 
walking over, unplugging the serial adapter, and re-inserting it.


I haven't tried using e.g. screen(1) from ports, I've only tested using 
cu(1) in base so far.  I can try something else if there's a reason 
to... on the OpenBSD box anyway, it'll be a bit harder on the FreeBSD 
appliance.


As I said, I've even seen this on FreeBSD, so I expect I just need 
someone to provide an explanation of what nuance of tty(4) usage I'm 
missing.


Help?

Thanks,
-Adam

Re: help understanding cua/tty EBUSY behaviour?

[Adam Thompson]

On 2019-08-03 18:14, Theo de Raadt wrote:

Adam Thompson  wrote:


Summary:  I open cua0 with cu(1), quit cu(1), try to re-open with
cu(1) but now it immediately fails with EBUSY.  *Usually* doesn't
happen with USB-to-serial (cuaU[0-9]) but have still seen it once or
twice.

[...]

You are observing the forcing-down of DTR and RTS for a long enough
period that the other side of the link notices the event.

Therefore it is not suprising you are finding this behaviour very
similar in many drivers and operating systems.


Thanks!
I feel as though I'm seeing something that's slightly off (see below), 
but I was focused on the open() end of things not the close() end - I 
now need to do more reading.



FWIW, things that don't *feel* right about this:
* "long enough" doesn't typically describe multiple days, in DTE/DCE 
signalling.  Far-end device is the same when testing using cua00 vs. 
cuaU0.
* isn't cua(4) supposed to be the device we use to ignore line signals?  
Why would closing it fiddle with line status?  Maybe I'm reading too 
much into hupcl in stty(1) manpage...

* and now I'm wondering if cu(1) fiddles with [-]clocal and/or [-]hupcl

As I said, more reading now you've given me a direction and I'll 
probably come back with at least one or two of my own answers.


-Adam

bgpctl(8) community question

[Adam Thompson]

[OpenBSD 6.5-STABLE, up to date]

When using bgpctl(8), I'm able to do almost everything I need, but I'm 
having trouble figuring out how to do one thing:


How do I show routes that do NOT have a community (or ext-community, or 
large-community) attribute?


The best I can come up with so far is a fairly ugly AWK script that 
buffers the detailed route output, then emits it if it doesn't see a 
Communities: line.  Am I missing a better way?


Thanks,
-Adam

N.B. manually looking through N sets of DFZ route tables isn't going to 
happen, I need a mostly-automatic solution.

Re: Tools for writers

[Adam Thompson]

On 2019-11-02 11:14, Peter Nicolai Mathias Hansteen wrote:

2. nov. 2019 kl. 16:00 skrev Oliver Leaver-Smith :

What tools do people find useful for writing on OpenBSD? By writing I 
mean long form such as novels and technical books, including plot and 
character development, outlining, and formatting for publishing (not 
all the same application necessarily)


I have found a number which boast Linux support, but not really 
anything that stands out which supports OpenBSD (aside from the 
obvious LaTeX et al.)


I really can’t speak to plot and character development, but all three
editions of The Book of PF were written using OpenOffice and later
LibreOffice write on OpenBSD snapshots.

Earlier versions of that manuscript were developed using DocBook SGML
(editing with emacs), but the publisher (fortunately) did not want any
truck with that.

For any new projects I would likely look half-heartedly for something
markdown based but would probably end up going the LibreOffice route
again.



FWIW, Brian Kernighan's new book was written using groff(1), with final 
rasterization done by gs(1), obviously there's a number of other tools 
involved.


On the other hand, unless you name is Brian Kernighan (or possibly 
Kristaps, Ingo, or jmc@) I doubt that toolset will satisfy you :-).


A few people around here have used TeX, LaTeX, and LyX (a LaTeX 
front-end) all of which are very much capable of large projects split 
into sections/chapters/etc.
 AFAIK OpenBSD's LaTeX / TeX packages are all more than adequate to the 
task, and all of the necessary tools are in ports.


-Adam

Re: Is there an easier way to browse ports?

[Adam Thompson]

On 2019-11-01 06:12, Mischa wrote:

On 1 Nov 2019, at 12:08, Alfred Morgan  wrote:

My current workflow looks something like this:

$ cd /usr/ports
$ make print-index | less
I search and scroll through and find something interesting such as
opensonic.
I read the Info: game based on the Sonic the Hedgehog universe
^Z
$ cat games/opensonic/pkg/DESCR # I can't get make describe to work
I read more about it.
I google opensonic for screenshots.
$ pkg_add opensonic
$ opensonic
$ fg

Ideally I would like a graphical ports browser with name, screenshots, 
and
description that I can scroll and search through. Curation would be 
nice:

ports suggestions, popular ports, dev team ports picks, etc.

-alfred


Have a look at: https://openports.pl 

I think it ticks some of your boxes. :)

Mischa


Also http://openports.se/ and http://ports.su/ .

-Adam

Re: Is there an easier way to browse ports?

[Adam Thompson]

Oh, ok... Do you recall an example offhand?  (I haven't noticed systemic
problems with either, but then I'm hardly a ports expert!)Thanks,-Adam
On Nov. 7, 2019 07:18, Marc Espie  wrote:

  On Wed, Nov 06, 2019 at 04:44:48PM -0600, Adam Thompson wrote:
  > Also http://openports.se/ and http://ports.su/ .

  Don't use those, they don't know how the openbsd ports are named.

Re: Is there an easier way to browse ports?

[Adam Thompson]

Ah, there's a good answer to the question I just asked Marc, thanks!-Adam

password-less user (without bothering security(8))?

[Adam Thompson]

Hi,
On 6.6-STABLE, I'm looking at security(8) and it's not immediately 
obvious to me how I can have an SSH key-only user who does not have a 
password, that also does not trigger daily security warnings.


The goal is to have a user that can never log in on the console, or via 
password any other way (FTP, SMTP auth, POP, etc., etc.), but only via 
the RSA key provided.


Is there a way to placate security(8) that I'm just not seeing?  Or is 
my goal fundamentally misguided for some reason I'm not seeing?  The 
user in this case is semi-trusted (e.g. yes, we'll let you login using 
an unprivileged account to run bgpctl in pipelines) but not 
organizationally-trusted (i.e. but that's ALL we want you to do on this 
system).


Thanks,
-Adam

syspatch(8) return values?

[Adam Thompson]

There's no mention of what syspatch(8) returns, in the manpage.

I can prove quickly enough that it exits(0) when there's nothing to do, 
but I'm more interested in knowing (for automation purposes) what the 
return values are in other circumstances, and all my systems are already 
up to date.  Before standing up yet another system, I figured I'd ask 
here.


I can think of four scenarios syspatch(8) perhaps ought to distinguish, 
at least I'm interested in these 4 outcomes:

  1. nothing to do
  2. patches waiting, but didn't do anthing
  3. patches applied
  4. something went wrong

Can I reliably tell based on $? or do I have to parse the output?

Thanks,
-Adam

Re: suggestions for USB printer (maybe even with scanner)?

[Adam Thompson]

On 2020-02-05 13:56, Claus Assmann wrote:

I need to buy a printer to connect to one of my OpenBSD machines
and I prefer a USB connection (as I don't control the network at
my current place).  Can I just buy any USB printer or are there
printers which do not work with OpenBSD? If so, what do I need
to check / avoid?

Any suggestion for something "cheap" (to print just a few documents
as needed)? I never had to buy a printer before, so I'm not familiar
with this area -- if possible I would like to get a printer/scanner
but I have no idea what I can buy locally :-(
A HP laserjet (which was a gift but broke today) worked only with
one of my OpenBSD machines which seemingly was related to the USB
HW, using a printcap entry like this:
usb:lp=/dev/ulpt0:sd=/var/spool/output/usb:sf:sh:tr=^D:


I don't know what you need in a printer, and I don't know what you mean 
by cheap, so... YMMV.


However, I've found Brother **LASER** printers to be very good, and most 
of them support PCL6 and/or PS3.
For example, the HL-L2370DW can only connect via USB, and supports PCL6, 
and currently sells for ~C$150-160.


Just don't try to use their MFC-* line of color printers under UNIX 
(except MacOS).  FWIW, if you're in a situation where you have a spare 
Mac, the Mac can bridge from CUPS/PDF format to Brother proprietary 
format... bit pf a pain but it works.


-Adam

Re: Dell Latitude e6400 OpenBSD Drive Issue

[Adam Thompson]

On 2020-02-09 06:58, Michael G Workman wrote:

Hello,

Shout out to the OpenBSD developers for making a great OS!

I was able to install OpenBSD 6.6 on a Dell Latitude e6400 laptop, with 
a

USB Install. Sent the dmesg in already.

The installer would not recognize the hard drive, a brand new SSD 
drive.
The solution to that, from stack exchange, was to change the SATA 
settings

in BIOS from IRRTL to AHCI, that fixed the problem.

However if my laptop is powered off for a while, the SATA setting 
changes
back to IRRTL instead of AHCI, very annoying, not sure why the BIOS 
would

not make my changes persistent. I think it may be a hardware issue, but
just wanted to know if anyone else has encountered this before?

Thanks.

*Michael G. Workman*
(321) 432-9295
michael.g.work...@gmail.com


I have run several laptops from that series with OpenBSD.  The other 
replies are correct, your BIOS battery is dead.  Unfortunately, on many 
of the Latitudes, the BIOS battery is of the variety that's embedded in 
the RTC chip, and is not separately replaceable.
Some, however, including - the 6430 for example - have a regular coin 
cell, albeit wrapped in a proprietary cover with a non-standard 
connector, but at least is *is* replaceable without insane amounts of 
work.
I have the owner's manuals for many of the 6400 series, email me 
directly if you can't find the guide to replacing parts for your 
particular model.

-Adam

Re: Dell Latitude e6400 OpenBSD Drive Issue

[Adam Thompson]

On 2020-02-10 09:36, Michael G Workman wrote:

Ok, thanks for the info.


For your E6400, see this guide: 
https://www.parts-people.com/blog/2012/10/16/dell-latitude-e6420-cmos-battery-removal-and-installation/


I found E6400 CMOS batteries from multiple vendors on the first page of 
Google results, most around US$10.  The only recommendation I can give 
is that I've used Parts-People.com in the past, no complaints with them.


Make sure you get the right replacement - check if your battery is 
2-wire or 3-wire *before* ordering a replacement - many of the listings 
don't worry about such minor details, g.


The older the Latitude, the harder it is to open, but even an E6400 is 
pretty easy, even if you've never opened up a laptop before.


Good luck,
-Adam

Re: syspatch(8) return values?

[Adam Thompson]

On 2020-02-08 06:03, Antoine Jacoutot wrote:

On Fri, Jan 31, 2020 at 09:03:59AM -0600, Adam Thompson wrote:

There's no mention of what syspatch(8) returns, in the manpage.

I can prove quickly enough that it exits(0) when there's nothing to 
do, but
I'm more interested in knowing (for automation purposes) what the 
return
values are in other circumstances, and all my systems are already up 
to

date.  Before standing up yet another system, I figured I'd ask here.

I can think of four scenarios syspatch(8) perhaps ought to 
distinguish, at

least I'm interested in these 4 outcomes:
  1. nothing to do
  2. patches waiting, but didn't do anthing
  3. patches applied
  4. something went wrong

Can I reliably tell based on $? or do I have to parse the output?


Most likely parse, yes.

"patches waiting, but didn't do anything" might be interesting (i.e 
patches are

available); dunno...


Equally if not more interesting would be "I tried to apply patches but 
failed somehow".  Which happened to me, and which is why I asked in the 
first place.
*sigh*  I'll try to propose some bad diffs [if I'm writing them, they'll 
be bad] someday soon.

-Adam

door handles

[Adam Thompson]

None of the Taymor levers are quite right.  So I went looking, and I 
found some of what I'm looking for.


Short list:

(top pick)
1. Omnia 762, plus privacy bolt.  I love it but holy shit that's 
expensive @ ~US$180ea!

https://www.omniaindustries.com/product/762/

2. Rocky Mountain Hardware's "BELLA" lever, included in door set, price 
unknown, distributor/retailer unknown.

https://www.rockymountainhardware.com/products/door-hardware/handles/levers/door-levers/bella-lever-l150


3. Emtek (an Assa Abloy company)'s "Spencer" Lever.  Price unknown.  
Lots of local retailers.

https://emtek.com/passage-privacy-levers/spencer-brass-lever

4. Schlage/Dexter J-series "Dover" lever.  Price: cheap.  Local 
availability unknown.

https://www.schlagecanada.com/en/home/products/J40DOVFFF.html#

5. Schlage/Dexter "Jazz" level.  Price: cheapish.  Local availability 
unknown.

https://www.schlagecanada.com/en/home/products/F40JAZFFF.html#

(bottom pick)
6. Weslock "Bristol" or "Bristol UL"
http://weslock.com/product/bristol-2/ or 
http://weslock.com/product/bristol-ul-2/

Re: What hardware spec would I need to push 20 gigabit of network traffic on an OpenBSD server?

[Adam Thompson]

On 15-10-25 03:46 AM, Some Developer wrote:
I'm just wondering what hardware spec I'd need push 20 gigabits of 
network traffic on an OpenBSD server?


Short answer:  It's not generally possible today, at least for your use 
case.


Medium answer:  Contact Esdenera Networks to find out.  They manage to 
do it somehow.  I'm sure they'll be happy to make it happen for you in 
exchange for suitable amounts of money...



Longer answer:

Network performance research numbers have presented by gnn at various 
conferences over the last year or so, and they consistently showed that 
OpenBSD, while performing well for a single-threaded stack, fell badly 
behind in multi-core, and wasn't able to keep up to 10Gbps.  The OpenBSD 
team is (currently, AFAIK) working on making the network stack 
multi-threaded, or at least not giant-locked, which should (eventually) 
dramatically increase performance scalability.


On top of that, there are substantial optimizations possible; research 
in the FreeBSD camp (and experience under OpenBSD as well) has shown 
that seemingly-similar hardware can perform radically differently.  
Drivers make a big difference.


You talk about storing the data - *writing* data to disk at 10Gbps 
(sustained) is currently in the realm of high-energy physics, with 
multi-million-dollar budgets for the storage arrays.  A 7200rpm disk can 
charitably be said to write at up to 100MBytes/sec, but that's not 
necessarily sustained speed, so minimum 10-unit array assuming 100% 
ideal throughput, which doesn't actually exist in the real world.  More 
likely you'd have to buy a large HDS array to get that kind of 
throughput.  Plus, that's about 2.5PB (yes, PETAbytes) of data every 
month.  Are you building this for the NSA?!?


You do realize that this means you're now trying to push *30* Gbits/sec 
on a single server, right?  (10 in, 10 out, 10 logged) Even Netflix, who 
spend a ridiculous amount of time doing optimization, have only recently 
gotten FreeBSD servers with tons of custom code and tweaks to pass the 
65Gbps-per-socket mark.


Lastly, Gbits/sec isn't the bottleneck.  The bottleneck is 
packets-per-second.  If you're pushing 10Gbps worth of 1500-byte 
packets, then this is possible today. (Not sure about 30Gbps.)  If 
you're trying to push 10Gbps worth of 64-byte packets on commodity 
hardware, forget about this pipe dream for another few years until the 
fully-MP network stack is finished and optimized.


Good luck... but you might want to consider doing this on a Juniper MX 
series or Cisco ASR instead - those platforms can at least maybe do the 
tunnelling part for around $250k, then feed the output into a 10GE 
switch with port mirroring (~$10k), then a Network Flight Recorder or 
similar to actually capture that much data (~$150k).


-Adam

[OT] Re: What hardware spec would I need to push 20 gigabit of network traffic on an OpenBSD server?

[Adam Thompson]

On 15-10-27 02:53 PM, Martin Schröder wrote:

And then there are SSDs. PCIE SSDs do up to 3000 MB/s write throughput.
https://www-ssl.intel.com/content/www/us/en/solid-state-drives/solid-state-drives-dc-p3608-series.html

And I'm sure there are tape libraries that can write that, too. :-)

I disregarded that part, based on the unsustainable assumptions elsewhere.
Plus, how much would 2.5PB of PCIe SSD cost?!?!?

As to tape libraries... interesting idea.  An LTO-6 drive can write at 
~160MBytes/sec, and that's pretty much the top end of 
generally-available tape drives today, so you'd still need an 8-drive 
RAIT-0 setup.  After taking into account library changer latency, I'd 
guess you'd need a custom buffering solution, maybe one system to shard 
the data, then ~10 systems to buffer the data to ~10 tape drives with a 
big enough buffer to sustain the ~2-4min it can take to eject and load a 
new tape in a big library.  Perhaps that's where the PCIe SSDs come in? :-)
(10Gbits/sec ~= 1 Gbyte/sec ~= 60 Gbytes/min... you could probably get 
away with a 256MB SSD)


A 12-drive StorageTek LTO6 library with ~1000 tapes costs around $2M 
last I heard, but at least the tape cost is reasonable once you get 
going.  Of course, you also need the multi-path FC networking gear for 
that, and I think OpenBSD just dropped FC support.


An alternative might be a z-series mainframe, I hear they have massive 
I/O capabilities.  But they don't (AFAIK) run OpenBSD.


-Adam

Re: What hardware spec would I need to push 20 gigabit of network traffic on an OpenBSD server?

[Adam Thompson]

Thank you for the reply. I see now that my request was wildly 
unrealistic.

Not "wildly", just unrealistic unless you have a massive budget.

Basically I'm trying to write a business plan and am trying to plan 
for the worst case scenario so I don't fall over if traffic somehow 
spikes to such levels. My expected level of traffic is probably in the 
300 megabits a second range but the incoming links from my upstream 
provider are 10GbE so I need to have some plan just in case I get a 
spike of 10 gigabits (as unlikely as that may be I still need to plan 
for it).
I would suggest simply limiting your incoming traffic... and paying for 
less bandwidth.  Most providers are quite happy to sell you, e.g. 4Gbps 
capacity on a 10Gbps physical line.  That moves the problem back a layer 
to the "someone else's problem" domain.


I haven't ruled out the possibility of using Cisco / Juniper for some 
of my requirements but obviously would like to use OpenBSD if possible 
because I've used it in the past and it includes everything that I 
need and best of all the documentation is excellent.
I think you might be able to do 10Gbps of L2TP traffic on Linux or 
FreeBSD (on commodity hardware) if you use one of the highly-optimized 
networking stacks that are available (e.g. DPDK). But even they won't 
handle it gracefully.
I'm not even certain that Cisco or Juniper gear will decapsulate L2tP 
traffic at that rate.


I can drop the logging requirement pretty easily. That isn't really 
important at all.


I wonder what other people do when they are disaster planning for 
their new services? At the same time I wonder how the internet 
backbone is handled. As far as I am aware they handle speeds of about 
100 gigabits a second so the hardware / software must be available for 
handling such speeds. I just guess they are ridiculously expensive to 
buy and maintain (well outside my budget anyway).

You build-to-budget.
The 100Gbps backbones/routers you hear about are a) few and far between, 
b) insanely expensive (multi-millions of dollars), and c) only shuttling 
packets as-is from one port to another.
Everyone else just buys what they can afford, and then they take steps 
(like throttling) to ensure their servers don't fall over.  Of course, a 
lot of people just let their servers fall over, too...


Also: load-balancers.

-Adam

Re: pfctl -f /etc/pf.conf fails on boot when DNS-resolved symbolic names are used

[Adam Thompson]

On 15-11-10 01:45 PM, Giancarlo Razzolini wrote:
As a general rule you should avoid using dns names on anything that 
might cause the boot process to fail. Even more, you should really 
avoid using names on hostname.if files.

Anybody run into this before?  - is the fix to add all the symbolic
names to /etc/hosts?

Well, if the hosts have fixed addresses, you'd be better using macros on
pf.conf that translate to their IP address. This way you won't run into
boot issues (or reload issues, in case your resolver is down). This has
the added inconvenience that you need to update your pf.conf file
manually every time one address changes.

Now, if you really, really need to use fqdn's on pf.conf, my suggestion
is that you use ifstated to detected if your link is up and your
resolver working, and them load the rules into an anchor afterwards.
Also, you can update the anchor to reflect any uplink unavailability. Or
you can use unbound with local-zones or a unbound + nsd combo, if you
also need authoritative. I think you'll need to hack your /etc/rc file
to load them before your pf.conf is loaded.



FWIW, yes, putting the entries into /etc/hosts *will* work, and it 
avoids the need to use pf.conf macros, ifstated, etc.


However, it now means that you have to ensure /etc/hosts remains 100% 
accurate... although I shudder to think of using ifstated and anchors to 
do this, it does avoid the /etc/hosts maintenance problem.  And make no 
mistake: you *will* eventually forget to update /etc/hosts.  Absolutely, 
100% guaranteed.


-Adam

Re: Problem with CARP interfaces not responding until VHID is changed.

[Adam Thompson]

On 16-01-21 04:02 PM, rizz2pro . wrote:

I know the CARP interface's MAC address is generated by the VHID so I am
sort of leaning towards it be an ARP issue and possibly not an issue with
the OBSD system. But I am hoping for some hints or ideas from you guys.
I have a suspicion... what kind of switches are the OpenBSD hosts in 
question connected to?  If they are managed switches, have you looked at 
the logs on the switch?


Some switches interpret CARP (and VRRP, for that matter) as either an 
attack or a misconfigured system, and will temporarily block traffic to 
one of the two ports.  Essentially, they think it's "MAC flapping"... 
which it sort of is, in fact.


-Adam

Re: VAX - are we dropping support in 5.9?

[Adam Thompson]

On 16-01-23 08:34 PM, Ted Unangst wrote:
I will add that one of the reasons we have support for all these 
museum pieces is that people can build their very own museum and run 
something interesting on it. But running on emulators doesn't really 
satisfy that goal. If there are, in fact, no museum pieces left in the 
world, we no longer need to supply an OS to run on them. 


Huh.  Previous discussions had led me to believe that the OpenBSD 
project's rationale for supporting all these various architectures was 
that it ultimately resulted in much-higher-quality code because 
platforms like VAX and SPARC64 acted as canaries for suboptimal coding 
practices?  (Endian issues, stack issues, framing issues, alignment 
issues, etc., etc., etc.)


Besides, I thought the run-on-everything-and-anything (including the 
verging-on-absurd) was NetBSD's thing, not OpenBSD's?  See 
http://netbsd.org/ports/, make your own opinions on which platforms 
verge on the absurd...


-Adam

[PATCH INCLUDED] Re: Problem with CARP interfaces not responding until VHID is changed.

[Adam Thompson]

On 16-01-25 03:43 PM, rizz2pro . wrote:
> Ok we've figured it out.
>
> We have a couple identical environments all attached to one switch and
> they are all advertising the same VHIDs to each other and it looks to
> be causing some arp problems. (Environment A was getting CARP
> advertisements from Environment B and vice versa)
>
> After specifying a "carppeer" on each CARP interface attached to that
> switch in all 3 environments the issue went away. All 3 environments
> are using the same VHID.
>
> Thanks for the help

Umm... per http://www.openbsd.org/faq/pf/carp.html

> /vhid/
> The Virtual Host ID. This is a unique number that is used to
> identify the redundancy group to other nodes in the group, and to
> distinguish between groups on the same network. Acceptable values
> are from 1 to 255. This must be the same on all members of the group. 

Note the word "unique".  Basically, by having multiple clusters with the 
same VHID, you deliberately broke carp(4).  VRRP would have broken in 
exactly the same way, and I think HSRP would also have broken in same 
way, too.

However, I see that neither carp(4) nor ifconfig (8) indicates that VHID 
should be unique and not shared, except by very weak implication.
If specifying a common VHID is not only contraindicated, but causes 
actual breakage on the network (as seen here), then the manpage(s) 
should say so, IMO.  (IIRC, the manpages are canonical, not the FAQ...?)

Drat, I don't have a copy of -current on this system... following is a 
proposed diff against 5.8-RELEASE.  Sorry for not doing a proper diff 
against -current but if I wait, I'll forget.  At least this way I might 
remember later.

I know in VRRP, the VHID is used to generate the MAC address, but I 
don't recall if carp(4) works the same way.  If it does, then the 
language I suggest for carp(4) below may be too permissive, in that use 
of carppeer will stop each system from complaining, but external clients 
will still encounter difficulties with ARP.

-Adam

--- carp.4.dist 2016-01-25 18:04:39.152065865 -0600
+++ carp.4  2016-01-25 18:08:14.326975564 -0600
@@ -58,6 +58,10 @@
  a common virtual host ID (VHID) and
  virtual host IP address on each machine which is to take part in the virtual
  group.
+The VHID uniquely identifies a cluster locally within a broadcast domain
+(network segment), but may be reused on other networks.  The
+.Cm carppeer
+option may also be used avoid conflicting VHID multicasts.
  Additional parameters can also be set on a per-interface basis:
  .Cm advbase
  and

--- ifconfig.8.dist 2016-01-25 18:15:07.089080002 -0600
+++ ifconfig.8  2016-01-25 18:14:35.745168361 -0600
@@ -849,6 +849,7 @@
  Set the virtual host ID to
  .Ar n .
  Acceptable values are 1 to 255.
+Clusters on the same network should use unique IDs.
  .El
  .Pp
  Taken together, the

Re: Willing to help

[Adam Thompson]

On 16-01-26 10:32 AM, Peter Hessler wrote:

On 2016 Jan 26 (Tue) at 08:13:22 -0600 (-0600), Edgar Pettijohn wrote:
:> * adduser(8)/useradd(8):
:>   Needs to be unified into one single

One binary, with symlinks.  Both methods should still work, however.


$0.02:

s/sym/hard /g

might satisfy a larger percentage of people...

;-)

-Adam

Re: Can I accelerate my magnet HDD using a SSD in any way?? E.g. softraid patch/ARC, dedicated hardware e.g. Intel RCS25ZB040LX="Nytro MegaRAID", anything

[Adam Thompson]

On 16-02-01 12:19 PM, Tinker wrote:

My purpose with asking for SSD-accelerated HDD was DOUBLE:

 1) I need some SSD storage but don't like that it could break 
together - I mean, a bug in your system will feed your SSD at full 
bandwidth for ~7h-7 days, it's completely fried - that's not OK, so 
putting a "redundance layer" in the from of an underlying magnet 
storage layer is really justified.


...so what?  I don't understand what your concern is here.  If you buy a 
batch of HDDs from the same lot, they'll probably all die around the 
same time, too.  I don't think I understand what you're trying to say.


If you're worried about SSD write endurance, I'd say... don't.  Or buy 
better-quality SSDs ;-).


See http://ssdendurancetest.com/  for real-world stress-test data. 
Especially 
http://ssdendurancetest.com/ssd-endurance-test-report/Intel-SSD-520-60.


The Intel DC P3700 series should last about 1.5yrs in an absolutely 
worst-case scenario.  Even the worst-case DC series I could find should 
still last about a month and a half under murderous stress-test conditions.


http://estimator.intel.com/ssdendurance/

This report 
(http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead) 
confirms that even if you completely fill an SSD, that's no big deal.


If your application has a "bug" it's unlikely the sort of bug that would 
fill the entire SSD, erase it all, fill it again, erase it all again, 
etc... and most SSDs can handle ~3000 full-device writes nowadays.


Also remember to factor in the fact that UFS/UFS2 doesn't (usually) 
saturate a block device with I/O write requests.  Maybe if it was just 
writing to a single enormous file it might, but could that actually 
happen in your situation?



2) I need some bulk storage, and I want the terabytes to be really 
cheap so that i NEVER will run out of archival space. An 8TB magnet 
HDD costs in the range USD 500.


Here, I like it to be stored "symmetrically" with how I store the 
other stuff, that is having separate disks, directories, mount points 
etc. for the two doesn't really appeal to me in this particular case -


Simply knowing that the less frequently accessed data will be 
taken from magnet and the more frequently accessed data from SSD seems 
both convenient and practical for my usecase, and, I'll try to have 
SSD volume to cover for *MORE THAN ALL* of my frequently stored data.


Perhaps knowing the prioritization algorithm as to be able to 
calculate more closely what's in the SSD and what's on the HDD only 
would make some sense, BUT, training it by telling it what's to load 
fast using "cat `find the-relevant-data/` > /dev/null" single-shot and 
perhaps via crontab, should deliver really well.


Cheap storage is a valid argument.  HDD vs SSD is still easily an order 
of magnitude difference in price.


So go run FreeNAS (or TrueNAS, or Solaris, or FreeBSD, or even 
Linux...), run ZFS with both a L2ARC and ZIL device on fast SSD, and 
remote-mount the directory.  You even have a choice of protocols :-).  
(NFS or iSCSI.  I suspect you'd want to use NFS, but YMMV.)


Short answer: I don't think any part of OpenBSD does what you're asking, 
natively.  Some supported hardware devices (like the MegaRAID Cachecade, 
et al.) do this for you.  But at that point, what's the difference 
between having the disks hooked up to some speciality hardware in an 
OpenBSD box, versus having the disks hooked up to generic, not-special 
hardware on a non-OpenBSD box? (Particularly given that UFS/UFS2 on 
OpenBSD isn't exactly known for its award-winning performance in the 
first place?)


FYI, even if Intel SRT was available on a Xeon board, you'd still be out 
of luck, because it requires device driver support  - it relies on the 
Intel fake-raid technology which, IIRC, is not supported at all under 
OpenBSD.


I'd also like to see something like this available in OpenBSD's VFS 
layer, but I'm not holding my breath.


IIRC there was some interest in integrating Hammer/Hammer2 into OpenBSD 
instead of ZFS due to licensing issues, but even Hammer2 doesn't appear 
to have tiered storage in its feature list.


Some more random thoughts:
- the Dell H800 supports up to 1GB NVRAM, which is just a CacheCade 
implementation.
- CacheVault is the newer version of CacheCade (AFAICT), and is 
supported on newer MegaRAID cards.  See 
http://www.avagotech.com/products/server-storage/raid-controllers/cache-protection#overview 
for some pretty skimpy info.
- moving your data from HDD to SSD isn't going to dramatically speed up 
READ operations.  Carefully examine the specsheets on both the HDDs and 
SSDs and compare read bandwidths.  Access time (latency) is where SSDs 
are useful, along with acting as write-back caches.
- Areca raid controllers support up to 8GB of cache and a sh*tload of 
SAS ports; this might be good enough for your needs - have a look.



Either iSCSI or NFS against a ZFS-based NAS is probably still your best 
bets f

Re: OpenBSD SPARC T4-1 softraid boot issues

[Adam Thompson]

On 2017-12-26 14:56, Jordan wrote:

I've recently gotten my hands on a couple shiny new SPARC T4-1 and
T3-1 servers and I was looking to install OpenBSD with a softraid
mirror on them for production use. The problem is, is that I end up
with this upon following the install instructions and rebooting:


FWIW...
AFAIK every single T4-1 (probably T3-1s but not certain) is capable of 
hardware RAID-0/1/10 on the SCSI controller, and that's how they're 
shipped from Sun/Oracle even when running Solaris.  I'm not absolutely 
certain that it's supported by OpenBSD, but I do recall seeing that it 
could be used as-is when I was investigating running OpenBSD on my 
T-series gear.  There's Oracle documentation on how to access the 
on-card firmware to set up and manage the RAID set in a pre-OS 
situation.


YMMV.  Unfortunately, both of mine are still running Solaris so I can't 
confirm right now.

-Adam

Re: OpenBSD SPARC T4-1 softraid boot issues

[Adam Thompson]

I'm afraid I don't know the answer to that, sorry.  Not sure if anyone does... 
I think the SCSI HBA driver is the same on SPARC64 as on AMD64, but then Sun 
had a habit of "improving" control interfaces for Solaris so who knows.  I do 
know under Solaris there is a bioctl-like tool to manage it, for what that's 
worth.
-Adam

On December 28, 2017 1:32:40 PM CST, Jordan Geoghegan  
wrote:
>Yes I had considered using the onboard hardware raid, but I don't 
>particularly trust it. I also need the ability to rebuild my arrays 
>while the machine is online and was hoping to be able to do the 3 disk 
>RAID1 offered by OpenBSD softraid. Do you know if bioctl(8) is capable 
>of controlling the onboard raid controller, or will I need to do all 
>raid rebuilds via the hardware raid bios on the T4?
>
>
>On 12/28/17 08:58, Adam Thompson wrote:
>> On 2017-12-26 14:56, Jordan wrote:
>>> I've recently gotten my hands on a couple shiny new SPARC T4-1 and
>>> T3-1 servers and I was looking to install OpenBSD with a softraid
>>> mirror on them for production use. The problem is, is that I end up
>>> with this upon following the install instructions and rebooting:
>>
>> FWIW...
>> AFAIK every single T4-1 (probably T3-1s but not certain) is capable
>of 
>> hardware RAID-0/1/10 on the SCSI controller, and that's how they're 
>> shipped from Sun/Oracle even when running Solaris. I'm not absolutely
>
>> certain that it's supported by OpenBSD, but I do recall seeing that
>it 
>> could be used as-is when I was investigating running OpenBSD on my 
>> T-series gear.  There's Oracle documentation on how to access the 
>> on-card firmware to set up and manage the RAID set in a pre-OS
>situation.
>>
>> YMMV.  Unfortunately, both of mine are still running Solaris so I 
>> can't confirm right now.
>> -Adam

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Dell Latitude E6540 OpenBSD 6.2 amd64 freezes when adjusting refresh rate using xrandr

[Adam Thompson]

On 2018-03-20 15:18, Xianwen Chen wrote:

Dear Mihai,
Although your tone in your email was not pleasant,


You are posting to OpenBSD-misc.  Objectionable tone is very common, 
particularly for users who *appear* to be complaining about 
immeasurably-small problems that aren't actually significant in the real 
world.  If you wish to remain an OpenBSD user, get used to people being 
rude.  (I am not going to speculate whether this is good or bad, but it 
is the case.)



If you are right that humans are not able to see the difference
between 60 Hz and 59.95 Hz, then something is wrong with xrandr that
the actual refresh rate is quite below 60, not as much as 59.95 as
reported by xrandr, because I can clearly see the flickering. I do not
think that this is a minimal thing, because the flickering screen
makes my head dizzy.


Then my suggestion would be to replace the lights in your room, not try 
to fix a 0.05Hz deviation.  The vast majority of systems I own report 
that the LCDs actually run at 59.95Hz; I've only seen one or two that 
ever reported 60Hz.  This is normal.


In a worst-case scenario, room lighting that is at a very slightly 
different frequency can cause odd effects, this is known as 
interference, and can produce a "beat frequency".  If the difference is 
small it's possible you could experience this as flickering.  (I believe 
the flickering is actually an neuro-optical illusion, but you might 
still be experiencing it.)



I do not think that there is problem with the connection cable, as
there was no such problem when the same external monitor is connected
to a ThinkPad R52 using the same VGA cable a couple of days ago. I can
check the cable connection again tomorrow.


That is irrelevant.  You have measured one VGA driver against a 
completely different VGA driver.  Different laptops = different 
electronics = different results.



I am sceptical whether there is any other source of distortion. I
don't know where to start if there should be distortion.


Fluorescent lights, or cheap LED lights - anything that needs a ballast 
or rectifier.  Any of these things can cause not only the beat frequency 
optical interference mentioned above, but ALSO can cause electromagnetic 
interference in the cable.  Oh, and cheap USB chargers are another 
common source of really bad EM interference.


And guess what's almost immune to this type of EM interference?  HDMI, 
DVI-D or DP cables.  Guess what's REALLY susceptible? VGA cables.  Hmm.



The exact frequency of a monitor, regardless of type, is almost always 
irrelevant to human eyes.  Whether it's 30Hz, 47.8Hz, 59.95Hz or 60.0Hz


As you are apparently experiencing real problems, I would check, in 
order:
1. your cable - switch to HDMI (or HDMI->DVI or HDMI->DP) and get rid of 
VGA **immediately**.
2. your lights - try it with all the lights EXCEPT regular incandescent 
/ halogen lights turned off.
3. your eyes - get a thorough examination by a doctor; there are some 
rare conditions that could cause odd things like this.
4. your brain - make sure you don't have a brain tumour (yes, I'm 
serious, it can cause things like this!)



As to the VGA cable - this is such an important point that I agree with 
Nick - please go away and don't complain further until you have switched 
to a digital connection of some sort.  I recall that my Dell E6430 was 
quite capable of producing so-called "120Hz" digital signals, and yours 
is a generation newer than that.  I am 99% certain that your problem 
will go away with a different cable.  (If you want DVI + DP connectors 
instead of HDMI, buy a Dell E-series dock on eBay for $50.)


-Adam

Re: How to unlock a serial port

[Adam Thompson]

[Replying directly as well, as I believe my MTA is still blacklisted by 
the OpenBSD mail server. Guess we'll find out! -Adam]


On 2021-01-17 20:09, Tilo Stritzky wrote:

On 14/01/21 17:38  Andrew Grillet wrote:

Hi

I am running OpenBSD on a T2000 (Sparc64).
I was trying to use the serial port from the primary domain, connected 
via

ssh, and my network lost the connection.
My tty00 is now locked:
jay# stty -f /dev/tty00
stty: /dev/tty00: Device busy
I do not want to reboot the primary, as the guests are running various 
live

services. I cannot find evidence of a lock file in /dev/spool/lock.
Is there a way out of this predicament?


fstat(1) is your friend here.
Note that each tty has a corresponding cua device, they're both under 
the

same lock.

tilo


I ran into this exact problem last year.  It'll be in the list archives.
According to Theo (if I understood him correctly) it's partly due to the 
way BSD serial ports have always worked, i.e. in a rather 
under-specified manner.
Apparently the core tty(4) code around this particular symptom hasn't 
really changed at all since OpenBSD forked.


My solution was to install Linux, sorry - I never did find a way around 
the problem on OpenBSD.


-Adam

Re: How to unlock a serial port

[Adam Thompson]

On 2021-01-19 19:15, Nick Holland wrote:

On 1/19/21 4:35 PM, Adam Thompson wrote:
I ran into this exact problem last year.  It'll be in the list 
archives.
According to Theo (if I understood him correctly) it's partly due to 
the

way BSD serial ports have always worked, i.e. in a rather
under-specified manner.
Apparently the core tty(4) code around this particular symptom hasn't
really changed at all since OpenBSD forked.


I'm curious what you think "this exact problem" is.


Based on later messages in the thread, it's clear I did *not* have the 
exact same problem, after all.
I was experiencing ttys that remained busy after the process that opened 
them had closed.


So, for example, if I used "screen /dev/cua0 9600", after exiting 
screen(1), confirming with ps(1) that all screen processes were gone, 
and confirming with lsof/fstat that nothing still had either cua00 or 
tty00 open, if I re-ran "screen /dev/cua00 9600" - or ANY other 
application, including cu(1) - the new process would simply block.
ktrace(8) confirmed that the open(2) call was simply blocking... 
forever.  I could reproduce it with cu(1) as well as screen.  Don't 
think I tried any other apps.


I recall that someone suggested at the time that it might be signal line 
related, but it didn't matter whether I had full RS-232 CTS/RTS and/or 
DCD/DSR signalling, basic 3-wire signalling only, or even if I unplugged 
the cable entirely, the blocking behaviour stuck around until I 
rebooted.  Granted, rebooting on a single-port terminal server isn't the 
end of the world, but it was a PITA that I got tired of.  (Oh, if I used 
a USB serial adapter, unplugging and replugging the USB dongle also 
cleared the problem.)




I'm not going to say serial support in OpenBSD is perfect, but it works
Darned Well,


Mostly, I agree.  I still don't know what was different about this one 
use case.  Pretty sure it wasn't hardware - I reproduced it on three 
vastly different amd64 systems, and with both traditional UARTs and USB 
dongles.  Pretty sure it wasn't (Exclusively) a PEBCAK issue.  But still 
not sure what it was.


I'll try again at some point, and I'll post if/when it happens to me 
again.


-Adam

Re: Join two overlapping subnets with two way NAT/BINAT

[Adam Thompson]

> Due to a company take over I have two networks, NetA and NetB, that
> I need to link together for bi directional data sharing etc.
> Unfortunately both networks use the same IP addressing scheme i.e.
> 172.16.10.0/24 and neither can changed within the timespan available
> to me.


For what it's worth, this is one thing that Cisco PIX firewalls actually 
do very well.  I won't use them for ordinary internet-facing firewalls, 
but for your scenario they're perfect.

-Adam Thompson
 athom...@athompso.net

Re: responding to buttonpress ACPI event sent by KVM/Qemu (same behavior in v5.2)

[Adam Thompson]

> Resurrecting this thread, since I also got a total freeze in
> Qemu/KVM after sending ACPI shutdown using OpenBSD 5.3.
> 
> Testing, I also tried 5.3 in VirtualBox and VMWare, both gave clean
> shutdown.  This makes me suspect KVM is at fault...
> 
>  QEMU/KVM (echo system_powerdown | nc -U /tmp/test1.monitor
> )
> ---> total freeze
>  VirtualBox ("ACPI shutdown") --> clean shutdown
>  VMWare Fusion ("Shutdown") --> clean shutdown
> 
> FWIW, I'm using an illumos/omnios host on AMD hardware.

Probably not purely a KVM problem; I'm running OpenBSD 5.3-RELEASE inside KVM 
(ProxmoxVE 3.1 platform) on Intel CPUs, and ACPI handling works correctly.  I 
also recall testing OpenBSD 5.2-RELEASE inside RedHat Enterprise Virtualization 
(also KVM-based) and it worked fine there on AMD chips.
The only HVM-related issues I've been able to find are a) speed [no surprise 
there], and b) the long-standing red-console-background effect when switching 
VTs.

This implies that you're seeing a bizarre interaction between your guest, your 
host, and the VM software.  It's remotely possible your hardware has something 
to do with it, but unlikely.  I haven't tested anything illumos-based.

-Adam Thompson
 athom...@athompso.net

Re: Looking for good, small, canadian version laptop suggestions

[Adam Thompson]

Hi guys,

I am looking for some suggestions for a good, small quite laptop. I was
looking at futureshop.ca and bestbuy.ca. I currently have an HP dv3


For OpenBSD, I would never buy something at FutureShop or BestBuy; those 
are all consumer-oriented "Designed For Windows 8" laptops.


I either buy Lenovo ThinkPads from an authorized reseller (e.g. the 
x201t sitting in front of me, and many of the OpenBSD developers use 
various models of Thinkpad), or I buy off-lease (trailing-edge) Dell 
Latitude/Precision laptops directly from Dell - see www.dfsdirect.ca for 
their off-lease selection.


The Latitude E4000 series are all quite small and light, are readily 
available, and AFAIK are fully supported.  Right now I'm running 
5.3-RELEASE on a Latitude D630 with no issues at all, and IIRC the E4500 
should be fully supported as well.


Many people cringe at the thought of a used laptop, but note that DFS 
will offer a 1-year warranty, which is exactly what you get buying 
consumer-grade laptops from a retail big-box store anyway. My favourite 
part of the Latitude E series (and most Precision models, too) is that 
if you get the optional docking base, you can then run dual-DVI off the 
laptop!


--
-Adam Thompson
 athom...@athompso.net

Re: Looking for good, small, canadian version laptop suggestions

[Adam Thompson]

On 2013-10-12 06:01, g.lister wrote:
Interesting. I always feel that I am getting ripped off when buying 
something refurbished but then again I find my stuff which I bought 
many years ago still works and is easier to install stuff on (things I 
care about anyway) and now when looking around I find the new stuff 
has some major improvements which might come in handy (graphics, CPU, 
faster RAM) if I settle for the off the shelf stuff (Win* or OS X) but 
since I don't I have to poke around more to find what I like.


I guess I should look as well on refurbished stuff and they come with 
a warranty, isn't it usually shorter? Replacing a hard drive and 
adding some more ram plus the right OS may make it into a livable 
solution. At the end one uses the software. My old Sony is kind of 
like that lots of things will never work, read webcam, but overall it 
has proven to be a well made laptop. I also got a more recent Dell, 
XPS I think, for my significant other and that one is also quite good 
it has sustained mass impact from some kid handling and is still running.




As I said already, buying a consumer-grade laptop new from your local 
big box retailer generally gets you a one-year warranty.
Whereas buying a refurb laptop from a reputable supplier (such as Dell 
Financial Services, in both Canada & USA) gets you a ... one-year 
warranty :-).


You are not getting cutting-edge equipment.  But in the case of running 
*anything* other than OS that comes loaded on the laptop, that's a 
*good* thing, not a bad thing.  I can't even run Windows 7 properly on 
the vast majority of laptops I can buy at Best Buy today, why would I 
expect to be able to run OpenBSD?  Whereas anything refurb is generally 
far enough behind the trailing edge that the drivers are already 
built-in to the OS.  I can install Win7 onto a Latitude E4500 and 99% of 
the drivers will work out of the box.  Maybe I don't get the absolute 
maximum set of functionality, but everything works.  I can also install 
OpenBSD onto a Latitude E4500 and get the same level of functionality.  
(Assuming you connect to Ethernet at first, to auto-download the 
Broadcom wireless firmware during first boot.)


Keep in mind that although you aren't getting the latest CPU, that's 
mostly irrelevant today - and especially so for OpenBSD.
You aren't getting "ripped off" when buying from DFS, because they're 
only charging you (roughly) 1/n of the original price, where n = 
laptop_age_in_years.  Those $299 deals they have for 3-year-old laptops 
are mostly for units that cost around $1500 brand new!


Right now, DFS Canada has several laptops with 8GB of RAM for under 
$800.  How much more would you like to put into it?!?  Only the very 
newest laptops can take more than that anyway!


Also, buying business-grade laptops is a sound investment because you 
don't have to replace them as often.  In my experience, the average 
consumer-grade laptop (including Dell Inspiron and Lenovo IdeaPad) lasts 
one year, or maybe two if you don't carry it around and don't abuse it 
at all.  The average business-grade laptop (Dell Latitude, Lenovo 
ThinkPad) lasts about three years under heavy use and abuse, and can 
last up to five years if handled gently.


I do recommend switching out the HDD and installing an SSD just so you 
never have to worry about crashing the disk if you drop it. Also, a Core 
2 Duo with an SSD and enough RAM (4Gb+) usually "feels" like a quad-core 
i7 with a 5400rpm HDD and 2Gb RAM... reinforcing my point about CPU 
horsepower, above.


I *prefer* to buy refurb because I know I'm not going to get ripped open 
on the "cutting edge", especially when it comes to running various 
UNIXes on the hardware.


Good luck with your quest, regardless.  (FYI: that solar-powered laptop, 
while nifty, is unlikely to work 100% with OpenBSD - the components will 
likely be too new and support will be lacking. OTOH, the screenshots 
show Ubuntu Linux, so I could be wrong here.)


--
-Adam Thompson
 athom...@athompso.net

Re: Looking for good, small, canadian version laptop suggestions

[Adam Thompson]

On 13-10-17 01:15 AM, Bernte wrote:

On 10/16/13 12:27 PM, Gilbert Sanford wrote:

I have purchased over 20 machines (about 50% laptop) from
dfsdirectsales.com over the last 5 years, and most of them had
next day business support still in effect from Dell. I had only one
machine that needed service (a Latitude E6510,) and it was
repaired at no charge within 2 days.
Nice advertisement for Dell here on the list, but not applicable to 
the original question. From the FAQ:

> Does DFS Export?
> No.
> DFS does not ship items purchased from www.dfsdirectsales.com 
outside of the 50 United States.

Last time I checked, Canada was not yet among them.
Although a number of economists are certain this will happen eventually, 
it isn't the case quite yet!


As I posted prevously, Dell Financial Services Canada has its own 
website: www.dfsdirect.ca.  Their stuff tends to be a bit older, and a 
bit more expensive than the US site, and support isn't *quite* as good, 
but they still provide a pretty good deal for people with Canadian 
shipping addresses.


Refurb laptops also show up from time to time on Tigerdirect.ca and 
Newegg.ca, usually at competitive prices.


--
-Adam Thompson
 athom...@athompso.net

Re: virtio network driver multicast support

[Adam Thompson]

On 13-10-16 07:30 AM, Jorge Luiz Silva Peixoto wrote:

Hello, folks!

Is IP multicast supported by virtio network driver on OpenBSD 5.3?

pfsync is not working when using vio interface with IP multicast. When
I set pfsync using syncpeer it works fine.

pfsync works when using em interface with IP multicast.

The test bed is a couple of virtual machine running on Linux KVM.
I'm not having any problems with CARP on vio(4)s inside KVM [Proxmox VE 
3.1], which uses multicast, so I don't think there's an inherent problem 
with all multicast.  Could still be a corner case with pfsync, though.


--
-Adam Thompson
 athom...@athompso.net

BGP & CARP - suggestions?

[Adam Thompson]

I've got two OpenBSD boxes acting as my border router[s], talking BGP to 
a small # (~4) of peers.
At the moment, I've got them using carp(4) on every interface, and 
bgpd.conf has for each neighbor{} stanza, a "depend on carpX" line.
This works, more or less, but failover is anything but instantaneous - 
at least one upstream loses my advertisements for a couple of minutes in 
a failover event.  Also, their default gateway points to a non-BGP 
router so they have a "back door" if bgp fails completely for some 
reason (e.g. typo in bgpd.conf, not sure what else), so I lose outbound 
connectivity until bgpd establishes new sessions and pulls in an entire 
routing table.


I think I can solve the outbound loss of connectivity during failover 
simply by changing the default gateway to point at a BGP peer.


The loss of inbound would, at first glance, appear to be caused by my 
peer not having soft-reconfig enabled, but they say it is enabled for 
them, and it's supposed to be on by default in bgpd(8) on my side.


Any ideas/suggestions/recommendations?

For at least one peer, I can probably get them to peer with both routers 
simultaneously - but a) does this add much value?, and b) would it work 
at all if the "LAN" interface [so to speak] is currently not the CARP 
master?


--
-Adam Thompson
 athom...@athompso.net

Re: DNS Hosting & Managed DNS

[Adam Thompson]

On 13-10-24 09:35 AM, Predrag Punosevac wrote:

Are there any strong opinions on DNS Hosting & Managed DNS providers. We
are small Lab currently using ZoneEdit. I believe ZoneEdit  was chosen
at the time they were free. We are looking to move to something little
bit more secure with DNSSEC support out of box. We have one domain name,
small web server and a mail server.

Thank you,
Predrag Punosevac

Certainly not the cheapest, but the best reliability and service I've 
found comes from EasyDNS (www.easydns.com).
Their DNSSEC implementation is a bit weak, they haven't implemented any 
of the stronger ciphers yet, but it works and is easy to use.
They're in Canada, so are somewhat resistant to the random take-down 
orders emanating from both the USA and the UK.  (As long as you don't 
have a .com or a .uk domain, anyway.)
They have lots - way too much, they'd say - of experience weathering 
DDOS storms and attacks.  Depending on the level of service you pay for, 
your DNS is distributed across up to 6(?) anycast strands, which 
translates to a maximum of something insane like ~300 DNS servers 
world-wide.
Their customer service and general cluefulness is so far unequalled in 
my book.  I've worked with them since ~1999 and I haven't yet seen any 
cause to doubt them.
I do wish they were a little bit more price-competitive, but at least 
you get what you pay for.


--
-Adam Thompson
 athom...@athompso.net

Re: RAID Crypt dual booting

[Adam Thompson]

On 13-10-25 01:29 PM, Ted Unangst wrote:
With crypto softraid: installboot copies /boot into a reserved area at 
the beginning of the softraid partition. In this case, the /boot file 
isn't used during booting. Then it copies biosboot into place, with 
the block array filled in with the locations of the copy of /boot. 
/boot will ask for your passphrase and decrypt the disk before loading 
/bsd. The usual trick for dual booting is to jump to or copy the PBR 
somewhere. That doesn't change with crypto. 


Does it also function that way with RAID 1 softraid volumes?
It would be nice to load the kernel directly off the root filesystem 
instead of having two auxiliary filesystems just to hold the kernel.


(And, if it does work this way with RAID 1, when did that start happening?)

--
-Adam Thompson
 athom...@athompso.net

Re: RAID Crypt dual booting

[Adam Thompson]

On 13-10-26 09:43 AM, Stefan Sperling wrote:
>> Does it also function that way with RAID 1 softraid volumes?
> Yes.

That's great news!  (I re-did a system with root-on-raid1 moments ago 
and lo - it "just works".)

But...
Even greater news would be if anyone other than a couple of developers 
knew about it.
I suggest altering FAQ section 14.21.1 
(http://www.openbsd.org/faq/faq14.html#softraid) to show an example that 
includes root on softraid instead of wd0a as / and wd0m as RAID.

I recently followed the most up-to-date guidance I could find on booting 
OpenBSD off softraid(4), and *everything* I found showed that I needed 
the kernel on a separate slice so boot(8) could find it.

For that matter, boot(8), boot_amd64(8) and biosboot(8) all fail to 
mention the special behaviour of installboot(8) on softraid(4), so I 
have - AFAIK - no way of discovering that OpenBSD can boot directly off 
a softraid(4) root partition.

Yeah, I know - submit a patch.  I'm having trouble figuring out 
precisely what to say where.  I'm hoping it's obvious to someone else...

I know OpenBSD borrows useful concepts from Linux from time to 
time, but not keeping the manpages up to date isn't a "feature" I'd like 
to see brought over to the BSD world. One of the fundamentals of OpenBSD 
has historically been correct documentation.

-- 
-Adam Thompson
  athom...@athompso.net

Re: RAID Crypt dual booting

[Adam Thompson]

On 13-10-27 03:49 PM, Predrag Punosevac wrote:

This is definitelly a thread which I followed with great interest. In
the light of Stefan's diff I have three questions.

1. Does that make

http://blog.cochard.me/2012/03/openbsd-51-installation-on-sofraid4.html

accurate? That would mean that I can crypt everything on my laptop
including /.


That link, one of the ones I found, is now *partially* obsolete, see below.
The undeadly article is also now *partially* obsolete in the same way.
The individual steps and tools used are still 100% accurate, but the 
underlying assumption of where /boot and /bsd must be placed is now wrong.



2. My second question is in the light of Stephan's

"It writes data to each chunk in sequence to provide increased capacity.
CONCAT does not provide redundancy."

Does above mean that if I want to use RAID 1/altroot for redundancy
purposes I still have to follow FAQ and create to identical root and
swap partitions on two separate HDD before using RAID 1 for the rest?


RAID1 and /altroot are two different, and complementary strategies.
The point of this thread is that you do *not* have to create identical 
root and swap partitions, despite what the FAQ implies.



As of (at least) 5.4-RELEASE (and probably much earlier), after booting 
from CD (or PXE, or bsd.rd), instead of choosing "Install" right away at 
the first step, choose "shell".

Follow the same MAKEDEV steps that all the FAQ and blog entries talk about.
Zero out the MBR/partition table/disklabel (using "fdisk -iy") just like 
before.

=== Here's where it changes: ===
Using "disklabel -E", create *one* partition, which can fill the entire 
disk, of type "raid", on each disk.
No need for wd0a=/, wd0b=swap, wd0m=raid, just create a single wd0a=raid 
and nothing else.

=== That's the change. ===
In my case, a system with three SATA drives, once I used bioctl(8) to 
create the RAID1 volume, it attached as "sd3".
Now exit the shell, and enter the Installer.  (I think you can reboot at 
this point, if you need to, but I didn't test that.)
Tell the installer to use the newly-attached softraid(4) volume, not any 
of the physical disks.
Auto-partitioning works, or you can do custom partitioning (in which 
case the simplest is usually an 'a' slice covering most of the RAID 
volume, and a 'b' slice for swap).



3. Are there any strong opinions on CARP/pfsync vs RAID 1/altroot for
firewall redundancy for small office use.


In my case, I've chosen to do sd0+sd1 as RAID1, and sd2 will mount as 
/altroot.  Then I have another entire identical system using both CARP 
and iBGP.
This gives me two layers of *redundancy* (RAID1 and CARP), and one layer 
of *recoverability* (altroot).  In other words, I should never be 
without a router in the first place, and if I completely !@#$%^& them up 
(by human error), I still might be able to just pull the RAID1 disks and 
boot off /altroot instead, and be back up and running in <5 minutes.


(Rationale: I don't feel like paying Cisco or Juniper $250k for this 
capability when I can build it myself using OpenBSD.  OK, those boxes do 
several things much better than OpenBSD, but not $250k better, at least 
not in my use case.  The cheapest-possible way to do this with hardware 
routers that I know of is to buy two refurb Catalyst 6500/SUP-720-VSE 
chassis, or two Catalyst 3750X L3 switches, either of which would still 
cost me ~$20k+.  The OpenBSD solution is better in many ways and the 
refurb servers running this only cost $1700 total including spare parts.)


RAID1, CARP, and /altroot solve three different problems.
They all happen to address the problem of a hard disk dying, but in 
different ways.

RAID1 prevents a dead disk from affecting a running system.
CARP prevents an entire dead system from affecting a running network.  
(in theory, anyway :-)
Altroot prevents corruption of a root partition from being completely 
fatal; it's a point-in-time backup copy of / that you can restore from - 
and in the worst-case scenario, replace the entire root disk with.


--
-Adam Thompson
 athom...@athompso.net

Re: Request to OpenBSD Dev's - Beer on offer

[Adam Thompson]

On 13-10-28 11:54 AM, Andy wrote:
Would any of the esteemed OpenBSD developers be interested in adding 
support for BFD (Bidirectional Forward Detection) to OpenBSD.

[...]
'+1's welcome from others who would be interested to show signs of 
support/interest..


I can only agree, BFD support would be a very nice thing to have, 
considering that in other ways OpenBSD is already a very capable 
router.  I'm not in a position right now to pay someone properly to 
implement it, but I can sustain the cost of another case or three of beer.


--
-Adam Thompson
 athom...@athompso.net

Re: Request to OpenBSD Dev's - Beer on offer

[Adam Thompson]

On 13-10-29 10:01 AM, Andy wrote:

Amazing!

So we just need to find an alcoholic developer and we're on our way ;) 
Could maybe send some caffeine and pro plus in the mean time ..


Are there any OpenBSD developers who don't like beer and/or caffeine?

Mind you, many of them are getting as old as I am, so large quantities 
of beer and caffeine may no longer be ideal.


--
-Adam Thompson
 athom...@athompso.net

downing vlan(4) doesn't remove routes

[Adam Thompson]

(Posted last week to tech@, no bites there.  Re-summarizing here.)

I've noticed that downing a vlan(4) interface does not remove the 
associated link-local route from the default routing table.
This seems to directly contradict the ifconfig(8) manpage, which says 
"This action automatically disables routes using the interface."


I can achieve the desired behaviour by deleting the vlan(4) interface, 
but I really don't want to do that.
I can also achieve the deisred behaviour by setting the IP address to 
0.0.0.0, but that also is undesirable.


Am I missing something, or is this broken?

--
-Adam Thompson
 athom...@athompso.net

bgpd(8) EGP vs IGP question

[Adam Thompson]

I've got two border gateways that peer (eBGP) with the same external AS; 
they also peer with each other (iBGP) as per normal BGP design.


Naturally, the BGP RIB contains two copies of every route; one learned 
from the external peer and one learned from the internal peer.


However, when I run "bgpctl show", both routes are marked with origin 
"i" (i.e. IGP).


Do I have to use "set origin egp" in the external neighbour's stanza in 
/etc/bgpd.conf?  Doing so works, and produces the expected output, but 
should it be necessary?


--
-Adam Thompson
 athom...@athompso.net

matching single-part label in ssh_config ?

[Adam Thompson]

Reading the ssh_config manpage, I don't see a way to do this...

I want to match single-part labels, e.g. "servername" without matching 
everything "servername.somewhere.else".

(I do rely on my local resolver's search functionality.)

So far, the best I can come up with is "*,!*.*" which doesn't seem to work.

Is there a way to do this?


FYI, I'm trying to set a less-secure cipher for machines on my local 
network, which I customarily address using non-qualified hostnames.  
I've just remembered that I can easily double my file transfer speed and 
reduce CPU load (both important in my case) by switching to "arcfour".


So far, the only workaround is to specify the FQDN or IP address, both 
of which are less than ideal.


--
-Adam Thompson
 athom...@athompso.net

Re: BGP changes to support CARP better

[Adam Thompson]

On 13-11-11 06:18 AM, Andy wrote:

On Sat 09 Nov 2013 15:57:14 GMT, athom...@athompso.net wrote:

PS; We are against 'sloppy state' so much because we cannot sanitize
the sessions anywhere else (these firewalls connect to raw Transit).

In the meantime I think we're going to be forced to use ifstated to
shutdown OpenBGPd on the backup :(

Ugly and very slow, but I would rather this than risk insecurity..

Thanks for reading :)


I have (I think) almost exactly the same issue; doesn't pfsync 
between the

redundant BGP routers solve your state-tracking problem?

In my case, I have two BGP routers communicating with one upstream peer
(one session per router), *not* using the CARP IP to establish BGP
sessions.  I had started with one BGP session originating from the CARP
IP, but every time I failed over, all my announcements went away and
instead of a ~60sec outage I had a ~4hr partial outage while my routes
re-propagated and foreign ASes stopped damping the change.

At the moment, I've disabled pf altogether, but preliminary testing 
shows
that using pfsync solves my problem - both routers maintain the same 
state

table, so it no longer matters which router processes any given packet
(AFAICT).

-Adam Thompson
  athom...@athompso.net




Hi Adam,

It almost works.. Sadly I believe the pfsync delay can be higher than 
the sessions RTT and so it wont always work. I.e. the internal server 
replies before the other firewall has got the state..


The only way that I know of to make it work properly is to use either 
'defer on' which slows down all connections as I believe it waits for 
the other firewall to commit and confirm the state update, or 'sloppy 
state' which is a big security risk as these are our only firewalls..


Is my understanding correct?

We have an internal network with many VLANs and these two firewalls 
with connections to a Transit. We need to dual stack (start working 
with routed v6 (and some routed v4) traffic etc in addition to the 
legacy NAT) but I cannot see that this is currently possible without 
employing one of the above methods which is extremely undesirable. As 
such we can't dual stack (packet filtering and BGP routing with CARP) 
using OpenBSD at the moment.


The ability to control BGP attribute values based on CARP state would 
resolve this.


NB; These firewalls will be shifting about 1-2 Gbps and near 1 million 
pps, so the pfsync latency is going to be /high/ even if I increase 
the pfsync MTU..




Well, you could - perhaps - flip this on its head.  Instead of changing 
BGP, what about forcing one router to be the master (via 
advbase/advskew), advertising a lower BGP preference (probably by using 
both localpref for iBGP and path prepending for eBGP) from the slave, 
using pfsync (default, not defer) to sync the state tables, and simply 
assuming that if the slave becomes the master it's because the master is 
dead, so losing a few packets isn't the end of the world?


You could use ifstated(8) on the master to ensure that bgpd is killed 
(-HUP, so the advertisements are withdrawn) if any of the other 
interfaces go down.
Finally, bgpd(8) already has support for the carp demotion counter, such 
that it won't restart the session(s) until the carp interface comes up.


Not sure if that all hangs together 100%, typing out loud here (so to 
speak).


--
-Adam Thompson
 athom...@athompso.net

Re: BGP changes to support CARP better

[Adam Thompson]

Oh.  Duh.  That makes perfect sense...
I can't test it until tomorrow morning but that solves all the problems, I 
think.
-Adam


Chris Cappuccio  wrote:

>Adam Thompson [athom...@athompso.net] wrote:
>> 
>> Well, you could - perhaps - flip this on its head.  Instead of changing BGP,
>> what about forcing one router to be the master (via advbase/advskew),
>> advertising a lower BGP preference (probably by using both localpref for
>> iBGP and path prepending for eBGP) from the slave, using pfsync (default,
>> not defer) to sync the state tables, and simply assuming that if the slave
>> becomes the master it's because the master is dead, so losing a few packets
>> isn't the end of the world?
>
>If you're talking about eBGP..or even iBGP for that matter, an interesting
>way to go could be:
>
>Two BGP sessions from different IPs (no CARP)
>BGP next-hop pointing to CARP-protected IP

Re: BGP changes to support CARP better

[Adam Thompson]

On 13-11-11 11:48 PM, Chris Cappuccio wrote:

Adam Thompson [athom...@athompso.net] wrote:

Well, you could - perhaps - flip this on its head.  Instead of changing BGP,
what about forcing one router to be the master (via advbase/advskew),
advertising a lower BGP preference (probably by using both localpref for
iBGP and path prepending for eBGP) from the slave, using pfsync (default,
not defer) to sync the state tables, and simply assuming that if the slave
becomes the master it's because the master is dead, so losing a few packets
isn't the end of the world?

If you're talking about eBGP..or even iBGP for that matter, an interesting
way to go could be:

Two BGP sessions from different IPs (no CARP)
BGP next-hop pointing to CARP-protected IP


I'm trying this, but I'm not sure it's actually working.  I suspect 
bgpd.conf cluelessness on my part, suggestions appreciated.


Existing situation:
 - dedicated VLAN 207 and X.X.X.160/28 for BGP interconnect
 - one router on their side (perhaps one logical router, using VRRP), 
two routers on my side (R{1,2}).

 - carp207 (X.X.X.162) on vlan207 (X.X.X.{163,164}) on trunk0
 - existing bgpd.conf works, each router (R1,R2) has 2 IPv4 neighbours: 
its redundant copy (R2/R1) and the upstream (.161). Very simple config, 
will email off-list if desired.


Change:
 - added "match to X.X.X.161 set nexthop X.X.X.162" before the existing 
"allow to any" on both R1 & R2
(I thought about doing "network x.x.x.x/xx set nexthop X.X.X.162" 
globally, but worried that will affect too many things later.)


Result:
 - no change, according to upstream provider
 - traceroute inbound shows replies from .163 (right now, anyway)


Am I misunderstanding the syntax, or perhaps making some more 
fundamental error?  I can also anonymize and post bgpd.conf here if needed.


--
-Adam Thompson
 athom...@athompso.net

Qsynth midi latency not low enough... what to do?

[Adam Thompson]

PROBLEM STATEMENT: driving FluidSynth from a MIDI controller produces ~1/4sec 
delay between keypress and sound.

NARRATIVE:

I finally got Qsynth working under Xfce (it freezes X under twm!) so I can 
control fluidsynth in a reasonably-obvious way... but I am now experiencing 
substantial latency.
The good news is that it feels just like playing an old pneumatic or tracker 
organ, where there’s a ~0.25sec delay between keypress and sound.
The bad news is that it feels just like playing an old pneumatic or tracker 
organ...

I’m not a good enough musician to handle the roughly quarter-second delay when 
playing live.  I know from many musicians that near-zero-latency from MIDI 
softsynths (even when using soundfonts) is possible... although no-one I know 
of uses OpenBSD.

Is sndio(4) suitable for real-time(-ish) performance?  Or do I need a (OS) 
platform that does ASIO or JACK?  (I mostly play by ear so I'm targeting 
<<0.1sec latency.)

If sndio core or umidi(4) support isn’t the problem, the only obvious thing to 
blame is FluidSynth... but the CPU in this laptop should be more than up to the 
task, and – again – I know this particular piece of software handles 
low-latency live performance in other configurations (i.e. on Linux, using 
JACK).
I don't suspect Qsynth at the moment, as it only controls how fluidsynth 
launches, it doesn't put itself in the data path.

Unsurprisingly, I’ve been unable to find any useful information on running this 
kind of setup under OpenBSD.  But as mentioned before, I’m trying to avoid the 
(to me) insanity that is JACK.

Dmesg follows, just in case anyone spots anything useful in there…

Hardware setup, broadly:
* Dell Latitude E6430 laptop
  - booting in EFI mode to work around a weird bootloader bug
* onboard azalia(4) audio (for now) using onboard speakers (for now)
* Roland A500PRO MIDI controller, connected via USB
* M-audio Uno USB-MIDI, nothing connected to it yet
* No-name USB 5.1ch Audio DAC from Amazon, nothing connected to it yet
  - (leaving the M-audio umidi and the "ABC" uaudio devices disconnected makes 
no difference)


Advice / pointers gratefully accepted, including pointers to documentation or 
threads I may have missed.

Thanks,
-Adam


Dmesg << __EOF__ (so to speak)
 OpenBSD 6.4 (GENERIC.MP) #1: Mon Nov 26 10:18:14 CET 2018

r...@syspatch-64-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 8471482368 (8079MB)
avail mem = 8205459456 (7825MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xebf40 (101 entries)
bios0: vendor Dell Inc. version "A21" date 05/08/2017
bios0: Dell Inc. Latitude E6430
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP APIC FPDT MCFG SSDT HPET SSDT SSDT SSDT ASF! SLIC BGRT 
SSDT
acpi0: wakeup devices P0P1(S4) USB1(S3) USB2(S3) USB3(S3) USB5(S3) USB6(S3) 
USB7(S3) PXSX(S4) RP01(S4) PXSX(S4) RP02(S4) PXSX(S4) RP05(S4) PXSX(S4) 
RP06(S4) PXSX(S4) [...]
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2193.29 MHz, 06-3a-09
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1.2, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2192.89 MHz, 06-3a-09
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 0, core 1, package 0
cpu2 at mainbus0: apid 4 (application processor)
cpu2: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2192.89 MHz, 06-3a-09
cpu2: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,FSGSBASE,SMEP,ERMS,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 2, package 0
cpu3 at mainbus0: apid 6 (application processor)
cpu3: Intel(R) Core(TM) i7-3632QM CPU @ 2.20GHz, 2192.89 MHz, 06-3a

6.4-release tset(1) really slow, what have I missed?

[Adam Thompson]

I've successfully installed OpenBSD 6.4-RELEASE at OVH, but I'm noticing 
one thing there that's different from everywhere else I've used 6.4.


tset(1) takes approximately 12-15 seconds to execute, (almost) every 
time.


On a DigitalOcean VPS running 6.3-STABLE (via openup) tset sensibly 
takes about 1 or 2 seconds:

  athom...@mail.athompso.net:~$ time tset -s
  TERM=xterm;
  0m01.01s real 0m00.00s user 0m00.01s system
  athom...@mail.athompso.net:~$ uname -r
  6.3

On the OVH VPS running 6.4-STABLE (via openup), the same command takes 
15 seconds:

  athom...@mail2.athompso.net:~$ time tset -s
  TERM=xterm;
  0m15.19s real 0m00.00s user 0m00.01s system
  athom...@mail2.athompso.net:~$ uname -r
  6.4


That's from two SSH sessions from the same client with the same 
parameters.


I've captured ktrace(1) output, which shows tset(1) doing, well, 
nothing:

...
 57429/443422  tset 0.035908 CALL  
kbind(0x7f7f7678,24,0xecf2201fc1aab9ca)

 57429/443422  tset 0.035933 RET   kbind 0
 57429/443422  tset 0.035950 CALL  
nanosleep(0x7f7f7760,0x7f7f7750)

 57429/443422  tset 0.035967 STRU  struct timespec { 1 }
 57429/443422  tset 15.809238 STRU  struct timespec { 0 }
 57429/443422  tset 15.809272 RET   nanosleep 0
 57429/443422  tset 15.809303 CALL  
kbind(0x7f7f76c8,24,0xecf2201fc1aab9ca)

 57429/443422  tset 15.809380 RET   kbind 0
...

I don't think this is a bug in 6.4, it's clearly environment-specific... 
but I have no idea what on earth could be causing it.


(dmesg, etc. omitted in this first message, since it's so ridiculously 
specific.)


Oh, and to make it even weirder, it doesn't ALWAYS happen.  It ran 
quickly twice earlier today, but never again.


Normally I'd say "it's DNS", and I thought it was due to the slow login 
times, but ktrace(1) says otherwise.


Any ideas?

Thanks,
-Adam

Re: 6.4-release tset(1) really slow, what have I missed?

[Adam Thompson]

On 2018-12-02 20:50, Philip Guenther wrote:

> On Sun, Dec 2, 2018 at 2:15 PM Adam Thompson  wrote: 
> 
>> I've successfully installed OpenBSD 6.4-RELEASE at OVH, but I'm noticing 
>> one thing there that's different from everywhere else I've used 6.4.
>> 
>> tset(1) takes approximately 12-15 seconds to execute, (almost) every 
>> time.
>> 
>> On a DigitalOcean VPS running 6.3-STABLE (via openup) tset sensibly 
>> takes about 1 or 2 seconds:
>> athom...@mail.athompso.net:~$ time tset -s
>> TERM=xterm;
>> 0m01.01s real 0m00.00s user 0m00.01s system
>> athom...@mail.athompso.net:~$ uname -r
>> 6.3
>> 
>> On the OVH VPS running 6.4-STABLE (via openup), the same command takes 
>> 15 seconds:
>> athom...@mail2.athompso.net:~$ time tset -s
>> TERM=xterm;
>> 0m15.19s real 0m00.00s user 0m00.01s system
>> athom...@mail2.athompso.net:~$ uname -r
>> 6.4
>> 
>> That's from two SSH sessions from the same client with the same 
>> parameters.
>> 
>> I've captured ktrace(1) output, which shows tset(1) doing, well, 
>> nothing:
>> ...
>> 57429/443422  tset 0.035908 CALL  
>> kbind(0x7f7f7678,24,0xecf2201fc1aab9ca)
>> 57429/443422  tset 0.035933 RET   kbind 0
>> 57429/443422  tset 0.035950 CALL  
>> nanosleep(0x7f7f7760,0x7f7f7750)
>> 57429/443422  tset 0.035967 STRU  struct timespec { 1 }
>> 57429/443422  tset 15.809238 STRU  struct timespec { 0 }
>> 57429/443422  tset 15.809272 RET   nanosleep 0
>> 57429/443422  tset 15.809303 CALL  
>> kbind(0x7f7f76c8,24,0xecf2201fc1aab9ca)
>> 57429/443422  tset 15.809380 RET   kbind 0
>> ...
>> 
>> I don't think this is a bug in 6.4, it's clearly environment-specific... 
>> but I have no idea what on earth could be causing it.
> 
> It requested a sleep of 1 second and 15 seconds passed.  That's a kernel 
> timetracking issue, so the output of "sysctl kern.timecounter" would be a 
> good place to start.  Is this is an MP kernel using the CPU TSC, but on a VM 
> where the virtual CPU's TSCs aren't in sync? 
> 
> Philip Guenther

Thanks for the pointer!  I noticed, once, that the system clock was way
off, but I assumed that was one of the boots where I didn't have
networking at startup so ntpd(8) -s failed to operate as expected.  Bad
kernel timekeeping would also produce that result, though. 

Anyway: 

athom...@mail2.athompso.net:~$ sysctl kern.timecounter
kern.timecounter.tick=1
kern.timecounter.timestepwarnings=0
kern.timecounter.hardware=acpitimer
kern.timecounter.choice=i8254(0) acpitimer0(1000) dummy(-100) 

and it's an SP kernel running on one vCPU.  No way of knowing what's
happening under the hood, other than that it's OpenStack Nova, which
IIRC means KVM virtualization.  Note the lack of advertised TSC support.


I'm unsure if my test is valid, but I switched to i8254 (confirmed
successful via sysctl), and tset(1) continues to pause for an
unnaturally long time.  But then I rebooted and re-tested the same
sysctl vaules, and this time tset(1) took 1sec, as expected.  WTF... 

Well, putting that into sysctl.conf seems to be a reasonable workaround
for now.  I also enabled timestepwarnings, and they are occurring,
although I don't yet know how often. 

Is this likely to be a big enough problem that I should ditch this VPS
platform for now? 

dmesg output, to verify SP kernel with no TSC: 

OpenBSD 6.4 (GENERIC) #1: Mon Nov 26 09:32:17 CET 2018
r...@syspatch-64-amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
real mem = 4177379328 (3983MB)
avail mem = 4041621504 (3854MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf6890 (10 entries)
bios0: vendor SeaBIOS version "2:1.10.2-58953eb7" date 04/01/2014
bios0: OpenStack Foundation OpenStack Nova
acpi0 at bios0: rev 0
acpi0: sleep states S3 S4 S5
acpi0: tables DSDT FACP SSDT APIC
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel Core Processor (Haswell, no TSX, IBRS), 2394.81 MHz,
06-3c-01
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,SSE3,PCLMUL,VMX,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,IBRS,IBPB,ARAT,XSAVEOPT,MELTDOWN
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache
cpu0: ITLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: DTLB 255 4KB entries direct-mapped, 255 4MB entries direct-mapped
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpicpu0 at acpi0: C1(@1 halt!)
...etc... 

Thanks, 

-Adam

Re: 6.4-release tset(1) really slow, what have I missed?

[Adam Thompson]

On 2018-12-02 22:12, Adam Thompson wrote:

> I'm unsure if my test is valid, but I switched to i8254 (confirmed successful 
> via sysctl), and tset(1) continues to pause for an unnaturally long time.  
> But then I rebooted and re-tested the same sysctl vaules, and this time 
> tset(1) took 1sec, as expected.  WTF... 
> 
> Well, putting that into sysctl.conf seems to be a reasonable workaround for 
> now.  I also enabled timestepwarnings, and they are occurring, although I 
> don't yet know how often.

I understand why I got inconsistent results... I had two different hosts
open in the two windows I was using.  Thank goodness they were both just
personal systems! 

I'll re-test tomorrow morning when I'm more awake! 

-Adam

bgplg doesn't work with wildcard httpd servers

[Adam Thompson]

Running 6.4 (-stable, via openup/mtier).
I have bgpd(8) talking to my border router, acting as a route collector. 
 That part seems fine.
I now have httpd(8) configured trivially to run bgplg(8) (per the 
bgplg(8) manpage) but it's not working, and I can't tell why.  **EDIT: 
yes, I can, see below**


httpd.conf:
===start===
server "*" {
listen on * port 80
location "/cgi-bin/*" {
fastcgi
root ""
}
}
===end===

On the client end, I get:

  bgpmirror# wget -v http://localhost/cgi-bin/bgplg
  --2019-01-11 10:12:05--  http://localhost/cgi-bin/bgplg
  Resolving localhost (localhost)... 127.0.0.1, ::1
  Connecting to localhost (localhost)|127.0.0.1|:80... connected.
  HTTP request sent, awaiting response... 200 No headers, assuming 
HTTP/0.9

  Length: unspecified
  Saving to: 'bgplg'
(it never completes until I kill it)

Ktrace'ing slowcgi and httpd in -d mode reveals that bgplg execve's 
properly, loads, spits out "invalid character in input" and dies.  
Slowcgi and/or httpd do not handle this... well, at all, really.  That 
error message also does not get logged anywhere nor is visible anywhere 
except ktrace logs.


Looking at the bgplg source code, this means there's something funky in 
its environment that it doesn't like.  Ah.  It looks like it's the "*" 
in server_name, as passed in by slowcgi:

  slowcgi: env[18], SERVER_NAME=*

Yup.  That's the problem, all right: /usr/src/usr.bin/bgplg/bgplg.c:115 
excludes '*'.  But I want my looking glass to be accessible from at 
least two different hostnames, and I really would prefer to not have to 
define them all manually in httpd.conf(5).


The naive local fix is trivial (adding '*' to the strchr call in line 
115), but what else might I be breaking or letting in?  Clearly this is 
supposed to ensure the environment is sanitized before continuing, but 
is "*" forbidden because it's unsafe, or simply because it never 
occurred to anyone?


Thoughts / suggestions ?

Thanks,
-Adam

smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

[Cross-posting here before I give up and switch to Postfix  -Adam]


I have an old instance that uses smtpd's virtual  to rewrite *sender* 
addresses.
Reading the 6.4-STABLE version of the smtpd.conf(5) manpage, I can't see how to 
accomplish my goal any more - it looks impossible.

I don't want to upgrade a working mail relay server to something that might be 
broken, so I'm seeking assistance first.

The purpose of this system is purely to relay mail from internal, semi-broken-ish systems 
out to our Office365 tenant, but I need to clean up bogus MAIL FROM / "From:" 
headers first, lest they be flagged as spam.

In general, I think I'm asking how to use virtual  with the "relay" action 
in the new syntax - the manual tells me this is impossible!?

Thanks,
-Adam


Old smtpd.conf:

===start===
listen on 0.0.0.0
listen on ::
table aliases db:/etc/opensmtpd/aliases.db
table vmap db:/etc/opensmtpd/vmap.db
table localnets { 192.168.10.0/24, 192.168.100.0/24, 192.168.157.0/24, 
192.168.158.0/24, 192.168.101.0/24, 10.158.0.0/16 }
accept from local for anyrelay via 
smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca
accept from source 192.168.158.63 for domain 192.168.158.63 virtual  
deliver to lmtp localhost:25
accept from source 192.168.100.63 for domain 192.168.100.63 virtual  
deliver to lmtp localhost:25
accept from source 192.168.158.63 for anyrelay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
remote.XXX.ca
accept from source 192.168.100.63 for anyrelay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
remote.XXX.ca

accept from source for anyrelay via 
smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca
===end===

old vmap:

===start===
ilom-alert@192.168.100.63:  sys...@xxx.ca
sys...@xxx.ca:   sys...@xxx.ca
sys...@ad.xxx.ca: sys...@xxx.ca
root@XXX.local: sys...@xxx.ca
===end===

Re: smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

As I said, I haven't tried anything yet as I don't want to break a working 
system, and I don't have a good way to test this in parallel right now. 

The manpage says "The local delivery methods support additional options: [...] 
virtual" without specifying which delivery methods are "local".  My assumption 
was that only "mbox" and "mda" were local, as lmtp can, and often does, point 
to another server.

Some brief experiments with a VM only got me syntax errors, so I didn't pursue 
that very thoroughly before asking for clarification.

-Adam

-Original Message-
From: Edgar Pettijohn  
Sent: Wednesday, January 16, 2019 8:12 AM
To: Adam Thompson ; misc@openbsd.org
Subject: Re: smtpd - help needed tranlsating to new virtual map syntax

It would be helpful if you show what you have tried.

Should be as simple as:

action "relay-01" lmtp /var/run/lmtp.sock virtual 

match from src  action "relay-01"

Edgar
On Jan 16, 2019 7:37 AM, Adam Thompson  wrote:
>
> [Cross-posting here before I give up and switch to Postfix  -Adam]
>
>
> I have an old instance that uses smtpd's virtual  to rewrite *sender* 
> addresses.
> Reading the 6.4-STABLE version of the smtpd.conf(5) manpage, I can't see how 
> to accomplish my goal any more - it looks impossible.
>
> I don't want to upgrade a working mail relay server to something that might 
> be broken, so I'm seeking assistance first.
>
> The purpose of this system is purely to relay mail from internal, 
> semi-broken-ish systems out to our Office365 tenant, but I need to clean up 
> bogus MAIL FROM / "From:" headers first, lest they be flagged as spam.
>
> In general, I think I'm asking how to use virtual  with the "relay" 
> action in the new syntax - the manual tells me this is impossible!?
>
> Thanks,
> -Adam
>
>
> Old smtpd.conf:
>
> ===start===
> listen on 0.0.0.0
> listen on ::
> table aliases db:/etc/opensmtpd/aliases.db table vmap 
> db:/etc/opensmtpd/vmap.db table localnets { 192.168.10.0/24, 
> 192.168.100.0/24, 192.168.157.0/24, 192.168.158.0/24, 
> 192.168.101.0/24, 10.158.0.0/16 } accept from local 
> for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca 
> accept from source 192.168.158.63 for domain 192.168.158.63 virtual 
>  deliver to lmtp localhost:25 accept from source 192.168.100.63 
> for domain 192.168.100.63 virtual  deliver to lmtp localhost:25 
> accept from source 192.168.158.63 for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
> remote.XXX.ca accept from source 192.168.100.63 for any
> relay via smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca 
> hostname remote.XXX.ca accept from source for any  
>   
> relay via smtp://XXX-ca.mail.protection.outlook.com hostname 
> remote.XXX.ca ===end===
>
> old vmap:
>
> ===start===
> ilom-alert@192.168.100.63:  sys...@xxx.ca
> sys...@xxx.ca:   sys...@xxx.ca
> sys...@ad.xxx.ca: sys...@xxx.ca
> root@XXX.local: sys...@xxx.ca
> ===end===
>
>

Re: smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

As it turns out, no, that doesn't work.
Trying to fix up broken sender mail domain-parts only simply gets me a "5.2.4 
Mailing list expansion problem" error, with no debug output to suggest why.

In this test case, my translations map had:

@bad.athompso.net @good.athompso.net

in it.  Obviously, this is a test setup :).
Smtpd.conf itself consisted of:

listen on all received-auth
smtp max-message-size 100M
table translations file:/etc/mail/translations  # ORIG->NEW 
mappings
table allowed-hosts file:/etc/mail/allowed-hosts# Who can 
connect?  (bare IP addresses or CIDR subnets)
action translate lmtp "/var/run/lmtp.sock" virtual
# 1st pass on allowed rewrite mail
action forward forward-only 
# and now it's not our problem anymore
match for any from local action forward # 2nd pass for 
reinjected mail, this time just forward it
match for any from src  action translate # inbound mail 
- hand it to LMTP, translating as we go

A cursory glance at the source code (yikes, it's been a long time since I was a 
programmer) suggests that virtual now only works on recipients, not senders.  
Which is too bad for me, as that means I'll have to switch at least one box to 
use Postfix.

I'm not convinced the new smtpd.conf grammar improves anything at all, but I 
assume it must help someone or it wouldn't have changed... but I believe my use 
case got thrown out with the bathwater, so to speak.  Oh, well.  :-(

(If anyone cares, the bad sender addresses are mostly alerts coming from older 
Sun ALOMs and at least one Lexmark printer that also sends email with broken 
From addresses.)

-Adam

 
-Original Message-
From: owner-m...@openbsd.org  On Behalf Of Adam Thompson
Sent: Wednesday, January 16, 2019 8:26 AM
To: 'Edgar Pettijohn' ; misc@openbsd.org
Subject: Re: smtpd - help needed tranlsating to new virtual map syntax

As I said, I haven't tried anything yet as I don't want to break a working 
system, and I don't have a good way to test this in parallel right now. 

The manpage says "The local delivery methods support additional options: [...] 
virtual" without specifying which delivery methods are "local".  My assumption 
was that only "mbox" and "mda" were local, as lmtp can, and often does, point 
to another server.

Some brief experiments with a VM only got me syntax errors, so I didn't pursue 
that very thoroughly before asking for clarification.

-Adam

-Original Message-
From: Edgar Pettijohn 
Sent: Wednesday, January 16, 2019 8:12 AM
To: Adam Thompson ; misc@openbsd.org
Subject: Re: smtpd - help needed tranlsating to new virtual map syntax

It would be helpful if you show what you have tried.

Should be as simple as:

action "relay-01" lmtp /var/run/lmtp.sock virtual 

match from src  action "relay-01"

Edgar
On Jan 16, 2019 7:37 AM, Adam Thompson  wrote:
>
> [Cross-posting here before I give up and switch to Postfix  -Adam]
>
>
> I have an old instance that uses smtpd's virtual  to rewrite *sender* 
> addresses.
> Reading the 6.4-STABLE version of the smtpd.conf(5) manpage, I can't see how 
> to accomplish my goal any more - it looks impossible.
>
> I don't want to upgrade a working mail relay server to something that might 
> be broken, so I'm seeking assistance first.
>
> The purpose of this system is purely to relay mail from internal, 
> semi-broken-ish systems out to our Office365 tenant, but I need to clean up 
> bogus MAIL FROM / "From:" headers first, lest they be flagged as spam.
>
> In general, I think I'm asking how to use virtual  with the "relay" 
> action in the new syntax - the manual tells me this is impossible!?
>
> Thanks,
> -Adam
>
>
> Old smtpd.conf:
>
> ===start===
> listen on 0.0.0.0
> listen on ::
> table aliases db:/etc/opensmtpd/aliases.db table vmap 
> db:/etc/opensmtpd/vmap.db table localnets { 192.168.10.0/24, 
> 192.168.100.0/24, 192.168.157.0/24, 192.168.158.0/24,
> 192.168.101.0/24, 10.158.0.0/16 } accept from local 
> for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com hostname remote.XXX.ca 
> accept from source 192.168.158.63 for domain 192.168.158.63 virtual 
>  deliver to lmtp localhost:25 accept from source 192.168.100.63 
> for domain 192.168.100.63 virtual  deliver to lmtp localhost:25
> accept from source 192.168.158.63 for anyrelay via 
> smtp://XXX-ca.mail.protection.outlook.com as sys...@xxx.ca hostname 
> remote.XXX.ca accept from source 192.168.100.63 for any
> relay via smtp://XXX-ca.mail.protection.outlook.com as sys

Re: smtpd - help needed tranlsating to new virtual map syntax

[Adam Thompson]

I found the "-T" (trace) flag to smtpd(8), and it gives me this, which AFAICT 
confirms my suspicions:
[...]
rule #2 matched: match from src allowed-hosts for any => translate
lookup: lookup "athom...@athompso.net" as ALIAS in table 
static:translations -> 0
lookup: lookup "athompso" as ALIAS in table static:translations -> 0
lookup: lookup "@athompso.net" as ALIAS in table static:translations -> 0
lookup: lookup "@" as ALIAS in table static:translations -> 0
expand: lka_expand: no aliases for virtual
mproc: lka -> pony : 53 IMSG_SMTP_EXPAND_RCPT
expand: 0x154201b89018: clearing expand tree
imsg: pony <- lka: IMSG_SMTP_EXPAND_RCPT (len=53)
smtp: 0x1127a72e6000: >>> 524 5.2.4 Mailing list expansion problem
[...]

complete output attached below, I've changed to @old.athompso.net and 
@new.athompso.net during testing since the last email.

On the sending side, from another host (listed in ), I'm running:
swaks --to athom...@athompso.net --from athom...@old.athompso.net 
--server 
which faithfully reports the 5.2.4 error.

I'm slightly disappointed, I still like OpenSMTPd's concise configuration 
syntax.  Postfix could still rewrite source addresses last time I checked, I 
hope it's still there - I do NOT want to run sendmail, thank you very much.

-Adam



Smtpd(8) trace output including invocation:
===from here to end of message===
bhs# smtpd -d -T all -v -d
debug: init ssl-tree
debug: init ca-tree
debug: init ssl-tree
debug: using "fs" queue backend
debug: using "ramqueue" scheduler backend
debug: using "ram" stat backend
info: OpenSMTPD 6.4.0 starting
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ca-tree
debug: init ca-tree
debug: init ca-tree
debug: init ca-tree
debug: init ca-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ssl-tree
debug: init ca-tree
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: using "fs" queue backend
debug: init ssl-tree
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "ramqueue" scheduler backend
debug: using "fs" queue backend
debug: using "ram" stat backend
setup_peer: klondike -> control[63932] fd=4
setup_peer: klondike -> pony express[40666] fd=5
setup_done: ca[35575] done
debug: using "ram" stat backend
setup_peer: control -> klondike[35575] fd=4
setup_peer: control -> lookup[49698] fd=5
setup_peer: control -> pony express[40666] fd=6
setup_peer: control -> queue[21502] fd=7
setup_peer: control -> scheduler[14152] fd=8
setup_done: control[63932] done
debug: using "ram" stat backend
setup_peer: lookup -> control[63932] fd=4
setup_peer: lookup -> pony express[40666] fd=5
setup_peer: lookup -> queue[21502] fd=6
setup_done: lka[49698] done
debug: using "ram" stat backend
setup_peer: pony express -> control[63932] fd=4
setup_peer: pony express -> klondike[35575] fd=5
setup_peer: pony express -> lookup[49698] fd=6
setup_peer: pony express -> queue[21502] fd=7
setup_done: pony[40666] done
debug: using "ram" stat backend
setup_peer: queue -> control[63932] fd=4
setup_peer: queue -> pony express[40666] fd=5
setup_peer: queue -> lookup[49698] fd=6
setup_peer: queue -> scheduler[14152] fd=7
setup_done: queue[21502] done
debug: using "ramqueue" scheduler backend
debug: using "ram" stat backend
setup_peer: scheduler -> control[63932] fd=4
setup_peer: scheduler -> queue[21502] fd=5
setup_done: scheduler[14152] done
smtpd: setup done
mproc: parent -> control: enabled
mproc: parent -> lka: enabled
mproc: parent -> queue: enabled
mproc: parent -> ca: enabled
mproc: parent -> pony: enabled
debug: parent_send_config_ruleset: reloading
mproc: parent -> lka : 0 IMSG_CONF_START
mproc: parent -> lka : 0 IMSG_CONF_END
debug: parent_send_config: configuring pony process
mproc: parent -> pony : 0 IMSG_CONF_START
mproc: parent -> pony : 0 IMSG_CONF_END
debug: parent_send_config: configuring ca process
mproc: parent -> ca : 0 IMSG_CONF_START
mproc: parent -> ca : 0 IMSG_CONF_END
setup_proc: klondike done
setup_proc: control done
setup_proc: lookup done
setup_proc: pony express done
setup_proc: queue done
setup_proc: scheduler done
mproc: ca -> control: enabled
debug: bounce warning after 4h
mproc: ca -> parent: enabled
mproc: ca -> pony: enabled
mproc: ca -> pony: disabled
mproc: pony -> parent: enabled
mproc: scheduler -> control: enabled
imsg: ca <- parent: IMSG_CONF_START (len=0)
mproc: lka -> parent: enabled
mproc: pony -> queue: enabled
mproc: scheduler -> queue: enabled
scheduler: getting batch: mask=0x3f, count=10
debug: /--- ramqueue: scheduler_ram_batch()
debug: \---
scheduler: got r=0, delay=-1, count=10
scheduler: sleeping
imsg: 

Re: smtpd - help needed tranlsating to new virtual map syntax [FIXED]

[Adam Thompson]

On 2019-01-21 04:08, Gilles Chehade wrote:

In this test case, my translations map had:

What is a translation map ?
There is no such thing in OpenSMTPD (as of today).


A virtual map that happened to be called .



You're feeding the virtual table with invalid values.


Apparently, yes.


Also, this is a recipient translation mechanism, similar to aliases, 
and

not a sender rewriting mechanism which we do not have at this point.
[...]
virtual _now_ only works on recipients, not senders ?
the virtual code hasn't changed, it works the way it always did.

there is no way it could ever do what you're describing or attempting 
to
do given that it doesn't operate at all anywhere near the message. 
there

is no way it has ever parsed:


This is all very surprising to hear.  The existing system works 
(somehow).  So I am apparently misunderstanding what is happening, 
because with the configuration as shown, telling the various broken 
email senders to use that box as their mailhost _somehow_ fixes the 
bogus From: headers and envelopes.


Oh, this just occurred to me as I'm writing:  I really hope I didn't 
switch to a different MTA on that system years ago, and then just forgot 
to check which MTA was actually running.  If that's the case, I'm not 
going to bother posting an update, because I'll be busy banging my head 
on the wall and then hiding in shame.



I'm not convinced the new smtpd.conf grammar improves anything at all, 
but I assume it must help someone or it wouldn't have changed... but I 
believe my use case got thrown out with the bathwater, so to speak.  
Oh, well.  :-(

This is bullshit.
The grammar doesn't reduce the functional scope, it can only expand it.


I'm taking your word for it - you will know far better than I do!



What you are describing has never existed in smtpd, there's never been
code to translate sender addresses and there's a good reason for that:


Good reasons aside, I still need to accommodate other vendor's broken 
mail implementations, because I can't fix them.  I know of multiple 
reasons source rewriting is a bad idea, in general, but I get paid to 
make stuff work, not just say that it's broken.




it not considered doable before the grammar change...
But sure, blame it on the grammar.


I believed that the grammar change had rendered my use case impossible 
because  was now limited to local delivery methods.  Clearly I 
was wrong... and not even in the way I thought I might be wrong.



I may sound a bit harsh, but starting a thread with "this is my last 
try

or I'll switch" (as if it actually matters)


My apologies - that was meant to sound more like "I have a plan B so if 
this isn't possible, that's OK but I've wasted so much time on this I'm 
kinda running out of time, please tell me if I should just stop now and 
switch".  I know *exactly* how much OpenBSD devs care if I use their 
code or not!  I do not want to be "that asshole", although it seems I've 
succeeded again - sorry.


Thank you for taking the time to reply.  Now I'm going to go check that 
mail server a 7,000,000th time, this time to see what MTA is actually 
*running*, not just *configured*.  I'm not sure whether I want it to be 
such a blatant mistake on my part or not... if yes, this all makes sense 
but I'm an idiot, whereas if no, then WTF, how is it working at all?


FWIW: I am much happier with OpenSMTPd than with other MTAs because of 
its forward-declarative configuration syntax.  Thank you for your work 
on bringing a modern, lean, secure(-er) MTA into existence.


-Adam

purpose of bgpd.conf dump "timeout" parameter?

[Adam Thompson]

In bgpd.conf(5), for the "dump" directive there is an optional "timeout" 
parameter.  What is its purpose?  I assume from the examples that it's 
denominated in seconds...


my first guess was to time out on attempting to write to the dump file, 
but that doesn't seem realistic.  It looks like it's a cycle, i.e. the 
dump file will be recreated every X seconds, but if so, why is it called 
"timeout" and not "interval" ?


I see in the source that MRT_MAX_TIMEOUT is set to 7200 - does this mean 
that if I leave the parameter unset, the MRT file will be re-dumped 
every 2 hrs?


Yes, I'll produce a patch for the manpage if someone can explain what 
the parameter is supposed to do / how it works.


Thanks,
-Adam

Re: purpose of bgpd.conf dump "timeout" parameter?

[Adam Thompson]

Yes, that clarifies it, thank you.  I will attempt to explain that better and 
provide patch for bgpd.conf.5 next week.
May I suggest a future function for bgpctl/bgpd - creating an MRT dump on 
demand?  Even if only able to run in the context of bgpd(8), I think it could 
be helpful.  (Certainly it would be helpful to me.  Which is probably obvious 
since I'm suggesting it...)  Something else to tack onto the to-do list, I 
guess.
Thanks,
-Adam

On February 8, 2019 5:23:24 PM CST, Claudio Jeker  
wrote:
>On Fri, Feb 08, 2019 at 03:56:12PM -0600, Adam Thompson wrote:
>> In bgpd.conf(5), for the "dump" directive there is an optional
>"timeout"
>> parameter.  What is its purpose?  I assume from the examples that
>it's
>> denominated in seconds...
>
>Yes it is.
> 
>> my first guess was to time out on attempting to write to the dump
>file, but
>> that doesn't seem realistic.  It looks like it's a cycle, i.e. the
>dump file
>> will be recreated every X seconds, but if so, why is it called
>"timeout" and
>> not "interval" ?
>
>Because naming is hard :)
> 
>> I see in the source that MRT_MAX_TIMEOUT is set to 7200 - does this
>mean
>> that if I leave the parameter unset, the MRT file will be re-dumped
>every 2
>> hrs?
>
>No, it just limits the poll timeout but it will only fire once the time
>really ran out.
> 
>> Yes, I'll produce a patch for the manpage if someone can explain what
>the
>> parameter is supposed to do / how it works.
>
>After timeout seconds the file is reopened (or maybe a new file is
>opened
>depending on the strftime expand of the filename). For all and update
>dumps this just puts new messages into a new file. For table dumps it
>will
>issue a new dump. e.g.
>   dump table "/tmp/rib-dump-%H%M" 300
>will create a new table dump every 5 minutes.
>
>Hope that helps.
>-- 
>:wq Claudio

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: keeping track of MAC addresses

[Adam Thompson]

On 2019-02-14 02:01, mailingli...@dotbit.ro wrote:

I would like to keep tabs on the MAC/IP addresses in my secure net.
I do know how to do this, but keeping track of ethernet MAC addresses 
seems
quite cumbersome in OpenBSD, not that it is more convenient in any 
other
general purpose operating system but many interfaces for ex. routers 
make it

easy to manage, especially MAC filtering.


Perhaps look at the "arpwatch" package in ports, which may be 
applicable.


But... you know that both ARP and MAC addresses can be trivially 
spoofed, right?  Just using /etc/ethers instead of ARP does *not* make 
your network secure.


Some "intelligent" switches do ARP sniffing to populate their internal 
hardware FIBs.  (Yes, that's a dumb idea.  Switch vendors still do it.)  
Disabling ARP on your hosts is... not generally a good idea.


PS: after running ifconfig em0 -arp my Allied Telesis AT-GS950-16 
managed
switch took the link down and refuses to bring it back up on the same 
port

without a reset. Other ports work fine.


I won't say this is impossible, but it seems unlikely.  I think it's 
more likely the lack of ARP traffic on the port caused the switch to do 
something "interesting" with IP traffic destined for this host.  Or 
maybe something else triggered storm-prevention features in the switch?  
Running an ifconfig(8) command should not be able to persistently shut 
down a switch port in any network environment.  Did you observe the link 
lights on the NIC and switch actually turn off and stay off?


As I have already mentioned I can manage by myself, but it seems to me 
that

this is something that a lot of people would want.


Not so much, AFAIK.  Disabling core IP protocols usually generates more 
problems than it solves.  Let us know how disabling/blocking ICMPv6 
works out for you... ;-)  [Hint: that's a trick question.  You can't run 
IPv6 without ICMPv6.]


You could filter on MAC addresses instead of restricting ARP:  
https://www.openbsd.org/faq/pf/tagging.html#ethernet   That requires 
using bridge(4) which apparently is on its way out, and I don't know if 
the replacement (switch(4)) supports filtering packets based on MAC 
address or not - it's OpenFlow-compliant, so there has to be a way, but 
it may or may not be easily accessible from inside OpenBSD.


You may also want to assign new MAC addresses to your hosts, both to 
eliminate the need to gather the MACs, and to simplify maintenance (e.g. 
the labour involved in replacing a NIC on a server or a motherboard is 
O(n^2) with hardware-bound MAC addresses in your setup, instead of 
O(1)).  There are special LAAs (Locally-Assigned Addresses) that you can 
use for this.  OpenBSD supports setting a locally-assigned MAC address 
with ifconfig(8) "lladdr" option.


Good luck on your strange quest,
-Adam

cvsweb.openbsd.org - same as cvsweb in ports?

[Adam Thompson]

I know this has been asked before, but my google-fu cannot unearth any 
trace of it, so I have to ask again - sorry!


What version of cvsweb does cvsweb.openbsd.org run?  And where is that 
software available?  It appears to not quite be the same as cvsweb in 
ports, so... ?


Thanks,
-Adam

security - preferred way to make check_access_file happy?

[Adam Thompson]

Hi,
I'm getting daily insecurity (i.e. security(8)) nags about userids that 
are off but still have a valid shell and access files.  (Specifically, 
I'm getting the nag from check_access_files() in /usr/libexec/security.)


Since ports (at least in my experience) regularly creates userids that 
will trigger this warning, what's the "best" way to disable the warning? 
 I'm reluctant to mess with permissions on directories created by 
packages, but maybe that's the best way?


Otherwise, it looks like I can disable the warning by setting a password 
on the userid in question.


However, that leads to the question: what if I don't *want* a password 
on the account, because it's supposed to be a SFTP-only, 
public-key-authentication-only account, but still needs to be readable 
and needs a valid shell for various cron jobs to be happy?  If I'm 
following the logic correctly, one of the warnings I'm getting is for 
~/.ssh being readable on a userid with no password - exactly the 
scenario I just mentioned.  But AFAIK they can't login if I take away 
S_IRUSR on ~/.ssh?


The most distasteful option is to hack /usr/libexec/security to ignore 
certain userids, but ... it's there for a reason.


The cleanest example I have right now from ports is _rancid, created by 
the rancid package, and triggered by the existence of ~_rancid/.ssh with 
S_IRUSR (u+r) permissions.


Suggestions / advice?

Thanks,
-Adam

Re: security - preferred way to make check_access_file happy?

[Adam Thompson]

Whoops... I'm getting the messages from 3 systems, all running 
6.4-STABLE, with no local modifications, under both VMware and 
Openstack, using openup to keep systems updated.  Dmesg available if 
anyone thinks it's relevant.

-Adam


On 2019-02-25 08:50, Adam Thompson wrote:

Hi,
I'm getting daily insecurity (i.e. security(8)) nags about userids
that are off but still have a valid shell and access files.
(Specifically, I'm getting the nag from check_access_files() in
/usr/libexec/security.)

Since ports (at least in my experience) regularly creates userids that
will trigger this warning, what's the "best" way to disable the
warning?  I'm reluctant to mess with permissions on directories
created by packages, but maybe that's the best way?

Otherwise, it looks like I can disable the warning by setting a
password on the userid in question.

However, that leads to the question: what if I don't *want* a password
on the account, because it's supposed to be a SFTP-only,
public-key-authentication-only account, but still needs to be readable
and needs a valid shell for various cron jobs to be happy?  If I'm
following the logic correctly, one of the warnings I'm getting is for
~/.ssh being readable on a userid with no password - exactly the
scenario I just mentioned.  But AFAIK they can't login if I take away
S_IRUSR on ~/.ssh?

The most distasteful option is to hack /usr/libexec/security to ignore
certain userids, but ... it's there for a reason.

The cleanest example I have right now from ports is _rancid, created
by the rancid package, and triggered by the existence of ~_rancid/.ssh
with S_IRUSR (u+r) permissions.

Suggestions / advice?

Thanks,
-Adam

Re: security - preferred way to make check_access_file happy?

[Adam Thompson]

Use vipw to put 13 * in the password field

From passwd(5)
[...]
 authentication, conventionally have 13 asterisks in the password 
field.


Thank you!  Now that I know what I'm looking for, I can see the relevant 
code in security(8), too.


I wonder if there's a way for ports to do that for me while calling 
useradd?  Another rabbit hole to go down.


Thanks again,
-Adam

Re: security - preferred way to make check_access_file happy?

[Adam Thompson]

On 2019-02-25 11:14, Stuart Henderson wrote:

On 2019/02/25 09:13, Adam Thompson wrote:

> Use vipw to put 13 * in the password field
>
> From passwd(5)
> [...]
>  authentication, conventionally have 13 asterisks in the password field.

Thank you!  Now that I know what I'm looking for, I can see the 
relevant

code in security(8), too.

I wonder if there's a way for ports to do that for me while calling 
useradd?

Another rabbit hole to go down.

Thanks again,
-Adam



It normally does already. Do you have an unusual "password" line in
/etc/usermgmt.conf?


Nope.
Ugh, although I said ports does this often in my experience, the only 
system/port I have right now where it's doing it is the RANCID package 
on that dedicated VM.  I have two other VMs where I have manually 
created single-purpose userids without passwords that also complain (one 
for RT [not from ports], one for webhosting).
So it could just be that package; at least I can't demonstrate any 
others right now.

-Adam

Re: How to restrict ip to access a directory in OpenBSD's httpd

[Adam Thompson]

On 2019-04-03 11:30, Stuart Henderson wrote:

On 2019-04-03, =?utf-8?B?RnVuZw==?=  wrote:

apache support somthing like

Order Allow,Deny
Allow from all
Deny from 1.2.3.4


How to achieve in OpenBSD's httpd?
We are using OpenBSD 6.4.




There is no built-in simple way.

It can be done by having httpd listen on two different ports,
one allowing access to this directory, the other denying access,
and using a PF rdr-to rule to send traffic to the "allow access"
port if it has the correct source IP address. But this is a bit
of a mess.


I vaguely recall hearing someone (possibly Reyk, several years ago?) 
mention that relayd can  handle access control for httpd, if httpd is 
listening only on loopback.
This seems like overkill, but does fit the "UNIX philosophy" of doing 
one thing well.


I'm not at all sure it was Reyk, and I'm sure not 100% confident of this 
solution, but a quick glance at the man pages suggests it's not totally 
insane, either.


-Adam

post-6.5-upgrade bgpd(8) problem

[Adam Thompson]

I've upgraded my looking glass from 6.4 to 6.5, and an experiencing an 
unexpected problem - routes learned from one (iBGP) peer are not being 
automatically exported to other (eBGP) peers.


I did not change /etc/bgpd.conf, but behaviour seems to have changed 
nonetheless.  The upgrade from 6.4 to 6.5 appeared smooth otherwise, 
nothing to suggest subtle breakage under the hood.


As you can see below, this bgpd.conf is almost so simple it *can't* have 
problems.  Apparently "almost" being the operative word.


Under 6.4, this behaved as though "export none" only applied to the 
first group.  Under 6.5, it behaves as though "export none" is a global 
setting.


Under 6.5, bgpctl show produces:
root@bgpmirror:~# bgpctl sh
Neighbor   ASMsgRcvdMsgSent  OutQ 
Up/Down  State/PrfRcvd
Hermes IPv4 16796 128773 85 0 
00:41:40 753748
Hermes IPv6 16796  29727 85 0 
00:41:40  68228
MBNOG IPv4  65204 97 85 0 
00:41:40  0
MBNOG IPv6  65204 97 85 0 
00:41:40  0
BGPMon.io IPv4   6447  0 21 0 Never  
  Active
isolario.it IPv465517 86 85 0 
00:41:39  0
isolario.it IPv665517 86 85 0 
00:41:39  0
and the operator of the MBNOG route collector confirms that I stopped 
sending him a full routing table at the same time I did the OS upgrade.


Any ideas?  What other information would help diagnose this problem?

Thanks,
-Adam



Dmesg & bgpd.conf:  
https://gist.github.com/athompso/e334d8621ce458925e25bb44b8068341



bgpd.conf, duplicated here for convenience:

===BOF===
route-collector yes
socket "/var/www/run/bgpd.rsock" restricted # for bgplg

# settings
nexthop qualify via default
fib-update no
dump table-v2 "/var/www/htdocs/bgplg/mrt/rib-dump.mrt" 3600
dump updates in "/var/www/htdocs/bgplg/mrt/updates-in-%H%M" 300
dump all in "/var/www/htdocs/bgplg/mrt/all-in-%H%M" 300

# myself
AS X
router-id X.X.X.X

# neighbors

group hermes {
enforce local-as no
enforce neighbor-as no
export none

neighbor X.X.X.X {
remote-as X
descr "Hermes IPv4"
}
neighbor X:X:X:X::X {
remote-as X
descr "Hermes IPv6"
}
}

group bgpresearch {
multihop 32
enforce local-as no
enforce neighbor-as no

neighbor 192.160.102.196 {
remote-as 65204
descr "MBNOG IPv4"
}
neighbor 2620:132:3003:300::196 {
remote-as 65204
descr "MBNOG IPv6"
}
neighbor 129.82.138.6 {
remote-as 6447
descr "BGPMon.io IPv4"
}
neighbor 146.48.78.12 {
remote-as 65517
descr "isolario.it IPv4"
}
neighbor 2a00:1620:c0:4e:146:48:78:12 {
remote-as 65517
descr "isolario.it IPv6"
}
}

# policies
allow quick from group hermes
allow quick to group bgpresearch
===EOF===

(if you want to see the unredacted version of bgpd.conf, ask and I'll 
email it directly to you, I just don't want internal addresses in the 
public archive.)

Re: post-6.5-upgrade bgpd(8) problem

[Adam Thompson]

On 2019-05-09 13:53, Sebastian Benoit wrote:

bgpctl sh rib neigh  out
for all neighbors.


All empty.



Also look at
bgpctl sh rib best


Completely empty.



if any routes are actually selected - maybe the "nexthop qualify via
default" isnt working.


I see two things...
1) when run as "bgpd -dv" I get repeated notifications about the same 
next-hops, dunno if that's normal or not.

And 2) from bgpd.conf(5):

route-collector (yes|no)
 If set to yes, the route selection process is turned 
off.  The

 default is no.

and I have it set to yes, since this is supposed to behave like a route 
collector.  Granted, with a single source of routes at the moment, I 
could turn that off... let's see what happens when I do:


Yup.  That behaviour has definitely changed, somehow - taking a 
not-so-wild guess, it's likely _somehow_ related to claudio@'s 
introduction of Adj-Rib-Out but I'm not enough of a C programmer (or 
kernel routing expert) to say for sure.


For now, commenting out the "route-collector yes" line in bgpd.conf will 
work, but there's a (for me) minor regression.  I have no idea which way 
the code is supposed to behave, so can't tell if this is a bug or a 
bugfix!


Based on CVS logs, it's probably a change introduced in rde.c after 
v1.442.


-Adam

relayd without pf?

[Adam Thompson]

OK, I'm pretty sure this is a dumb question, but...

Does relayd work properly, or at all with pf disabled?  (in 6.5-RELEASE)

It looks like it should as long as I use "relay" instead of "redirect", 
but I'm having trouble, and don't want to keep banging my head against a 
wall if it's something this simple.


Thanks,
-Adam


--begin relayd.conf--
http protocol rtproxy {
pass quick
}
relay rt4 {
listen on 0.0.0.0 port 80
protocol rtproxy
forward to 127.0.0.1 port 8080
}
relay rt6 {
listen on :: port 80
protocol rtproxy
forward to ::1 port 8080
}
--end relayd.conf--

Re: relayd without pf?

[Adam Thompson]

On 2019-05-14 15:42, Adam Thompson wrote:

OK, I'm pretty sure this is a dumb question, but...

Does relayd work properly, or at all with pf disabled?  (in 
6.5-RELEASE)



I have partially answered my own question.  That last message was posted 
prematurely, in more than one way, sorry!


1. the relayd.conf in the previous message was copied-and-pasted from 
the wrong window, in mid-edit.


2. relayd(8) does not work with pf(4) disabled.  I'm unclear if this is 
a bug, or by design.  With pf disabled, it outputs:

root@rt:~# relayd -dv
startup
relayd: pfe: pf is disabled
parent: proc_open: imsg_flush: Broken pipe
ca exiting, pid 37187
ca exiting, pid 79962
ca exiting, pid 95113
root@rt:~# hce exiting, pid 91576
relay exiting, pid 26432
relay exiting, pid 6966
relay exiting, pid 50166

The message "pfe: pf is disabled" looks like an informational message to 
me, I'm not using any pf features, so it shouldn't matter... but it very 
much does matter, and relayd exits shortly after starting if pf is 
disabled.


Pinging @reyk - is this a bug or deliberate?

-Adam

Re: relayd without pf?

[Adam Thompson]

FWIW, I also encountered some slightly different error messages, I'll see if I 
can reproduce those.
-Adam

On May 14, 2019 4:48:29 p.m. CDT, Reyk Floeter  wrote:
>
>> Am 14.05.2019 um 23:06 schrieb Adam Thompson :
>> 
>>> On 2019-05-14 15:42, Adam Thompson wrote:
>>> OK, I'm pretty sure this is a dumb question, but...
>>> Does relayd work properly, or at all with pf disabled?  (in
>6.5-RELEASE)
>> 
>> 
>> I have partially answered my own question.  That last message was
>posted prematurely, in more than one way, sorry!
>> 
>> 1. the relayd.conf in the previous message was copied-and-pasted from
>the wrong window, in mid-edit.
>> 
>> 2. relayd(8) does not work with pf(4) disabled.  I'm unclear if this
>is a bug, or by design.  With pf disabled, it outputs:
>> root@rt:~# relayd -dv
>> startup
>> relayd: pfe: pf is disabled
>> parent: proc_open: imsg_flush: Broken pipe
>> ca exiting, pid 37187
>> ca exiting, pid 79962
>> ca exiting, pid 95113
>> root@rt:~# hce exiting, pid 91576
>> relay exiting, pid 26432
>> relay exiting, pid 6966
>> relay exiting, pid 50166
>> 
>> The message "pfe: pf is disabled" looks like an informational message
>to me, I'm not using any pf features, so it shouldn't matter... but it
>very much does matter, and relayd exits shortly after starting if pf is
>disabled.
>> 
>> Pinging @reyk - is this a bug or deliberate?
>> 
>
>It’s a historical reason because redirects existed first. And most
>OpenBSD systems keep pf enabled by default.
>
>But you’re right; it should be easy to fix.
>
>Reyk

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: OpenBSD on VMware ESXi

[Adam Thompson]

On 2019-05-22 09:25, mxb wrote:
I think FreeBSD or any Linux template will work just fine and add 
vmxnet3.

However, last I checked (1year ago) vmxnet3 been less stable than
e1000 under pressure.


Don't use the Linux templates.  I would recommend against using the 
FreeBSD templates, and go with "Other (64-bit)" instead.  YMMV on using 
FreeBSD vs Other, I haven't seen consistent results here yet... just 
don't pick Linux, or DOS, or Windows - in some situations, that allows 
VMware to take certain shortcuts that are based on assumptions about the 
Linux/Win/etc. kernel & device drivers that (probably) aren't valid 
under OpenBSD.


Various people have reported different problems with vmxnet3; I'm aware 
of at least 4 or 5 different environment-specific issues (i.e. can't be 
reproduced on any other vSphere/ESXi system).  I have some of those 
problems, and I cannot reproduce them outside my production environment, 
but they don't prevent me from running OpenBSD.


Workarounds:
* use vmxnet2
* use e1000

If vmxnet3 and pvscsi work for you (you'll know pretty darn fast!), use 
them.  When they work, which is usually (in my experience), they're 
generally very stable and high-performing compared to the emulated h/w 
(e1000, lsisas, lsiscsi, buslogic).


I also experience timer issues, and I've had to specify 
kern.timecounter.hardware=i8254 in sysctl.conf.  This is likely a VMware 
problem, not an OpenBSD problem, but it's non-trivial to diagnose.  
(Even i8254 doesn't work perfectly: e.g. usleep() isn't very accurate in 
my VMs!)  I'm also running these VMs on very heavily-loaded hosts, which 
is probably a factor.


My disk write throughput is horrible, but that's an interaction between 
how OpenBSD does writes, how VMware handles writes into thin-provisioned 
disks, and how my NFS storage handles writes on thin-provisioned 
volumes; it's not an OpenBSD problem, strictly speaking, although that's 
the only place it's normally visible.


Overall, OpenBSD works well under ESXi, but there are semi-random 
problems that do have workarounds.


Several years ago, Theo noted (approximately, I'm going from memory here 
AND paraphrasing) that it was hard enough for OpenBSD to handle broken 
hardware implementations, it's exponentially harder to handle an 
incorrect software emulation of hardware that was incorrect in the first 
place.  This has proven accurate, and VMware doesn't really care much 
about OpenBSD, since I doubt it even registers on their radar so they're 
not terribly interested in fixing VMware bugs that are only visible 
under OpenBSD.  (If you have a support contract, please submit bug 
reports to VMware.  If enough of us do so, they might start fixing some 
of the problems.)


-Adam

"Invalid argument" when exec'ing and/or ktrace'ing a file?

[Adam Thompson]

I have a binary - built on this 6.5-STABLE amd64 system by an automatic 
build process as part of a CPAN module installation, that will not 
execute:



rt@rt$ /var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf
ksh: /var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf: 
Invalid argument


or even allow itself to be ktrace'd:

rt@rt$ ktrace 
/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf
ktrace: exec of 
'/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf' 
failed: Invalid argument


At least ktrace(8) told me it was an exec(3) problem.  So off I go to 
exec(3) which leads me to execve(2), wherein I see:



[EINVAL] argv did not contain at least one element.


errno(2) confirms the english expansion of "Illegal argument"

file(1) gives me a clue, but I don't know what to do with it:


rt@rt$ file `locate bin/wkhtmltopdf`
/usr/local/bin/wkhtmltopdf:   ELF 
64-bit LSB shared object, x86-64, version 1
/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf: ELF 
64-bit LSB executable, x86-64, version 1


where the version in /usr/local/bin is from ports.

the RT::Extension::TicketPDF version is significantly larger, ~4x, which 
led me to:



rt@rt$ ldd `locate bin/wkhtmltopdf`
/usr/local/bin/wkhtmltopdf:
StartEnd  Type  Open Ref GrpRef Name
041737f61000 04173ac77000 exe   20   0  
/usr/local/bin/wkhtmltopdf
0419ee50 0419ee599000 rlib  01   0  
/usr/local/lib/libjpeg.so.70.0

[...elided...]
/var/www/rt/local/plugins/RT-Extension-TicketPDF/bin/wkhtmltopdf:
not a dynamic executable


So... how do I figure out why a static binary (whose build process is 
pretty opaque) won't/can't run?


Clueless at this level, my detailed knowledge of how exec worked under 
OpenBSD ended in the a.out era.


(oh, the reason I don't use the version from ports?  It's too new for 
the TicketPDF code. *sigh*)


Thanks for any pointers,
-Adam

openup failing?

[Adam Thompson]

I've seen a large number failures recently from m:tier's openup tool, 
complaining of:


ftp: connect: Host is down
!!! Cannot retrieve https://stable.mtier.org/openup
!!! Please verify your Internet connection, proxy settings and 
firewall.


I'm seeing this from two different networks/providers/companies, so I'm 
assuming it's not me, but am posting this to validate that assumption.


Assuming it's not just me, does anyone know what's going on with them?  
The relevant routes appear in the DFZ, but none of the *.mtier.org IPs I 
know of respond.


-Adam

Re: The su manual doesn't mention use root account by default

[Adam Thompson]

On 2019-06-12 03:55, Ingo Schwarze wrote:

Even though su(1) can still be used today to relinquish privilege
when you are already root, no more development is done on it and people
rarely look at the manual page.  The last time new functionality was
added to the su(1) manual page was almost a decade ago, and the
last time before that 17 years ago.


Well, su(8) also is used to obtain root privileges in the first place.

FWIW, I regularly use "su" on OpenBSD because it's a relatively 
consistent cross-platform way to have root run a command as someone 
else.  I recall a good number of ports using su(8) internally in, e.g. 
process-control scripts - but that was years ago, not sure if it's still 
true or not.


doas simply isn't available anywhere else (yet).  (IMHO, I don't think a 
portable version of doas has a lot of potential - it's not complicated 
enough! )


During initial system installation & deployment, before doas is 
configured, and assuming you haven't [yet] added your SSH keys to 
~root/.ssh/allowed_keys, it's quite impossible to avoid using su.  
(AFAIK.  If there's another way, let me know!)


I hope you're just saying that su(8) is a mature, stable utility that 
needs no further work right now.  It kind of sounds like you might be 
saying that su(8) could be on the chopping block, much like sudo(8)... 
have I misread that?


-Adam

Re: How does OpenBSD probe for I/O devices?

[Adam Thompson]

On 2019-06-12 13:12, ¯\__/¯ ¯\__/¯ wrote:

I've search for the answer to this question, but I can't find it.
I also read the source code, but I still don't get how it works.
Help pl0x


Not sure exactly what you're looking for...

On modern architectures, most OSes (including OpenBSD) "walk the 
hardware device tree".  The possible topologies and nodes of the device 
tree are controlled by the kernel source code.  OpenBSD does 99% of it 
at boot time, with a few notable exclusions (PCMCIA, PC CARD, USB, can't 
remember what else).


Look under /usr/src/sys/arch/* for functions with "_attach_" in their 
names, which should give you a very rough idea of where to start 
looking.


For both historical and somewhat-current documentation on how this 
works, check out 
https://en.wikipedia.org/wiki/Marshall_Kirk_McKusick#Bibliography .
(I'm unaware of any OpenBSD-specific publications covering that sort of 
thing, but OpenBSD *is* derived from the same BSD UNIX that Kirk wrote 
about.  Lessons learned about one BSD can, usually, have their concepts 
applied to their cousins - although the implementation details have 
diverged quite a bit by now.)


-Adam

security(8) question - how to skip a single file?

[Adam Thompson]

I have RTFMed and googled, but I still can’t figure out how to do one simple
thing: make security(8) ignore a single file that changes on a daily basis,
where that file is otherwise monitored due to /etc/mtree/4.4BSD.dist.



The file in question is /var/unbound/db/root.key, which I have auto-updating.



Yes, I understand why this file is important, but on this particular system,
being lulled into complacency by a daily false-positive security(8) report is
more of a danger than someone managing to hack the root DNS key.



Suggestions or pointers or interpretation of the docs appreciated.



Thanks,

-Adam

Re: security(8) question - how to skip a single file?

[Adam Thompson]

Thanks, Vijay. That’s exactly what I couldn’t find in the documentation.
(Now that I know what to look for, I see the line in security(8)’s manpage
that I overlooked.)

CC’ing list to help the next person with this question…

-Aadm



From: Vijay Sankar [mailto:vsan...@foretell.ca]
Sent: October 6, 2016 10:20
To: Adam Thompson 
Subject: Re: security(8) question - how to skip a single file?



Hi Adam,

Not replying to list in case I did not understand the question.

I have the following towards the end of /etc/changelist

.
.
.
/var/nsd/etc/nsd.conf
# /var/unbound/etc/root.key
/var/unbound/etc/unbound.conf
/var/yp/Makefile.main
/var/yp/Makefile.yp

Is that what you are looking for?

Vijay

Quoting Adam Thompson mailto:athom...@athompso.net>
>:

I have RTFMed and googled, but I still can’t figure out how to do one
simple
thing: make security(8) ignore a single file that changes on a daily basis,
where that file is otherwise monitored due to /etc/mtree/4.4BSD.dist.



The file in question is /var/unbound/db/root.key, which I have auto-updating.



Yes, I understand why this file is important, but on this particular system,
being lulled into complacency by a daily false-positive security(8) report is
more of a danger than someone managing to hack the root DNS key.



Suggestions or pointers or interpretation of the docs appreciated.



Thanks,
-Adam



Vijay Sankar, M.Eng., P.Eng.
ForeTell Technologies Limited
vsan...@foretell.ca
<https://server3.foretell.ca/post/imp/dynamic.php?page=mailbox>

Re: Can't install -current on a Dell precision t3500

[Adam Thompson]

On 2017-02-13 07:11, STeve Andre' wrote:

I'm puzzled and am asking for help.  I'm attempting to install
the -current snapshot (feb 12) on a Dell precision t3500.  The
install formats a 6T disk very quickly, like in 25 seconds.  Hmm.

   After installing the tar files, installboot fails with a
"Bad magic number in superblock".  If I mount the a partition I
see real data.  Changing to a 160G disk everything works & boots,
but not with the 6T disk.

   The t3500 is a sata 2 machine, as is the 160G disk. The 6T disk
is sata 3, but since I see the OS written to the 6T disk it's been
written out OK so thats not it.  I'm missing something with regards
the size of the disk?   Probably I'm forgetting to include something
relevant but I've been dealing with this last night and am tired.
Clues?


You're missing the fact that a T3500 cannot boot off a disk larger than 
2TB, because it supports legacy BIOS boot only, not UEFI.  It is 
possible you might occasionally get lucky but AFAIK the disk geometry 
messes up the BIOS bootloader pretty much 100% of the time.


The Precision line didn't gain UEFI (i.e. >2TB boot disk support) until 
the Tx6xx series, AFAIK.
If your T3500 has a UEFI boot option, then I'm wrong, and you should be 
able to solve your problem by switching to UEFI.


In theory, creating a 2TB partition would be sufficient, but in practice 
it usually isn't.


My solution to these problems is to leave a small USB stick permanently 
installed and boot off that; the easiest way to manage that in OpenBSD 
that I know of is to mount it at /boot and replace /bsd with a symlink 
to /boot/bsd (after moving /bsd, of course).  This will require some 
manual work before rebooting off the ramdisk, naturally, and works IMHO 
best if you use diskids instead of device paths.


-Adam

Re: Topics for revised PF and networking tutorial

[Adam Thompson]

On 2017-04-07 16:41, Mihai Popescu wrote:

I don;t want to offend you folks, but I'm curious and I will ask: is
this BSDCon so useful? Does it pay the efforts?

If someone has time and knowledge to do a PF tutorial he/she can do it
and post. Do you need the Con?

I'm asking this having in my mind Google Summer of (no)Code thread from 
misc@.

Again, i'm asking, i've never been to a Con to sense the feeling.

Thanks.


I'll take a stab at this...

* BSDCan (not Con) is cheap.  Stupidly cheap, in fact: $195/person if 
you're paying your own way.
* The PF tutorial is not free - there is an additional cost ($75) to 
attend the tutorial.
* Peter clearly has the time and knowledge to do it, he has huge amounts 
of raw material on his website, including what amounts to last year's 
tutorial slides, for free - but also chooses to deliver this tutorial.  
Based on the fee, the number of attendees, and the number of presenters, 
no-one's getting rich off this.
* The tutorial is a focused, half-day session where you get to interact 
with the top PF trainer in the world, and ask the questions specific to 
your network.
* Peter keeps his tutorial up to date, unlike most if not all of the 
resources you'll find online, some of which predate the change in syntax 
from several years ago.


You've asked almost the same question as "why does anyone need 
tutorials? just read the man pages!" just at the next level up.  The 
answer is because the man pages aren't adequate to cover every scenario, 
and not everyone can read man pages effectively.  People have different 
learning styles, if nothing else.  I learn best by seeing examples and 
asking questions.  (In fact, the lack of good examples is a pet peeve of 
mine with the OpenBSD man pages, but that's another story.)


I've attended Peter's seminar two?, maybe three? times now, and got 
something new out of it each time - some nuance that wasn't obvious just 
from reading pf.conf(5).  Sometimes it was something Peter said, 
sometimes it was something another attendee said.  That's the value of 
attending any training class or seminar, not just this one for PF.


The tutorial is aimed not at people who would go and produce another 
tutorial, but at ordinary system administrators who don't have time to 
pore over the entire manpage, who want the most relevant information to 
them distilled and delivered efficiently.


Plus, this year it appears that Peter is co-delivering the seminar with 
Massimiliano Stucchi from RIPE, so it will presumably cover a lot of 
IPv6 topics as well, which are poorly represented in existing materials 
and yet increasingly relevant.


Disclaimer: I now help organize (one small) part of BSDCan & PgCon, so 
I'm not *entirely* unbiased, but this is pretty much what I would have 
said the first two years I attended, anyway.


-Adam

Re: Topics for revised PF and networking tutorial

[Adam Thompson]

> -Original Message-
> From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On
> Behalf Of bytevolc...@safe-mail.net
> Sent: April 10, 2017 19:31
>
> > Plus, this year it appears that Peter is co-delivering the seminar
> > with Massimiliano Stucchi from RIPE, so it will presumably cover a lot
> > of IPv6 topics as well, which are poorly represented in existing
> > materials and yet increasingly relevant.
>
> And for those of us who cannot attend, hopefully it will be on video.

I can't say with 100% certainty, but it's unlikely.  The tutorials are not
typically recorded.  (Among other things, AFAIK the people who do the
recording are only present for the conference itself.)  There's also the
matter of the tutorials not  necessarily being covered by the same broadcast
license (hmm, I wonder if Henning will consent this year?).  I don't have
anything to do with any of those parts of the conference, so I can't speak to
the details.

The slides and material are sometimes - not always - made available afterward,
and that depends on the individual presenters.  Max is working for RIPE -
which makes large amounts of their material available for free - and Peter
historically makes his material available online for free, so I therefore have
at least moderate hopes that they'll be able to find a way to sort out the
copyright issues and get the slides put up somewhere.

-Adam

Re: acme-client(1) and http_proxy

[Adam Thompson]

By definition, you will (probably) not be able to use the ACME protocol - it 
only works (normally) when your system is connected directly to the public 
internet with a static IP address.

Simply because you say "behind a corporate firewall", I already know (or at 
least assume) that ACME will not work for you, ever.

ACME, and LetsEncrypt, only handles public websites.  There are ways around 
this, but they are painful and likely not worthwhile - it *will* be cheaper to 
just buy a regular SSL certificate than to get a LetsEncrypt certificate for an 
internal server.

-Adam

> -Original Message-
> From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On
> Behalf Of Manuel Giraud
> Sent: April 21, 2017 07:37
> To: misc@openbsd.org
> Subject: acme-client(1) and http_proxy
> 
> Hi,
> 
> I'm trying to use the new acme-client on a server behind a corporate
> proxy (i.e. I have to set a http_proxy to get out). It seems (from reading
> the code) that acme-client(1) does not honor http_proxy.
> 
> Is this on purpose? If so, can someone point me to another acme client
> that does this?
> --
> Manuel Giraud