Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Nick Holland
On 5/3/19 2:32 PM, Strahil Nikolov wrote:
> On May 3, 2019 10:49:55 PM GMT+03:00, Nick Holland
>  wrote:
>> On 5/2/19 1:52 AM, Consus wrote:
>>> Hi,
>>> 
>>> I've upgraded my systems from 6.4 to 6.5 without a glitch, but I
>>> see that /etc/networks and some other files (like malloc.conf.5)
>>> are
>> still
>>> present, although there is no use for them in the new release.
>>> 
>>> Is there a reason why these files are not listed in "FIles to
>> remove"?
>>> Is there a way to track them? It's not like something gonna
>>> break,
>> but
>>> old configuration files (and manual pages) lying around can make 
>>> someone's life harder during the debug session.
>> 
>> There is no promise that an upgraded machine will be file-for-file 
>> identical to a fresh install.  Here is the list of problems this
>> might cause you, as you can see, it's a long list and quite
>> horrible:
>> 
>> * If you use the same hw for 20 years, you might run out of disk
>> space?
>> 
>> Ok, not very long and not very horrible.
>> 
>> You are trying to solve a non-problem.  And sometimes, 'specially
>> on an upgraded machine, it's great to see how things WERE when the
>> machine was set up.  If you really care, go ahead, delete stuff.
>> 
>> Nick.
> 
> Hi All,
> 
> As I linux guy (my experience in openBSD can be easily measured in
> days) I can share the view  of less experienced user that was planing
> to upgrade from 6.4 to 6.5 and that eneded with a full reinstall.
> 
> I tried to update a VM (stock setup) with a 10 GB disk from 6.4 to
> 6.5  and thus it seemed that booting from the 6.5 DVD will do the
> trick. Sadly the installer never checked the avalable space , but
> just started to do it's stuff until reporting that not enough space
> is available.

The installer didn't check. Neither did you.  Let's blame the installer.

Ok, sure, might be nice, but when there are a snootload of different
platforms with radically different size binaries, it's not trivial.  But
feel free to send in a patch.  Test on two or three different platforms,
first, though, please.

And ... considering the number of times I've seen and heard about Linux
systems hose themselves with upgrades, I question your implication.
Major Linux upgrade?  Most people I know just say "Screw it.  Rebuild,
reload".  Linux might have the edge on incremental upgrades, but
eventually, you are going to need to move to the more current
release...and then OpenBSD starts looking REALLY GOOD.

10g disk?  When I first started working with OpenBSD, that was really
big.  But then, I had to manually partition the disk.  20 years later,
10G is tiny.  The installer auto-partioner is really intended for bigger
disks.   Yeah, you are in "Special Case" territory, which isn't a good
spot to be as a new user.

> Why did the installer allow installation despite the available space
> is low ( even windows checks available space :) )???

The average windows user doesn't know what the units of storage mean.

> Why should the end-user delete old unnecessary/problematic files ?

That's my question.  What's the big deal?  On a modern disk, just ignore
them.  They won't be a problem until long after your rotate out the hw.
 Problem is, you used a 2001 vintage size disk.  You should have rotated
that out around 2005.

And I'm curious how a CentOS 6 to Centos 7 upgrade would go on a 10G
disk.  I have my suspicions, and I suspect it would be entertaining to
watch...assuming it wasn't something you were dependent upon.

> Usually we do have package management system to take care of that (or
> at least to rename those files in case we really need them).

Yeah, you need to wait until Linux "package management" screws itself
into a knot for you.

> For me, system upgrade is a very complicated  and  error prone
> procedure.

OpenBSD has what I call a "Learning Curb".  You gotta lift your feet.
Not a lot, it's not hard, but you can't just shuffle along mindlessly
and expect to be carried to the next level without your engaging your brain

If you used Linux for a little bit and figured that OpenBSD is "just
like Linux, but different", yeah, no, you are going to be disappointed.
 Different beast.  From a management perspective, I'd say Linux and
Windows are much more alike than Linux and OpenBSD.  Linux is written
for and by those frustrated with Windows ("Reinventing Windows,
poorly").  OpenBSD is Unix.  It's probably the simplest Unix out there
to use and manage, but it's not Windows (or Linux).

Or...  Think of Linux (and windows) as the big cushy luxury car.  Easy
to drive, assuming you work within the anticipated parameters, but you
really have no idea what's going on under the hood.  "you aren't
supposed to".  That's the design goal, and it works pretty well...until
it doesn't.  OpenBSD is more like a semi-primative small car with tight
suspension and a stick-shift trans.  It's got antilock brakes, but for
the most part, it assumes you know what you are doing when you get
behind the wheel.  When it gets 

Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Predrag Punosevac
Strahil Nikolov wrote:

> 
> On May 3, 2019 10:49:55 PM GMT+03:00, Nick Holland 
>  \
> wrote:
> > On 5/2/19 1:52 AM, Consus wrote:
> > > Hi,
> > > 
> > > I've upgraded my systems from 6.4 to 6.5 without a glitch, but I see
> > > that /etc/networks and some other files (like malloc.conf.5) are
> > still
> > > present, although there is no use for them in the new release.
> > > 
> > > Is there a reason why these files are not listed in "FIles to
> > remove"?
> > > Is there a way to track them? It's not like something gonna break,
> > but
> > > old configuration files (and manual pages) lying around can make
> > > someone's life harder during the debug session.
> > 
> > There is no promise that an upgraded machine will be file-for-file
> > identical to a fresh install.  Here is the list of problems this might
> > cause you, as you can see, it's a long list and quite horrible:
> > 
> > * If you use the same hw for 20 years, you might run out of disk space?
> > 
> > Ok, not very long and not very horrible.
> > 
> > You are trying to solve a non-problem.  And sometimes, 'specially on an
> > upgraded machine, it's great to see how things WERE when the machine
> > was
> > set up.  If you really care, go ahead, delete stuff.
> > 
> > Nick.
> 
> Hi All,
> 
> As I linux guy (my experience in openBSD can be easily measured in days)
> I can share the view  of less experienced user that was planing  to
> upgrade from 6.4 to 6.5 and that eneded with a full reinstall.
> 

I just upgraded 18 servers running mission critical network
infrastructure and services for a research group of 150 people.
Everything went without a glitch. Some of the servers have been
continuously upgraded since OpenBSD 5.4. That is a solid 5 years which
is a typical lifespan of a production server. 

Just as a comparison, I am still afraid to upgrade dozen or so file
servers and jail hosts running FreeBSD 11.2 to 12.0 in-spite of root on
the ZFS mirror and beadm. I typically wait at least year and a half
after initial release of Red Hat to do fresh re-installation of our
computing nodes. Red Hat as you know doesn't support upgrade between the
major releases. Ubuntu (deep learning guys love that crap) upgrade from
16.04 to 18.04 should not be attempted on the production server. On the
top of it network stack on Ubuntu 18.04 is completely broken (at lease
running as Xen DomU. I was too afraid to try on our AWS instances).


> I tried to update a VM (stock setup) with a 10 GB disk from 6.4 to 6.5
> and thus it seemed that booting from the 6.5 DVD will do the trick.
> Sadly the installer never checked the avalable space , but just started
> to do it's stuff until reporting that not enough space is available.
> 
> Why did the installer allow installation despite the available space is
> low ( even windows checks available space :) )???
> 
> Why should the end-user delete old unnecessary/problematic files ?


Because Theo's misplaced his crystal ball and without it, it's
impossible for him to tell which of your files are old and unnecessary
and which once are your local modifications and important data files. 


> Usually we do have package management system to take care of that (or at
> least to rename those files in case we really need them).
> 
> For me, system upgrade is a very complicated  and  error prone
> procedure.
> 

Just move on. Stick to what you know and feel comfortable working with.

Cheers,
Predrag

> P.S.: No offence here, just sharing my thoughts.
> 
> Best Regards,
> Strahil Nikolov



Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Strahil Nikolov
On May 3, 2019 10:49:55 PM GMT+03:00, Nick Holland 
 wrote:
>On 5/2/19 1:52 AM, Consus wrote:
>> Hi,
>> 
>> I've upgraded my systems from 6.4 to 6.5 without a glitch, but I see
>> that /etc/networks and some other files (like malloc.conf.5) are
>still
>> present, although there is no use for them in the new release.
>> 
>> Is there a reason why these files are not listed in "FIles to
>remove"?
>> Is there a way to track them? It's not like something gonna break,
>but
>> old configuration files (and manual pages) lying around can make
>> someone's life harder during the debug session.
>
>There is no promise that an upgraded machine will be file-for-file
>identical to a fresh install.  Here is the list of problems this might
>cause you, as you can see, it's a long list and quite horrible:
>
>* If you use the same hw for 20 years, you might run out of disk space?
>
>Ok, not very long and not very horrible.
>
>You are trying to solve a non-problem.  And sometimes, 'specially on an
>upgraded machine, it's great to see how things WERE when the machine
>was
>set up.  If you really care, go ahead, delete stuff.
>
>Nick.

Hi All,

As I linux guy (my experience in openBSD can be easily measured in days) I can 
share the view  of less experienced user that was planing  to upgrade from 6.4 
to 6.5 and that eneded with a full reinstall.

I tried to update a VM (stock setup) with a 10 GB disk from 6.4 to 6.5  and 
thus it seemed that booting from the 6.5 DVD will do the trick.
Sadly the installer never checked the avalable space , but just started to do 
it's stuff until reporting that not enough space is available.

Why did the installer allow installation despite the available space is low ( 
even windows checks available space :) )???

Why should the end-user delete old unnecessary/problematic files ? Usually we 
do have package management system to take care of that (or at least to rename 
those files in case we really need them).

For me, system upgrade is a very complicated  and  error prone procedure.

P.S.: No offence here, just sharing my thoughts.

Best Regards,
Strahil Nikolov



Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Steve Williams

On 02/05/2019 6:23 a.m., Stephen Gregoratto wrote:

On 2019-05-02 11:46, Noth wrote:

I set up a script for sysclean:

cat sysclean65.txt | while read line ; do rm -rf "${line}" ; done

Nitpick, but this could be shortened to:

   xargs rm -rf < sysclean??.txt

Just tested this on my server, so it should work fine.


If there are filenames with spaces in them, I think that command won't 
work as expected.


Cheers,
Steve Williams



Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Noth



On 03/05/2019 10:48, Gonzalo L. Rodriguez wrote:

On Thu, 02 May 2019 at 11:46:20 +0200, Noth wrote:

On 02/05/2019 11:02, Consus wrote:

On 10:27 Thu 02 May, Markus Hennecke wrote:

Am 02.05.2019 um 09:52 schrieb Consus:

I've upgraded my systems from 6.4 to 6.5 without a glitch, but I see
that /etc/networks and some other files (like malloc.conf.5) are still
present, although there is no use for them in the new release.

Is there a reason why these files are not listed in "FIles to remove"?
Is there a way to track them? It's not like something gonna break, but
old configuration files (and manual pages) lying around can make
someone's life harder during the debug session.

Take a look at the sysutils/sysclean port.

That's pretty much how I discovered this. But I want to know the
"official" way. Maybe there is a reason why e.g. perl files are to be
removed, but man pages are not.


I set up a script for sysclean:

cat sysclean65.txt | while read line ; do rm -rf "${line}" ; done

You probably want some /etc/sysclean.ignore bits before that
Agreed, thanks for the suggestion. Hadn't read the manpage properly, 
just for a change. With that you can just pipe sysclean's output to a 
delete script...




Re: umb0 ucom0 umodem0 detached

2019-05-03 Thread Theo de Raadt
Malte Wedel  wrote:

> Hello Misc,
> 
> I have OpenBSD running on a Thinkpad X1 Carbon, 3rd Gen, everything is
> working great, except for the builtin 4G modem "umb0" which detaches and
> disappears from time to time and needs a reboot to become available
> again. I found this thread which seems to be exactly my issue:
> 
> http://openbsd-archive.7691.n7.nabble.com/Re-Current-197-Nov-5-umb0-ucom0-umodem0-detached-td330969.html
> 
> When the issue occurs, the following is written into /var/log/messages:
> 
> May  3 14:42:09 carbon /bsd: umb0 detached
> May  3 14:42:09 carbon /bsd: ucom0 detached
> May  3 14:42:09 carbon /bsd: umodem0 detached

I suspect this detachment is caused by the driver violating the protocol,
and the device gives up.

> May  3 14:42:26 carbon /bsd: uhub0: port 4, set config 0 at addr 2 failed
> May  3 14:42:26 carbon /bsd: uhub0: device problem, disabling port 4

This is a different problem, and is believed to be a race condition on
port power-up.  This problem has been moving around for years...



umb0 ucom0 umodem0 detached

2019-05-03 Thread Malte Wedel
Hello Misc,

I have OpenBSD running on a Thinkpad X1 Carbon, 3rd Gen, everything is
working great, except for the builtin 4G modem "umb0" which detaches and
disappears from time to time and needs a reboot to become available
again. I found this thread which seems to be exactly my issue:

http://openbsd-archive.7691.n7.nabble.com/Re-Current-197-Nov-5-umb0-ucom0-umodem0-detached-td330969.html

When the issue occurs, the following is written into /var/log/messages:

May  3 14:42:09 carbon /bsd: umb0 detached
May  3 14:42:09 carbon /bsd: ucom0 detached
May  3 14:42:09 carbon /bsd: umodem0 detached
May  3 14:42:26 carbon /bsd: uhub0: port 4, set config 0 at addr 2 failed
May  3 14:42:26 carbon /bsd: uhub0: device problem, disabling port 4

The issue also occurs with GENERIC.MP kernel. I guess you need the issue
reproduced with XHCI_DEBUG enabled? Is there any other way than
rebooting to enable port 4 on uhub0 again?

Regards,
Malte

OpenBSD 6.5 (CARBON) #4: Thu Apr 25 01:59:26 CEST 2019
    ma...@carbon.malte.de:/usr/src/sys/arch/amd64/compile/CARBON
real mem = 8446996480 (8055MB)
avail mem = 8189042688 (7809MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.7 @ 0xccbfd000 (65 entries)
bios0: vendor LENOVO version "N14ET32W (1.10 )" date 08/13/2015
bios0: LENOVO 20BTS33S01
acpi0 at bios0: rev 2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP SLIC ASF! HPET ECDT APIC MCFG SSDT SSDT SSDT
SSDT SSDT SSDT SSDT SSDT SSDT SSDT PCCT SSDT TCPA SSDT UEFI MSDM BATB
FPDT UEFI DMAR
acpi0: wakeup devices LID_(S4) SLPB(S3) IGBE(S4) EXP2(S4) XHCI(S3) EHC1(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpihpet0 at acpi0: 14318179 Hz
acpiec0 at acpi0
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz, 2195.26 MHz, 06-3d-04
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.2.4.1.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz, 2194.93 MHz, 06-3d-04
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
cpu2: Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz, 2194.93 MHz, 06-3d-04
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
cpu3: Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz, 2194.93 MHz, 06-3d-04
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,3DNOWP,PERF,ITSC,FSGSBASE,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,RDSEED,ADX,SMAP,PT,IBRS,IBPB,STIBP,L1DF,SSBD,SENSOR,ARAT,XSAVEOPT,MELTDOWN
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
ioapic0 at mainbus0: apid 2 pa 0xfec0, version 20, 40 pins
acpimcfg0 at acpi0
acpimcfg0: addr 0xf800, bus 0-63
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus -1 (PEG_)
acpiprt2 at acpi0: bus 3 (EXP1)
acpiprt3 at acpi0: bus 4 (EXP2)
acpiprt4 at acpi0: bus -1 (EXP3)
acpiprt5 at acpi0: bus -1 (EXP6)
acpicpu0 at acpi0: C3(200@233 mwait.1@0x40), C2(200@148 mwait.1@0x33),
C1(1000@1 mwait.1), PSS
acpicpu1 at acpi0: C3(200@233 mwait.1@0x40), C2(200@148 mwait.1@0x33),
C1(1000@1 mwait.1), PSS
acpicpu2 at acpi0: 

Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Consus
On 15:08 Thu 02 May, Ingo Schwarze wrote:
> Hi Nick,
> 
> Nick Holland wrote on Thu, May 02, 2019 at 08:04:32AM -0400:
> 
> > There is no promise that an upgraded machine will be file-for-file
> > identical to a fresh install.  Here is the list of problems this might
> > cause you, as you can see, it's a long list and quite horrible:
> > 
> > * If you use the same hw for 20 years, you might run out of disk space?
> > 
> > Ok, not very long and not very horrible.
> > 
> > You are trying to solve a non-problem.  And sometimes, 'specially on an
> > upgraded machine, it's great to see how things WERE when the machine was
> > set up.  If you really care, go ahead, delete stuff.
> 
> There is (at least) one slight issue that doesn't have an official
> solution yet: manual pages.
> 
> It might be a good idea to do
> 
>   # rm -rf /usr/share/man/* /usr/X11R6/man/*
> 
> immediately before an upgrade.
> 
> If you don't do that, man(1) might serve you stale manual pages
> afterwards that were removed from the sets, containing information
> that no longer applies.
> 
> All the same, so far, we don't officially recommend it, and even i
> usually forget about it when doing upgrades.
> 
> Should that be automated?  Or are there risks of downsides or side
> effects?  I'm not sure.  Either way, it's hardly a very serious
> problem, it's merely slightly annoying.
> 
> Yours,
>   Ingo

Maybe it's a good idea to note this on the upgrade page? Something like
"the upgrade procedure may leave some files behing; you can manually
clean them up using sysclean package"?



Re: bgpd acting up, dropping connected/static network statements

2019-05-03 Thread Claudio Jeker
On Fri, May 03, 2019 at 11:52:07AM +0200, open...@kene.nu wrote:
> Much appreciated, will test. Did this also affect previous versions
> (specifically thinking about 6.3 and 6.4)?

No. This code was changed after 6.4
 
> On Fri, May 3, 2019 at 11:43 AM Claudio Jeker  
> wrote:
> >
> > On Fri, May 03, 2019 at 09:59:40AM +0200, open...@kene.nu wrote:
> > > Hello,
> > >
> > > I am seeing strange behaviour of bgpd in 6.5.
> > >
> > > Not sure what causes the networks in bgpd to disappear but they do
> > > disappear and performing a netstart pick the network back up again in
> > > bgpd. I cannot see this in either 6.4 or 6.3. One triggering factor
> > > seems to be restarting the bgpd process.
> > >
> > > Excerpt form the daemon logs (bgpd restart or reload):
> > > May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
> > > (LOCAL) AS64712: announce 10.1.150.0/24
> > > May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
> > > (LOCAL) AS64712: withdraw announce 10.1.150.0/24
> > >
> > > If one performs a netstart, of relevant vlan interfaces, the
> > > announcements seem to survive a bgpd reload. Static routes never
> > > survive a restart or reload.
> > >
> > > Some additional commands to show behaviour:
> > > # uname -a
> > > OpenBSD host 6.5 GENERIC.MP#3 amd64
> > > # ifconfig vlan190
> > > vlan190: flags=8943 mtu 
> > > 1500
> > > lladdr 
> > > index 33 priority 0 llprio 3
> > > encap: vnetid 190 parent em0 txprio packet
> > > groups: vlan
> > > media: Ethernet autoselect (1000baseT full-duplex,master)
> > > status: active
> > > inet 10.1.150.2 netmask 0xff00 broadcast 10.1.150.255
> > > # grep connected /etc/bgpd.conf
> > > network inet connected set community 65000:64712
> > > # bgpctl sh ip bgp 10.1.150.0/24
> > > flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
> > >S = Stale, E = Error
> > > origin validation state: N = not-found, V = valid, ! = invalid
> > > origin: i = IGP, e = EGP, ? = Incomplete
> > >
> > > flags ovs destination  gateway  lpref   med aspath origin
> > > # sh /etc/netstart vlan150
> > > # bgpctl sh ip bgp 10.1.150.0/24
> > > flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
> > >S = Stale, E = Error
> > > origin validation state: N = not-found, V = valid, ! = invalid
> > > origin: i = IGP, e = EGP, ? = Incomplete
> > >
> > > flags ovs destination  gateway  lpref   med aspath origin
> > > AI*>N 10.1.150.0/240.0.0.0100 0 i
> > >
> > >
> > > My bgpd.conf:
> > > # GLOBALS
> > > AS 64712
> > > router-id 172.30.198.4
> > > holdtime 9
> > > log updates
> > >
> > > prefix-set internal { 10.0.0.0/8 prefixlen >= 16, 10.60.0.0/15,
> > > 172.20.0.0/16 prefixlen <= 32, 172.29.0.0/16 prefixlen >= 24,
> > > 172.29.248.10/31 prefixlen = 32, 172.30.0.0/16 prefixlen >= 24 }
> > >
> > > # DEFAULT FILTERING
> > > deny from any
> > > deny to any
> > >
> > > # NETWORK STATEMENTS
> > > network 172.30.198.4/32 set community 65000:64712
> > > network inet connected set community 65000:64712
> > > network inet static set community 65000:64712
> > >
> > > # NEIGHBORS
> > > group "vpn" {
> > > announce IPv6 none
> > > route-reflector
> > > remote-as 64712
> > >
> > > neighbor 10.1.230.9 {
> > > descr "vpn1"
> > > }
> > > neighbor 10.1.230.10 {
> > > descr "vpn2"
> > > }
> > > }
> > >
> > > # SOURCE FILTERING
> > > allow to group "vpn" prefix-set internal community 65000:64712
> > > # DEST FILTERING
> > > allow from group "vpn" prefix-set internal
> > > # TRAFFIC ENGINEERING
> > > match to group "vpn" set nexthop 10.1.230.12
> > > match to any prefix 172.30.198.4/32 set nexthop self
> > >
> >
> > Thanks for the detailed report. I quick workaround is to reload the config
> > twice. Then the networks are added again. The proper fix is attached.
> >
> > The problem was that when already present networks were readded the
> > function kr_net_redist_add() returned 0 which was propegated to
> > kr_net_match() which then caused kr_redistribute() to actually remove the
> > prefix.
> >
> > I changed the code to only return 0 when there is actually the case that
> > the network being added is shadowed by another one and therefor this
> > prefix should be removed. While there I also fixed a memory leak ;)
> >
> > Please test.
> > --
> > :wq Claudio
> >
> > Index: kroute.c
> > ===
> > RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
> > retrieving revision 1.235
> > diff -u -p -r1.235 kroute.c
> > --- kroute.c7 Mar 2019 07:42:36 -   1.235
> > +++ kroute.c3 May 2019 09:32:10 -
> > @@ -1230,19 +1230,19 @@ kr_net_redist_add(struct ktable *kt, str
> >
> > xr = RB_INSERT(kredist_tree, >kredist, r);
> > if (xr != NULL) {
> > -   if (dynamic == xr->dynamic || dynamic) {
> > +   free(r);
> > +
> > +  

Re: bgpd acting up, dropping connected/static network statements

2019-05-03 Thread openbsd
Much appreciated, will test. Did this also affect previous versions
(specifically thinking about 6.3 and 6.4)?

On Fri, May 3, 2019 at 11:43 AM Claudio Jeker  wrote:
>
> On Fri, May 03, 2019 at 09:59:40AM +0200, open...@kene.nu wrote:
> > Hello,
> >
> > I am seeing strange behaviour of bgpd in 6.5.
> >
> > Not sure what causes the networks in bgpd to disappear but they do
> > disappear and performing a netstart pick the network back up again in
> > bgpd. I cannot see this in either 6.4 or 6.3. One triggering factor
> > seems to be restarting the bgpd process.
> >
> > Excerpt form the daemon logs (bgpd restart or reload):
> > May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
> > (LOCAL) AS64712: announce 10.1.150.0/24
> > May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
> > (LOCAL) AS64712: withdraw announce 10.1.150.0/24
> >
> > If one performs a netstart, of relevant vlan interfaces, the
> > announcements seem to survive a bgpd reload. Static routes never
> > survive a restart or reload.
> >
> > Some additional commands to show behaviour:
> > # uname -a
> > OpenBSD host 6.5 GENERIC.MP#3 amd64
> > # ifconfig vlan190
> > vlan190: flags=8943 mtu 1500
> > lladdr 
> > index 33 priority 0 llprio 3
> > encap: vnetid 190 parent em0 txprio packet
> > groups: vlan
> > media: Ethernet autoselect (1000baseT full-duplex,master)
> > status: active
> > inet 10.1.150.2 netmask 0xff00 broadcast 10.1.150.255
> > # grep connected /etc/bgpd.conf
> > network inet connected set community 65000:64712
> > # bgpctl sh ip bgp 10.1.150.0/24
> > flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
> >S = Stale, E = Error
> > origin validation state: N = not-found, V = valid, ! = invalid
> > origin: i = IGP, e = EGP, ? = Incomplete
> >
> > flags ovs destination  gateway  lpref   med aspath origin
> > # sh /etc/netstart vlan150
> > # bgpctl sh ip bgp 10.1.150.0/24
> > flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
> >S = Stale, E = Error
> > origin validation state: N = not-found, V = valid, ! = invalid
> > origin: i = IGP, e = EGP, ? = Incomplete
> >
> > flags ovs destination  gateway  lpref   med aspath origin
> > AI*>N 10.1.150.0/240.0.0.0100 0 i
> >
> >
> > My bgpd.conf:
> > # GLOBALS
> > AS 64712
> > router-id 172.30.198.4
> > holdtime 9
> > log updates
> >
> > prefix-set internal { 10.0.0.0/8 prefixlen >= 16, 10.60.0.0/15,
> > 172.20.0.0/16 prefixlen <= 32, 172.29.0.0/16 prefixlen >= 24,
> > 172.29.248.10/31 prefixlen = 32, 172.30.0.0/16 prefixlen >= 24 }
> >
> > # DEFAULT FILTERING
> > deny from any
> > deny to any
> >
> > # NETWORK STATEMENTS
> > network 172.30.198.4/32 set community 65000:64712
> > network inet connected set community 65000:64712
> > network inet static set community 65000:64712
> >
> > # NEIGHBORS
> > group "vpn" {
> > announce IPv6 none
> > route-reflector
> > remote-as 64712
> >
> > neighbor 10.1.230.9 {
> > descr "vpn1"
> > }
> > neighbor 10.1.230.10 {
> > descr "vpn2"
> > }
> > }
> >
> > # SOURCE FILTERING
> > allow to group "vpn" prefix-set internal community 65000:64712
> > # DEST FILTERING
> > allow from group "vpn" prefix-set internal
> > # TRAFFIC ENGINEERING
> > match to group "vpn" set nexthop 10.1.230.12
> > match to any prefix 172.30.198.4/32 set nexthop self
> >
>
> Thanks for the detailed report. I quick workaround is to reload the config
> twice. Then the networks are added again. The proper fix is attached.
>
> The problem was that when already present networks were readded the
> function kr_net_redist_add() returned 0 which was propegated to
> kr_net_match() which then caused kr_redistribute() to actually remove the
> prefix.
>
> I changed the code to only return 0 when there is actually the case that
> the network being added is shadowed by another one and therefor this
> prefix should be removed. While there I also fixed a memory leak ;)
>
> Please test.
> --
> :wq Claudio
>
> Index: kroute.c
> ===
> RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
> retrieving revision 1.235
> diff -u -p -r1.235 kroute.c
> --- kroute.c7 Mar 2019 07:42:36 -   1.235
> +++ kroute.c3 May 2019 09:32:10 -
> @@ -1230,19 +1230,19 @@ kr_net_redist_add(struct ktable *kt, str
>
> xr = RB_INSERT(kredist_tree, >kredist, r);
> if (xr != NULL) {
> -   if (dynamic == xr->dynamic || dynamic) {
> +   free(r);
> +
> +   if (dynamic != xr->dynamic && dynamic) {
> /*
> -* ignore update, equal announcement already present,
> -* or a non-dynamic announcement is already present
> -* which has preference.
> +* ignore update a non-dynamic announcement is
> + 

Re: bgpd acting up, dropping connected/static network statements

2019-05-03 Thread Claudio Jeker
On Fri, May 03, 2019 at 09:59:40AM +0200, open...@kene.nu wrote:
> Hello,
> 
> I am seeing strange behaviour of bgpd in 6.5.
> 
> Not sure what causes the networks in bgpd to disappear but they do
> disappear and performing a netstart pick the network back up again in
> bgpd. I cannot see this in either 6.4 or 6.3. One triggering factor
> seems to be restarting the bgpd process.
> 
> Excerpt form the daemon logs (bgpd restart or reload):
> May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
> (LOCAL) AS64712: announce 10.1.150.0/24
> May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
> (LOCAL) AS64712: withdraw announce 10.1.150.0/24
> 
> If one performs a netstart, of relevant vlan interfaces, the
> announcements seem to survive a bgpd reload. Static routes never
> survive a restart or reload.
> 
> Some additional commands to show behaviour:
> # uname -a
> OpenBSD host 6.5 GENERIC.MP#3 amd64
> # ifconfig vlan190
> vlan190: flags=8943 mtu 1500
> lladdr 
> index 33 priority 0 llprio 3
> encap: vnetid 190 parent em0 txprio packet
> groups: vlan
> media: Ethernet autoselect (1000baseT full-duplex,master)
> status: active
> inet 10.1.150.2 netmask 0xff00 broadcast 10.1.150.255
> # grep connected /etc/bgpd.conf
> network inet connected set community 65000:64712
> # bgpctl sh ip bgp 10.1.150.0/24
> flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
>S = Stale, E = Error
> origin validation state: N = not-found, V = valid, ! = invalid
> origin: i = IGP, e = EGP, ? = Incomplete
> 
> flags ovs destination  gateway  lpref   med aspath origin
> # sh /etc/netstart vlan150
> # bgpctl sh ip bgp 10.1.150.0/24
> flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
>S = Stale, E = Error
> origin validation state: N = not-found, V = valid, ! = invalid
> origin: i = IGP, e = EGP, ? = Incomplete
> 
> flags ovs destination  gateway  lpref   med aspath origin
> AI*>N 10.1.150.0/240.0.0.0100 0 i
> 
> 
> My bgpd.conf:
> # GLOBALS
> AS 64712
> router-id 172.30.198.4
> holdtime 9
> log updates
> 
> prefix-set internal { 10.0.0.0/8 prefixlen >= 16, 10.60.0.0/15,
> 172.20.0.0/16 prefixlen <= 32, 172.29.0.0/16 prefixlen >= 24,
> 172.29.248.10/31 prefixlen = 32, 172.30.0.0/16 prefixlen >= 24 }
> 
> # DEFAULT FILTERING
> deny from any
> deny to any
> 
> # NETWORK STATEMENTS
> network 172.30.198.4/32 set community 65000:64712
> network inet connected set community 65000:64712
> network inet static set community 65000:64712
> 
> # NEIGHBORS
> group "vpn" {
> announce IPv6 none
> route-reflector
> remote-as 64712
> 
> neighbor 10.1.230.9 {
> descr "vpn1"
> }
> neighbor 10.1.230.10 {
> descr "vpn2"
> }
> }
> 
> # SOURCE FILTERING
> allow to group "vpn" prefix-set internal community 65000:64712
> # DEST FILTERING
> allow from group "vpn" prefix-set internal
> # TRAFFIC ENGINEERING
> match to group "vpn" set nexthop 10.1.230.12
> match to any prefix 172.30.198.4/32 set nexthop self
> 

Thanks for the detailed report. I quick workaround is to reload the config
twice. Then the networks are added again. The proper fix is attached.

The problem was that when already present networks were readded the
function kr_net_redist_add() returned 0 which was propegated to
kr_net_match() which then caused kr_redistribute() to actually remove the
prefix.

I changed the code to only return 0 when there is actually the case that
the network being added is shadowed by another one and therefor this
prefix should be removed. While there I also fixed a memory leak ;)

Please test.
-- 
:wq Claudio

Index: kroute.c
===
RCS file: /cvs/src/usr.sbin/bgpd/kroute.c,v
retrieving revision 1.235
diff -u -p -r1.235 kroute.c
--- kroute.c7 Mar 2019 07:42:36 -   1.235
+++ kroute.c3 May 2019 09:32:10 -
@@ -1230,19 +1230,19 @@ kr_net_redist_add(struct ktable *kt, str
 
xr = RB_INSERT(kredist_tree, >kredist, r);
if (xr != NULL) {
-   if (dynamic == xr->dynamic || dynamic) {
+   free(r);
+
+   if (dynamic != xr->dynamic && dynamic) {
/*
-* ignore update, equal announcement already present,
-* or a non-dynamic announcement is already present
-* which has preference.
+* ignore update a non-dynamic announcement is
+* already present which has preference.
 */
-   free(r);
return 0;
}
/*
-* only the case where xr->dynamic == 1 and dynamic == 0
-* ends up here and in this case non-dynamic announcments
-* are preferred. Override dynamic flag.
+* only 

Re: Upgrade procedure (6.4 -> 6.5)

2019-05-03 Thread Gonzalo L. Rodriguez
On Thu, 02 May 2019 at 11:46:20 +0200, Noth wrote:
> 
> On 02/05/2019 11:02, Consus wrote:
> > On 10:27 Thu 02 May, Markus Hennecke wrote:
> > > Am 02.05.2019 um 09:52 schrieb Consus:
> > > > I've upgraded my systems from 6.4 to 6.5 without a glitch, but I see
> > > > that /etc/networks and some other files (like malloc.conf.5) are still
> > > > present, although there is no use for them in the new release.
> > > > 
> > > > Is there a reason why these files are not listed in "FIles to remove"?
> > > > Is there a way to track them? It's not like something gonna break, but
> > > > old configuration files (and manual pages) lying around can make
> > > > someone's life harder during the debug session.
> > > Take a look at the sysutils/sysclean port.
> > That's pretty much how I discovered this. But I want to know the
> > "official" way. Maybe there is a reason why e.g. perl files are to be
> > removed, but man pages are not.
> > 
> I set up a script for sysclean:
> 
> cat sysclean65.txt | while read line ; do rm -rf "${line}" ; done

You probably want some /etc/sysclean.ignore bits before that

> sysclean65.txt is obtained by running sysclean -a >>sysclean65.txt . I don't
> run that line in sysclean65.sh because the files have to be reviewed to
> prevent deletion of any additional files you may have added, like certs or
> scripts.
> 
> HTH
> 
> Noth
> 


-- 

- gonzalo



bgpd acting up, dropping connected/static network statements

2019-05-03 Thread openbsd
Hello,

I am seeing strange behaviour of bgpd in 6.5.

Not sure what causes the networks in bgpd to disappear but they do
disappear and performing a netstart pick the network back up again in
bgpd. I cannot see this in either 6.4 or 6.3. One triggering factor
seems to be restarting the bgpd process.

Excerpt form the daemon logs (bgpd restart or reload):
May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
(LOCAL) AS64712: announce 10.1.150.0/24
May  3 07:44:25 host bgpd[94972]: Rib Loc-RIB: neighbor 172.30.198.4
(LOCAL) AS64712: withdraw announce 10.1.150.0/24

If one performs a netstart, of relevant vlan interfaces, the
announcements seem to survive a bgpd reload. Static routes never
survive a restart or reload.

Some additional commands to show behaviour:
# uname -a
OpenBSD host 6.5 GENERIC.MP#3 amd64
# ifconfig vlan190
vlan190: flags=8943 mtu 1500
lladdr 
index 33 priority 0 llprio 3
encap: vnetid 190 parent em0 txprio packet
groups: vlan
media: Ethernet autoselect (1000baseT full-duplex,master)
status: active
inet 10.1.150.2 netmask 0xff00 broadcast 10.1.150.255
# grep connected /etc/bgpd.conf
network inet connected set community 65000:64712
# bgpctl sh ip bgp 10.1.150.0/24
flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
   S = Stale, E = Error
origin validation state: N = not-found, V = valid, ! = invalid
origin: i = IGP, e = EGP, ? = Incomplete

flags ovs destination  gateway  lpref   med aspath origin
# sh /etc/netstart vlan150
# bgpctl sh ip bgp 10.1.150.0/24
flags: * = Valid, > = Selected, I = via IBGP, A = Announced,
   S = Stale, E = Error
origin validation state: N = not-found, V = valid, ! = invalid
origin: i = IGP, e = EGP, ? = Incomplete

flags ovs destination  gateway  lpref   med aspath origin
AI*>N 10.1.150.0/240.0.0.0100 0 i


My bgpd.conf:
# GLOBALS
AS 64712
router-id 172.30.198.4
holdtime 9
log updates

prefix-set internal { 10.0.0.0/8 prefixlen >= 16, 10.60.0.0/15,
172.20.0.0/16 prefixlen <= 32, 172.29.0.0/16 prefixlen >= 24,
172.29.248.10/31 prefixlen = 32, 172.30.0.0/16 prefixlen >= 24 }

# DEFAULT FILTERING
deny from any
deny to any

# NETWORK STATEMENTS
network 172.30.198.4/32 set community 65000:64712
network inet connected set community 65000:64712
network inet static set community 65000:64712

# NEIGHBORS
group "vpn" {
announce IPv6 none
route-reflector
remote-as 64712

neighbor 10.1.230.9 {
descr "vpn1"
}
neighbor 10.1.230.10 {
descr "vpn2"
}
}

# SOURCE FILTERING
allow to group "vpn" prefix-set internal community 65000:64712
# DEST FILTERING
allow from group "vpn" prefix-set internal
# TRAFFIC ENGINEERING
match to group "vpn" set nexthop 10.1.230.12
match to any prefix 172.30.198.4/32 set nexthop self



Re: Upgrading a CARP firewall cluster

2019-05-03 Thread mabi
‐‐‐ Original Message ‐‐‐
On Tuesday, April 30, 2019 9:29 PM, Lyndon Nerenberg  wrote:

> On our systems, we run the 'a' machine as primary and the 'b' machine
> as backup. When upgrading, we do the 'b' machine first, since this
> doesn't disrupt the primary. After the 'b' machine is fully configured,
> monitor its state table to ensure it's consistent with the 'a'
> machine. Once you are convinced pf is staying in sync, demote the
> 'a' machine and upgrade it.

Thanks for your procedure tips, that's pretty much the same procedure I use 
except that I didn't even demote the "a" machine before upgrading it. Now I 
guess demoting the machine in question before upgrading it is the best practice 
and so I checked the OpenBSD FAQ about CARP and see different methods. The 
"carpdemote" way seems the cleanest way so as I have 8 carp interfaces all in 
the default carp group, should I simply run the following command:

$ ifconfig -g carp carpdemote 50

or what is your way of demoting the server before upgading it?

Regards,
Mabi