Re: OpenBSD.calendar patch
On Sat, Jun 27, 2020 at 11:56:49AM -0700, jungle boogie wrote: > Hi Friends, > > Here's a small patch to the OpenBSD.calendar. I didn't want to spend too > much time on this until I find out if it would be accepted. > morning. i think if such a file is worth having, it's worth updating. having said that, i'm counting myself out from looking after this one - hopefully someone else will pick it up. jmc > Here's my changes: > > > --- /usr/share/calendar/calendar.openbsd Fri Jun 26 21:01:56 2020 > +++ calendar.openbsd Sat Jun 27 01:37:40 2020 > @@ -10,15 +10,19 @@ > Jan 06 IPF gets integrated into the OpenBSD kernel, 1996 > Jan 06 NRL IPv6 addition to OpenBSD, 1999 > Jan 09 n2k10: Network hackathon, Melbourne, Australia, 17 developers, > 2010 > +Jan 12 u2k20: Uckermark hackathon, Urckermark, Germany, 14 developers, > 2020 > Jan 13 n2k13: Network hackathon, Dunedin, New Zealand, 17 developers, > 2013 > +Jan 17 a2k19: Antipodean hackathon, Wellington, New Zealand, 18 > developers, 2019 > Jan 18 n2k14: Mini-hackathon, Dunedin, New Zealand, 15 developers, 2014 > Jan 20 Bind 9 goes into the tree, 2003 > +Jan 20 a2k20: Antipodean hackathon, Hobart, Tasmania, 17 developers, > 2020 > Jan 26 Anoncvs service inaugurated, 1996 > Jan 26 n2k9: Network hackathon, Basel, Switzerland, 19 developers, 2009 > Jan 27 OpenBSD/amd64 port is added, from NetBSD, 2004 > Jan 29 "second anoncvs server is 100 miles from the first", 1996 > Jan 31 OpenBSD/cats port is added, from NetBSD, 2004 > Feb 03 Describe the ports mechanism [in OpenBSD], 1997 > +Feb 05 a2k18: Dunedin, New Zeland, 19 developers, 2018 > Feb 13 Unpatented fast block cipher for new password hashing, 1997 > Feb 14 GNU RCS expired from source tree, replaced with OpenRCS, 2007 > Feb 19 IPsec package by John Ioannidis and Angelos D. Keromytis, 1997 > @@ -27,6 +31,7 @@ > Feb 28 Cryptographic services framework in OpenBSD, 2000 > Mar 09 Support for the VAX architecture removed, 2016 > Mar 10 OpenBSD/WWW translation started -- German, Spanish, Dutch, 2000 > +Mar 28 t2k19: Taipei mini hackathon, Taipei, Taiwan, 16 developers, > 2019 > Apr 01 OpenBSD/hppa64 port is added, 2005 > Apr 01 k2k11: Kernel hackathon, Hafnarfjordur, Iceland, 15 developers, > 2011 > Apr 10 f2k7: First filesystem hackathon, Vienna, Austria, 14 > developers, 2007 > @@ -40,10 +45,12 @@ > Apr 27 i386/PAE work integrated, 2006 > May 01 OpenBSD 3.3 released, exploiting W^X, 2003 > May 05 n2k8: Network hackathon, Ito, Japan, 18 developers, 2008 > +May 07 g2k19: General hackathon, Ottawa, Canada, 43 developers, 2019 > May 08 c2k3 General hackathon, Calgary, Alberta, 51 developers, 2003 > May 09 First commit to OpenBSD stable branch, OPENBSD_2_7, 2000 > May 09 OpenBSD/aviion port is added, 2006 > May 19 OpenBSD 2.3 released, including "ports" system, 1998 > +May 19 OpenBSD 6.7 released, 48th release, 2020 > May 21 c2k5: General hackathon, Calgary, Alberta, 60 developers, 2005 > May 21 c2k6: General hackathon, Calgary, Alberta, 47 developers, 2006 > May 24 OpenBSD gets a trunk(4), 2005 > @@ -57,6 +64,7 @@ > Jun 04 c99: First hackathon (IPsec), Calgary, Alberta, 10 developers, > 1999 > Jun 04 c2k2: General hackathon, Calgary, Alberta, 42 developers, 2002 > Jun 06 c2k8: General hackathon, Edmonton, Alberta, 55 developers, 2008 > +Jun 21 WireGuard imported into kernel, 2020 > Jun 14 r2k6: First network hackathon, Hamburg, Germany, 6 developers, > 2006 > Jun 15 OpenBSD 2.7 released, including OpenSSH, 2000 > Jun 15 c2k: First general hackathon, Calgary, Alberta, 18 developers, > 2000 > @@ -70,6 +78,7 @@ > Jul 02 c2k11: General hackathon, Edmonton, Alberta, Canada, 36 > developers, 2011 > Jul 07 g2k12: General hackathon, Budapest, Hungary, 41 developers, 2012 > Jul 08 g2k14: General hackathon, Ljubljana, Slovenia, 49 developers, > 2014 > +Jul 08 g2k18: General hackathon, Ljubljana, Slovenia, 39 developers, > 2018 > Jul 11 OpenBSD goes wireless w/ if_wi addition, 1999 > Jul 23 OpenBSD goes multimedia with Brooktree 848 support, 1998 > Jul 24 Non-executable stack on most architectures, 2002 > @@ -83,6 +92,7 @@ > Aug 28 k2k6: IPsec hackathon, Schloss Kransberg, Germany, 14 > developers, 2006 > Sep 01 Support for the sparc (32bit) architecture removed, 2016 > Sep 03 Support for the zaurus architecture removed, 2016 > +Sep 06 n2k18: Network hackathon, Usti nad Labem, Czech Republic, 11 > developers, 2018 > Sep 16 s2k11: General hackathon, Ljubljana, Slovenia, 25 developers, > 2011 > Sep 17 n2k12: Network hackathon, Starnberg, Germany, 23 developers, > 2012 > Sep 19 j2k10: Mini-hackathon, Sakae Mu
Re: Stuck in Needbuf state, trying to understand (6.7)
On Sun, Jun 28, 2020 at 2:40 AM Bryan Linton wrote: > On 2020-06-27 19:29:31, Bob Beck wrote: > > > > No. > > > > I know *exactly* what needbuf is but to attempt to diagnose what your > > problem is we need exact details. especially: > > > > 1) The configuration of your system including all the details of the > filesystems > > you have mounted, all options used, etc. > > > > 2) The script you are using to generate the problem (Not a paraphrasing > of what > > you think the script does) What filesystems it is using. > > > > Not the OP, but this problem sounds almost exactly like the bug I > reported last year. > > There is a detailed list of steps I used to reproduce the bug in > the following bug report. > > https://marc.info/?l=openbsd-bugs&m=156412299418191 > > I was even able to bisect and identify the commit which first > caused the breakage for me. > > > ---8<--- > > CVSROOT:/cvs > Module name:src > Changes by: b...@cvs.openbsd.org2019/05/08 06:40:57 > > Modified files: > sys/kern : vfs_bio.c vfs_biomem.c > > Log message: > Modify the buffer cache to always flip recovered DMA buffers high. > > This also modifies the backoff logic to only back off what is requested > and not a "mimimum" amount. Tested by me, benno@, tedu@ anda ports build > by naddy@. > > ok tedu@ > > ---8<--- > > However, I have since migrated away from using vnd(4)s since I was > able to find other solutions that worked for my use cases. So I > may not be able to provide much additional information other than > what is contained in the above bug report. > > -- > Bryan > > > > > > > Reproduction of BUG. # optional mkdir /tmpfs mount_mfs -o rw -s 2500M swap /tmpfs # i mounted through fstab so this line is not tested #the bug /bin/dd if=/dev/zero of=/tmpfs/img.dd count=0 bs=1 seek=25 vnconfig vnd3 /tmpfs/img.dd printf "a a\n\n\n\nw\nq\n" | disklabel -E vnd3 newfs vnd3a mount /dev/vnd3a /mnt cd /tmp && ftp https://cdn.openbsd.org/pub/OpenBSD/6.7/amd64/base67.tgz cd /mnt #will occur here (the mkdir was ub beedbuf state for a while ... for v in 1 2 3 4 5 6 7 8 9; do mkdir /tmp/$v; tar xzvf /tmp/base67.tgz -C /mnt/$v; done Ready to test patches. -- -- - Knowing is not enough; we must apply. Willing is not enough; we must do
Re: sparc64.html: Mention T4-2 crashes on older firmware
> Date: Sun, 28 Jun 2020 13:38:37 +0200 > From: Klemens Nanni > > On Sun, Jun 28, 2020 at 12:59:10PM +0200, Mark Kettenis wrote: > > Would be nice if we could give a hint about the firmware revisions > > that are known to work. > https://www.oracle.com/servers/technologies/firmware/release-history-jsp.html#T4-2 > > SysFW 8.9.11 from 10.01.2019 produces the following versions on my T4-2: > > hypervisor_version = Hypervisor 1.15.16 2018/11/28 07:41 > obp_version = OpenBoot 4.38.16 2018/11/28 07:24 > sysfw_version = Sun System Firmware 8.9.11 2018/11/28 07:59 > > I *assume* 8.9.x to work as well because that is what used to run on my > machine prior to updating it, buts it's guess-work and none of the > publicly available information puts actual dates next to the SysFW > versions listed on above mentioned link, so I have no means to pick a > version based on `-> show /HOST' output from Koakuma listing their known > to be broken versions. > > Here's a diff that explicitly mentions "SysFW 8.9.11" becuase that is > all I can guarantee; users with other firmware versions are encouraged > to help bisect the breaking^Wfixing change. > > OK? yeah, I think that's the best we can do. > Index: sparc64.html > === > RCS file: /cvs/www/sparc64.html,v > retrieving revision 1.403 > diff -u -p -r1.403 sparc64.html > --- sparc64.html 19 May 2020 02:02:10 - 1.403 > +++ sparc64.html 28 Jun 2020 11:38:18 - > @@ -230,6 +230,11 @@ and SPARC Enterprise T5120/T5220 may req > before OpenBSD can be successfully installed. The SPARC Enterprise > T5120/T5220 needs at least OBP 4.28.0. > > +On machines like the Oracle SPARC T4-2, older firmware versions from > +around 2011 are known to cause kernel panics and crashes in > +https://man.openbsd.org/sparc64/ldomd.8";>ldomd; later > +versions, at least SysFW 8.9.11 from 2018, are known to work. > + > Supported devices > > >
route add ::/0 ...
Hi,
When "::/0" is used as "default",
# route add ::/0 fe80::1%em0
add net ::/0: gateway fe80::1%em0: Invalid argument
route command trims the sockaddr to { .len = 2, .family = AF_INET6 }
for "::/0", but rtable_satoplen() refuses it. I think it should be
accepted.
ok?
Allow sockaddr for prefix length be trimmed before the key(address)
field. Actually "route" command trims at the address family field for
"::/0"
Index: sys/net/rtable.c
===
RCS file: /cvs/src/sys/net/rtable.c,v
retrieving revision 1.69
diff -u -p -r1.69 rtable.c
--- sys/net/rtable.c21 Jun 2019 17:11:42 - 1.69
+++ sys/net/rtable.c28 Jun 2020 11:30:54 -
@@ -887,8 +887,8 @@ rtable_satoplen(sa_family_t af, struct s
ap = (uint8_t *)((uint8_t *)mask) + dp->dom_rtoffset;
ep = (uint8_t *)((uint8_t *)mask) + mlen;
- if (ap > ep)
- return (-1);
+ if (ap >= ep)
+ return (0);
/* Trim trailing zeroes. */
while (ap < ep && ep[-1] == 0)
Re: sparc64.html: Mention T4-2 crashes on older firmware
On Sun, Jun 28, 2020 at 12:59:10PM +0200, Mark Kettenis wrote: > Would be nice if we could give a hint about the firmware revisions > that are known to work. https://www.oracle.com/servers/technologies/firmware/release-history-jsp.html#T4-2 SysFW 8.9.11 from 10.01.2019 produces the following versions on my T4-2: hypervisor_version = Hypervisor 1.15.16 2018/11/28 07:41 obp_version = OpenBoot 4.38.16 2018/11/28 07:24 sysfw_version = Sun System Firmware 8.9.11 2018/11/28 07:59 I *assume* 8.9.x to work as well because that is what used to run on my machine prior to updating it, buts it's guess-work and none of the publicly available information puts actual dates next to the SysFW versions listed on above mentioned link, so I have no means to pick a version based on `-> show /HOST' output from Koakuma listing their known to be broken versions. Here's a diff that explicitly mentions "SysFW 8.9.11" becuase that is all I can guarantee; users with other firmware versions are encouraged to help bisect the breaking^Wfixing change. OK? Index: sparc64.html === RCS file: /cvs/www/sparc64.html,v retrieving revision 1.403 diff -u -p -r1.403 sparc64.html --- sparc64.html19 May 2020 02:02:10 - 1.403 +++ sparc64.html28 Jun 2020 11:38:18 - @@ -230,6 +230,11 @@ and SPARC Enterprise T5120/T5220 may req before OpenBSD can be successfully installed. The SPARC Enterprise T5120/T5220 needs at least OBP 4.28.0. +On machines like the Oracle SPARC T4-2, older firmware versions from +around 2011 are known to cause kernel panics and crashes in +https://man.openbsd.org/sparc64/ldomd.8";>ldomd; later +versions, at least SysFW 8.9.11 from 2018, are known to work. + Supported devices
Re: sparc64.html: Mention T4-2 crashes on older firmware
> Date: Sun, 28 Jun 2020 12:36:25 +0200 > From: Klemens Nanni > > As Koakuma confirmed on bugs@, their machine exhibited strange behaviour > with such outdated firmware while at least the latest version runs fine > as expected. > > I'd like to mention this in the FAQ. > > Feedack? OK? Would be nice if we could give a hint about the firmware revisions that are known to work. > Index: sparc64.html > === > RCS file: /cvs/www/sparc64.html,v > retrieving revision 1.403 > diff -u -p -r1.403 sparc64.html > --- sparc64.html 19 May 2020 02:02:10 - 1.403 > +++ sparc64.html 28 Jun 2020 10:33:46 - > @@ -230,6 +230,11 @@ and SPARC Enterprise T5120/T5220 may req > before OpenBSD can be successfully installed. The SPARC Enterprise > T5120/T5220 needs at least OBP 4.28.0. > > +On machines like the Oracle SPARC T4-2, older firmware versions from > +around 2011 are known to cause kernel panics and crashes in > +https://man.openbsd.org/sparc64/ldomd.8";>ldomd; later > +versions, at least from 2018, are known to work. > + > Supported devices > > > >
sparc64.html: Mention T4-2 crashes on older firmware
As Koakuma confirmed on bugs@, their machine exhibited strange behaviour with such outdated firmware while at least the latest version runs fine as expected. I'd like to mention this in the FAQ. Feedack? OK? Index: sparc64.html === RCS file: /cvs/www/sparc64.html,v retrieving revision 1.403 diff -u -p -r1.403 sparc64.html --- sparc64.html19 May 2020 02:02:10 - 1.403 +++ sparc64.html28 Jun 2020 10:33:46 - @@ -230,6 +230,11 @@ and SPARC Enterprise T5120/T5220 may req before OpenBSD can be successfully installed. The SPARC Enterprise T5120/T5220 needs at least OBP 4.28.0. +On machines like the Oracle SPARC T4-2, older firmware versions from +around 2011 are known to cause kernel panics and crashes in +https://man.openbsd.org/sparc64/ldomd.8";>ldomd; later +versions, at least from 2018, are known to work. + Supported devices
Re: pipex(4): use reference counters for `ifnet'
On 27/06/20(Sat) 17:58, Vitaliy Makkoveev wrote: > > [...] > > Look at r1.329 of net/if.c. Prior to this change if_detach_queues() was > > used to free all mbufs when an interface was removed. Now lazy freeing > > is used everytime if_get(9) rerturns NULL. > > > > This is possible because we store an index and not a pointer directly in > > the mbuf. > > > > The advantage of storing a session pointer in `ph_cookie' is that no > > lookup is required in pipexintr(), right? Maybe we could save a ID > > instead and do a lookup. How big can be the `pipex_session_list'? > > > > It's unlimited. In pppac(4) case you create the only one interface and > you can share it between the count of sessions you wish. In my practice > I had machines with 800+ active ppp interfaces in 2005. We can have > dosens of cores and hundreds gigs of ram now. How big can be real > count of active ppp interfaces on VPN provider's NAS? With that number of items a linear list might not be the best fit if we decide to stop using pointers. So if we want to use a "lookup" a different data structure might be more appropriate. > I looked at r1.328 of net/if.c. > Imagine we have a lot of connected clients with high traffic. One on them > starts connect/disconnect games. It's not malicious hacker, just mobile > phone in area with low signal. You need: > > 1. block pipexintr() (and netisr too). > 2. block insertion to queues from concurrent threads. > 3. Walk through very loaded queues and compare. And most or may be all >packets are foreign. > 4. Repeat it every time you lost connection. > > Yes, now it's all serealized. And pipexintr() is already blocked while > we do session destruction(). But what is the reason to make your future > life harder? pipex(4) session already has pointer to it's relared > `ifnet'. `ifnet' already has reference counters. Why don't use them? We decided as a team to not use reference counting because they are hard to debug. Using if_get(9)/if_put(9) allows us to check that our changes where correct with static analysis tools. If we start using reference counting for ifps in pipex(4) they we now have an exception in the network stack. That means the techniques applied are not coherent and it makes it harder to work across a huge amount of code. But I don't care, if there's a consensus that it is the way to go, then go for it. > In the way I suggest to use refcounters for pipex(4) session you don't > need to block your packet processing. You don't need to do searchs. You > just need to be sure the `ifnet' you has is still alive. You already has > mechanics for that. Why don't use this? The same can be said with the actual machinery. Using a if_get(9)-like approach doesn't block packet processing, you don't need to to search, there's no reference counting bug that can lead to deadlock, it is consistent with the existing network stack and there are already example of implementation like: if_get() and rtable_get(). > You don't like if_ref() to be > used outside net/if.c? Ok, what's wrong with referenced pointers > obtained by if_get(9)? While session is alive it *uses* it's `ifnet'. It > uses `ifnet' all lifetime *not* only while output. And we should be > shure we didn't destroy `ifnet' while someone uses it. What's wrong with > referenced pointers? If there's a bug, if the reference counting is messed up, it is hard to find where that happened. It is like a use-after-free, where the crash happens is not where the bug is. If you see a leak you don't know where the reference drop is missing. Sure they are many ways to deal with that, but since we did not embrace them. I'm not sure that it makes sense to change direction or introduce a difference now. > The way I wish to go used in file(9). May be it is > totally wrong and we need global in-kernel descriptor table where `fp' > will be referenced by index too :) ? That's not what I'm saying. if_get(9) already exists and is already used in a certain way, I don't see why not embrace that and keep the network stack coherent. But once again, I don't want to block anyone in its development, if you and others agree that's the way to go, then let's go this way.
Re: Stuck in Needbuf state, trying to understand (6.7)
On 2020-06-27 19:29:31, Bob Beck wrote:
>
> No.
>
> I know *exactly* what needbuf is but to attempt to diagnose what your
> problem is we need exact details. especially:
>
> 1) The configuration of your system including all the details of the
> filesystems
> you have mounted, all options used, etc.
>
> 2) The script you are using to generate the problem (Not a paraphrasing of
> what
> you think the script does) What filesystems it is using.
>
Not the OP, but this problem sounds almost exactly like the bug I
reported last year.
There is a detailed list of steps I used to reproduce the bug in
the following bug report.
https://marc.info/?l=openbsd-bugs&m=156412299418191
I was even able to bisect and identify the commit which first
caused the breakage for me.
---8<---
CVSROOT:/cvs
Module name:src
Changes by: b...@cvs.openbsd.org2019/05/08 06:40:57
Modified files:
sys/kern : vfs_bio.c vfs_biomem.c
Log message:
Modify the buffer cache to always flip recovered DMA buffers high.
This also modifies the backoff logic to only back off what is requested
and not a "mimimum" amount. Tested by me, benno@, tedu@ anda ports build
by naddy@.
ok tedu@
---8<---
However, I have since migrated away from using vnd(4)s since I was
able to find other solutions that worked for my use cases. So I
may not be able to provide much additional information other than
what is contained in the above bug report.
--
Bryan
>
>
> On Sat, Jun 27, 2020 at 08:09:18PM -0400, sven falempin wrote:
> > On Fri, Jun 26, 2020 at 7:35 PM sven falempin
> > wrote:
> >
> > >
> > >
> > > On Fri, Jun 26, 2020 at 5:22 PM Stuart Henderson
> > > wrote:
> > >
> > >> On 2020/06/26 15:30, sven falempin wrote:
> > >> > behavior confirmed on current.
> > >> >
> > >> > Once the process stalls, ( could be anything writing to the vnconfig
> > >> disk,
> > >> > cp , umount )
> > >> > a few other calls like df , or ps, etc may hang, never the same
> > >> > sp or mp kernel, reproduced on today's snapshots.
> > >>
> > >> vnconfig is used as part of "make release", many builds are done every
> > >> week using this so it's not a general problem with vnconfig.
> > >>
> > >> Can you show some commands or a script to trigger the behaviour?
> > >>
> > >
> > > the perl script use the system to call :
> > >
> > > vnconfig.
> > > mount.
> > > umount. <- saw hanged
> > > cp.<- saw hanged
> > > tar.<- saw hanged
> > > svn up.<- saw hanged
> > > and dd.
> > > newfs.
> > >
> > > really nothing fancy, only stuff writing to disk got stuck.
> > >
> > > At one point it does a chroot but it never hangs near that , most of the
> > > time it hangs before.
> > >
> > > The script has been used like 1000 times on 6.0 and maybe twice more on
> > > 6.4.
> > >
> > > I have absolutely no idea what the 'needbuf' of top is .
> > >
> > > the script hangs at random position , always writing into vnconfig.
> > >
> > > I have no idea how to reproduce outside the perl script , so maybe it is
> > > related
> > > to some devious perl stdin/stdout buffer .
> > >
> > > Nevertheless there's like a 5% chance that's the script will work( slowly
> > > )
> > >
> > > Most of the system call are inside a routine to log
> > >
> > > sub debug_system {
> > > $logger->debug('running: '.join(' ', @_));
> > > return system(@_);
> > > }
> > >
> > > so i can easily put things inside to try to understand the issue.
> > >
> > > It is really a strange behavior, and the device must be shut down
> > > electrically.
> > > Something really odd, i run syslogd on a buffer, and syslogc buffer is
> > > stuck too
> > > when the device stuck (but it supposed to be mostly already allocated
> > > memory ).
> > >
> > > It's really like the vm does not want to give anymore bucket (<- i
> > > don't know what i m talking about here,
> > > but i looks like that anything that doesn't malloc is ok , computer reply
> > > to ping , can do a few things for a while , and then complete
> > > hang )
> > >
> > > I ran the 6.7 release on a VM somewhere and another device with many perl
> > > script and they work.
> > >
> > > Only this fails 95% of the time and is VERY VERY slow when ok.
> > > compared to what i saw in /usr/src the vnconfig is big , ( forgot to copy
> > > df -h ),
> > > like 2GB
> > >
> >
> >
> > i put ktrace in front of the perl system call
> >
> > An di was able to recover a 800MB trace
> >
> > $ kdump -f ./trace.out | tail -20
> > kdump: realloc: Cannot allocate memory
> > 25955 UNKNOWN(1634890859)
> > 72466 ? CALL syscall()
> >
> >
> > could that be of some use ?
> >
> >
> > --
> > --
> > -
> > Knowing is not enough; we must apply. Willing is not enough; we must do
>
>
