Re: sysupgrade to 6.6 failed at comp66.tgz
On 23:21 Sat 23 Nov, cho...@jtan.com wrote:
> > You can't seriously be calling "-x* -game*" an unsupported configuration ?
> > Seems to me
> > like a sensible thing to do on any box that's going to be headless for its
> > entire life
> > and only ever accessed via SSH (or text console at a push).
>
> Lines 159-160 of /usr/sbin/sysupgrade read as follows:
>
> SETS=$(sed -n -e 's/^SHA256 (\(.*\)) .*/\1/' \
> -e '/^INSTALL\./p;/^bsd/p;/\.tgz$/p' SHA256)
>
> This is followed by ~45 lines which download, verify and extract
> them. The entire thing from that point takes up less than two
> thirds of this small laptop screen.
>
> It would have been quicker to write a patch to include the desired
> functionality than create this email thread.
>
> Here's a quick-and-dirty thing I just made up:
>
> ed /usr/sbin/sysupgrade
> /^SETS=/s//: ${SETS:=/
> +s/$/}/
> wq
>
> Now you can, possibly, set SETS in the environent to override what
> sysupgrade will even consider.
U - usability :D
I guess Theo is right and it's best to merge all sets into baseXX.tgz
(except for siteXX.tgz) to prevent "it works until it's not" situations.
Re: sysupgrade to 6.6 failed at comp66.tgz
On 24/11/2019 09:20, Rachel Roch wrote: You can't seriously be calling "-x* -game*" an unsupported configuration ? Seems to me like a sensible thing to do on any box that's going to be headless for its entire life and only ever accessed via SSH (or text console at a push). I agree in principle. However... Many software packages include dependencies on X libraries (not merely in OpenBSD but in general). Personally I don't think it is worth going to all the trouble of eliminating every unnecessary dependency on X libraries, especially considering that many of these packages are complex, complicated, and deeply integrated in to the OS. The effort is better spent elsewhere. I haven't looked but I expect that games packages don't take a lot of storage relatively speaking. I once experienced a situation where something broke -- on a Linux system I think -- where a non-games package failed to install or execute because I hadn't installed any games packages and therefore the expected directory structure that would have otherwise been created wasn't in place. Different users will have different interpretations of what comprises the "minimal set of software packages and their configurations for a functional headless server". The two broad examples above are descriptions of depedencies that some people would find harmless. Others would find them messy. Others would find them harmless /and/ messy. Personally I don't mind. I would prefer a stable system where all the dependencies are in place, the system is supported, and I am able to seek support from the community. Resources are too scarce to spend fixing this (in my opinion) non-problem. Andrew -- OpenPGP key: EB28 0338 28B7 19DA DAB0 B193 D21D 996E 883B E5B9
Re: sysupgrade to 6.6 failed at comp66.tgz
> You can't seriously be calling "-x* -game*" an unsupported configuration ?
> Seems to me
> like a sensible thing to do on any box that's going to be headless for its
> entire life
> and only ever accessed via SSH (or text console at a push).
Lines 159-160 of /usr/sbin/sysupgrade read as follows:
SETS=$(sed -n -e 's/^SHA256 (\(.*\)) .*/\1/' \
-e '/^INSTALL\./p;/^bsd/p;/\.tgz$/p' SHA256)
This is followed by ~45 lines which download, verify and extract
them. The entire thing from that point takes up less than two
thirds of this small laptop screen.
It would have been quicker to write a patch to include the desired
functionality than create this email thread.
Here's a quick-and-dirty thing I just made up:
ed /usr/sbin/sysupgrade
/^SETS=/s//: ${SETS:=/
+s/$/}/
wq
Now you can, possibly, set SETS in the environent to override what
sysupgrade will even consider.
Matthew
Re: sysupgrade to 6.6 failed at comp66.tgz
On 2019-11-23 14:20, Rachel Roch wrote: This topic has been beat to death. deraadt@ and other have made it clear that if you do not install all the sets, you are running an unsupported configuration. It has been stated that if people keep bitching, they're just going to merge the release sets into one set. I like the fact that there are separate sets. A number of times I've had to squeeze an install onto a <2GB disk, and it was useful being able to select only the specific sets I wanted/needed, while at the same time acknowledging that it was indeed an unsupported configuration. If people are going to try and be edgelords by refusing to install all the sets, then it's up to them to maintain and diagnose their unsupported configuration. You can't seriously be calling "-x* -game*" an unsupported configuration ? Seems to me like a sensible thing to do on any box that's going to be headless for its entire life and only ever accessed via SSH (or text console at a push). Running without the X sets is most certainly an unsupported configuration. Many packages/ports have dependencies on things in the X sets, I've been bit by this issue my self. Certain ports have "no_x11" flavours, but it's not a guarantee. With regards to the game sets, they're less than 5MB, so its pretty irrelevant. Forgoing the X sets does nothing for security, and ultimately further removes you from a standard supported install. Unless you're trying to do an install on a super small disk, just install all the sets. If you're running any sort of modern or important production machine, you're going to have more than enough disk space to install all the sets. If you want to be an edgelord and not install all the sets, then by all means, please enjoy your unsupported system. Don't come back bitching and moaning if something breaks.
Re: sysupgrade to 6.6 failed at comp66.tgz
This topic has been beat to death. deraadt@ and other have made it clear that if you do not install all the sets, you are running an unsupported configuration. It has been stated that if people keep bitching, they're just going to merge the release sets into one set. I like the fact that there are separate sets. A number of times I've had to squeeze an install onto a <2GB disk, and it was useful being able to select only the specific sets I wanted/needed, while at the same time acknowledging that it was indeed an unsupported configuration. If people are going to try and be edgelords by refusing to install all the sets, then it's up to them to maintain and diagnose their unsupported configuration. You can't seriously be calling "-x* -game*" an unsupported configuration ? Seems to me like a sensible thing to do on any box that's going to be headless for its entire life and only ever accessed via SSH (or text console at a push). It's an unsupported configuration.
Re: sysupgrade to 6.6 failed at comp66.tgz
> This topic has been beat to death. deraadt@ and other have made it clear that > if you do not install all the sets, you are running an unsupported > configuration. It has been stated that if people keep bitching, they're just > going to merge the release sets into one set. > > I like the fact that there are separate sets. A number of times I've had to > squeeze an install onto a <2GB disk, and it was useful being able to select > only the specific sets I wanted/needed, while at the same time acknowledging > that it was indeed an unsupported configuration. > > If people are going to try and be edgelords by refusing to install all the > sets, then it's up to them to maintain and diagnose their unsupported > configuration. > You can't seriously be calling "-x* -game*" an unsupported configuration ? Seems to me like a sensible thing to do on any box that's going to be headless for its entire life and only ever accessed via SSH (or text console at a push).
Re: sysupgrade to 6.6 failed at comp66.tgz
On 2019-11-23 13:45, Rachel Roch wrote: - maybe sysupgrade needs to be patched to avoid this issue? Probably not. sysupgrade has assumptions baked in to it which have evidently been rendered invalid either by another tool or by the person using them. That tool is where the patch most likely ought to be directed. If I may make a little comment here. Surely it is a little bit questionable to "bake assumptions" into sysupgrade that everybody is going to do a complete install when the OpenBSD installer itself gives you the option to select what is going to be installed. At the very least, may I suggest that even if the developers don't want to increase the intelligence of sysupgrade that they at least code in some sanity checks (e.g. "pick a file - or two - at random from the core tgz files that you would normally expect to be present on the system if a 'full-default' install was done. If file not present, then throw a horrid error message and abort). It strikes me as a little silly to put a tool out there that you know will trash (or at least severely brick) a user's system just because of some severely opinionated "baked assumptions" coded into it. This topic has been beat to death. deraadt@ and other have made it clear that if you do not install all the sets, you are running an unsupported configuration. It has been stated that if people keep bitching, they're just going to merge the release sets into one set. I like the fact that there are separate sets. A number of times I've had to squeeze an install onto a <2GB disk, and it was useful being able to select only the specific sets I wanted/needed, while at the same time acknowledging that it was indeed an unsupported configuration. If people are going to try and be edgelords by refusing to install all the sets, then it's up to them to maintain and diagnose their unsupported configuration.
Re: sysupgrade to 6.6 failed at comp66.tgz
>> - maybe sysupgrade needs to be patched to avoid this issue? >> > > Probably not. sysupgrade has assumptions baked in to it which have > evidently been rendered invalid either by another tool or by the > person using them. That tool is where the patch most likely ought > to be directed. > > If I may make a little comment here. Surely it is a little bit questionable to "bake assumptions" into sysupgrade that everybody is going to do a complete install when the OpenBSD installer itself gives you the option to select what is going to be installed. At the very least, may I suggest that even if the developers don't want to increase the intelligence of sysupgrade that they at least code in some sanity checks (e.g. "pick a file - or two - at random from the core tgz files that you would normally expect to be present on the system if a 'full-default' install was done. If file not present, then throw a horrid error message and abort). It strikes me as a little silly to put a tool out there that you know will trash (or at least severely brick) a user's system just because of some severely opinionated "baked assumptions" coded into it.
Re: sysupgrade to 6.6 failed at comp66.tgz
‐‐‐ Original Message ‐‐‐ On Friday, November 22, 2019 11:45 AM, Stuart Henderson wrote: > A combination of things: > > - You didn't install the comp set before Thank you Stuart for your detailed mail. That's exactly it, I did not have comp65.tgz set installed as I just recently read on this mailing list that the best practice would be to install all sets, including the x* sets even if I don't need X on my servers. This is the only way that guarantees that such tools like sysupgrade can work properly. Lesson learnt live here ;-) So thanks to your instructions I managed to upgrade to 6.6 using sysupgrade and it all worked well. Great work behind this sysupgrade tool!!
Re: sysupgrade to 6.6 failed at comp66.tgz
On 2019-11-22, mabi wrote:
> Hi,
>
> I just tried out sysupgrade on one of my OpenBSD 6.5 servers in order to
> upgrade automatically to 6.6 but unfortunately it failed at the comp66.tgz
> and rebooted (upgrade log below).
>
> It looks like I am now running a half-upgraded hybrid OpenBSD 6.5/6.6 system.
> It also didn't manage to relink the kernel after reboot (log file below).
>
> So I was wondering if anyone had any recommendations or insights to my
> following points:
>
> - reason why it failed?
A combination of things:
- You didn't install the comp set before
- syspatch65-003_mds.tgz resulted in minor breakage if comp wasn't
installed (the problem was in syspatch generation and has since been
rectified) - /usr/include/machine is meant to be a symlink to the
arch name e.g.
$ ls -l /usr/include/machine
lrwxr-xr-x 1 root bin 5 Nov 15 18:18 /usr/include/machine -> amd64
$ tar tvzf syspatch65-003_mds.tgz | grep usr/include
-r--r--r-- 1 root bin 2933 May 27 15:44
usr/include/amd64/codepatch.h
-r--r--r-- 1 root bin 13210 May 27 15:44 usr/include/amd64/cpu.h
-r--r--r-- 1 root bin 2467 May 27 15:44
usr/include/amd64/cpu_full.h
-r--r--r-- 1 root bin 56044 May 27 15:44
usr/include/amd64/specialreg.h
-r--r--r-- 1 root bin 27384 May 27 15:44
usr/include/amd64/vmmvar.h
-r--r--r-- 1 root bin 2933 May 27 15:44
usr/include/machine/codepatch.h
-r--r--r-- 1 root bin 13210 May 27 15:44 usr/include/machine/cpu.h
-r--r--r-- 1 root bin 2467 May 27 15:44
usr/include/machine/cpu_full.h
-r--r--r-- 1 root bin 56044 May 27 15:44
usr/include/machine/specialreg.h
-r--r--r-- 1 root bin 27384 May 27 15:44
usr/include/machine/vmmvar.h
Here is an example in the same dir built with the fixed process:
$ tar tvzf syspatch65-008_swapgs.tgz | grep usr/include
-r--r--r-- 1 root bin 3456 Aug 8 14:37
usr/include/amd64/codepatch.h
-r--r--r-- 1 root bin 3859 Aug 8 14:37
usr/include/amd64/frameasm.h
> - what should I do now? retry to upgrade with sysupgrade?
> - re-install the whole system?
rm -r /usr/include/machine and run the upgrade again.
If you want to use sysupgrade again for this, edit the script and force
NEXT_VERSION=6.6.
Otherwise boot bsd.rd and do it by hand - select "upgrade" and select all sets.
> - maybe sysupgrade needs to be patched to avoid this issue?
It _could_ be patched to do this ..
[ -d /usr/include/machine ] && rm -r /usr/include/machine
Though the problem also affects people who don't use sysupgrade,
modifying the installer is needed to fix things in that case e.g.
this would do the trick
Index: install.sub
===
RCS file: /cvs/src/distrib/miniroot/install.sub,v
retrieving revision 1.1145
diff -u -p -r1.1145 install.sub
--- install.sub 19 Oct 2019 13:14:23 - 1.1145
+++ install.sub 22 Nov 2019 10:44:29 -
@@ -1660,7 +1660,7 @@ install_files() {
fi
if isin comp$VERSION.tgz $_get_sets; then
rm -rf /mnt/usr/lib/{gcc-lib,clang}
- rm -rf /mnt/usr/include/g++
+ rm -rf /mnt/usr/include/*
fi
rm -rf /mnt/var/syspatch/*
fi
Re: sysupgrade to 6.6 failed at comp66.tgz
mabi writes: > Hi, > > - reason why it failed? It cannot remove /usr/include/machine because it is not empty. > - what should I do now? retry to upgrade with sysupgrade? Empty /usr/include/machine. > - re-install the whole system? If you like. It will certainly empty out /usr/include/machine. > - maybe sysupgrade needs to be patched to avoid this issue? Probably not. sysupgrade has assumptions baked in to it which have evidently been rendered invalid either by another tool or by the person using them. That tool is where the patch most likely ought to be directed. Matthew
sysupgrade to 6.6 failed at comp66.tgz
Hi,
I just tried out sysupgrade on one of my OpenBSD 6.5 servers in order to
upgrade automatically to 6.6 but unfortunately it failed at the comp66.tgz and
rebooted (upgrade log below).
It looks like I am now running a half-upgraded hybrid OpenBSD 6.5/6.6 system.
It also didn't manage to relink the kernel after reboot (log file below).
So I was wondering if anyone had any recommendations or insights to my
following points:
- reason why it failed?
- what should I do now? retry to upgrade with sysupgrade?
- re-install the whole system?
- maybe sysupgrade needs to be patched to avoid this issue?
Best regards,
Mabi
*** output of upgrade log ***
Terminal type? [vt220] vt220
Available disks are: sd0.
Which disk is the root disk? ('?' for details) [sd0] sd0
Checking root filesystem (fsck -fp /dev/sd0a)... OK.
Mounting root filesystem (mount -o ro /dev/sd0a /mnt)... OK.
Force checking of clean non-root filesystems? [no] no
fsck -p f8bd514855ccf1e5.f... OK.
fsck -p f8bd514855ccf1e5.d... OK.
fsck -p f8bd514855ccf1e5.e... OK.
fsck -p f8bd514855ccf1e5.g... OK.
/dev/sd0a (f8bd514855ccf1e5.a) on /mnt type ffs (rw, local)
/dev/sd0f (f8bd514855ccf1e5.f) on /mnt/home type ffs (rw, local, nodev, nosuid)
/dev/sd0d (f8bd514855ccf1e5.d) on /mnt/tmp type ffs (rw, local, nodev, nosuid)
/dev/sd0e (f8bd514855ccf1e5.e) on /mnt/usr type ffs (rw, local, nodev,
wxallowed)
/dev/sd0g (f8bd514855ccf1e5.g) on /mnt/var type ffs (rw, local, nodev, nosuid)
Let's upgrade the sets!
Location of sets? (cd0 disk http nfs or 'done') [http] disk
Is the disk partition already mounted? [yes] yes
Pathname to the sets? (or 'done') [6.6/amd64] /home/_sysupgrade/
Select sets by entering a set name, a file name pattern or 'all'. De-select
sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'.
[X] bsd [X] base66.tgz[X] game66.tgz[X] xfont66.tgz
[X] bsd.mp[X] comp66.tgz[X] xbase66.tgz [X] xserv66.tgz
[X] bsd.rd[X] man66.tgz [X] xshare66.tgz
Set name(s)? (or 'abort' or 'done') [done] done
Directory does not contain SHA256.sig. Continue without verification? [no] yes
Installing bsd 100% |**| 18250 KB00:00
Installing bsd.mp 100% |**| 18336 KB00:00
Installing bsd.rd 100% |**| 10058 KB00:00
Installing base66.tgz 100% |**| 236 MB00:36
Installing comp66.tgz81% |* | 58880 KB00:02
ETAtar: Unable to remove directory ./usr/include/machine: Directory not empty
Installing comp66.tgz 100% |**| 72109 KB00:14
Installation of comp66.tgz failed. Continue anyway? [no] no
*** output of /usr/share/relink/kernel/GENERIC/relink.log ***
(SHA256) /bsd: FAILED
Failed to verify /bsd's checksum, therefore a randomly linked kernel (KARL)
is not being built. KARL can be re-enabled for next boot by issuing as root:
sha256 -h /var/db/kernel.SHA256 /bsd
