Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 6/06/2010, at 1:27 PM, Uwe Dippel wrote: Philip Guenther guenther at gmail.com writes: You now have and now it seems the core discussion is just about whether (or where) an additional rm -rf /usr/obj/* should be added to help people that know enough to set up the source tree for building/patching by untaring src.tar.gz but don't know to remove the obj tree at the same time. So, no diff here, but a suggestion: If one needs to avoid stale stuff lying around in /usr/obj at applying a patch, the only logical consequence is, to clean out all /obj totally, even before applying a single patch. If I am correct, the instructions should be clear for 00N_ThisApp.patch: Apply by doing: cd /usr/src patch -p0 00N_ThisApp.patch Clean the build directories by issuing the command /usr/sbin/mk_build_clean And then rebuild and install the library and statically-linked binaries that depend upon it: cd lib/libThisApp make obj make depend make includes make make install cd ../../sbin make obj make depend make make install , where mk_build_clean is just the set of steps pointed out in 'man release', respectively in FAQ5. To me, and I guess Richard Toohey, the case is solved. Everyone who can read, and likes following instructions, can read and follow this easily. No, the point that people are making (over and above HOW you report an issue) is that blindly following the patch instructions is *not* good enough. It doesn't matter how much more gets put in the docs or the patch files, we didn't *think* enough about what we were doing - that's my mistake and I intend to learn from it. If we just end up blindly doing stuff and clicking on buttons to see what happens, we should probably be using another operating system. This *is* in the docs - if you read a little bit further than we did, and do a bit more thinking. Maybe another line in the patch file would help ... but then what about the next issue that comes up? The patch file doesn't tell you how to put the source there, so if you do *literally* follow cd /usr/src and then the patch line, you won't even have the right source, will you? You had to read elsewhere to know how to get the source in the first place. Or next someone will argue that Then build and install a new kernel is too brief and there should be full instructions in every patch file. If we go off and read the FAQ about building the kernel because the full instructions were not in the patch file, why didn't we go and read the FAQ about building userland? That's my take on it and I think the list members are rather bored of this thread! Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Jun 05 12:25:03, Uwe Dippel wrote: Philip Guenther guenther at gmail.com writes: Please point to the part of the Upgrade Guide which talks about building from source, untarring the src tar file, or applying errata. I can't seem to find any such reference, but I'm sure it's in there somewhere, because you originally said that you did the upgrade exactly following the upgrade guide and, as we found out later, your steps included building from source. You misread what I did. I was following the Upgrade Guide to the dot, following Applying patches in OpenBSD to the dot, and then the instructions in the patch files. To the dot. Applying the patches in OpenBSD says clearly that after applying the patches, you can _build_the_kernel_ as normal, which sends you to faq5.html#Building (a broken link BTW) that tells you about /usr/obj. This is where my unfortunate quarrel with Jacob came from, when he said I was building userland, and I insisted I was applying errata. Applying errata means patching the files and then building either kernel or userland as normal. Right?
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
patrick keshishian wrote: On Fri, Jun 4, 2010 at 7:49 PM, Jacob Meuser jake...@sdf.lonestar.org wrote: I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. Do you not agree barring broken makefiles and unreliable system clock (as someone pointed out), object files and binaries (in obj/) should have been rebuilt? --patrick ?? odds that OP found a bad date and fixed it (silently) ??
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Jacob Meuser wrote: ... On 5/06/2010, at 7:31 AM, Nick Holland wrote: a patch to the upgrade guide would be wrong. The problem is the patching process (a special case of the userland build process) assumes a clean obj dir. This has nothing to do with upgrades. If you try to rebuild the same userland utility more than once for /any/ reason without clearing the obj dir, you can run into problems. Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org ANYTHING left in /usr/obj will be a possible problem. ANYTHING left ANYWHERE will be a possible problem anytime anything assumes (or has/likes to assume) that it is working with a clean slate. Fixing minor problems (and bending everything else out of shape) does NOT make for better systems. For me, I prefer things (upgrade/update/whatever) that do as little collateral damage as possible. (And anytime you want/need to find out what went wrong you do NOT clean up everything first.)
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
IF YOU DON'T KNOW WHAT YOUR ARE DOING, INSTALL A NEW SNAPSHOT Theo de Raadt wrote: Miod, Dale, Kurt, Kettenis and I am quite often the first people to deal with bumping systems forward over bumps. Some bumps are so difficult that after they are done the rest of us jump over them using snapshots. When they happen, WE -- THE DEVELOPERS -- USE THE SNAPSHOTS! They happen in lots of releases. Why would we use snapshots, because we are stupid? Or are we smart enough to not waste our time doing things the hard way? IF YOU DON'T KNOW WHAT YOUR ARE DOING, INSTALL A NEW SNAPSHOT (Me, I never know what I am doing, but he KNOWS what he's talking about)
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 5/06/2010, at 5:51 PM, Theo de Raadt wrote: On Fri, Jun 4, 2010 at 7:49 PM, Jacob Meuser jake...@sdf.lonestar.org wrote: I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. Do you not agree barring broken makefiles and unreliable system clock (as someone pointed out), object files and binaries (in obj/) should have been rebuilt? It's a source tree with nearly 40,000 .[chyl] source code files, and probably another 40,000 further source code dependencies if you include manual pages and the perl parts. We try very hard, and the bsd.*.mk macro package helps a lot (enforcing consistancy-because-of-simplicity), but if you think we can get all the dependencies right every single time, it is a tough call. But this case is worse -- when the trash in the obj tree totally mis-matches the src tree since it is so far in the past... that is totally impossible. Dependencies don't help when they don't know about the files. Even make clean or cleandir won't help you then. This was not an installer bug. It had nothing to do with upgrades. We've said it before, and I guess we get to say it again: If don't know what you are doing, install a new snapshot. How many more times do we have to say that? Why are people defending a person who thinks they are smart enough, and has just proved that they're not? Miod, Dale, Kurt, Kettenis and I am quite often the first people to deal with bumping systems forward over bumps. Some bumps are so difficult that after they are done the rest of us jump over them using snapshots. When they happen, WE -- THE DEVELOPERS -- USE THE SNAPSHOTS! They happen in lots of releases. Why would we use snapshots, because we are stupid? Or are we smart enough to not waste our time doing things the hard way? Uwe thinks he's being really clever, but he's not clever at all. He's got a record of choosing the hardest paths. That's his problem. I just wish he wouldn't be such a loud whiner when he screws his system up. But I don't understand what he's doing differently to me. A new release is out, you want to upgrade from the previous release to the new one, and then you want to apply the errata patches. Not saying I'm doing it perfectly, but I'd like to understand where he's going wrong because I do something similar - and wrong, from the sound of it. I don't want to follow current, I just follow the release apply errata patches. (But it looks like Uwe, I am NOT doing it 100% right.) So I was on 4.5, 4.6 came out. I got the 4.6 CD, followed the upgrade process. I replaced /usr/src with the 4.6 code off the CD, downloaded the errata patches, built. 4.7 comes out, I upgrade from CD, copied the 4.7 source over, downloaded errata, built. Seems to work most of the time apart from a patch requiring a userland build (like openssl) so this is where I am obviously going wrong. A snapshot is a point-in-time, may-have-issues, best way to follow current build, isn't it? But I'd rather follow release. I appreciate you've got better things to do than hand-holding, so I'll work my way through it, but any cluesticks appreciated. Thanks.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Jun 04 16:22:35, Uwe Dippel wrote: Jacob Meuser jakemsr at sdf.lonestar.org writes: oh good grief. you had a dirty /usr/obj. just look at the pfctl snippet of the log you posted. do you see pfctl being built? do you see pfctl being installed from /usr/obj? Oh, yes. So the blame is on my side, I guess. Mea culpa maxima! I didn't know that the object directories need to be cleaned manually. Until yesterday, I would have taken a bet that the object directories lie within the source trees (/usr/xenocaram /usr/src), and be cleaned when cleaning the sources. Now I am aware that I need to know the location of the object directories and clean them manually. More importantly, you have now read all of http://www.openbsd.org/faq/faq5.html Right? Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? So you still _haven't_ read http://www.openbsd.org/faq/faq5.html#BldUserland ? Cleaning /usr/obj is the very first line. Why would anyone waste any more time on you? Thanks for the various people who helped me patiently at analysing this problem to the very end! You didn't read the documentation, screwed up, and wasted everyone's time with nonsensical speculationas. End of analysis. You are welcome.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Sat, Jun 05, 2010 at 06:48:18PM +1200, Richard Toohey wrote: But I don't understand what he's doing differently to me. A new release is out, you want to upgrade from the previous release to the new one, and then you want to apply the errata patches. Look, there are several flaws to the way make works. Those flaws are *totally integral* to the way make works. In most cases, you get lucky, sometimes you don't. Basically, the flaw is that make needs a list of dependencies to work. It can either be hardcoded in makefiles, or regenerated through .depend. For anything but the most trivial projects, it is regenerated dynamically through make depend. But if that information changes too much (especially when files get removed), then it fails. It fails HARD. As in, you get a make error because it can no longer find a dependency it used to find. If you're even less lucky, it fails silently. As in, it forgets to register a dependency/recompile a file even though things have changed. Make doesn't know about every dependency. Especially tweaks to compile flags, changes in defines, too deep changes don't get noticed. Oh yeah, you *could* add more meta-junk that says that whenever you change THAT file, then you have to recompile this and this. More often than not, GNU-hell projects (as known as autoshit, as known as automake/autoconf/libtool) err in the other direction. So you make a wee little tweak to a file in there, and suddenly, it recompiles the WHOLE WORLD. And assumes you have every crappy development tool the FSF ever shat on the world. Very offensive if you ask me (that's the whole point of the REORDER_DEPENDENCIES in ports land, btw. This allows up to sneak out of that problem every time we do a small patch to a Makefile.in in GNUlalaland). It's all about process. You got a process you can follow to rebuild OpenBSD. Follow it to the letter. If you take shortcuts, you will get bitten eventually. If you're smart enough, you will recover. But it is complicated. Hint: if the guy who's actually in charge of the OS, and the guy who's actually maintaining make are BOTH telling you it's complicated, then it probably is. Do you think you're smarter than Theo and I combined ? Go ahead, prove us wrong...
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 5/06/2010, at 7:45 PM, Jan Stary wrote: On Jun 04 16:22:35, Uwe Dippel wrote: Jacob Meuser jakemsr at sdf.lonestar.org writes: oh good grief. you had a dirty /usr/obj. just look at the pfctl snippet of the log you posted. do you see pfctl being built? do you see pfctl being installed from /usr/obj? Oh, yes. So the blame is on my side, I guess. Mea culpa maxima! I didn't know that the object directories need to be cleaned manually. Until yesterday, I would have taken a bet that the object directories lie within the source trees (/usr/xenocaram /usr/src), and be cleaned when cleaning the sources. Now I am aware that I need to know the location of the object directories and clean them manually. More importantly, you have now read all of http://www.openbsd.org/faq/faq5.html Right? Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? So you still _haven't_ read http://www.openbsd.org/faq/faq5.html#BldUserland ? Cleaning /usr/obj is the very first line. I wanted to understand where I was going wrong in my patching because it sounded similar to what Uwe is doing. In my case I was *just* following the errata patch instructions, and that's *not* enough, depending on the patch. So, RTFM, FAQs 10.15 and 5.3.5 in my case. Thanks ... I've learned something. Why would anyone waste any more time on you? Thanks for the various people who helped me patiently at analysing this problem to the very end! You didn't read the documentation, screwed up, and wasted everyone's time with nonsensical speculationas. End of analysis. You are welcome.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Sat, Jun 05, 2010 at 01:49:46AM -0400, Tony Abernethy wrote: Jacob Meuser wrote: ... On 5/06/2010, at 7:31 AM, Nick Holland wrote: a patch to the upgrade guide would be wrong. The problem is the patching process (a special case of the userland build process) assumes a clean obj dir. This has nothing to do with upgrades. If you try to rebuild the same userland utility more than once for /any/ reason without clearing the obj dir, you can run into problems. Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org ANYTHING left in /usr/obj will be a possible problem. ANYTHING left ANYWHERE will be a possible problem anytime anything assumes (or has/likes to assume) that it is working with a clean slate. Fixing minor problems (and bending everything else out of shape) does NOT make for better systems. For me, I prefer things (upgrade/update/whatever) that do as little collateral damage as possible. (And anytime you want/need to find out what went wrong you do NOT clean up everything first.) so Tony, tell me, how does 'rm -rf /usr/obj/*', after installing new binaries and new sources code (from a tarball - not an insignificant part of the issue, and exactly what the directions say to do) create collateral damage? you're already past the point of no return anyway, right? maybe I worded it wrongly but that's what I'm asking. is telling people to 'rm -rf /usr/obj/*' after they have completed the update really a necessary part of the upgrade process. no. but I bet if it would say that in the upgrade guide, this stupid thread would never have happened. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 5/06/2010, at 8:14 PM, Marc Espie wrote: On Sat, Jun 05, 2010 at 06:48:18PM +1200, Richard Toohey wrote: But I don't understand what he's doing differently to me. A new release is out, you want to upgrade from the previous release to the new one, and then you want to apply the errata patches. Look, there are several flaws to the way make works. Those flaws are *totally integral* to the way make works. In most cases, you get lucky, sometimes you don't. Basically, the flaw is that make needs a list of dependencies to work. It can either be hardcoded in makefiles, or regenerated through .depend. For anything but the most trivial projects, it is regenerated dynamically through make depend. But if that information changes too much (especially when files get removed), then it fails. It fails HARD. As in, you get a make error because it can no longer find a dependency it used to find. If you're even less lucky, it fails silently. As in, it forgets to register a dependency/recompile a file even though things have changed. Make doesn't know about every dependency. Especially tweaks to compile flags, changes in defines, too deep changes don't get noticed. Oh yeah, you *could* add more meta-junk that says that whenever you change THAT file, then you have to recompile this and this. More often than not, GNU-hell projects (as known as autoshit, as known as automake/autoconf/libtool) err in the other direction. So you make a wee little tweak to a file in there, and suddenly, it recompiles the WHOLE WORLD. And assumes you have every crappy development tool the FSF ever shat on the world. Very offensive if you ask me (that's the whole point of the REORDER_DEPENDENCIES in ports land, btw. This allows up to sneak out of that problem every time we do a small patch to a Makefile.in in GNUlalaland). It's all about process. You got a process you can follow to rebuild OpenBSD. Follow it to the letter. If you take shortcuts, you will get bitten eventually. If you're smart enough, you will recover. But it is complicated. Hint: if the guy who's actually in charge of the OS, and the guy who's actually maintaining make are BOTH telling you it's complicated, then it probably is. Do you think you're smarter than Theo and I combined ? Go ahead, prove us wrong... No way!!! I certainly wasn't saying it wasn't complicated, or that I had a better way, it was more that I thought I *was* following the right process. I appreciate OpenBSD and all the work you guys do. Especially taking the time to explain - the cluesticks have been applied (just leave off with the thumbscrews, please.) I got into a pattern of clear src, get new src from CD, apply patch, follow *literal* patch instructions (without thinking/reading about the patch *context*, or how I might actually need to do a little more work.) Apart from RTFMing, we learn from mistakes, and this has been one of mine. Thanks.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Jacob Meuser wrote: On Sat, Jun 05, 2010 at 01:49:46AM -0400, Tony Abernethy wrote: Jacob Meuser wrote: ... On 5/06/2010, at 7:31 AM, Nick Holland wrote: a patch to the upgrade guide would be wrong. The problem is the patching process (a special case of the userland build process) assumes a clean obj dir. This has nothing to do with upgrades. If you try to rebuild the same userland utility more than once for /any/ reason without clearing the obj dir, you can run into problems. Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org ANYTHING left in /usr/obj will be a possible problem. ANYTHING left ANYWHERE will be a possible problem anytime anything assumes (or has/likes to assume) that it is working with a clean slate. Fixing minor problems (and bending everything else out of shape) does NOT make for better systems. For me, I prefer things (upgrade/update/whatever) that do as little collateral damage as possible. (And anytime you want/need to find out what went wrong you do NOT clean up everything first.) so Tony, tell me, how does 'rm -rf /usr/obj/*', after installing new binaries and new sources code (from a tarball - not an insignificant part of the issue, and exactly what the directions say to do) create collateral damage? you're already past the point of no return anyway, right? maybe I worded it wrongly but that's what I'm asking. is telling people to 'rm -rf /usr/obj/*' after they have completed the update really a necessary part of the upgrade process. no. but I bet if it would say that in the upgrade guide, this stupid thread would never have happened. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org -- Ok, my take on this mess. If not this stupid thread, then some other stupid thread. You do not 'rm -rf /usr/obj/*' AFTER the update. You do the 'rm -rf /usr/obj/*' BEFORE you stick strange stuff into /usr/obj. Collateral damage is anything that gets in the way of finding out exactly what is or exactly what happened. This whole mess seems to be because some unstated something AFTER the update was claimed to be as a result of the update. How often should /tmp be obliterated? When you say after installing new sources, what exactly is left on the system? The new sources presumably are there, but what else is there and does it matter? The answer requires a directory listing of everything on the system that did not come from the new sources. Anything short of that and you cannot state what it is that you did. All I need to break any automated system you devise is to have some programs that I compile myself and use the system directories to hold the sources etc.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Sat, Jun 05, 2010 at 05:13:19AM -0400, Tony Abernethy wrote: All I need to break any automated system you devise is to have some programs that I compile myself and use the system directories to hold the sources etc. then you are on your own, not someone who is just following the directions. you'd know that it doesn't apply to you. but whatever. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Jacob Meuser wrote: On Sat, Jun 05, 2010 at 05:13:19AM -0400, Tony Abernethy wrote: All I need to break any automated system you devise is to have some programs that I compile myself and use the system directories to hold the sources etc. then you are on your own, not someone who is just following the directions. you'd know that it doesn't apply to you. but whatever. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org --- It is essential that I understand the difference. (Although I have some difficulty in understanding how anybody could possibly actually be just following the directions.) As soon as I depart from the directions, everything downstream is my responsibility. The developers are not and can not be responsible for guessing what I have or have not squirreled away wherever. On this silly thread, the upgrade did actually function as it should. Some unmentioned stuff AFTER the upgrade put things in the state BEFORE the upgrade. I can imagine scenarios where that is EXACTLY the results I would want, but that was not the case for this silly thread. For this silly thread, there is nothing that I see in the OpenBSD system that needs any fixing. (but some people who know better may/will disagree) Until and unless selecting all also gets the sources, I must assume that setting up the system for following -stable is a separate process.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Tony Abernethy tony at servasoftware.com writes: Might be better to read and comprehend ``man patch'' before assuming limitations on the scope of patch's reach. It is always so nice to trample on the person lying on the ground, ain't it! Where in 'man patch' is the underlying problem addressed? - oh, yeah, maybe mine is the old version, again ... .
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
wtf are you talking about tony? have you even read the upgrade guide? have you read any of this thread, at all, or did you see some long thread on misc@ and decide to jump in? we have users that say they follow the install and upgrade guides to the letter and they get fucked. there is a problem. they don't even know /usr/obj exists. why is enlightening them about /us/obj at the point in the process where /usr/obj, if they have been following the directions to the letter, is basically poison such a terrible idea? yes, blindly rm -rf /usr/obj/* is not the answer. I did not send a patch to nick@ that added that to the upgrade guide, that is not what I'm saying. and I certainly never ever said anything about anything being automated or changing anyting whatsoever in the installer. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Jacob Meuser wrote: we have users that say they follow the install and upgrade guides to the letter and they get fucked. there is a problem. they don't even know /usr/obj exists. What they say. What they did. Two different things. There's lots of things they do not know about. I fail to understand why it is important to warn them about /usr/obj and not warn them about /usr/src. Surely there's lots of other things they need to be warned about. Enough warnings and you might even attain Microsoft Windows.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
what a totally useless bunch of misc traffic only because a drunk OpenBSD user did not remove /usr/obj before building shit
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Theo de Raadt wrote: If [you] don't know what you are doing, install a new snapshot. We do this frequently. Works very well. bsd.rd makes it easy to move to a new snapshot. We buy -release CDs too, but seldom open them. Brad
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Philip Guenther guenther at gmail.com writes: Please point to the part of the Upgrade Guide which talks about building from source, untarring the src tar file, or applying errata. I can't seem to find any such reference, but I'm sure it's in there somewhere, because you originally said that you did the upgrade exactly following the upgrade guide and, as we found out later, your steps included building from source. You misread what I did. I was following the Upgrade Guide to the dot, following Applying patches in OpenBSD to the dot, and then the instructions in the patch files. To the dot. This is where my unfortunate quarrel with Jacob came from, when he said I was building userland, and I insisted I was applying errata. From what I have learned, Applying patches in OpenBSD should be removed from the FAQ, since we would all have been spared this thread. It obviously has carried a number of people away. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Sat, Jun 05, 2010 at 06:48:18PM +1200, Richard Toohey wrote: On 5/06/2010, at 5:51 PM, Theo de Raadt wrote: On Fri, Jun 4, 2010 at 7:49 PM, Jacob Meuser jake...@sdf.lonestar.org wrote: I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. Do you not agree barring broken makefiles and unreliable system clock (as someone pointed out), object files and binaries (in obj/) should have been rebuilt? It's a source tree with nearly 40,000 .[chyl] source code files, and probably another 40,000 further source code dependencies if you include manual pages and the perl parts. We try very hard, and the bsd.*.mk macro package helps a lot (enforcing consistancy-because-of-simplicity), but if you think we can get all the dependencies right every single time, it is a tough call. But this case is worse -- when the trash in the obj tree totally mis-matches the src tree since it is so far in the past... that is totally impossible. Dependencies don't help when they don't know about the files. Even make clean or cleandir won't help you then. This was not an installer bug. It had nothing to do with upgrades. We've said it before, and I guess we get to say it again: If don't know what you are doing, install a new snapshot. How many more times do we have to say that? Why are people defending a person who thinks they are smart enough, and has just proved that they're not? Miod, Dale, Kurt, Kettenis and I am quite often the first people to deal with bumping systems forward over bumps. Some bumps are so difficult that after they are done the rest of us jump over them using snapshots. When they happen, WE -- THE DEVELOPERS -- USE THE SNAPSHOTS! They happen in lots of releases. Why would we use snapshots, because we are stupid? Or are we smart enough to not waste our time doing things the hard way? Uwe thinks he's being really clever, but he's not clever at all. He's got a record of choosing the hardest paths. That's his problem. I just wish he wouldn't be such a loud whiner when he screws his system up. But I don't understand what he's doing differently to me. A new release is out, you want to upgrade from the previous release to the new one, and then you want to apply the errata patches. Not saying I'm doing it perfectly, but I'd like to understand where he's going wrong because I do something similar - and wrong, from the sound of it. I don't want to follow current, I just follow the release apply errata patches. (But it looks like Uwe, I am NOT doing it 100% right.) So I was on 4.5, 4.6 came out. I got the 4.6 CD, followed the upgrade process. I replaced /usr/src with the 4.6 code off the CD, downloaded the errata patches, built. 4.7 comes out, I upgrade from CD, copied the 4.7 source over, downloaded errata, built. Seems to work most of the time apart from a patch requiring a userland build (like openssl) so this is where I am obviously going wrong. A snapshot is a point-in-time, may-have-issues, best way to follow current build, isn't it? But I'd rather follow release. I appreciate you've got better things to do than hand-holding, so I'll work my way through it, but any cluesticks appreciated. Thanks. It appears simple enough to me, albeit it's early and I'm not caffeine fueled yet. As part of the any upgrade rm -rf /usr/obj/*, or wherever you are storing the object files. /usr/xobj/*, and /usr/pobj/* too!. By doing the upgrade you have just obliterated any chance of there being a reliable relationship between those object files and the installed programs. Any time you switch between compiling from source and manually dumping object codes and programs onto the system you should start the compile process from scratch to re-establish a known state. After an upgrade, and after you have recreated the /usr/obj, usr/xobj, and /usr/pobj directories as part of recompiling patches, ports, whatever, you should not need to nuke the object directories if you want to save a few minutes of time (on most semi-modern systems). The plan I would recommend is 1) Upgrade. 2) Follow release(8) to generate a coherent source - obj - program relationship at your desired -release, -stable, -current level. 3) Check that the entire system built without errors. 3) Apply whatever errata you now need. But really, just do a whole system build everytime. If you are fiddling with systems way far away why try to save a few minutes when you can be extra careful? Ken
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Sat, Jun 5, 2010 at 5:25 AM, Uwe Dippel udip...@gmail.com wrote: Philip Guenther guenther at gmail.com writes: Please point to the part of the Upgrade Guide which talks about building from source, untarring the src tar file, or applying errata. I can't seem to find any such reference, but I'm sure it's in there somewhere, because you originally said that you did the upgrade exactly following the upgrade guide and, as we found out later, your steps included building from source. You misread what I did. Unfortunately, I read your first five message exactly, in which say you upgraded but completely fail to mention building from source or patching. It was only in the last message of the original thread you mention patching, though you didn't describe the process you used at that point. I was following the Upgrade Guide to the dot, following Applying patches in OpenBSD to the dot, This thread perhaps wouldn't have happened if you hadn't waited until your 13th message to describe that last part. You now have and now it seems the core discussion is just about whether (or where) an additional rm -rf /usr/obj/* should be added to help people that know enough to set up the source tree for building/patching by untaring src.tar.gz but don't know to remove the obj tree at the same time. shurg I'm just glad to hear that it's PEBCAK instead of some bizarre bug in the kernel or installer. Philip Guenther
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
I was following the Upgrade Guide to the dot, following Applying patches in OpenBSD to the dot, This thread perhaps wouldn't have happened if you hadn't waited until your 13th message to describe that last part. You now have and now it seems the core discussion is just about whether (or where) an additional rm -rf /usr/obj/* should be added to help people that ^^ know enough to set up the source tree for building/patching by untaring src.tar.gz but don't know to remove the obj tree at the same time. shurg I'm just glad to hear that it's PEBCAK instead of some bizarre bug in the kernel or installer. Not people. One person, who (a) miscommunicated, (b) thought he was smart enough and wasn't. Now there's calls for more text to be added to various places. Will it be read by anyone? Nope.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Perhaps every section of the FAQ begin with an exhortation to read the entire FAQ. I am flabbergasted that someone who runs a 'production' box would put themeselves in this position.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Philip Guenther guenther at gmail.com writes: You now have and now it seems the core discussion is just about whether (or where) an additional rm -rf /usr/obj/* should be added to help people that know enough to set up the source tree for building/patching by untaring src.tar.gz but don't know to remove the obj tree at the same time. So, no diff here, but a suggestion: If one needs to avoid stale stuff lying around in /usr/obj at applying a patch, the only logical consequence is, to clean out all /obj totally, even before applying a single patch. If I am correct, the instructions should be clear for 00N_ThisApp.patch: Apply by doing: cd /usr/src patch -p0 00N_ThisApp.patch Clean the build directories by issuing the command /usr/sbin/mk_build_clean And then rebuild and install the library and statically-linked binaries that depend upon it: cd lib/libThisApp make obj make depend make includes make make install cd ../../sbin make obj make depend make make install , where mk_build_clean is just the set of steps pointed out in 'man release', respectively in FAQ5. To me, and I guess Richard Toohey, the case is solved. Everyone who can read, and likes following instructions, can read and follow this easily. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
So, no diff here, but a suggestion: If one needs to avoid stale stuff lying around in /usr/obj at applying a patch, the only logical consequence is, to clean out all /obj totally, even before applying a single patch. If I am correct, the instructions should be clear for 00N_ThisApp.patch: Apply by doing: cd /usr/src patch -p0 00N_ThisApp.patch Clean the build directories by issuing the command /usr/sbin/mk_build_clean And then rebuild and install the library and statically-linked binaries that depend upon it: cd lib/libThisApp make obj make depend make includes make make install cd ../../sbin make obj make depend make make install , where mk_build_clean is just the set of steps pointed out in 'man release', respectively in FAQ5. To me, and I guess Richard Toohey, the case is solved. Everyone who can read, and likes following instructions, can read and follow this easily. No. You must be at least this tall to use OpenBSD; sorry -- you are a midget.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Thu, Jun 3, 2010 at 7:18 PM, Richard Toohey richardtoo...@paradise.net.nz wrote: OK, I've tried it and cannot reproduce what you see. I've never done an upgrade from bsd.rd before, so wanted to give it a go. Obviously something different with your set-up, or where you got the files from, or factor X - but as other people have said, they can't guess what. In short - the basic bsd.rd follow these instructions worked for me here. OK, I start with 4.6 amd64 (either 4.6 or just pre-4.6 release) uname -a OpenBSD dellamd64.home 4.6 GENERIC#0 amd64 But before I upgrade, what's /sbin/pfctl? $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 492664 Dec 3 23:12 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 3e1fa4f69809adff432f9da62010a6a7 http://openbsd.org/faq/upgrade47.html One easy way to boot from the install kernel is to place the 4.7 version of bsd.rd in the root of your boot drive, then instruct the boot loader to boot using this new bsd.rd file. On amd64 and i386, you do this by entering boot bsd.rd at the initial boot prompt. OK, I'll get the bsd.rd from the 4.7 release CD (but could have used FTP.) /usr/bin/su root mv /bsd.rd /bsd46.rd mount /dev/cd0a /mnt/ cp /mnt/4.7/amd64/bsd.rd /bsd.rd umount /mnt eject /dev/cd0a reboot ... boot boot bsd.rd ... Welcome to the OpenBSD/amd64 4.7 installation program. ... I choose upgrade ... take defaults all the way until ... Location of sets? [What do I do here? I'll try http, and take the defaults. What did YOU do here?] bsd, bsd.rd, base47.tgz, misc47.tgz, comp47.tgz, man47.tgz, game47.tgz, xbase47.tgz xshare47.tgz, xfont47.tgz, xserv47.tgz ... all get to 100% no errors. ... rest of install, reboot ... $ uname -a OpenBSD dellamd64.home 4.7 GENERIC#112 amd64 $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 500856 Mar 18 15:36 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 7720c9a4dc100fe29d2d3c4a16954eb4 Thanks, Richard. No, you couldn't encounter it. It comes in later. I have now the whole upgrade session of my third machine, the log is 2 MB. Whenever I rebooted, it was okay: 1. reboot to start bsd.rd - okay 2. reboot directly after bsd.rd upgrade - okay 3. reboot after 'Final steps', before pkg_add - okay 4. reboot after 'Upgrading packages' - okay 5. reboot after patching - old files and wrong timestamps - bummer, as Theo might say. I wonder if I can put the file up into the open, or if it contains security-related matter.?? As bz2 it is just 91 k; I will of course make it available to individuals on request. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 2:18 PM, Uwe Dippel udip...@gmail.com wrote: 5. reboot after patching - old files and wrong timestamps - bummer, as Theo might say. Sorry, guys, (too quick as too often), just cat-grep pfctl shows where the old one comes in: pfctl: pf already enabled # ls -l # ls -l /etc/# ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 500856 Mar 18 10:36 /sbin/pfctl # ls -l # ls -l /sbin/ # ls -l # ls -l /sbin/pfctl # ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 500856 Mar 18 10:36 /sbin/pfctl -r-xr-xr-x 1 root bin500856 Mar 18 10:36 pfctl # pfctl -f # pfctl -f /etc/# pfctl -f # pfctl -f /etc/pf.conf# pfctl -f /etc/pf.conf # # pfctl -e pfctl: pf already enabled # # pfctl -d # # pfctl -e === pfctl /usr/src/sbin/pfctl/obj - /usr/obj/sbin/pfctl === pfctl === pfctl nroff -Tascii -mandoc /usr/src/sbin/pfctl/pfctl.8 pfctl.cat8 === pfctl install -c -s -o root -g bin -m 555 pfctl /sbin/pfctl install -c -o root -g bin -m 444 pfctl.cat8 /usr/share/man/cat8/pfctl.0 pfctl: DIOCBEGINADDRS: Operation not supported by device pfctl: DIOCBEGINADDRS: Operation not supported by device -r-xr-xr-x 1 root bin492664 Jun 4 13:28 pfctl Through patching outdated(?) source files; though with the proper time stamp: # cd /home/ftp/pub/OpenBSD/4.7 # ls -l -r-xr-xr-x 1 root ftp 131759003 Mar 21 19:17 src.tar.gz -rw-r--r-- 1 root ftp 20668814 Mar 21 19:17 sys.tar.gz # md5 src.tar.gz MD5 (src.tar.gz) = 5214cd951cac5b7fbd89c968d1b5f859 # md5 sys.tar.gz MD5 (sys.tar.gz) = 566c0cd7c3d2f28b17a9795324ead6ff Maybe TeXitoi was right, after all, when he mentioned corrupted files on some mirrors? I wouldn't bet on it, but usually our fastest mirror here is ftp://ftp.jaist.ac.jp. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Thu, Jun 3, 2010 at 11:18 PM, Uwe Dippel udip...@gmail.com wrote: On Thu, Jun 3, 2010 at 7:18 PM, Richard Toohey richardtoo...@paradise.net.nz wrote: OK, I've tried it and cannot reproduce what you see. I've never done an upgrade from bsd.rd before, so wanted to give it a go. Obviously something different with your set-up, or where you got the files from, or factor X - but as other people have said, they can't guess what. In short - the basic bsd.rd follow these instructions worked for me here. OK, I start with 4.6 amd64 (either 4.6 or just pre-4.6 release) uname -a OpenBSD dellamd64.home 4.6 GENERIC#0 amd64 But before I upgrade, what's /sbin/pfctl? $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 492664 Dec 3 23:12 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 3e1fa4f69809adff432f9da62010a6a7 http://openbsd.org/faq/upgrade47.html One easy way to boot from the install kernel is to place the 4.7 version of bsd.rd in the root of your boot drive, then instruct the boot loader to boot using this new bsd.rd file. On amd64 and i386, you do this by entering boot bsd.rd at the initial boot prompt. OK, I'll get the bsd.rd from the 4.7 release CD (but could have used FTP.) /usr/bin/su root mv /bsd.rd /bsd46.rd mount /dev/cd0a /mnt/ cp /mnt/4.7/amd64/bsd.rd /bsd.rd umount /mnt eject /dev/cd0a reboot ... boot boot bsd.rd ... Welcome to the OpenBSD/amd64 4.7 installation program. ... I choose upgrade ... take defaults all the way until ... Location of sets? [What do I do here? I'll try http, and take the defaults. What did YOU do here?] bsd, bsd.rd, base47.tgz, misc47.tgz, comp47.tgz, man47.tgz, game47.tgz, xbase47.tgz xshare47.tgz, xfont47.tgz, xserv47.tgz ... all get to 100% no errors. ... rest of install, reboot ... $ uname -a OpenBSD dellamd64.home 4.7 GENERIC#112 amd64 $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 500856 Mar 18 15:36 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 7720c9a4dc100fe29d2d3c4a16954eb4 Thanks, Richard. No, you couldn't encounter it. It comes in later. I have now the whole upgrade session of my third machine, the log is 2 MB. Whenever I rebooted, it was okay: 1. reboot to start bsd.rd - okay 2. reboot directly after bsd.rd upgrade - okay 3. reboot after 'Final steps', before pkg_add - okay 4. reboot after 'Upgrading packages' - okay 5. reboot after patching - old files and wrong timestamps - bummer, as Theo might say. you mean applying the errata47.html patches? If so, are you certain your source tree is tagged OPENBSD_4_7 and not anything else? --patrick I wonder if I can put the file up into the open, or if it contains security-related matter.?? As bz2 it is just 91 k; I will of course make it available to individuals on request. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 4/06/2010, at 6:41 PM, patrick keshishian wrote: On Thu, Jun 3, 2010 at 11:18 PM, Uwe Dippel udip...@gmail.com wrote: On Thu, Jun 3, 2010 at 7:18 PM, Richard Toohey richardtoo...@paradise.net.nz wrote: OK, I've tried it and cannot reproduce what you see. I've never done an upgrade from bsd.rd before, so wanted to give it a go. Obviously something different with your set-up, or where you got the files from, or factor X - but as other people have said, they can't guess what. In short - the basic bsd.rd follow these instructions worked for me here. OK, I start with 4.6 amd64 (either 4.6 or just pre-4.6 release) uname -a OpenBSD dellamd64.home 4.6 GENERIC#0 amd64 But before I upgrade, what's /sbin/pfctl? $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 492664 Dec 3 23:12 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 3e1fa4f69809adff432f9da62010a6a7 http://openbsd.org/faq/upgrade47.html One easy way to boot from the install kernel is to place the 4.7 version of bsd.rd in the root of your boot drive, then instruct the boot loader to boot using this new bsd.rd file. On amd64 and i386, you do this by entering boot bsd.rd at the initial boot prompt. OK, I'll get the bsd.rd from the 4.7 release CD (but could have used FTP.) /usr/bin/su root mv /bsd.rd /bsd46.rd mount /dev/cd0a /mnt/ cp /mnt/4.7/amd64/bsd.rd /bsd.rd umount /mnt eject /dev/cd0a reboot ... boot boot bsd.rd ... Welcome to the OpenBSD/amd64 4.7 installation program. ... I choose upgrade ... take defaults all the way until ... Location of sets? [What do I do here? I'll try http, and take the defaults. What did YOU do here?] bsd, bsd.rd, base47.tgz, misc47.tgz, comp47.tgz, man47.tgz, game47.tgz, xbase47.tgz xshare47.tgz, xfont47.tgz, xserv47.tgz ... all get to 100% no errors. ... rest of install, reboot ... $ uname -a OpenBSD dellamd64.home 4.7 GENERIC#112 amd64 $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 500856 Mar 18 15:36 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 7720c9a4dc100fe29d2d3c4a16954eb4 Thanks, Richard. No, you couldn't encounter it. It comes in later. I have now the whole upgrade session of my third machine, the log is 2 MB. Whenever I rebooted, it was okay: 1. reboot to start bsd.rd - okay 2. reboot directly after bsd.rd upgrade - okay 3. reboot after 'Final steps', before pkg_add - okay 4. reboot after 'Upgrading packages' - okay 5. reboot after patching - old files and wrong timestamps - bummer, as Theo might say. you mean applying the errata47.html patches? If so, are you certain your source tree is tagged OPENBSD_4_7 and not anything else? --patrick I think we are getting closer, aren't we? So, NOTHING to do with the actual upgrade, is it? Or the ports/packages. It is something to do with how you are PATCHING after an upgrade. You don't mention where/when you get the source you patch? Because that would be a separate step, wouldn't it? (I usually install from CD, so I scrub /usr/src load from src.tar.gz on the CD.) I wonder if I can put the file up into the open, or if it contains security-related matter.?? As bz2 it is just 91 k; I will of course make it available to individuals on request. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 2:41 PM, patrick keshishian pkesh...@gmail.com wrote: you mean applying the errata47.html patches? If so, are you certain your source tree is tagged OPENBSD_4_7 and not anything else? Do I understand you correctly? I am not building releases. I am installing/downloading the sets; then I do all the stuff in 'Upgrade guide', then rm -Rf * in /usr/src rm -Rf * in /usr/xenocara rm -Rf * in /usr/ports, and then tar ... the source files meticulously as pointed out in the guide: # cd /usr/src # tar xzf ../sys.tar.gz # tar xzf ../src.tar.gz # cd /usr # tar xzf xenocara.tar.gz # tar xzf ports.tar.gz and then download the patches into /usr/src, then applying them and this is what you would see in the serial log. So I don't tag, because I don't cvs; what i do is just download the 4 files. (Also see my other post, it points clearly to the sequence and the reboots done, with always checking pfctl after each reboot.) Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 3:00 PM, Richard Toohey richardtoo...@paradise.net.nz wrote: I think we are getting closer, aren't we? So, NOTHING to do with the actual upgrade, is it? No absolutely nothing. I withdraw the subject with regret. At least the 'base47'-part thereof. Or the ports/packages. I guess, not. It is something to do with how you are PATCHING after an upgrade. You don't mention where/when you get the source you patch? Because that would be a separate step, wouldn't it? (I usually install from CD, so I scrub /usr/src load from src.tar.gz on the CD.) Exactly. Just explained it in the previous post, and don't want to repeat myself. Except that I download, and then the actual files used were: # md5 /usr/src.tar.gz MD5 (/usr/src.tar.gz) = 5214cd951cac5b7fbd89c968d1b5f859 # md5 /usr/sys.tar.gz MD5 (/usr/sys.tar.gz) = 566c0cd7c3d2f28b17a9795324ead6ff (Here, contrary to the earlier post, at the actual location in /usr of the target machine before extraction: # ls -l /usr/s* -rw-r--r-- 1 root wheel 131759003 Mar 21 19:17 /usr/src.tar.gz -rw-r--r-- 1 root wheel 20668814 Mar 21 19:17 /usr/sys.tar.gz) Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 04, 2010 at 03:55:53PM +0800, Uwe Dippel wrote: On Fri, Jun 4, 2010 at 2:41 PM, patrick keshishian pkesh...@gmail.com wrote: you mean applying the errata47.html patches? If so, are you certain your source tree is tagged OPENBSD_4_7 and not anything else? Do I understand you correctly? I am not building releases. I am installing/downloading the sets; then I do all the stuff in 'Upgrade guide', then rm -Rf * in /usr/src rm -Rf * in /usr/xenocara rm -Rf * in /usr/ports, and then tar ... the source files meticulously as pointed out in the guide: # cd /usr/src # tar xzf ../sys.tar.gz # tar xzf ../src.tar.gz # cd /usr # tar xzf xenocara.tar.gz # tar xzf ports.tar.gz and then download the patches into /usr/src, then applying them and this is what you would see in the serial log. So I don't tag, because I don't cvs; what i do is just download the 4 files. (Also see my other post, it points clearly to the sequence and the reboots done, with always checking pfctl after each reboot.) Uwe Don't you have old stuff lying around in /usr/obj that gets installed over your new binaries?
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 12:55 AM, Uwe Dippel udip...@gmail.com wrote: On Fri, Jun 4, 2010 at 2:41 PM, patrick keshishian pkesh...@gmail.com wrote: you mean applying the errata47.html patches? If so, are you certain your source tree is tagged OPENBSD_4_7 and not anything else? Do I understand you correctly? I am not building releases. I am installing/downloading the sets; then I do all the stuff in 'Upgrade guide', then rm -Rf * in /usr/src rm -Rf * in /usr/xenocara rm -Rf * in /usr/ports, and then tar ... the source files meticulously as pointed out in the guide: # cd /usr/src # tar xzf ../sys.tar.gz # tar xzf ../src.tar.gz Where did you get those tar-balls from? Those are most likely not 4.7 sources. --patrick
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 4:32 PM, patrick keshishian pkesh...@gmail.com wrote: Where did you get those tar-balls from? Those are most likely not 4.7 sources. I gave the potential link and their md5 sums further up. Our link here is sooo slooow; I *am* currently downloading the archives from ftp://ftp.openbsd.org/pub/OpenBSD/4.7 to compare the checksums. That would explain a lot (though not everything, since the kernel looks pretty correct: 4.7). Maybe someone is faster and can confirm or refute the authenticity of the archives? Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 4:22 PM, Eric Faurot e...@faurot.net wrote: Don't you have old stuff lying around in /usr/obj that gets installed over your new binaries? That's probably the critical question now. Though, sorry to say, there is nowhere written that you have to rm -Rf it, when you - upgrade - patch Actually, I wasn't even aware of the existence of this directory until several minutes ago. (I expected it to be cleaned with wiping the source directories.) Then, according to what is written by a number of people further up, an number of people could be hit by this. I for one would expect the time stamps to take care for that. And, to stress it again: When you are under 'quality control', and responsible for the uptime of a system, you would never do anything out of the scope of instructions, naturally. especially not some rm -Rf * in a directory of your arbitrary choice. ;) And don't point me to man release, please. I am not doing releases, I am not doing stable, I do, like many others, 'Upgrade Guide X.Y to X.Z', and then get and apply the errata from http://openbsd.org/errataXZ.html; according to their instructions. If this happens to be wrong, by all means, then I make a mistake, and have been making this mistake for 5 years. So, rm -Rf * in /usr/obj is necessary? Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 4/06/2010, at 8:33 PM, Uwe Dippel wrote: On Fri, Jun 4, 2010 at 4:22 PM, Eric Faurot e...@faurot.net wrote: Don't you have old stuff lying around in /usr/obj that gets installed over your new binaries? That's probably the critical question now. Though, sorry to say, there is nowhere written that you have to rm -Rf it, when you - upgrade - patch Actually, I wasn't even aware of the existence of this directory until several minutes ago. (I expected it to be cleaned with wiping the source directories.) [cut] I do, like many others, 'Upgrade Guide X.Y to X.Z', and then get and apply the errata from http://openbsd.org/errataXZ.html; according to their instructions. If this happens to be wrong, by all means, then I make a mistake, and have been making this mistake for 5 years. So, rm -Rf * in /usr/obj is necessary? Uwe I've been doing upgrades from *around* the 4.2 release, but always from the release CDs. And then I've patched as per the errata instructions. And I've not (so far as I can recall) can to clear /usr/obj before applying patches. And as I type that I remembered! Hang on, there *was* once a patch that didn't apply cleanly ... think it was this one (but then I must have started with 3.9) - definitely an SSL one that wouldn't build without clearing /usr/obj first: http://marc.info/?l=openbsd-miscm=116335647308807w=2 So something for the upgrade docs or the patch file(s)? Maybe patch 3 for 4.7 might be responsible this time around? If you try the make clean step when applying patch 3 (something like in the link above - it might fix your problems.) Anyway, absolutely nothing to do with the installer or the 4.6 to 4.7 upgrade, so enough from me. Thanks.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 04, 2010 at 04:40:28PM +0800, Uwe Dippel wrote: On Fri, Jun 4, 2010 at 4:32 PM, patrick keshishian pkesh...@gmail.com wrote: Where did you get those tar-balls from? Those are most likely not 4.7 sources. I gave the potential link and their md5 sums further up. Our link here is sooo slooow; I *am* currently downloading the archives from ftp://ftp.openbsd.org/pub/OpenBSD/4.7 to compare the checksums. That would explain a lot (though not everything, since the kernel looks pretty correct: 4.7). Maybe someone is faster and can confirm or refute the authenticity of the archives? Uwe oh good grief. you had a dirty /usr/obj. just look at the pfctl snippet of the log you posted. do you see pfctl being built? do you see pfctl being installed from /usr/obj? -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 04, 2010 at 04:33:34PM +0800, Uwe Dippel wrote: When you are under 'quality control', and responsible for the uptime of a system, you would never do anything out of the scope of instructions, naturally. especially not some rm -Rf * in a directory of your arbitrary choice. ;) if this is so important: a) reinstall, don't upgrade b) learn to read logs and understand what you are doing And don't point me to man release, please. I am not doing releases, but, actually, you are. you're just skipping lots of it. have been making this mistake for 5 years. So, rm -Rf * in /usr/obj is necessary? learn the system and tools you use. is it necessary? depends on the history of the machine in question. I bet you built pfctl at least once before, no? -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 04, 2010 at 04:33:34PM +0800, Uwe Dippel wrote: When you are under 'quality control', and responsible for the uptime of a system, you would never do anything out of the scope of instructions, naturally. especially not some rm -Rf * in a directory of your arbitrary choice. ;) 'rm -rf /usr/obj/*' is not your arbitrary choice, it's a documented step when building the system. http://www.openbsd.org/faq/faq5.html#BldUserland
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, 4 Jun 2010, Jan Stary wrote: 'rm -rf /usr/obj/*' is not your arbitrary choice, it's a documented step when building the system. http://www.openbsd.org/faq/faq5.html#BldUserland Also documented there is keeping /usr/obj/ in its own partition and using 'newfs' to zap everything more quickly. /Lars
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Jacob Meuser jakemsr at sdf.lonestar.org writes: oh good grief. you had a dirty /usr/obj. just look at the pfctl snippet of the log you posted. do you see pfctl being built? do you see pfctl being installed from /usr/obj? Oh, yes. So the blame is on my side, I guess. Mea culpa maxima! I didn't know that the object directories need to be cleaned manually. Until yesterday, I would have taken a bet that the object directories lie within the source trees (/usr/xenocaram /usr/src), and be cleaned when cleaning the sources. Now I am aware that I need to know the location of the object directories and clean them manually. I was totally unaware that, in case of a patch, the installer would take the next best file of the correct name from there; irrespective of the underlying version. Though I feel in good company. I guess, a great number of people on this list were in a similar situation. Knowing the 'social contract' of OpenBSD, I only have to blame myself for ignorance. Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? Also, with respect to the 'errata', the patches, they describe in detail what needs to be done. Maybe here, it could as well be suggested, that before applying the first patch of a new version of OpenBSD, /usr/obj should be cleaned, or be verified to be clean? Thanks for the various people who helped me patiently at analysing this problem to the very end! Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Might be better to read and comprehend ``man patch'' before assuming limitations on the scope of patch's reach. -Original Message- From: owner-m...@openbsd.org [mailto:owner-m...@openbsd.org] On Behalf Of Uwe Dippel Sent: Friday, June 04, 2010 11:23 AM To: misc@openbsd.org Subject: Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64 Jacob Meuser jakemsr at sdf.lonestar.org writes: oh good grief. you had a dirty /usr/obj. just look at the pfctl snippet of the log you posted. do you see pfctl being built? do you see pfctl being installed from /usr/obj? Oh, yes. So the blame is on my side, I guess. Mea culpa maxima! I didn't know that the object directories need to be cleaned manually. Until yesterday, I would have taken a bet that the object directories lie within the source trees (/usr/xenocaram /usr/src), and be cleaned when cleaning the sources. Now I am aware that I need to know the location of the object directories and clean them manually. I was totally unaware that, in case of a patch, the installer would take the next best file of the correct name from there; irrespective of the underlying version. Though I feel in good company. I guess, a great number of people on this list were in a similar situation. Knowing the 'social contract' of OpenBSD, I only have to blame myself for ignorance. Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? Also, with respect to the 'errata', the patches, they describe in detail what needs to be done. Maybe here, it could as well be suggested, that before applying the first patch of a new version of OpenBSD, /usr/obj should be cleaned, or be verified to be clean? Thanks for the various people who helped me patiently at analysing this problem to the very end! Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 04, 2010 at 04:22:35PM +, Uwe Dippel wrote: Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? you can't supply a patch? can't even attempt one? all these posts and time wasted and you can't try to make a patch to maybe save someone else from the same fate? -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
* Uwe Dippel udip...@gmail.com [2010-06-04 18:26]: I didn't know that the object directories need to be cleaned manually. this should not be needed assuming system time didn't jump. it is still good practice tho. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Jacob Meuser wrote: On Fri, Jun 04, 2010 at 04:22:35PM +, Uwe Dippel wrote: Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? you can't supply a patch? can't even attempt one? all these posts and time wasted and you can't try to make a patch to maybe save someone else from the same fate? a patch to the upgrade guide would be wrong. The problem is the patching process (a special case of the userland build process) assumes a clean obj dir. This has nothing to do with upgrades. If you try to rebuild the same userland utility more than once for /any/ reason without clearing the obj dir, you can run into problems. Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. faq10.html, section 10.15 would be more appropriate, I think. It will probably require more than a couple line diff to work it in clearly. Nick.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
snip Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. oops. What's the recommended procedure for this? -B
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 9:22 AM, Uwe Dippel udip...@gmail.com wrote: ... Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? Please point to the part of the Upgrade Guide which talks about building from source, untarring the src tar file, or applying errata. I can't seem to find any such reference, but I'm sure it's in there somewhere, because you originally said that you did the upgrade exactly following the upgrade guide and, as we found out later, your steps included building from source. The other possibility is that you did stuff that was *not* in the upgrade guide, in which case adding text to the upgrade guide saying if you do this other stuff that's described elsewhere then you should read those other directions and follow them completely seems kinda bizarre. Also, with respect to the 'errata', the patches, they describe in detail what needs to be done. Maybe here, it could as well be suggested, that before applying the first patch of a new version of OpenBSD, /usr/obj should be cleaned, or be verified to be clean? It already is the first bullet in the directions for building userland from source: http://www.openbsd.org/faq/faq5.html#BldUserland Philip Guenther
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 5/06/2010, at 7:31 AM, Nick Holland wrote: Jacob Meuser wrote: On Fri, Jun 04, 2010 at 04:22:35PM +, Uwe Dippel wrote: Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? you can't supply a patch? can't even attempt one? all these posts and time wasted and you can't try to make a patch to maybe save someone else from the same fate? a patch to the upgrade guide would be wrong. The problem is the patching process (a special case of the userland build process) assumes a clean obj dir. This has nothing to do with upgrades. If you try to rebuild the same userland utility more than once for /any/ reason without clearing the obj dir, you can run into problems. Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. faq10.html, section 10.15 would be more appropriate, I think. It will probably require more than a couple line diff to work it in clearly. Nick. So something in or around here ... Not all patches are for the kernel. In some cases, you will have to rebuild individual utilities. At other times, will require recompiling all utilities statically linked to a patched library. Follow the guidance in the header of the patch, and if uncertain, rebuild the entire system. Along the lines of: In some cases (you'll know you have do this because x) you need to clean /usr/obj first, you do this by y. Let me learn what x and y are. Thanks.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Sat, Jun 05, 2010 at 10:29:57AM +1200, Richard Toohey wrote: On 5/06/2010, at 7:31 AM, Nick Holland wrote: Jacob Meuser wrote: On Fri, Jun 04, 2010 at 04:22:35PM +, Uwe Dippel wrote: Still, may I suggest, that the next Upgrade Guide gets an extra line, with a remark pointing out the existence of /usr/obj; and the suggestion to clean it? you can't supply a patch? can't even attempt one? all these posts and time wasted and you can't try to make a patch to maybe save someone else from the same fate? a patch to the upgrade guide would be wrong. The problem is the patching process (a special case of the userland build process) assumes a clean obj dir. This has nothing to do with upgrades. If you try to rebuild the same userland utility more than once for /any/ reason without clearing the obj dir, you can run into problems. Clearing the obj directory as part of the upgrade is like flushing your toilet based on the date -- may help, but after a while, things start to stink. It isn't the general (or proper) solution. faq10.html, section 10.15 would be more appropriate, I think. It will probably require more than a couple line diff to work it in clearly. Nick. So something in or around here ... Not all patches are for the kernel. In some cases, you will have to rebuild individual utilities. At other times, will require recompiling all utilities statically linked to a patched library. Follow the guidance in the header of the patch, and if uncertain, rebuild the entire system. Along the lines of: In some cases (you'll know you have do this because x) you need to clean /usr/obj first, you do this by y. Let me learn what x and y are. Thanks. I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. -- jake...@sdf.lonestar.org SDF Public Access UNIX System - http://sdf.lonestar.org
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 04, 2010 at 10:31:22PM -0700, patrick keshishian wrote: On Fri, Jun 4, 2010 at 7:49 PM, Jacob Meuser jake...@sdf.lonestar.org wrote: I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. Do you not agree barring broken makefiles and unreliable system clock (as someone pointed out), object files and binaries (in obj/) should have been rebuilt? --patrick Consider the follwoing scenario: I have a release n system, last built on May 1. So the obj files have a time stamp of May 1. The files of the tarbal of the new release n+1 have timestamps of Apr 1 or earlier, since it was made at that time. If I then install the tarball by extracting, according to makethe object files are still fresh, since the sources will be older than the object files. By default the files extracted form a tartball get the timestamps from the tarbal. There are some variations on this scenario. -Otto
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Fri, Jun 4, 2010 at 7:49 PM, Jacob Meuser jake...@sdf.lonestar.org wrote: I'm still curious how anything left in /usr/obj can be anything but a possible problem after updating system binaries and sources to a new release. especially for people who are just following the directions as they are written. Do you not agree barring broken makefiles and unreliable system clock (as someone pointed out), object files and binaries (in obj/) should have been rebuilt? It's a source tree with nearly 40,000 .[chyl] source code files, and probably another 40,000 further source code dependencies if you include manual pages and the perl parts. We try very hard, and the bsd.*.mk macro package helps a lot (enforcing consistancy-because-of-simplicity), but if you think we can get all the dependencies right every single time, it is a tough call. But this case is worse -- when the trash in the obj tree totally mis-matches the src tree since it is so far in the past... that is totally impossible. Dependencies don't help when they don't know about the files. Even make clean or cleandir won't help you then. This was not an installer bug. It had nothing to do with upgrades. We've said it before, and I guess we get to say it again: If don't know what you are doing, install a new snapshot. How many more times do we have to say that? Why are people defending a person who thinks they are smart enough, and has just proved that they're not? Miod, Dale, Kurt, Kettenis and I am quite often the first people to deal with bumping systems forward over bumps. Some bumps are so difficult that after they are done the rest of us jump over them using snapshots. When they happen, WE -- THE DEVELOPERS -- USE THE SNAPSHOTS! They happen in lots of releases. Why would we use snapshots, because we are stupid? Or are we smart enough to not waste our time doing things the hard way? Uwe thinks he's being really clever, but he's not clever at all. He's got a record of choosing the hardest paths. That's his problem. I just wish he wouldn't be such a loud whiner when he screws his system up.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Theo de Raadt deraadt at cvs.openbsd.org writes: A chit-chat on a public mailing list isn't going to find this supposed bug. Why discuss it? Why not just keep prove it happened. Yes, Theo. Though: How? This is what I tried to find out. I showed the list if files. Do you assume I tinkered with it? Why should I? pfctl wasn't working correctly. Without the help of the list, I wouldn't have been able to drill it down to some 70 files being of the previous version. Thanks to everyone who helped! Don't you see how tiring it is to discuss it when we've seen no evidence? It might be tiring, but what evidence do you want? Here, I want to solve a problem of files missing. Since I followed the Upgrade guide to the dot, rebooted to bsd.rd in the beginning, rebooted at the command prompt, we (I) need to find what went wrong. That's all. I don't even mind if the mistake was on my side, then I could learn. So, please, specify the evidence that you need. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Theo de Raadt deraadt at cvs.openbsd.org writes: A chit-chat on a public mailing list isn't going to find this supposed bug. Why discuss it? Why not just keep prove it happened. Yes, Theo. Though: How? This is what I tried to find out. I showed the list if files. Do you assume I tinkered with it? Why should I? pfctl wasn't working correctly. Without the help of the list, I wouldn't have been able to drill it down to some 70 files being of the previous version. Thanks to everyone who helped! Don't you see how tiring it is to discuss it when we've seen no evidence? It might be tiring, but what evidence do you want? Here, I want to solve a problem of files missing. Since I followed the Upgrade guide to the dot, rebooted to bsd.rd in the beginning, rebooted at the command prompt, we (I) need to find what went wrong. That's all. I don't even mind if the mistake was on my side, then I could learn. So, please, specify the evidence that you need. If everyone felt the need to debug the personal problem with their own machines on this giant mailing list in the fashion you just did, I will unsubscribe. It isn't tiring -- it is just plain ridiculous. Figure out what is wrong, THEN POST THAT.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Uwe Dippel wrote: drill it down to some 70 files being of the previous version. It might be tiring, but what evidence do you want? The error message(s) you are suppressing (or maybe didn't see) About the only way you can get some files but not all files from a tarball is some fatal error in the extraction of the tarball. Any such error tends to give an error message. I don't think this list likes to play guessing games as to exactly what mistakes you have made or what evidence you are suppressing.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Thu, Jun 3, 2010 at 3:20 PM, Tony Abernethy t...@servasoftware.com wrote: The error message(s) you are suppressing (or maybe didn't see) About the only way you can get some files but not all files from a tarball is some fatal error in the extraction of the tarball. Any such error tends to give an error message. I don't think this list likes to play guessing games as to exactly what mistakes you have made or what evidence you are suppressing. Oh, how beautiful! This is a sign of mutual trust. I documented everything from the first pfctl after reboot from upgrade not working, for what I am chided; and still, I am supposed to 'suppress evidence'. How nice of you! And if I present a serial log, I will have been suspected to have tempered with it? No, that seriously turns me off. I have given everything in detail that I came across, I have not been silent about any additional message, any unusual activity. I have stated a few times that I followed the upgrade procedure to the dot, I have confirmed that nothing unusual showed. Over. I might have made some mistake, yes. Even though these same boxes have been upgraded since 3.8, nevermind. I could at all times have made a mistake or overlooked something. But to start kind of asking for 'proof', that's what's ridiculous, to cite Theo. I am willing to give individuals unprivileged access to the boxes, I did this before, to look around. When you have a box that is relevant to your company, and you are responsible for it, and you noticed something unusual, why would you not want to come screaming to the list (like it did, my excuses), to look for help, but 'conveniently avoid' mentioning that serious error message during the upgrade? You need the box to be up and running, and adding this error message can only help; so why would you suppress it, maybe preventing efficient help to be offered? I fully agree that what I report sounds highly unlikely. But it is true, and by now I confirmed exactly the same having happened on that other box (i386), If I suppressed anything, why should I add to the improbability? Yes, it happened, and I applied the same method and have by now tar xzvphf-ed the 70+ sbin-files that were there and - identically to the previous amd64 - are from version 4.6. It is not excluded, as I wrote earlier, that the upgrade itself does everything 100% correct. Who knows, there can always be a rogue package. Not that I'm saying this has happened, but theoretically, any package could contain 4.6-files and write them back at pkg_add. Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 3/06/2010, at 8:42 PM, Uwe Dippel wrote: [cut] No, that seriously turns me off. I have given everything in detail that I came across, I have not been silent about any additional message, any unusual activity. I have stated a few times that I followed the upgrade procedure to the dot, I have confirmed that nothing unusual showed. [cut] quoteeverything in detail ... I have not been silent/unquote I think that is the point that people are trying to make - you've sent so much to the list that it is hard to keep up. The people who can help may not want to wade through 20 emails to keep up with you. Take a deep breath. Take a 4.6 CD install on a spare clean machine and upgrade via the CD. Problem there? Yes/no. Start again with the 4.6 CD a CLEAN install, upgrade via your normal process to 4.7. Is the issue there? Yes/no. And then follow the other suggestions from minds wiser than mine. So far as you are aware, if you upgrade amd64 from a 4.6 CD to a 4.7 via your normal upgrade process, there are issues - correct? If so, I've got a spare amd64 box to try on, but to be honest I'm a bit intimated by the volume of mail you've sent - I'm losing the thread. HTH. Thanks.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Jun 03 16:42:58, Uwe Dippel wrote: On Thu, Jun 3, 2010 at 3:20 PM, Tony Abernethy t...@servasoftware.com wrote: The error message(s) you are suppressing (or maybe didn't see) About the only way you can get some files but not all files from a tarball is some fatal error in the extraction of the tarball. Any such error tends to give an error message. I don't think this list likes to play guessing games as to exactly what mistakes you have made or what evidence you are suppressing. Oh, how beautiful! This is a sign of mutual trust. I documented everything from the first pfctl after reboot from upgrade not working, for what I am chided; and still, I am supposed to 'suppress evidence'. How nice of you! And if I present a serial log, I will have been suspected to have tempered with it? Where is the complete serial log of the install and subsequent md5 checks? Because that's the only way anybody can see what really happened, you know. But to start kind of asking for 'proof', that's what's ridiculous, to cite Theo. No. You claim the installer of 4.7 installs some, but not all, of the files in the install sets; or that some of the files instal 4.7 install sets are actually 4.6 files, according to their md5 hashes. That's what's highly suspectible (if not ridiculous). Don't be surprised that people want to be sure this actually happened before they invest time into investigating it. I am willing to give individuals unprivileged access to the boxes, I did this before, to look around. That's not a substitute for the complete, unedited serial log of everything that happend from the moment you put an official CD in to the moment you run the md5 test (included). Having root account on your machine will not give me this information. I fully agree that what I report sounds highly unlikely. But it is true, and by now I confirmed exactly the same having happened on that other box (i386), Where's the two complete serial logs? If I suppressed anything, why should I add to the improbability? Yes, it happened, and I applied the same method and have by now tar xzvphf-ed the 70+ sbin-files that were there and - That were _where_? Do you claim that the base47.tgz file contains 4.6 versions of certain files in /sbin? Where exactly does this strange base47.tgz file come from? identically to the previous amd64 - are from version 4.6. It is not excluded, as I wrote earlier, that the upgrade itself does everything 100% correct. Who knows, there can always be a rogue package. Not that I'm saying this has happened, but theoretically, any package could contain 4.6-files and write them back at pkg_add. Packages install under /usr/local and don't interfere with files from a base install. Jan
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 3/06/2010, at 9:02 PM, Richard Toohey wrote: On 3/06/2010, at 8:42 PM, Uwe Dippel wrote: [cut] No, that seriously turns me off. I have given everything in detail that I came across, I have not been silent about any additional message, any unusual activity. I have stated a few times that I followed the upgrade procedure to the dot, I have confirmed that nothing unusual showed. [cut] quoteeverything in detail ... I have not been silent/unquote I think that is the point that people are trying to make - you've sent so much to the list that it is hard to keep up. The people who can help may not want to wade through 20 emails to keep up with you. Take a deep breath. Take a 4.6 CD install on a spare clean machine and upgrade via the CD. Problem there? Yes/no. Start again with the 4.6 CD a CLEAN install, upgrade via your normal process to 4.7. Is the issue there? Yes/no. And then follow the other suggestions from minds wiser than mine. So far as you are aware, if you upgrade amd64 from a 4.6 CD to a 4.7 via your normal upgrade process, there are issues - correct? If so, I've got a spare amd64 box to try on, but to be honest I'm a bit intimated by the volume of mail you've sent - I'm losing the thread. OK, I've tried it and cannot reproduce what you see. I've never done an upgrade from bsd.rd before, so wanted to give it a go. Obviously something different with your set-up, or where you got the files from, or factor X - but as other people have said, they can't guess what. In short - the basic bsd.rd follow these instructions worked for me here. OK, I start with 4.6 amd64 (either 4.6 or just pre-4.6 release) uname -a OpenBSD dellamd64.home 4.6 GENERIC#0 amd64 But before I upgrade, what's /sbin/pfctl? $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 492664 Dec 3 23:12 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 3e1fa4f69809adff432f9da62010a6a7 http://openbsd.org/faq/upgrade47.html One easy way to boot from the install kernel is to place the 4.7 version of bsd.rd in the root of your boot drive, then instruct the boot loader to boot using this new bsd.rd file. On amd64 and i386, you do this by entering boot bsd.rd at the initial boot prompt. OK, I'll get the bsd.rd from the 4.7 release CD (but could have used FTP.) /usr/bin/su root mv /bsd.rd /bsd46.rd mount /dev/cd0a /mnt/ cp /mnt/4.7/amd64/bsd.rd /bsd.rd umount /mnt eject /dev/cd0a reboot ... boot boot bsd.rd ... Welcome to the OpenBSD/amd64 4.7 installation program. ... I choose upgrade ... take defaults all the way until ... Location of sets? [What do I do here? I'll try http, and take the defaults. What did YOU do here?] bsd, bsd.rd, base47.tgz, misc47.tgz, comp47.tgz, man47.tgz, game47.tgz, xbase47.tgz xshare47.tgz, xfont47.tgz, xserv47.tgz ... all get to 100% no errors. ... rest of install, reboot ... $ uname -a OpenBSD dellamd64.home 4.7 GENERIC#112 amd64 $ ls -l /sbin/pfctl -r-xr-xr-x 1 root bin 500856 Mar 18 15:36 /sbin/pfctl $ md5 /sbin/pfctl MD5 (/sbin/pfctl) = 7720c9a4dc100fe29d2d3c4a16954eb4 HTH. Thanks.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 2010-06-03, Uwe Dippel udip...@gmail.com wrote: No, that seriously turns me off. I have given everything in detail that I came across, The problem is obviously relating to something that you didn't come across otherwise you wouldn't be posting here - if you can post logs this will give more people a chance to spot it.. Ideally send everything right back to the boot messages from bsd.rd. The request for unedited is because some people will tidy things up before posting, which might remove something that other people would find important. On a hunch: what does stat /sbin/pfctl say? Who knows, there can always be a rogue package. Not that I'm saying this has happened, but theoretically, any package could contain 4.6-files and write them back at pkg_add. You'll get a clearly visible error if the hash doesn't match that stored in the install kernel.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Getting closer ... Extracted the archive being used for the upgrade to amd64 into my user-directory and calculated all 7484 md5 for the files included in base47, and redirected those into a file. Then, I calculated all the md5 for the files *installed* in the upgraded machine; the file names taken from the same base47. Then, I had two files of 7484 md5s each, and could diff them. Further down is the result. I'm stumped. Why would these files (around 100) not be the 4.7 version, but previous (4.6 I guess; I haven't checked all). (Add /etc/pfctl to the list of different files; I had already manually copied it to /sbin from the archive to get the firewall working) So *all* sets were installed, in principle, but some files were not. Huh? I am sure, some people with more insight can help me further to explain what is going on here. What makes or made these files below so 'special' that they fail to be 'just there' after the upgrade on amd64? Thanks for any further hint, Uwe $ diff md5sums_archive md5sums_install 1c1 ./usr/lib/libasn1.so.17.0 f07fcaad530dd9632ef7de1491ed6bd3 --- ./usr/lib/libasn1.so.17.0 aa2c929c805b55008bba1bc942483b01 3,4c3,4 ./usr/lib/libcom_err.so.17.0 f07fcaad530dd9632ef7de1491ed6bd3 ./usr/lib/libcrypto.so.18.0 4280f48657120e382c01ca4c1a8aafc4 --- ./usr/lib/libcom_err.so.17.0 aa2c929c805b55008bba1bc942483b01 ./usr/lib/libcrypto.so.18.0 5f38e49397b845acdf818c520953eb0e 14,15c14,15 ./usr/lib/libkafs.so.17.0 f07fcaad530dd9632ef7de1491ed6bd3 ./usr/lib/libkrb5.so.17.0 f07fcaad530dd9632ef7de1491ed6bd3 --- ./usr/lib/libkafs.so.17.0 aa2c929c805b55008bba1bc942483b01 ./usr/lib/libkrb5.so.17.0 aa2c929c805b55008bba1bc942483b01 34c34 ./usr/lib/libssl.so.15.1 baa09f0512fbe6ecb1519de10ed6a8a4 --- ./usr/lib/libssl.so.15.1 e3dcfdfc876252231bd8994c3f0c6f1d 192,195c192,195 ./sbin/atactl 6ba0fc88a2cf2ad11bf5bdfee76b5bc5 ./sbin/badsect 4914ef0ea057a00d8b9ae91eaf894af5 ./sbin/bioctl ce8467b4415309be9b447405938fcda0 ./sbin/ccdconfig 08123bd59420bf542011b5a72c6f896c --- ./sbin/atactl e214991d640840544a90595b661b6378 ./sbin/badsect a87ca3adea353c650c15ff7db2059c94 ./sbin/bioctl 6737ac6873c92917779282cf3f3e8cb5 ./sbin/ccdconfig af51c2c19995759fcfd4f8ed6c7ab64d 197,198c197,198 ./sbin/clri 721d8795ff051d72ee67175902c53dd6 ./sbin/dhclient 3e1f06ca19aa5aceec214d2891fa504e --- ./sbin/clri afef2851d33038cb9de7c21f2d6dc037 ./sbin/dhclient 5f6a023ce04f9fc3611a56c1752c5c30 200,219c200,219 ./sbin/disklabel 3162a316caada7f5ebdd8b07d5722cb2 ./sbin/dmesg 3acb23453982bdf9974c4f3abb8d6dfd ./sbin/dump e24875bd59e468780c4f53dc8685befc ./sbin/dumpfs 5cc5b40775147860423cddf45e264fac ./sbin/fdisk 6a1a875a41cceed2874e2b1e861b0257 ./sbin/fsck 6fd26d79982bfe5d91d2f450ea495a19 ./sbin/fsck_ext2fs 5dbb372ffbdb958bcfbd8d24811c4f87 ./sbin/fsck_ffs 7bba80258056a3a40b51d24a63d4e5de ./sbin/fsck_msdos 85eacde50cecb85043601e05aac5a606 ./sbin/fsdb e46a8fa824d753af715cae0f8e4a8049 ./sbin/fsirand 65018fa13f98e1de12f5dfdcfc59cafd ./sbin/growfs 7e7ba9034167529de5cef12497e2228b ./sbin/halt e8612dfe7b7703188cb887852b073fe7 ./sbin/ifconfig bc731472da980771e922604a7f76bb7e ./sbin/init a6e6bf349857e9addcce114f5cbeebea ./sbin/iopctl b1ffd69049a845e749f1fdff490045be ./sbin/ipsecctl acdee246db653efa457193d9d7be195b ./sbin/isakmpd 6e8462f8a4c3cfc2901dbe3163c9f857 ./sbin/kbd b7da651953889ab863042dd1e05976dc ./sbin/ldattach b0b97a2496c0c2593c437842cb29d9df --- ./sbin/disklabel b6455e58788253af334bda563c12ca12 ./sbin/dmesg 4a9f96f0a968f616a4dda156ec1572f4 ./sbin/dump bdbfcd38d79289f81f23059cfb6156ea ./sbin/dumpfs 847cde118bbff6e12981ec92270aabcc ./sbin/fdisk 7b0d0a7788e323811c91c92761c7244f ./sbin/fsck 8105d9fc124a57dd343ba97d19c9fc48 ./sbin/fsck_ext2fs ed161578a1777c598c10bb6963d0b7b4 ./sbin/fsck_ffs 1b978655ccdcbf54e78c8febc2b8808b ./sbin/fsck_msdos 43bc067c65f648041f8ade25ddd077d1 ./sbin/fsdb 8f720b110108c74f55b69935a20adfa6 ./sbin/fsirand d39bf0252bfaad9aa256dbf294ede7da ./sbin/growfs d129af4e9526b87992de226da5f1e184 ./sbin/halt 2d0046c3e383d785b856d1cb0dbe7e5a ./sbin/ifconfig 35e192bac398bf47ddf8e0a190f6b06a ./sbin/init 37d5ca74a94642c48f2278c17420bf76 ./sbin/iopctl 04b18862d04525f6a53324694180592f ./sbin/ipsecctl 0f78f6df80715707bcd0dca44199debe ./sbin/isakmpd 9093d66c257145221ce33f4114ca3507 ./sbin/kbd d0e6b82ecadad09eab297ce032fe1d70 ./sbin/ldattach 04eace371d1dc317b289da273a311c10 221,242c221,242 ./sbin/lmccontrol 2c9a1f7a4cb9af7d9ceaf47d9482eb8b ./sbin/mkfifo 7ebd0d605fb65d8acdce0b1542b7a949 ./sbin/mknod 7ebd0d605fb65d8acdce0b1542b7a949 ./sbin/modload bca677810f776226d24832fa2a118609 ./sbin/modunload 95e4adeda57e7c52f240f136e092eb7b ./sbin/mount 6903ddec325432d73f65c80a56a9aef3 ./sbin/mount_cd9660 cb343f92845ad398d2cc3e4262934030 ./sbin/mount_ext2fs dcb91a3f42126fb96ece261f9a3db010 ./sbin/mount_ffs 6fbd41195622e084f3b9ace630d73d2d ./sbin/mount_mfs c707d5acad7bc11fc5feeb4f4841a1e0 ./sbin/mount_msdos c1d5742cc20e13b2b195f9d8d32c7769 ./sbin/mount_nfs
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Now, with $ diff md5sums_archive md5sums_install | grep ^ | cut -d ' ' -f2 these are the files different on amd64, between what the archive supplied, and what the installer left behind: ./usr/lib/libasn1.so.17.0 ./usr/lib/libcom_err.so.17.0 ./usr/lib/libcrypto.so.18.0 ./usr/lib/libkafs.so.17.0 ./usr/lib/libkrb5.so.17.0 ./usr/lib/libssl.so.15.1 ./sbin/atactl ./sbin/badsect ./sbin/bioctl ./sbin/ccdconfig ./sbin/clri ./sbin/dhclient ./sbin/disklabel ./sbin/dmesg ./sbin/dump ./sbin/dumpfs ./sbin/fdisk ./sbin/fsck ./sbin/fsck_ext2fs ./sbin/fsck_ffs ./sbin/fsck_msdos ./sbin/fsdb ./sbin/fsirand ./sbin/growfs ./sbin/halt ./sbin/ifconfig ./sbin/init ./sbin/iopctl ./sbin/ipsecctl ./sbin/isakmpd ./sbin/kbd ./sbin/ldattach ./sbin/lmccontrol ./sbin/mkfifo ./sbin/mknod ./sbin/modload ./sbin/modunload ./sbin/mount ./sbin/mount_cd9660 ./sbin/mount_ext2fs ./sbin/mount_ffs ./sbin/mount_mfs ./sbin/mount_msdos ./sbin/mount_nfs ./sbin/mount_nnpfs ./sbin/mount_ntfs ./sbin/mount_portal ./sbin/mount_procfs ./sbin/mount_udf ./sbin/mount_vnd ./sbin/mountd ./sbin/ncheck ./sbin/ncheck_ffs ./sbin/newfs ./sbin/newfs_msdos ./sbin/nfsd ./sbin/nologin ./sbin/pfctl ./sbin/pflogd ./sbin/ping ./sbin/ping6 ./sbin/quotacheck ./sbin/raidctl ./sbin/rdump ./sbin/reboot ./sbin/restore ./sbin/route ./sbin/rrestore ./sbin/rtsol ./sbin/savecore ./sbin/scan_ffs ./sbin/scsi ./sbin/shutdown ./sbin/slattach ./sbin/swapctl ./sbin/swapon ./sbin/sysctl ./sbin/ttyflags ./sbin/tunefs ./sbin/umount ./sbin/vnconfig ./sbin/wpa-psk ./sbin/wsconsctl ./usr/libexec/kdc ./usr/sbin/sysctl Let's assume for a moment, that the differences of Kerberos and crypto stuff is a result of the patches and packages, everything else different is the majority of the files in /sbin. A yet closer inspection of the differences there leads to a confirmation of what was assumed before: of all files in /sbin, except of /sbin/ifconfig, /sbin/ipsecctl and /sbin/isakmpd, are the files of the 4.6 Release. Waiting for some further enlightment about what was going on; what happened to those 4.7 files, Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On 06/01/10 23:25, Uwe Dippel wrote: ... There is one more machine (amd64) that needs to be upgraded. Before I do this, I rather solicit suggestions on how to log the upgrade process, debug it, or otherwise. serial console. Log everything from the first chars out the serial port to the reboot. In fact, log the reboot. Don't edit anything. Use a public mirror or an official CD for the install, make sure it is obvious which. Stick the resulting file on a webserver. ... Nick.
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Nick Holland nick at holland-consulting.net writes: There is one more machine (amd64) that needs to be upgraded. Before I do this, I rather solicit suggestions on how to log the upgrade process, debug it, or otherwise. serial console. Log everything from the first chars out the serial port to the reboot. In fact, log the reboot. Don't edit anything. Use a public mirror or an official CD for the install, make sure it is obvious which. Stick the resulting file on a webserver. Thanks, Nick. Based on the latest results, the problem seems to exist only for most of the /sbin files. So, the upgrade runs through as programmed. With a public mirror, it will take hours. I really hope SHA256 is good enough to confirm the integrity of the archives. Serial console seems a good idea; I have to use it in any case. What I have in mind, is, before the reboot, to use the command prompt to check the files in the /sbin-to-be. I have a hunch, that they'll be there, then. Then I'll do the same after the reboot, and once again, after the package upgrade. Should the phenomenon show again, by now I can imagine that the changes are happening some time later. We'll see ... Uwe
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
On Wed, Jun 02, 2010 at 02:42:53PM +, Uwe Dippel wrote: Nick Holland nick at holland-consulting.net writes: There is one more machine (amd64) that needs to be upgraded. Before I do this, I rather solicit suggestions on how to log the upgrade process, debug it, or otherwise. serial console. Log everything from the first chars out the serial port to the reboot. In fact, log the reboot. Don't edit anything. Use a public mirror or an official CD for the install, make sure it is obvious which. Stick the resulting file on a webserver. Thanks, Nick. Based on the latest results, the problem seems to exist only for most of the /sbin files. So, the upgrade runs through as programmed. With a public mirror, it will take hours. I really hope SHA256 is good enough to confirm the integrity of the archives. Serial console seems a good idea; I have to use it in any case. What I have in mind, is, before the reboot, to use the command prompt to check the files in the /sbin-to-be. I have a hunch, that they'll be there, then. Then I'll do the same after the reboot, and once again, after the package upgrade. Should the phenomenon show again, by now I can imagine that the changes are happening some time later. We'll see ... Just for clarity: is everything that fails to change on the same disk? I.e. can you post the output of 'mount' (within bsd.rd) as well? And I presume you shut down in a sensible fashion, right? Joachim
Re: Installer bug? - Upgrade 4.6 to 4.7 failed to upgrade base47, on i386 and amd64
Based on the latest results, the problem seems to exist only for most of the /sbin files. So, the upgrade runs through as programmed. With a public mirror, it will take hours. I really hope SHA256 is good enough to confirm the integrity of the archives. Serial console seems a good idea; I have to use it in any case. What I have in mind, is, before the reboot, to use the command prompt to check the files in the /sbin-to-be. I have a hunch, that they'll be there, then. Then I'll do the same after the reboot, and once again, after the package upgrade. Should the phenomenon show again, by now I can imagine that the changes are happening some time later. We'll see ... Just for clarity: is everything that fails to change on the same disk? I.e. can you post the output of 'mount' (within bsd.rd) as well? And I presume you shut down in a sensible fashion, right? A chit-chat on a public mailing list isn't going to find this supposed bug. Why discuss it? Why not just keep prove it happened. Don't you see how tiring it is to discuss it when we've seen no evidence?