Re: uname weirdness after kernel/OS update
The following is my most recent email message to someone who was helping me with a very odd uname issue. I hope that this reporting of the final events (oh-god-pleaselet-this-be-done-and-over-with) helps someone else some day. The offer that I make at the end of my message is genuine. If a FreeBSD expert (Greg? *nudge*) wants the /boot files, they can have them. Jaime -- Forwarded message -- Date: Mon, 29 Dec 2003 15:05:07 -0500 (EST) From: [EMAIL PROTECTED] To: T Kellers [EMAIL PROTECTED] Subject: Re: compiled kernel file After lots of various ideas, including kernels compiled on different boxes (e.g. the one that you sent) nothing seemed to work. Then, I noticed that not everything in / was being listed when I typed ls at the boot manager. This is when I started getting creative. I used sysinstall's disk slice editor to put a new MBR onto the drive and removed /boot. The next attempt to boot refused to mount any of my SCSI drives and it showed a few files in / that were different than they should be. For example, /proc was missing, /homes (an older attempt to make home directories exist on /homes/students and /homes/staff left this directory behind) was back -- even though I thought that I removed it -- and /home was gone, and the most recent etc-*.tar.gz backup of /etc (which I made before the 12/23/03 cvsup) was missing. It was as if I suddenly took a trip backwards in time for this partition by at least a few months. My best guess is that someone had hidden the real / partition and put their own partition (or disk image?) in its place, using a compromised boot loader. This would explain why using ls at the boot loader produced a different list of files than ls at the single-user shell showed. It also explains why new kernels wouldn't load, making uname give bad results on a new kernel. It was reporting data about the kernel that the cracker had given it! I again removed /boot, /usr/src, and /usr/obj, just in case these were violated, too. I did a new cvsup, make buildworld, make buildkernel, make installkernel, and rebooted into single user mode. The / partition was the way I had left it, not the way it was when the symptoms were noticed. So I kept going and did a make installworld and a mergemaster and then rebooted again. Everything seems to be working well now. uname now says: zeus:jkikpoleuname -a FreeBSD zeus.cairodurham.org 4.9-STABLE FreeBSD 4.9-STABLE #0: Mon Dec 29 13:46:57 EST 2003 root@:/usr/obj/usr/src/sys/ZEUS i386 I have changed my root password a few weeks ago. I just removed the toor password (in vipw, I replaced the cypher with a *). My next step is to change the password of any account in the wheel group. I honestly think that someone had broken into this box and made some really creative cracks. I'm not sure about back doors at this point. Using chkrootkit doesn't show anything out of place. (An occasional possible LKM trojan report, but its not consistent and various people claim that apache can cause false positives on that test.) If ANY of the above rings some bells for you, please let me know. Any advice on securing this box would be appreciated, too. Unfortunately, formatting the drive and reinstalling the OS is not an option at this time. :( Feel free to pass this report along to FreeBSD report along to any FreeBSD power-user that can make the OS better by reading this. I'd be happy to provide assorted files off the system (including any of the /boots that I still have) if they will help. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
On Fri, 26 Dec 2003, Kent Stewart wrote: On Friday 26 December 2003 11:05 am, Jaime wrote: Are you sure that you are building and installing a kernel. That would be about the only thing that wouldn't update your boot message. I am completely certain. I've used make buildkernel KERNCONF=... and make installkernel KERNCONF=... as well as the older /usr/sbin/config method. An ls -l / shows a newer time stamp. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
On Fri, Dec 26, 2003 at 02:22:27PM -0500, [EMAIL PROTECTED] wrote: On Fri, 26 Dec 2003, Kent Stewart wrote: On Friday 26 December 2003 11:05 am, Jaime wrote: Are you sure that you are building and installing a kernel. That would be about the only thing that wouldn't update your boot message. I am completely certain. I've used make buildkernel KERNCONF=... and make installkernel KERNCONF=... as well as the older /usr/sbin/config method. An ls -l / shows a newer time stamp. And you did reboot as well, so as to actually use the new kernel? (Just asking since you didn't say explicitly that you had done that.) -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
On Fri, 26 Dec 2003, Erik Trulsson wrote: And you did reboot as well, so as to actually use the new kernel? Yes. (Just asking since you didn't say explicitly that you had done that.) Fair enough. We all would have felt pretty dumb if it was something that obvious and yet we didn't check. :) FWIW, I've been using the make-world process since 1997. The only other time that I've ever had a problem (including several years of updating the box in question) was when I had bad hardware. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
Did you do a make kernel KERNCONF=YOURKERNELFILE, too? I'm only asking because you mentioned make world, and while that rebuilds the OS, it doesn't make (or install) the kernel. I have to ask simple questions; the problem, if not simple, is flat-out weird. Tim Kellers CPE/NJIT On Friday 26 December 2003 02:40 pm, Jaime wrote: On Fri, 26 Dec 2003, Erik Trulsson wrote: And you did reboot as well, so as to actually use the new kernel? Yes. (Just asking since you didn't say explicitly that you had done that.) Fair enough. We all would have felt pretty dumb if it was something that obvious and yet we didn't check. :) FWIW, I've been using the make-world process since 1997. The only other time that I've ever had a problem (including several years of updating the box in question) was when I had bad hardware. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
On Fri, 26 Dec 2003, T Kellers wrote: Did you do a make kernel KERNCONF=YOURKERNELFILE, too? Yes. I followed the directions in the /usr/src/UPDATING file that I have followed at least 8 times previously and successfully on this very same server over the last few years. cvsup -g -L 2 stable-supfile (after editing) cd /usr/src make buildworld make buildkernel KERNCONF=ZEUS make installkernel KERNCONF=ZEUS reboot (single user) make installworld mergemaster reboot I have tried simple kernel recompiles since then. I am currently in the process of recompiling the entire OS via a third instance of the above procedure. I have to ask simple questions; the problem, if not simple, is flat-out weird. I understand. Its just frustrating. Let's start from the other end, though. From where does uname draw its data? With that information, I might be able to track down the problem. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
On Fri, 26 Dec 2003 15:11:20 -0500 (EST) [EMAIL PROTECTED] wrote: On Fri, 26 Dec 2003, T Kellers wrote: Did you do a make kernel KERNCONF=YOURKERNELFILE, too? Yes. I followed the directions in the /usr/src/UPDATING file that I have followed at least 8 times previously and successfully on this very same server over the last few years. cvsup -g -L 2 stable-supfile (after editing) cd /usr/src make buildworld make buildkernel KERNCONF=ZEUS make installkernel KERNCONF=ZEUS reboot (single user) make installworld mergemaster reboot I have tried simple kernel recompiles since then. I am currently in the process of recompiling the entire OS via a third instance of the above procedure. I have to ask simple questions; the problem, if not simple, is flat-out weird. I understand. Its just frustrating. Let's start from the other end, though. From where does uname draw its data? By interogating sysctl's mibs. See uname(3). With that information, I might be able to track down the problem. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] -- IOnut Unregistered ;) FreeBSD user ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: uname weirdness after kernel/OS update
On Fri, 26 Dec 2003, Tim Kellers wrote: I'm building world/kernel on a spare box right now to see if it shakes an idea or two loose. I'm also wondering if your /usr/src files are actually building a new world, too. Trying to think of what might break if you are running a STABLE world with a pre-release kernel. Top is the classic utility that breaks when your world and kernel don't match, but I'm not sure if that will happen if you don't cross version boundaries. Well, I've compiled with 4.9-RELEASE binaries and 4.9-STABLE (12/24/03) binaries and had no observable effects. Likewise, the symptoms were first noticed when the world and kernel were the same. They have repeted themselves within every combination that I've tried. FWIW, I tried rm -rf /usr/obj and recompile, but that didn't improve things. I also tried mv /usr/src /usr/src.old and then re-cvsup and recompile. That didn't help, either. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: uname weirdness after kernel/OS update
Try a rm -rf /usr/src/* and then rebuild using the config method from /usr/src/sys/i386/conf with make depend; make; make install after configuring. HTH Eric F Crist President AdTech Integrated Systems, Inc (612) 998-3588 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, December 26, 2003 1:22 PM To: Kent Stewart Cc: [EMAIL PROTECTED] Subject: Re: uname weirdness after kernel/OS update On Fri, 26 Dec 2003, Kent Stewart wrote: On Friday 26 December 2003 11:05 am, Jaime wrote: Are you sure that you are building and installing a kernel. That would be about the only thing that wouldn't update your boot message. I am completely certain. I've used make buildkernel KERNCONF=... and make installkernel KERNCONF=... as well as the older /usr/sbin/config method. An ls -l / shows a newer time stamp. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: uname weirdness after kernel/OS update
I forgot, you'll need to re-cvsup after you delete your src directory contents. :-O Eric F Crist President AdTech Integrated Systems, Inc (612) 998-3588 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Eric F Crist Sent: Friday, December 26, 2003 2:48 PM To: [EMAIL PROTECTED]; 'Kent Stewart' Cc: [EMAIL PROTECTED] Subject: RE: uname weirdness after kernel/OS update Try a rm -rf /usr/src/* and then rebuild using the config method from /usr/src/sys/i386/conf with make depend; make; make install after configuring. HTH Eric F Crist President AdTech Integrated Systems, Inc (612) 998-3588 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, December 26, 2003 1:22 PM To: Kent Stewart Cc: [EMAIL PROTECTED] Subject: Re: uname weirdness after kernel/OS update On Fri, 26 Dec 2003, Kent Stewart wrote: On Friday 26 December 2003 11:05 am, Jaime wrote: Are you sure that you are building and installing a kernel. That would be about the only thing that wouldn't update your boot message. I am completely certain. I've used make buildkernel KERNCONF=... and make installkernel KERNCONF=... as well as the older /usr/sbin/config method. An ls -l / shows a newer time stamp. Jaime ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
