RE: scripting the buildworld/installworld process
I don't think 10 is enough. Wash has a buildworld problem on current and, on his listing, you would only make it about half way through the error 1 messages and may miss the one that means something. I log for a different reason. If you had one die and I didn't, I want to be able to show you that I made it past where yours errored off. Good enough, but otherwise? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: scripting the buildworld/installworld process
Well, it worked! After a fashion... This morning I swapped my small primary HDD and larger secondary HDD, then created the new slices/mount points, and re-installed: Primary Master 8.something G: / 500M Swap256M /var500M /tmp500M /usrremainder over 6G Secondary Slave 2.1G: /usr/obj1000M /disk2 remainder about 1G Drat! I should have put the swap partition on the second HDD! Oh well. Getting the times for the different processes into a log worked well, the entire buildworld/installworld took about 2.5 hours on my Celeron 300 w/ 64M. I did have to re-edit the first script, because it wasn't able to process all the sections that had this: \time -aho $blog make buildworld | tee $tlog | tail -n 50 $tlog $bdir/buildworld.log It would just zoom through the whole script and write empty log files. Finished in about 2 seconds! :-) I left the sections of the script like this: \time -aho $blog make buildworld Once I got everything tweaked, it worked fine! I did have a problem with portupgrade -Na, it finished with the following message: The following packages were not installed or upgraded: ! Cvsup-without-gui-16.1g Unknown build error ! XFree86-4-libraries Unknown build error * mtoolsSkipped Any ideas what may have caused this? I've been trying to get mtools installed, and it always fails. Thanks to all who responded! I appreciate it! Scripts and build.log attached. Run in the following order: Update Update1 Update2 upgrade upgrade Description: Binary data build.log Description: Binary data update Description: Binary data update1 Description: Binary data update2 Description: Binary data ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: scripting the buildworld/installworld process
bw w -j5 893.563u 311.353s 30:31.44 65.7%1302+1591k 53927+138930io 2331pf+0w The wall clock time is really how efficient your process is. The fact that your user time or sys time is faster doesn't mean much if the wall clock time is 1/3 longer. Wall clock? Would that be 30:31.44 from above? I think that once you have it setup, I will probably make my KISS simple script more complicated :). I have one machine that the mouse doesn't Glad to share, if that's what you mean. After re-reading your page about scripting the buildworld, I see an opportunity to test my script. I never thought about putting /usr/src and /usr/obj on different HDD's. I have 2 ATAPI HDD in my box, on different IDE busses, so that should work out well. I'll also need to adjust some slice sizes as well. How much space do you recommend for /var if I'm logging like you do? It's currently: Disk1 - / 100M swap 256M /var 200M /usr remainder about 1.8G Disk2 - /disk2 entire disk 7.8G So I assume /usr/obj should be on the second HDD? About how much space should I reserve? Looks like I should also adjust my script to remove the -j4, eh? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: scripting the buildworld/installworld process
On Sunday 31 August 2003 04:05 pm, Charles Howse wrote: bw w -j5 893.563u 311.353s 30:31.44 65.7%1302+1591k 53927+138930io 2331pf+0w The wall clock time is really how efficient your process is. The fact that your user time or sys time is faster doesn't mean much if the wall clock time is 1/3 longer. Wall clock? Would that be 30:31.44 from above? I think that once you have it setup, I will probably make my KISS simple script more complicated :). I have one machine that the mouse doesn't Glad to share, if that's what you mean. That is what I meant. You learn by trying things. After re-reading your page about scripting the buildworld, I see an opportunity to test my script. I never thought about putting /usr/src and /usr/obj on different HDD's. I have 2 ATAPI HDD in my box, on different IDE busses, so that should work out well. I'll also need to adjust some slice sizes as well. How much space do you recommend for /var if I'm logging like you do? It's currently: Disk1 - / 100M swap 256M /var 200M /usr remainder about 1.8G Disk2 - /disk2 entire disk 7.8G My smallest HD is 13GB and the system had 4 of them. My typical system is on a 3x30GB ATA-100/133 HDs. Since, I have a lot of free space, my partitions are typically 1.5GB. FreeBSD 4.x creates an ../obj of around 300-400MB; however, FreeBSD 5.x produces a 700-800MB. I like to mount a partition as /usr/obj and /usr/src. The builds work just fine with a link to /disk2/obj as /usr/obj. The reason for separate partition is that I have messed up my system to the point that I had to nfs_mount /usr/src and /usr/obj from a different machine. If I mount /disk2/obj as /usr/obj, the make installworld dies because it knows the true path to the obj files. If I redo a system, I create a mount point as /usr/obj. Until then, I just link /usr/obj to /usr2/obj and etc. I think that just moving /usr/obj to your 2nd HD as a link would separate the I/O. For FreeBSD 5.x, you need a lot more space in /. When I installed 5.0-release, I created / as 500MB and it is about 24% used. If you have to manually download some tarballs, you can chew up space in a hurry. The files in /var/log/build can fill up a fs really fast because each system update is around 7-8MB. Since I log everything from cvsup to make index, I fill up space in a hurry :). If something breaks, I want to be able to point to the commit that broke it. So I assume /usr/obj should be on the second HDD? About how much space should I reserve? Looks like I should also adjust my script to remove the -j4, eh? Like you said, play with it. I use write caching and softupdates. They all made it faster. Depending on your system, you may find that -j is faster. From my experience, all of my systems (6 of various speeds and cpus), do a build world faster without a -j. When I was doing benchmarks on a Cray X/MP with 100MB/s 1000MB/s data channels to stripped disks capable of 20MB/s continuous, write caching doubled the throughput of the system. The gain was not obvious up front and everyone was surprised by the result. After we saw it, it was a shoulder shrug because it appeared obvious that you want a process to have faster access to data that it needs than forcing it to save processed data first. In the commercial world, protecting the processed data has a much higher assigned value. I went for speed. If someone said 4-stable was broken, I wanted to be able to test the builds in less than 30 minutes. I also didn't want to spend a lot of money to be able to do that :). BTW, those build times all had setiathome running in the background. Having it running in the background really doesn't change the elapsed time too much exept in 5.x. I kill seti before I do a timed run on it. Kent -- Kent Stewart Richland, WA http://users.owt.com/kstewart/index.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: scripting the buildworld/installworld process
Well, here are my current versions of the update scripts.
I hope they are complicated enough for everyone. grin
Contrary to Kent's suggestion, I'm only logging the last 10 lines of the
important commands. I defer to his experience and knowledge, but I
can't see the value in logging 7500 lines of code that executed
correctly. Let's just get the last 10 lines, since that's where any
errors will be.
Comments, suggestions, flames?
--
#!/usr/local/bin/bash
#
# Update the system, run in multi-user mode.
#
# Checks the exit status of all important commands,
# Exits on exit status of anything other than 0,
# Prints the name of the command that failed.
# Logs the last 10 lines of screen output for important commands.
bailout() {
if [ $? -ne 0 ] ; then
echo Update1 has bailed out on error $?
echo The command that failed was...
echo $cmd
exit
fi
}
echo -n Have you read /usr/src/UPDATING? [y/n]:
read a
if [ $a = n ] ; then
less /usr/src/UPDATING
exit
fi
echo -n Have you merged /etc/group and /usr/src/etc/group? [y/n]:
read b
if [ $b = n ] ; then
diff -c /etc/group /usr/src/etc/group | less
exit
fi
echo -n Have you merged /etc/master.passwd and
/usr/src/etc/master.passwd? [y/n]:
read c
if [ $c = n ] ; then
diff -c /etc/master.passwd /usr/src/etc/master.passwd
exit
fi
cmd=cp /disk2/larry/etc/make.conf /etc
cp /disk2/larry/etc/make.conf /etc
bailout
cmd=cp /disk2/larry/usr/src/sys/i386/conf/CUSTOM
/usr/src/sys/i386/conf
cp /disk2/larry/usr/src/sys/i386/conf/CUSTOM /usr/src/sys/i386/conf
bailout
cmd=Clean out /usr/obj
echo -n Clean out /usr/obj? [y/n]:
read d
if [ $d = y ] ; then
cd /usr/obj
chflags -R noschg *
rm -rf *
fi
bailout
echo -n Continue with build? [y/n]:
read e
if [ $e = n ] ; then
exit
fi
cmd=make buildworld
cd /usr/src
echo Time for buildworld: /var/log/build/build.log
\time -aho /var/log/build/build.log make buildworld 21 |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/buildworld-`date
+%Y%m%d-%H%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
cmd=make buildkernel KERNCONF=GENERIC
echo Time for buildkernel GENERIC: /var/log/build/build.log
\time -aho /var/log/build/build.log make buildkernel KERNCONF=GENERIC
21 |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/buildkernG-`date
+%Y%m%d-%H%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
cmd=make installkernel KERNCONF=GENERIC
echo Time for installkernel GENERIC: /var/log/build/build.log
\time -aho /var/log/build/build.log make installkernel KERNCONF=GENERIC
21 |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/installkernG-`date
+%Y%m%d-%H%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
cmd=make buildkernel KERNCONF=CUSTOM
cp /boot/kernel /boot/kernel.GENERIC
echo Time for buildkernel CUSTOM: /var/log/build/build.log
\time -aho /var/log/build/build.log make buildkernel KERNCONF=CUSTOM
21 |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/buildkernC-`date
+%Y%m%d-%H%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
cmd=make installkernel KERNCONF=CUSTOM
echo Time for installkernel CUSTOM: /var/log/build/build.log
\time -aho /var/log/build/build.log make installkernel KERNCONF=CUSTOM
21 |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/installkernC-`date
+%Y%m%d-%H%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
echo Reboot to single user mode and run /disk2/larry/bin/update2
--
#!/usr/local/bin/bash
#
# Update the system. Run in single user mode.
bailout () {
if [ $? -ne 0 ] ; then
echo Update2 has bailed out on exit status $?
echo The command that failed was...
echo $cmd
exit
fi
}
echo -n Continue with installworld? [y/n]:
read a
if [ $a = n ] ; then
exit
fi
cmd=make installworld
echo -n Time for installworld: /var/log/build/build.log
\time -aho /var/log/build/build.log make installworld |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/instworld-`date
+%Y%m%d-%H:%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
Cmd=cp -Rp /etc /etc.old
if [ -d /etc.old ] ; then
rm -r /etc.old
fi
cp -Rp /etc /etc.old
Bailout
Cmd=rm -r /var/tmp/temproot
if [ -d /var/tmp/temproot ] ; then
cd /var/tmp/temproot
chflags -R noschg *
cd
rm -r /var/tmp/temproot
Fi
bailout
Cmd=mergemaster -a
/usr/sbin/mergemaster -a
bailout
Cmd=./MAKEDEV all
cd /dev
./MAKEDEV all
bailout
cmd=make clean
cd /usr/src/release/sysinstall
echo -n Time for make clean: /var/log/build/build.log
\time -aho /var/log/build/build.log make clean |
tee /var/log/build/tmp.log |
tail /var/log/build/tmp.log /var/log/build/make_clean-`date
+%Y%m%d-%H:%M`.log
bailout
echo /var/log/build/build.log
rm /var/log/build/tmp.log
cmd=make
Re: scripting the buildworld/installworld process
On Monday 01 September 2003 11:13 am, Charles Howse wrote: Well, here are my current versions of the update scripts. I hope they are complicated enough for everyone. grin Contrary to Kent's suggestion, I'm only logging the last 10 lines of the important commands. I defer to his experience and knowledge, but I can't see the value in logging 7500 lines of code that executed correctly. Let's just get the last 10 lines, since that's where any errors will be. Comments, suggestions, flames? I don't think 10 is enough. Wash has a buildworld problem on current and, on his listing, you would only make it about half way through the error 1 messages and may miss the one that means something. I log for a different reason. If you had one die and I didn't, I want to be able to show you that I made it past where yours errored off. -- Kent Stewart Richland, WA http://users.owt.com/kstewart/index.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: scripting the buildworld/installworld process
Most of the time, you only need to see the last 4 or 5. I think that I only look at one of my builds to see the chmod 444 freebsd.cf and fire up the next script. OK, I understand. I think I will go ahead and change tail to tail -n 50. That should provide plenty of feedback, eh? BTW, If you add both of your kernels to /etc/make.conf, you would only need one buildkernel. The first one is the one that is installed. I have it commented now but I used to use #KERNCONF=RUBY GENERIC to build both and install just RUBY. I got so that I liked the logs I thought it would be a good idea to install both GENERIC and CUSTOM kernels, so that I could boot to GENERIC if necessary. So, if I add KERNCONF=GENERIC CUSTOM to make.conf, then the generic kernel is installed as /boot/kernel, and I can cp /boot/kernel /boot/kernel.GENERIC. Then I can do installkernel KERNCONF=CUSTOM, and that will install the custom kernel as /boot/kernel, and that will be the default to boot...? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: scripting the buildworld/installworld process
On Monday 01 September 2003 01:11 pm, Charles Howse wrote: BTW, If you add both of your kernels to /etc/make.conf, you would only need one buildkernel. The first one is the one that is installed. I have it commented now but I used to use #KERNCONF=RUBY GENERIC to build both and install just RUBY. I got so that I liked the logs I thought it would be a good idea to install both GENERIC and CUSTOM kernels, so that I could boot to GENERIC if necessary. So, if I add KERNCONF=GENERIC CUSTOM to make.conf, then the generic kernel is installed as /boot/kernel, and I can cp /boot/kernel /boot/kernel.GENERIC. Then I can do installkernel KERNCONF=CUSTOM, and that will install the custom kernel as /boot/kernel, and that will be the default to boot...? When I do that, I usually install GENERIC first and then install my custom kernel and mv kernel.old to kernel.GENERIC. I haven't thought about it beyond that. You probably also have to chflag it first. FWIW, I rarely use GENERIC except on the first install. I am more likely to boot to kernel.old. If the kernel dies at single user, you want the kernel.old to be a good one and if you have done a couple of kernelinstalls, that won't be true. Kent -- Kent Stewart Richland, WA http://users.owt.com/kstewart/index.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: scripting the buildworld/installworld process
I'll be glad to post the scripts for review if anyone's interested. I'll take a look and test them for you also. OK, any input or corrections welcome. I definitly would be interested in more error checking and reporting, and I'm thinking about using \time to append the time it takes the long utilities to run to a file for comparison purposes. -- This is the first one -- #!/usr/local/bin/bash # # Update the system, run in multi-user mode echo -n Have you read /usr/src/UPDATING? [y/n]: read a if [ $a = n ] ; then less /usr/src/UPDATING exit fi echo -n Have you merged /etc/group and /usr/src/etc/group? [y/n]: read b if [ $b = n ] ; then diff -c /etc/group /usr/src/etc/group | less exit fi echo -n Have you merged /etc/master.passwd and /usr/src/etc/master.passwd? [y/n]: read c if [ $c = n ] ; then diff -c /etc/master.passwd /usr/src/etc/master.passwd exit fi # copy my customized make.conf cp /disk2/larry/etc/make.conf /etc # copy my customized kernel config file cp /disk2/larry/usr/src/sys/i386/conf/CUSTOM /usr/src/sys/i386/conf echo -n Clean out /usr/obj? [y/n]: read d if [ $d = y ] ; then cd /usr/obj chflags -R noschg * rm -rf * fi echo -n Continue with build? [y/n]: read e if [ $e = n ] ; then exit fi cd /usr/src make -j4 buildworld make buildkernel KERNCONF=GENERIC make installkernel KERNCONF=GENERIC cp /boot/kernel /boot/kernel.GENERIC make buildkernel KERNCONF=CUSTOM make installkernel KERNCONF=CUSTOM echo Reboot to single user mode and run /disk2/larry/bin/update2 Exit -- This is the second one -- #!/usr/local/bin/bash # # Update the system, run in single user mode echo -n Continue with installworld? [y/n]: read a if [ $a = n ] ; then exit fi make installworld if [ -d /etc.old ] ; then rm -r /etc.old fi cp -Rp /etc /etc.old /usr/sbin/mergemaster -a cd /dev ./MAKEDEV all cd /usr/src/release/sysinstall make clean make all install echo Finished! Do ps and top work? Exit 2. After reading man mergemaster, I see that the -a flag will run mergemaster automatically and leave any new files in /var/tmp/temproot. I'd like to confirm that if I opt to run mergemaster -a, then I should check/merge any files before leaving single user mode. It seems like the right thing to do...? Correct, but I'll advise you run mergemaster manually. Margemaster is such a dangerous thing to run with -a on a system that has had so many mods to /etc When I upgraded from 4.8-p3 to p4 recently, I ran mergemaster manually. The only file I had to make a decision on was (I forget the name) the one that prints the uname information when booting. Just for the sake of discussion, let's say I did opt to run mergemaster -a. All I would have to do is check every file in /var/tmp/temproot and be sure it didn't break anything by overwriting the older version, correct? ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: scripting the buildworld/installworld process
On Sunday 31 August 2003 01:25 pm, Charles Howse wrote: I'll be glad to post the scripts for review if anyone's interested. I'll take a look and test them for you also. OK, any input or corrections welcome. I definitly would be interested in more error checking and reporting, and I'm thinking about using \time to append the time it takes the long utilities to run to a file for comparison purposes. -- This is the first one -- #!/usr/local/bin/bash # # Update the system, run in multi-user mode echo -n Have you read /usr/src/UPDATING? [y/n]: read a if [ $a = n ] ; then less /usr/src/UPDATING exit fi echo -n Have you merged /etc/group and /usr/src/etc/group? [y/n]: read b if [ $b = n ] ; then diff -c /etc/group /usr/src/etc/group | less exit fi echo -n Have you merged /etc/master.passwd and /usr/src/etc/master.passwd? [y/n]: read c if [ $c = n ] ; then diff -c /etc/master.passwd /usr/src/etc/master.passwd exit fi # copy my customized make.conf cp /disk2/larry/etc/make.conf /etc # copy my customized kernel config file cp /disk2/larry/usr/src/sys/i386/conf/CUSTOM /usr/src/sys/i386/conf echo -n Clean out /usr/obj? [y/n]: read d if [ $d = y ] ; then cd /usr/obj chflags -R noschg * rm -rf * fi echo -n Continue with build? [y/n]: read e if [ $e = n ] ; then exit fi cd /usr/src make -j4 buildworld make buildkernel KERNCONF=GENERIC make installkernel KERNCONF=GENERIC cp /boot/kernel /boot/kernel.GENERIC make buildkernel KERNCONF=CUSTOM make installkernel KERNCONF=CUSTOM echo Reboot to single user mode and run /disk2/larry/bin/update2 Exit -- This is the second one -- #!/usr/local/bin/bash # # Update the system, run in single user mode echo -n Continue with installworld? [y/n]: read a if [ $a = n ] ; then exit fi make installworld if [ -d /etc.old ] ; then rm -r /etc.old fi cp -Rp /etc /etc.old /usr/sbin/mergemaster -a cd /dev ./MAKEDEV all cd /usr/src/release/sysinstall make clean make all install echo Finished! Do ps and top work? Exit 2. After reading man mergemaster, I see that the -a flag will run mergemaster automatically and leave any new files in /var/tmp/temproot. I'd like to confirm that if I opt to run mergemaster -a, then I should check/merge any files before leaving single user mode. It seems like the right thing to do...? Correct, but I'll advise you run mergemaster manually. Margemaster is such a dangerous thing to run with -a on a system that has had so many mods to /etc When I upgraded from 4.8-p3 to p4 recently, I ran mergemaster manually. The only file I had to make a decision on was (I forget the name) the one that prints the uname information when booting. Just for the sake of discussion, let's say I did opt to run mergemaster -a. All I would have to do is check every file in /var/tmp/temproot and be sure it didn't break anything by overwriting the older version, correct? One area I would check is /etc/master.passwd and groups. It would be really interesting for you if your user accounts all disappeared. I also won't let mergemaster touch my hosts and printcap files. I would also go to the trouble of tee'ing the makes and create logs of the makes and installs. Since you are going on, you want to know what you have done. What I typically do is make buildworld 21 | tee /var/log/build/bworld-`date +%Y%m%d-%H%M`.log Sorry about the wrap but I have kmail set at col 72. I have relatively large /var and /tmp and just added ../build to /var/log. The date required minutes in order to be unique. I also log all of my cvsups. Kent -- Kent Stewart Richland, WA http://users.owt.com/kstewart/index.html ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: scripting the buildworld/installworld process
I'll be glad to post the scripts for review if anyone's interested. I'll take a look and test them for you also. OK, any input or corrections welcome. I definitly would be interested in more error checking and reporting, and I'm thinking about using \time to append the time it takes the long utilities to run to a file for comparison purposes. -- This is the first one -- #!/usr/local/bin/bash # # Update the system, run in multi-user mode echo -n Have you read /usr/src/UPDATING? [y/n]: read a if [ $a = n ] ; then less /usr/src/UPDATING exit fi echo -n Have you merged /etc/group and /usr/src/etc/group? [y/n]: read b if [ $b = n ] ; then diff -c /etc/group /usr/src/etc/group | less exit fi echo -n Have you merged /etc/master.passwd and /usr/src/etc/master.passwd? [y/n]: read c if [ $c = n ] ; then diff -c /etc/master.passwd /usr/src/etc/master.passwd exit fi # copy my customized make.conf cp /disk2/larry/etc/make.conf /etc # copy my customized kernel config file cp /disk2/larry/usr/src/sys/i386/conf/CUSTOM /usr/src/sys/i386/conf echo -n Clean out /usr/obj? [y/n]: read d if [ $d = y ] ; then cd /usr/obj chflags -R noschg * rm -rf * fi echo -n Continue with build? [y/n]: read e if [ $e = n ] ; then exit fi cd /usr/src make -j4 buildworld make buildkernel KERNCONF=GENERIC make installkernel KERNCONF=GENERIC cp /boot/kernel /boot/kernel.GENERIC make buildkernel KERNCONF=CUSTOM make installkernel KERNCONF=CUSTOM echo Reboot to single user mode and run /disk2/larry/bin/update2 Exit -- This is the second one -- #!/usr/local/bin/bash # # Update the system, run in single user mode echo -n Continue with installworld? [y/n]: read a if [ $a = n ] ; then exit fi make installworld if [ -d /etc.old ] ; then rm -r /etc.old fi cp -Rp /etc /etc.old /usr/sbin/mergemaster -a cd /dev ./MAKEDEV all cd /usr/src/release/sysinstall make clean make all install echo Finished! Do ps and top work? Exit 2. After reading man mergemaster, I see that the -a flag will run mergemaster automatically and leave any new files in /var/tmp/temproot. I'd like to confirm that if I opt to run mergemaster -a, then I should check/merge any files before leaving single user mode. It seems like the right thing to do...? Correct, but I'll advise you run mergemaster manually. Margemaster is such a dangerous thing to run with -a on a system that has had so many mods to /etc When I upgraded from 4.8-p3 to p4 recently, I ran mergemaster manually. The only file I had to make a decision on was (I forget the name) the one that prints the uname information when booting. Just for the sake of discussion, let's say I did opt to run mergemaster -a. All I would have to do is check every file in /var/tmp/temproot and be sure it didn't break anything by overwriting the older version, correct? One area I would check is /etc/master.passwd and groups. It would be really interesting for you if your user accounts all disappeared. I also won't let mergemaster touch my hosts and printcap files. I'm not sure exactly what you mean. I'm asking whether or not to diff the master.passwd and group files ( see above ). If I run mergemaster -a, my understanding is that it doesn't make any changes, it just leaves the new files in /var/tmp/temproot to be dealt with later, so it wouldn't touch my hosts and other important custom files, right? I would also go to the trouble of tee'ing the makes and create logs of the makes and installs. Since you are going on, you want to know what you have done. What I typically do is make buildworld 21 | tee /var/log/build/bworld-`date +%Y%m%d-%H%M`.log Excellent idea, if I was tight on disk space, I could even gzip them. required minutes in order to be unique. I also log all of my cvsups. Another great idea! Thanks! I've just been working on timing the different processes, here's a snippet... Printf Time for buildworld:\t /root/build.log \time -aho /root/build.log make buildworld Printf \n /root/build.log BTW, which field do I want from the output of time? Real, user, whatever? I'll also need to do some cut'ing in order to just get the proper field from time...or I could just let it rip...? For instance... Time for buildworld: 2:34:15 real 2:35:12 useretc... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: scripting the buildworld/installworld process
On Sunday 31 August 2003 02:19 pm, Charles Howse wrote: I'll be glad to post the scripts for review if anyone's interested. I'll take a look and test them for you also. OK, any input or corrections welcome. I definitly would be interested in more error checking and reporting, and I'm thinking about using \time to append the time it takes the long utilities to run to a file for comparison purposes. -- This is the first one -- #!/usr/local/bin/bash # # Update the system, run in multi-user mode echo -n Have you read /usr/src/UPDATING? [y/n]: read a if [ $a = n ] ; then less /usr/src/UPDATING exit fi echo -n Have you merged /etc/group and /usr/src/etc/group? [y/n]: read b if [ $b = n ] ; then diff -c /etc/group /usr/src/etc/group | less exit fi echo -n Have you merged /etc/master.passwd and /usr/src/etc/master.passwd? [y/n]: read c if [ $c = n ] ; then diff -c /etc/master.passwd /usr/src/etc/master.passwd exit fi # copy my customized make.conf cp /disk2/larry/etc/make.conf /etc # copy my customized kernel config file cp /disk2/larry/usr/src/sys/i386/conf/CUSTOM /usr/src/sys/i386/conf echo -n Clean out /usr/obj? [y/n]: read d if [ $d = y ] ; then cd /usr/obj chflags -R noschg * rm -rf * fi echo -n Continue with build? [y/n]: read e if [ $e = n ] ; then exit fi cd /usr/src make -j4 buildworld make buildkernel KERNCONF=GENERIC make installkernel KERNCONF=GENERIC cp /boot/kernel /boot/kernel.GENERIC make buildkernel KERNCONF=CUSTOM make installkernel KERNCONF=CUSTOM echo Reboot to single user mode and run /disk2/larry/bin/update2 Exit -- This is the second one -- #!/usr/local/bin/bash # # Update the system, run in single user mode echo -n Continue with installworld? [y/n]: read a if [ $a = n ] ; then exit fi make installworld if [ -d /etc.old ] ; then rm -r /etc.old fi cp -Rp /etc /etc.old /usr/sbin/mergemaster -a cd /dev ./MAKEDEV all cd /usr/src/release/sysinstall make clean make all install echo Finished! Do ps and top work? Exit 2. After reading man mergemaster, I see that the -a flag will run mergemaster automatically and leave any new files in /var/tmp/temproot. I'd like to confirm that if I opt to run mergemaster -a, then I should check/merge any files before leaving single user mode. It seems like the right thing to do...? Correct, but I'll advise you run mergemaster manually. Margemaster is such a dangerous thing to run with -a on a system that has had so many mods to /etc When I upgraded from 4.8-p3 to p4 recently, I ran mergemaster manually. The only file I had to make a decision on was (I forget the name) the one that prints the uname information when booting. Just for the sake of discussion, let's say I did opt to run mergemaster -a. All I would have to do is check every file in /var/tmp/temproot and be sure it didn't break anything by overwriting the older version, correct? One area I would check is /etc/master.passwd and groups. It would be really interesting for you if your user accounts all disappeared. I also won't let mergemaster touch my hosts and printcap files. I'm not sure exactly what you mean. I'm asking whether or not to diff the master.passwd and group files ( see above ). If I run mergemaster -a, my understanding is that it doesn't make any changes, it just leaves the new files in /var/tmp/temproot to be dealt with later, so it wouldn't touch my hosts and other important custom files, right? Your diff would have caught it. I scrolled past it too fast and zeroed in on other things :). I would also go to the trouble of tee'ing the makes and create logs of the makes and installs. Since you are going on, you want to know what you have done. What I typically do is make buildworld 21 | tee /var/log/build/bworld-`date +%Y%m%d-%H%M`.log Excellent idea, if I was tight on disk space, I could even gzip them. required minutes in order to be unique. I also log all of my cvsups. Another great idea! Thanks! I've just been working on timing the different processes, here's a snippet... Printf Time for buildworld:\t /root/build.log \time -aho /root/build.log make buildworld Printf \n /root/build.log BTW, which field do I want from the output of time? Real, user, whatever? I'll also need to do some cut'ing in order to just get the proper field from time...or I could just let it rip...? For instance... Time for buildworld: 2:34:15 real2:35:12 useretc... When I started timing my buildworlds, I figured out that you need everything but the right hand stuff in the following AMD Athlon 2000+
RE: scripting the buildworld/installworld process
When I started timing my buildworlds, I figured out that you need everything but the right hand stuff in the following AMD Athlon 2000+ XP 877.636u 233.835s 23:02.33 80.4%1350+1662k 46008+7469io 2359pf+0w bw w -j2 891.151u 303.327s 35:02.77 56.8%1305+1600k 52256+138679io 2749pf+0w bw w -j3 894.429u 305.373s 31:29.87 63.4%1305+1595k 50306+138970io 1914pf+0w bw w -j4 893.259u 310.180s 32:43.57 61.2%1308+1597k 56102+138705io 2145pf+0w bw w -j5 893.563u 311.353s 30:31.44 65.7%1302+1591k 53927+138930io 2331pf+0w The wall clock time is really how efficient your process is. The fact that your user time or sys time is faster doesn't mean much if the wall clock time is 1/3 longer. I haven't automated timing this part because I time the whole script and not just the build. I would cut off the right hand side but it is too trivial to use just time mkworld and cut and paste the time into a cat times.log of my builds. This also lets me add comments in addition to the build time information. For example, the AMD 2000+ was important because that was the first build after an upgrade from an Intel P III 866. The upgrade was needed because the old mobo went flaky. I think that once you have it setup, I will probably make my KISS simple script more complicated :). I have one machine that the mouse doesn't work in single user mode and including the timing into my mkworld script would save time. I can always go back and edit my time log to include the comments. If I have a choice, I would add the date field to the right hand side so that I can connect the time to the build log. I read briefly your web page about the -j urban legend. From the stats above, it looks like it runs faster without the -j flag...? That might be something interesting to play with on my Celeron 300 w/ 64MB. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
