TLDR; : logging what gets installed or updated can be painful
Well, this has been a fun few days. Last week I was working on some
random perl module update in BLFS, and discovered that my logs of
what got installed or modified were not listing the module itself.
I've been setting a marker when I start to build a package, and then
finding out which installed files were newer than that.
Unfortunately, for any package which ships a script (typically, perl
modules, but might also apply in other cases), the script gets
installed as-shipped, so it is dated in the past.
So, I decided to rework my scripts - they now seem to be sorted (but
still need to be tested in a fresh build to check what I do in
chapter 5 of LFS), but it was doubleplus ungood getting from there
to here! (acknowledgements to the late Eric Arthur Blair).
The first problem was that I now need to not only find what files
exist, but to run ls -l on them (files - for directories, find is
adequate). With my latest system, that leads to a problem - cmake
installs a few files with spaces in their names (I knew that cmake
can be unpleasant, but really!), In the end, I've used (another)
sed here, so that Silly Name.ext becomes Silly\ Name.ext before it
gets pushed to ls -l : sed 's/\x20/\\\ /g' Plus, I then wanted to
put the filename at the front of the ls -l output, to merge in with
the directories - tried awk, but there were too many complications,
tried cut, then realised that once again sed was my friend :
sed 's%\(.*\) /\(.*\)%/\2 \1%'
And then I found that, at least with current coreutils, the
whitespace in the ls -l output seemed to be random (sometimes more,
sometimes less), so, you guessed : sed 's/ \+/ /g' when I was piping
the files to sort.
Seems to work, so my logging must now be 110% correct ;-)
Unfortunately, on what I regard as a completed desktop system (more
than 330,000 files or directories in '/', excluding those on
separate fielsystems such as /dev, /sys, /tmp, /home) this now takes
about 18 seconds to run on a reasonably-fast machine. Oh well,
what's a few hours when building a new system ( over 600 steps on
this particular desktop - I separate things like adding users/groups,
and a separate step per package, even for things like the audacious
plugins and poppler data, or for the individual Xorg packages).
I then tried to set up similar logging for those packages which I
build and install manually (mostly, these days, Python things which
use odd methods, where I'm unclear if I can do a DESTDIR or
equivalent). And yet again, I had cause to regret using 'set -e'
(in theory, I do that in my "driver" scripts which run the "client"
scripts to build packages or add a user or group, but I'm not
exactly convinced that everything is using it, perhaps because I
load the functions as son as possible in the build and only turn on
set -e later. [ So, posting CAN help me understand my problems :-) ]
Anyway, for a normal build the diff of the before and after logs was
ok, after I had sorted out exactly what to do (trad diff: 'c'
records to change 1 or many lines [1], deletions[2], '---'[3],
additions [4] ; other a or d headers [5]; additions [6] or deletions
[7] - processes listed below.
In the end I copied the functions into separate log-before and
log-after scripts (because log-after was stopping on non-zero status
from diff, and also in some nested if statements, so I reworked them
to a much-slower process) and these work - slowly, perhaps 20
seconds to do the comparison on a full system.
But I'm still baffled by some of the failures I had when reworking
them - instead of nested if tests, I tried to clear a variable, test
the input line and set the variable if the test was true [ OK ],
run another test if the variable was not set, i.e. -z [ skipped ],
yet another test [ also skipped, but echoing "+$VAR+" reported '++'
as if not set ], then a last test if the variable was not set [ that
ran ]. This sounds suspiciously like problems I saw when I first
started to script in bash, but it's probalby something silly that
I'm doing. But I've worked around it, so I'm happy(-ish) and can go
back to tagging things for gcc5.
Thanks for listening!
Process status on the diff, short version
1. set up 'in change'
2. log the name to 'modified' (if in change)
3. set up 'ignore'
4. ignore them!
5. set up 'other'
6. log the name to 'installed'
7. log the name to 'deleted' - hopefully, something extraneous to
the build, such as log rotation.
For those not familiar with 'trad' diff, it uses control records
(number{,number} operation number{,number} - where operation is a,c,
or d; '< ' (deleted - for a change, delete the old line), '---' (in a
change, this separates deleted lines from added lines), '> ' added
line (or new version in a change). It's fairly easy to see why diff
-u is better for most normal purposes.
ĸen
--
Nanny Ogg usually went to bed early. After all, she was an old lady.
Sometimes she went to bed as early as 6 a.m.
--
http://lists.linuxfromscratch.org/listinfo/lfs-chat
FAQ: http://www.linuxfromscratch.org/blfs/faq.html
Unsubscribe: See the above information page
