Am Fri, Dec 09, 2022 at 01:13:50PM +0000 schrieb Michael: > > > I get the impression Dale isn't actually PLANNING his disk storage. It's > > > just a case of "help I'm downloading all this stuff where do I put it!!!" > > > > Haha, thanks for the laugh. > > Actually this had me thinking what is the need to back up the ... Internet? > […] > > I appreciate some of these video files may be rare finds, or there may be a > risk some of these may be taken off the interwebs sooner or later. This > should leave a rather small subset of all downloads, which may merit a local > backup, just in case. I'd thought the availability of higher fiber download > speeds negates the need for local backups, of readily downloadable media.
Good points. I am a big fan of having stuff locally as well, because I don’t want to be dependent on a company’s servers and a working Internet connection. But this mostly applies to my mobile device, because I don’t have a data plan for mobile Internet. > > Well, ts uses mpeg2 encoding, just like old video DVDs, which is very > > inefficient when compared with modern h264/h265. Modern digital TV broadcast > > uses h264 by now. > > Depending on the PVR make/model I've seen 1080p resolution recordings with > .m2ts and .ts file extensions, while the codecs inside them are the same. I wasn’t aware that ts could contain h264. But then again—I never really bothered with live TV recordings in recent years. These days, if I find something interesting, I download the show form the TV channel’s website (called Mediathek in Germany, a word play on Bibliothek, meaning library). Interestingly though, the picture quality is noticably worse than what I receive via DVB-T. > > ¹ I do have several external USB disks, plus the big NAS. All of which don’t > > run very often. And I don’t want to turn them on just to look for a certain > > file. That’s why I have another little script. ;-) It uses the `tree` > > command to save the complete content listing of a directory into a text > > file and names the file automatically by the name of the directory it > > crawls. So if I want to find a file, I just need to grep through my text > > files. > > Backup scripts utilising rsync, tar, etc. can output a log file which contains > (some) details of all the backed up files. Nothing as sophisticated as > Frank's script, but it allows for a quick search against the name of the file > or directory, before extraction. Naturally, I just discovered two bugs in the script while I was re-reading my mail. One of them broke the creation of the symlink which points to the most recent version of a script output. The other prevented normal operation if only gzip was available amongst the used compressors. -- Grüße | Greetings | Salut | Qapla’ Please do not share anything from, with or about me on any social network. We promise nothing, but that we keep.
#!/usr/bin/env bash
# history
# 2018-02-10 initial version
# 2020-06-10 added -a option
# ????-??-?? gzip result
# 2021-03-22 put current date into output filename
# 2021-04-06 code refactoring with more functions and input sanitisation
# 2021-12-22 prefer zstd over gzip
# 2022-01-23 don't create symlink if there is no previous file
# 2022-01-30 added xz to compressors
# 2022-12-08 some cleanup, added -C and -K options
# 2022-12-09 bug fixes
die() {
echo "${@}" > /dev/stderr
exit 1
}
usage() {
cat <<-EOF
Usage: $(basename "$0") [-o NAME] [DIR]
A wrapper to tree, it writes the content of DIR into a text file.
The file is named after DIR and the current date, and symlink to the
most recent version is set. The file is automatically compressed to
zstd, xz or gzip, whichever is available in that order.
Options:
-a access attributes: owner, group, permissions
-C do not compress result file
-K do not keep backup of existing file in case of overwriting
-o The destination where to write the trees.
Default: .
If it is a directory: write in there
If it is a filename: use that as base. If not,
use the name of the directory as name base.
EOF
}
test_writable() {
touch "$1" 2>/dev/null || die "Cannot create file in $(dirname "$1")."
}
# run tree and redirect output to destination file
call_to_tree() {
WHICH="$1"
OUTPATH="$2-$1"
shift 2
declare TREE_ARGS
TREE_ARGS+=("$@")
TREE_ARGS+=("-o")
TREE_ARGS+=("$DATED_PATH")
local DATED_PATH="$OUTPATH-$TODAY"
local EXT
local PACK
local CREATE_SYMLINK=no
if [ "$COMPRESS" = "no" ]; then
0
elif command -v zstd > /dev/null; then
PACK="zstd --rm -q -13"
EXT=".zst"
elif command -v xz > /dev/null; then
PACK="xz"
EXT=".xz"
else
PACK="gzip"
EXT=".gz"
fi
ls "$OUTPATH-"* > /dev/null 2>&1 && CREATE_SYMLINK=yes
# pack yet unpacked file
if [ "$COMPRESS" != "no" ]; then
[ -f "$DATED_PATH" ] && [ ! -f "$DATED_PATH$EXT" ] && $PACK
"$DATED_PATH"
fi
# move away old file
if [ -f "$DATED_PATH$EXT" ]; then
[ "$KEEP_OLD" = "no" ] && rm -f "$DATED_PATH$EXT" || mv -f
"$DATED_PATH$EXT" "$DATED_PATH.old$EXT"
fi
test_writable "$DATED_PATH"
echo "Writing $WHICH tree to $(realpath "$DATED_PATH")$EXT"
BEGIN_TIME=$(date +%s)
"${TREE_ARGS[@]}"
END_TIME=$(date +%s)
echo >> "$DATED_PATH"
df -BM . | awk 'NR==2 { print "Size: " $3 " Available: " $4 " Usage: "
$5; }' >> "$DATED_PATH"
[ "$PACK" ] && $PACK "$DATED_PATH"
if [ "$CREATE_SYMLINK" = yes ]; then
if [ -e "$OUTPATH" ] && [ ! -L "$OUTPATH" ]; then
echo "Cannot set symlink, filename already taken."
else
echo "Setting symlink $OUTPATH -> $(basename "$DATED_PATH")"
ln -sfn "$(basename "$DATED_PATH$EXT")" "$OUTPATH$EXT"
fi
fi
echo "Reading of directory took $((END_TIME - BEGIN_TIME)) seconds."
}
# parse arguments
unset ACCESSRIGHTS
unset COMPRESS
unset KEEP_OLD
while getopts "CKaho:" OPTION
do
case $OPTION in
C) COMPRESS=no ;;
K) KEEP_OLD=no ;;
a) ACCESSRIGHTS=yes ;;
h) usage; exit 0 ;;
o) OUTARG="$OPTARG" ;;
esac
done
shift $((OPTIND-1)); OPTIND=1
# input sanitisation and derivation of destination paths
DIR="${1:-$(pwd)}"
[ -d "$DIR" ] || die "The given directory does not exist."
[ "$OUTARG" ] && OUTARG="$(realpath "$OUTARG")" || OUTARG="$(pwd)"
if [ -d "$OUTARG" ]; then
OUTDIR="$OUTARG"
OUTNAME="$(basename "$(realpath "$DIR")")" # realpath to catch `videotree .`
[ "$OUTNAME" = "/" ] && OUTNAME=root
else
OUTDIR="$(dirname "$OUTARG")"
OUTNAME="$(basename "$OUTARG")"
fi
TODAY="$(date +%F)"
cd "$DIR"
# write a very verbose file with permission and owner information
call_to_tree "detailed" "$OUTDIR/$OUTNAME" tree -af -DFins --dirsfirst
--du --timefmt "%Y-%m-%d %T" ${ACCESSRIGHTS:+-pug}
# write a smaller version with only file size information
#call_to_tree "simple" "$OUTDIR/$OUTNAME-tree" tree -ax -n --du -h --dirsfirst
echo "Done."
signature.asc
Description: PGP signature
