> -----Original Message----- > From: openwrt-devel [mailto:[email protected]] > On Behalf Of Paul Spooren > Sent: Freitag, 31. Juli 2020 05:15 > To: [email protected] > Cc: [email protected]; Paul Spooren <[email protected]> > Subject: [PATCH] scripts: add size_compare.sh > > As package size changes are a continuous topic on the mailing list this > scripts > helps developers to compare their local package modifications against latest > upstream. > > The script downloads the latest package indexes based on env variables or > the `.config` file. The script compares the actual installed size > (data.tar.gz) or the IPK package size.
Just gave this a quick try, and generally it looks good. However, I got 82 packages listed for " Compare packages of ath79/generic/mips_24kc", but only 66 were in the final comparison. Will check the code in detail later. A few comment below. > > An example output is found below: > > ``` > user@dawn:~/src/openwrt/openwrt$ ./scripts/size_compare.sh Compare > packages of ath79/tiny/mips_24kc: > dropbear busybox iw ubus > > Checking installed size > > Fetching latest package indexes... > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left Speed > 100 80634 100 80634 0 0 33499 0 0:00:02 0:00:02 --:--:-- 33485 > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left Speed > 100 54082 100 54082 0 0 24252 0 0:00:02 0:00:02 --:--:-- 24252 > > Comparing package sizes... > Change Local Package > 611B 208715B busybox > 39B 5612B ubus > -42B 34940B iw > -14916B 89265B dropbear > ``` > > I plan to integrate this script into the CI so we have a summary how sizes > change over different architectures. > > Signed-off-by: Paul Spooren <[email protected]> > --- > scripts/size_compare.sh | 117 > ++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 117 insertions(+) > create mode 100755 scripts/size_compare.sh > > diff --git a/scripts/size_compare.sh b/scripts/size_compare.sh new file mode > 100755 index 0000000000..b310a085a4 > --- /dev/null > +++ b/scripts/size_compare.sh > @@ -0,0 +1,117 @@ > +# SPDX-License-Identifier: GPL-2.0-or-later # # Copyright (C) 2020 Paul > +Spooren <[email protected]> # ### ### size_compare - compare size of > +OpenWrt packages against upstream ### ### The script compares locally > +compiled package with the package indexes ### available upstream. This > +way the storage impact of optimizations or ### feature modifiactions is > +easy to see. > +### > +### If no environmental variables are set the scritp reads the current > +### .config file. The evaluated env variables are the following: > +### > +### TARGET SUBTARGET ARCH PACKAGES BIN_DIR BASE_URL > CHECK_INSTALLED > +### > +### Usage: > +### ./scripts/size_compare.sh > +### > +### Options: > +### -p --package-size Check IPK package size and not installed size > +### -h --help This message > + > +CONFIG_TARGET=$(sed -n 's/^CONFIG_TARGET_BOARD="\(.*\)"$/\1/p' > .config) > +CONFIG_SUBTARGET=$(sed -n > 's/^CONFIG_TARGET_SUBTARGET="\(.*\)"$/\1/p' > +.config) CONFIG_ARCH=$(sed -n > +'s/^CONFIG_TARGET_ARCH_PACKAGES="\(.*\)"$/\1/p' .config) > +CONFIG_PACKAGES=$(sed -n 's/^CONFIG_PACKAGE_\(.*\)=y$/\1/p' > .config | > +tr '\n' ' ') CONFIG_BIN_DIR=$(sed -n > +'s/^CONFIG_BINARY_DIR="\(.*\)"$/\1/p' .config) > + > +TARGET=${TARGET:-$CONFIG_TARGET} > +SUBTARGET=${SUBTARGET:-$CONFIG_SUBTARGET} > +ARCH=${ARCH:-$CONFIG_ARCH} > +PACKAGES=${PACKAGES:-$CONFIG_PACKAGES} > +BIN_DIR=${CONFIG_BIN_DIR:-./bin} > +BASE_URL="${BASE_URL:-https://downloads.openwrt.org/snapshots}"; > +CHECK_INSTALLED="${CHECK_INSTALLED:-y}" > + > +TARGET_URL="$BASE_URL/targets/$TARGET/$SUBTARGET/packages/Pack > ages.gz" > +PACKAGES_URL="$BASE_URL/packages/$ARCH/base/Packages.gz" > + > +help() { > + sed -rn 's/^### ?//;T;p' "$0" > +} > + > +package_size () { > + FOUND_PACKAGE= > + if [ -z "$CHECK_INSTALLED" ]; then > + SEARCH_PATTERN="Size" > + else > + SEARCH_PATTERN="Installed-Size" > + fi > + while IFS= read -r line; do > + if [ "$line" = "Package: $2" ]; then > + FOUND_PACKAGE=y > + fi > + if [ -n "$FOUND_PACKAGE" ]; then > + case $line in > + "$SEARCH_PATTERN"*) > + echo "$line" | cut -d ' ' -f 2 > + break > + ;; > + esac > + fi > + done < "$1" > +} > + > +compare_sizes () { > + for PACKAGE in $PACKAGES; do > + if [ "$PACKAGE" = "libc" ]; then > + continue > + fi > + PACKAGE_FILE=$(find "$BIN_DIR/packages/$ARCH/" > "$BIN_DIR/targets/$TARGET/$SUBTARGET/" -name "${PACKAGE}_*.ipk") When I first run this, I was in x86 target where only a kernel refresh has happened before (so, no build). This then produces a lot of warning due to missing packages folder: find: ‘./bin/targets/x86/generic/’: No such file or directory find: ‘./bin/packages/i386_pentium4/’: No such file or directory So, one should either suppress the message here for the find command, or alternatively check for the folders' existence beforehand. > + if [ -z "$PACKAGE_FILE" ]; then > + continue > + fi > + if [ -z "$CHECK_INSTALLED" ]; then > + SIZE_LOCAL=$(stat -c '%s' "$PACKAGE_FILE") > + else > + SIZE_LOCAL=$(tar tzvf "$PACKAGE_FILE" | grep > data.tar.gz | awk '{ print $3 }') > + fi > + SIZE_UPSTREAM=$(package_size "$TMP_INDEX" > "$PACKAGE") > + SIZE_DIFF="$((SIZE_LOCAL-SIZE_UPSTREAM))" > + echo "${SIZE_DIFF}B ${SIZE_LOCAL}B $PACKAGE" Personally, I think the "B" looks ugly. As we only have bytes, we could consider to drop it. One could also consider to add a "+" for the first column when change is positive. Both changes here are lowest priority. > + done > +} > + > +if [ "$1" = "-h" ]; then > + help > + exit 0 > +fi > + > +if [ "$1" = "-p" ]; then > + CHECK_INSTALLED= > +fi > + > +echo "Compare packages of $TARGET/$SUBTARGET/$ARCH": > +echo "$PACKAGES" > +echo > + > +if [ -z "$CHECK_INSTALLED" ]; then > + echo "Checking IPK package size" > +else > + echo "Checking installed size" > +fi > +echo > + > +echo "Fetching latest package indexes..." > +TMP_INDEX=$(mktemp /tmp/size_compare_package_index.XXXXXX) > +curl "$TARGET_URL" | gzip -d > "$TMP_INDEX" || exit 1 curl > +"$PACKAGES_URL" | gzip -d >> "$TMP_INDEX" || exit 1 echo > + > +echo "Comparing package sizes..." > +echo "Change Local Package" Since we have enough space, I'd go for "Change Local Remote Package" here. Best Adrian > +compare_sizes | sort -n -r > + > +rm "$TMP_INDEX" > -- > 2.25.1 > > > _______________________________________________ > openwrt-devel mailing list > [email protected] > https://lists.openwrt.org/mailman/listinfo/openwrt-devel
openpgp-digital-signature.asc
Description: PGP signature
_______________________________________________ openwrt-devel mailing list [email protected] https://lists.openwrt.org/mailman/listinfo/openwrt-devel
