On Wed, Mar 11, 2009 at 10:32 AM, Aaron Griffin <[email protected]> wrote: > On Wed, Mar 11, 2009 at 8:27 AM, Abhishek Dasgupta <[email protected]> wrote: >> This patch adds initial support for any architecture. >> It assumes that there'll be a repo-any directory in repos/ >> from which it'll pull the PKGBUILDs. Also the architectureo >> independent packages are assumed to be in $reponame/os/any. >> >> This commit does not fix anything going wrong in the cron-jobs >> or misc-scripts folder. Also this commit contain any helper >> scripts for the any architecture corresponding to db-core or >> testing2x. >> >> Signed-off-by: Abhishek Dasgupta <[email protected]> >> --- >> db-move | 48 ++++++++++++++++++++++++---------------- >> db-remove | 33 +++++++++++++++++---------- >> db-update | 72 >> +++++++++++++++++++++++++++++++++++++++++++++--------------- >> 3 files changed, 104 insertions(+), 49 deletions(-) >> >> diff --git a/db-move b/db-move >> index ff9677b..aac0e4e 100755 >> --- a/db-move >> +++ b/db-move >> @@ -1,6 +1,27 @@ >> #!/bin/bash >> # Originally from Pierre's testing2extra script >> >> +process_db_move() { >> + >> + # copy the db file into our working area >> + if [ -f "$ftppath_from/$1/$repofrom.db.tar.$DB_COMPRESSION" ]; then >> + /bin/cp "$ftppath_from/$1/$repofrom.db.tar.$DB_COMPRESSION" . >> + /usr/bin/repo-remove -q "$repofrom.db.tar.$DB_COMPRESSION" >> $packagename || die "Error in repo-remove" >> + #use '*' to move the old DB too >> + mv $repofrom.db.tar.$DB_COMPRESSION* $ftppath_from/$1 >> + echo " Package files will be cleaned up automatically" >> + fi >> + >> + if [ -f "$ftppath_to/$1/$repoto.db.tar.$DB_COMPRESSION" ]; then >> + /bin/cp "$ftppath_to/$1/$repoto.db.tar.$DB_COMPRESSION" . >> + fi >> + >> + /bin/cp "$ftppath_from/$1/$_pkgfile" . >> + /usr/bin/repo-add -q "$repoto.db.tar.$DB_COMPRESSION" $_pkgfile || die >> "Error in repo-add" >> + #use '*' to move the old DB too >> + mv $repoto.db.tar.$DB_COMPRESSION* $_pkgfile $ftppath_to/$1 >> + >> +} >> if [ $# -ne 4 ]; then >> echo "usage: $(basename $0) <pkgname> <repo-from> <repo-to> <arch>" >> exit 1 >> @@ -17,8 +38,8 @@ _arch="$4" >> >> export CARCH="$_arch" >> >> -ftppath_from="$FTP_BASE/$repofrom/os/$_arch/" >> -ftppath_to="$FTP_BASE/$repoto/os/$_arch/" >> +ftppath_from="$FTP_BASE/$repofrom/os/" >> +ftppath_to="$FTP_BASE/$repoto/os/" >> svnrepo_from="$repofrom-$_arch" >> svnrepo_to="$repoto-$_arch" >> >> @@ -62,7 +83,7 @@ if [ -d "$packagename/repos/$svnrepo_from" ]; then >> . "$packagename/repos/$svnrepo_from/$BUILDSCRIPT" >> _pkgfile="$pkgname-$pkgver-$pkgrel-$_arch$PKGEXT" >> >> - if [ ! -f "$ftppath_from/$_pkgfile" ]; then >> + if [ ! -f "$ftppath_from/$_arch/$_pkgfile" ]; then >> die "error: package file '$_pkgfile' not found in repo '$repofrom'" >> fi >> >> @@ -81,23 +102,12 @@ if [ -d "$packagename/repos/$svnrepo_from" ]; then >> [ -d build/ ] || mkdir build >> cd build/ >> >> - # copy the db file into our working area >> - if [ -f "$ftppath_from/$repofrom.db.tar.$DB_COMPRESSION" ]; then >> - /bin/cp "$ftppath_from/$repofrom.db.tar.$DB_COMPRESSION" . >> - /usr/bin/repo-remove -q "$repofrom.db.tar.$DB_COMPRESSION" >> $packagename || die "Error in repo-remove" >> - #use '*' to move the old DB too >> - mv $repofrom.db.tar.$DB_COMPRESSION* $ftppath_from >> - echo " Package files will be cleaned up automatically" >> + if [ "$_arch" == "any" ]; then >> + process_db_move i686 >> + process_db_move x86_64 >> + else >> + process_db_move $_arch >> fi >> - >> - if [ -f "$ftppath_to/$repoto.db.tar.$DB_COMPRESSION" ]; then >> - /bin/cp "$ftppath_to/$repoto.db.tar.$DB_COMPRESSION" . >> - fi >> - >> - /bin/cp "$ftppath_from/$_pkgfile" . >> - /usr/bin/repo-add -q "$repoto.db.tar.$DB_COMPRESSION" $_pkgfile || die >> "Error in repo-add" >> - #use '*' to move the old DB too >> - mv $repoto.db.tar.$DB_COMPRESSION* $_pkgfile $ftppath_to >> else >> die "Error: $packagename is not in repo $repofrom" >> fi >> diff --git a/db-remove b/db-remove >> index acc849d..6432d94 100755 >> --- a/db-remove >> +++ b/db-remove >> @@ -1,5 +1,21 @@ >> #!/bin/bash >> >> +process_db_files() { >> + # copy the db file into our working area >> + if [ -f "$ftppath/$1/$reponame.db.tar.$DB_COMPRESSION" ]; then >> + /bin/cp "$ftppath/$1/$reponame.db.tar.$DB_COMPRESSION" build/ >> + else >> + echo "No database found at '$ftpdir', nothing more to do" >> + exit 0 >> + fi >> + >> + echo " Removing from $reponame DB file" >> + cd build/ >> + /usr/bin/repo-remove -q "$reponame.db.tar.$DB_COMPRESSION" >> $packagename >> + >> + /bin/cp "$reponame.db.tar.$DB_COMPRESSION" "$ftppath/$1/" >> +} >> + >> if [ $# -ne 3 ]; then >> echo "usage: $(basename $0) <pkgname> <reponame> <arch>" >> exit 1 >> @@ -15,7 +31,7 @@ arch="$3" >> >> export CARCH="$arch" >> >> -ftppath="$FTP_BASE/$reponame/os/$arch/" >> +ftppath="$FTP_BASE/$reponame/os/" >> svnrepo="$reponame-$arch" >> >> [ "$UID" = "" ] && UID=$(uid) >> @@ -64,20 +80,13 @@ fi >> cd "$WORKDIR" >> [ -d build/ ] || mkdir build >> >> -# copy the db file into our working area >> -if [ -f "$ftppath/$reponame.db.tar.$DB_COMPRESSION" ]; then >> - /bin/cp "$ftppath/$reponame.db.tar.$DB_COMPRESSION" build/ >> +if [ "$arch" == "any" ]; then >> + process_db_files i686 >> + process_db_files x86_64 >> else >> - echo "No database found at '$ftpdir', nothing more to do" >> - exit 0 >> + process_db_files $arch >> fi >> >> -echo " Removing from $reponame DB file" >> -cd build/ >> -/usr/bin/repo-remove -q "$reponame.db.tar.$DB_COMPRESSION" $packagename >> - >> -/bin/cp "$reponame.db.tar.$DB_COMPRESSION" "$ftppath/" >> - >> echo "Package files will be cleaned up automatically" >> >> cleanup >> diff --git a/db-update b/db-update >> index 77af39f..f51f470 100755 >> --- a/db-update >> +++ b/db-update >> @@ -15,6 +15,7 @@ _arch="$2" >> export CARCH="$_arch" >> >> ftppath="$FTP_BASE/$reponame/os/$_arch/" >> +ftppath_base="$FTP_BASE/$reponame/os/" >> svnrepo="$reponame-$_arch" >> stagedir="$STAGING/$reponame" >> >> @@ -72,10 +73,21 @@ repo_lock $reponame $_arch >> cd "$WORKDIR" >> >> # copy the db file into our working area >> -if [ -f "$ftppath/$reponame.db.tar.$DB_COMPRESSION" ]; then >> - /bin/cp "$ftppath/$reponame.db.tar.$DB_COMPRESSION" build/ >> +if [ "$_arch" == "any" ]; then >> + for architecture in i686 x86_64; do >> + if [ -f >> "$ftppath_base/$architecture/$reponame.db.tar.gz.$DB_COMPRESSION" ]; then >> + /bin/cp >> "$ftppath_base/$architecture/$reponame.db.tar.gz.$DB_COMPRESSION" >> build/$reponame-i686.db.tar.gz >> + else >> + touch >> "build/$reponame-$architecture.db.tar.$DB_COMPRESSION" >> + fi >> + done >> +else >> + if [ -f "$ftppath/$reponame.db.tar.$DB_COMPRESSION" ]; then >> + /bin/cp "$ftppath/$reponame.db.tar.$DB_COMPRESSION" build/ >> + else >> + touch "build/$reponame.db.tar.$DB_COMPRESSION" >> + fi >> fi >> - >> echo "Updating DB for $reponame $_arch" >> >> if [ -d "${stagedir}64" ]; then >> @@ -106,11 +118,11 @@ fi >> if [ -n "$ADDPKGS" ]; then >> >> echo "==> Copying DB file from '$reponame'..." >&2 >> - if [ -f "$ftppath/$reponame.db.tar.$DB_COMPRESSION" ]; then >> - /bin/cp "$ftppath/$reponame.db.tar.$DB_COMPRESSION" build/ >> - else >> - touch "build/$reponame.db.tar.$DB_COMPRESSION" >> - fi >> +# if [ -f "$ftppath/$reponame.db.tar.$DB_COMPRESSION" ]; then >> +# /bin/cp "$ftppath/$reponame.db.tar.$DB_COMPRESSION" build/ >> +# else >> +# touch "build/$reponame.db.tar.$DB_COMPRESSION" >> +# fi >> >> echo "==> Processing new/updated packages for repository >> '$reponame'..." >&2 >> >> @@ -147,10 +159,19 @@ if [ -n "$ADDPKGS" ]; then >> >> pkgs="" >> for pkg in $to_add; do pkgs="$pkgs $(basename $pkg)"; done >> - >> - /usr/bin/repo-add -q "$reponame.db.tar.$DB_COMPRESSION" $pkgs >> + if [ "$_arch" == "any" ]; then >> + /usr/bin/repo-add -q >> "$reponame-i686.db.tar.$DB_COMPRESSION" $pkgs >> + /usr/bin/repo-add -q >> "$reponame-x86_64.db.tar.$DB_COMPRESSION" $pkgs >> + else >> + /usr/bin/repo-add -q >> "$reponame.db.tar.$DB_COMPRESSION" $pkgs >> + fi >> else >> - rm -f "build/$reponame.db.tar.$DB_COMPRESSION" >> + if [ "$_arch" == "any" ]; then >> + rm -f "build/$reponame-i686.db.tar.gz" >> + rm -f "build/$reponame-x86_64.db.tar.gz" >> + else >> + rm -f "build/$reponame.db.tar.$DB_COMPRESSION" >> + fi >> echo "Errors found when adding packages" >> fi >> else >> @@ -159,13 +180,28 @@ fi >> >> # if non empty, move all build dirs >> if [ $(/bin/ls "$WORKDIR/build/" 2>/dev/null | wc -l) != 0 ]; then >> - echo "Copying new files to '$ftppath'" >> - for f in "$WORKDIR/build/"*; do >> - if ! /bin/cp "$f" "$ftppath"; then >> - die "error: failure while copying files to $ftppath" >> - fi >> - done >> - >> + if [ "$_arch" != "any" ]; then >> + echo "Copying new files to '$ftppath'" >> + for f in "$WORKDIR/build/"*; do >> + if ! /bin/cp "$f" "$ftppath"; then >> + die "error: failure while copying files to >> $ftppath" >> + fi >> + done >> + else >> + echo "Copying architecture-independent packages..." >> + for f in "$WORKDIR/build/"*.pkg.tar.gz; do >> + if ! /bin/cp "$f" "$ftppath"; then >> + die "error: failure while copying $f to >> $ftppath" >> + fi >> + # Make link to arch-any package from os/{i686,x86_64} >> directories >> + ln -s "$ftppath/$f" "$ftppath_base/i686/" >> + ln -s "$ftppath/$f" "$ftppath_base/x86_64/" >> + for architecture in i686 x86_64; do >> + if ! /bin/cp >> "$reponame-$architecture.db.tar.$DB_COMPRESSION" >> "$ftppath_base/$architecture/$reponame.db.tar.$DB_COMPRESSION"; then >> + die "error: failed to transfer $architecture >> repository database." >> + fi >> + done >> + fi >> else >> echo "Nothing to copy, no work done" >> fi >> >> -- >> Abhishek Dasgupta <http://abhidg.mine.nu> >> GPG 67972DOF pgp.mit.edu > > Hmm the aur-general list is a weird place for this patch :) > > The only thing that makes me hesitant about this is that I was going a > slightly different route. > > This patch here: > http://code.phraktured.net/cgit.cgi/dbscripts/commit/?h=any-arch&id=1ef058dad91a5ca82db2744776251a2ab68bcf90 > changes the dbscripts to loop over all architectures for each run. I > wanted to simply add "any" into the file detection part, but not > delete them until the very end > > So this way, if I have 3 packages in my staging dir: foo(i686), > bar(x86_64), and baz(any), running db-update does the following: > for A in ARCHES (where ARCHES=(i686 x86_64)): > get all files with "A" arch _or_ "any" arch > do loop as normal > delete all files we processed correctly
I guess the only thing this DOESN'T take into account is the symlinking part, but that would just be a single "special case" when copying
