On 09/13/10 03:08 PM, Shawn Walker wrote:
On 09/13/10 02:37 PM, Keith Mitchell wrote:
Hi,
I've run into an issue where the scripts I'm using to merge the x86 and
SPARC versions of slim_source packages into a single repo stopped
working if the source repo was created on build 147. It appears to be
because the on disk format changed, which means I'm probably merging the
wrong way.
The general steps used for the build process are:
- Build slim_source on each architecture. This generates an IPS repo for
each. (The repo is created using pkgsend*)
- Zip up the IPS repo for archiving and transport
- Copy the x86.zip and sparc.zip repos to a common area, and unzip them
(The next few steps were based heavily on ON's use of merge.py)
- Use pkg.depotd to start the repos:
47
<http://indiana-build.central:8001/file/b34598f05b64/hudson/merge_repo.ksh#l47>
/usr/lib/pkg.depotd -d ${x86_REPO_DIR} --add-content -p
${TMP_PORT_x86} --set-property publisher.prefix=${PUB_PREFIX} >
${LOG_i386} 2>&1 &
48
<http://indiana-build.central:8001/file/b34598f05b64/hudson/merge_repo.ksh#l48>
i386_pid=${!}
49
<http://indiana-build.central:8001/file/b34598f05b64/hudson/merge_repo.ksh#l49>
/usr/lib/pkg.depotd -d ${SPARC_REPO_DIR} --add-content -p
${TMP_PORT_SPARC} --set-property publisher.prefix=${PUB_PREFIX} >
${LOG_sparc} 2>&1 &
50
<http://indiana-build.central:8001/file/b34598f05b64/hudson/merge_repo.ksh#l50>
sparc_pid=${!}
- Use merge.py to pull from the temporary depots and generate blended
packages - this is where I varied from ON, as they use "make
clobber/merge/publish" I believe.
57
<http://indiana-build.central:8001/file/b34598f05b64/hudson/merge_repo.ksh#l57>
${PKG_GATE}/src/util/publish/merge.py -d ${MERGED_PKG_DIR} -v
i386,http://localhost:${TMP_PORT_x86} -v
sparc,http://localhost:${TMP_PORT_SPARC} arch
- Using pkgsend*, create a new repo, and publish the blended packages
to it
- Using pkg.depotd --add-content --exit-ready, prepare the repo for use
(then zip it up to be ready for integration)
- Additionally, rsync the content of the new repo with a "long-standing"
repo with each nightly build of the packages, and use pkg.depotd
--add-content --exit-ready, then restart the SMF service running the
repo.
* I'm aware that pkgsend's use as a method for creating repos is
deprecated, but haven't had a chance to replace it with pkgrepo yet.
I suspect I could *probably* get around the issue by pointing at
$REPO_DIR/publisher/install-nightly instead of $REPO_DIR when pulling
down the x86-only / sparc-only packages, but I get the feeling that's
not the best way to do it. I also suspect that my usage of pkg.depotd
can and should be replaced with some pkgrepo commands. Any tips on what
direction I should take to make this work more elegantly with pkg tools
would be appreciated!
What isn't clear is at which point you're encountering an issue.
Specifically, at what point were your scripts breaking, and why?
I'm assuming at the point you were rsync'ing content...
-Shawn
Sorry about that, seems I missed a not-so-minor detail there.
From what I recollect of the logs, the issue is before the rsync. It
seems to be at or just before merge.py is run (i.e., the issue *seems*
to be with either how I call merge.py, or how I set-up the repo).
The symptom I'm seeing from the output is that merge.py is finding the
x86 packages, but not the sparc ones, when attempting to merge. The x86
packages/repo were built on a system with 146 pkg bits; the sparc stuff
was made on a system running stock 147. (When I look at the x86 repo, I
see what appears to my untrained eye to be a version 3 repo; when I look
at the sparc repo I see a publisher/install-nightly directory which I'm
guessing is version 4?).
I can add more detail as soon as I get my x86 system back up so I can
grab the output from the entire process and dig through the logs a bit
more, but I happen to have a log excerpt available here from the first
half of the process:
+ [ -d /export/home/hudson/workspace/update_install_repo/repo/merged_pkgs ]
+ mkdir -p /export/home/hudson/workspace/update_install_repo/repo/merged_pkgs
+ [ -d /export/home/hudson/workspace/update_install_repo/repo/merged_repo ]
+ mkdir -p /export/home/hudson/workspace/update_install_repo/repo/merged_repo
+ PKGS='consolidation/install/install-incorporation
consolidation/install/install-redistributable install/beadm
install/distribution-constructor install/installadm SUNWgui-install
SUNWslim-utils system/install system/install/auto-install
system/install/auto-install/auto-install-common system/install/tests
system/install/text-install system/library/install'
+ TMP_PORT_x86=10001
+ TMP_PORT_SPARC=10002
+ i386_pid=631172
+ sparc_pid=631173
+ sleep 30
+ /usr/lib/pkg.depotd -d
/export/home/hudson/workspace/update_install_repo/repo/i386 --add-content -p
10001 --set-property publisher.prefix=install-nightly
+ 1> /export/home/hudson/workspace/update_install_repo/i386_tmp.log 2>& 1
+ /usr/lib/pkg.depotd -d
/export/home/hudson/workspace/update_install_repo/repo/sparc --add-content -p
10002 --set-property publisher.prefix=install-nightly
+ 1> /export/home/hudson/workspace/update_install_repo/sparc_tmp.log 2>& 1
+ /export/gk/pkg-gate/src/util/publish/merge.py -d
/export/home/hudson/workspace/update_install_repo/repo/merged_pkgs -v
i386,http://localhost:10001 -v sparc,http://localhost:10002 arch
Processing 21 packages
Merging SUNWauto-install for ['i386']
Merging SUNWauto-install-common for ['i386']
Merging SUNWdistro-const for ['i386']
Merging SUNWgui-install for ['i386']
Merging SUNWinstall for ['i386']
Merging SUNWinstall-libs for ['i386']
Merging SUNWinstall-test for ['i386']
Merging SUNWinstalladm-tools for ['i386']
Merging SUNWslim-utils for ['i386']
Merging consolidation/install/install-incorporation for ['i386']
Merging consolidation/install/install-redistributable for ['i386']
Merging install/distribution-constructor for ['i386']
Merging install/installadm for ['i386']
Merging system/install for ['i386']
Merging system/install/auto-install for ['i386']
Merging system/install/auto-install/auto-install-common for ['i386']
Merging system/install/gui-install for ['i386']
Merging system/install/media/internal for ['i386']
Merging system/install/tests for ['i386']
Merging system/install/text-install for ['i386']
Merging system/library/install for ['i386']
+ /usr/bin/kill 631172 631173
Thanks,
Keith
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
