On 16/03/10 09:11 AM, James C. McPherson wrote:
> On 16/03/10 01:12 AM, Mark J. Nelson wrote:
>> On 03/15/10 06:34 AM, James C. McPherson wrote:
>>> On 15/03/10 10:33 AM, James C. McPherson wrote:
>>>>
>>>> I've noticed that our sparc nightly builds are now
>>>> taking about 9 hours, sometimes closer to 10.
>>>>
>>>> That's a massive increase - when we were generating
>>>> SysV packages the sparc nightlies took about 6 hours.
>>
>> This is much more dramatic increase than I would expect.
>>
>>>> I know there are some operations on sparc that python
>>>> chokes on (memcmp, iirc), but is there anything else
>>>> that might be having an impact here?
>>
>> Are you setting your PKG_ARCHIVE path to something nfs mounted on the
>> gate machine? That was never great, but now it's terrible. The effort
>> to build packages locally and then rsync the repos is well worth the
>> build time performance improvement you'll see.
>
> ah! We've got PKGARCHIVE set to ${PARENT_WS}/packages/${MACH}/nightly
> and PARENT_WS is /ws/onnv-gate.
>
> I'll change that, and add some post-nightly stuff to
> push the bits across to the /ws location.
So with two brief changes to the env file, we now
have package repo generation taking between 30 and
40 minutes, and pushing of those repos to /ws/onnv-gate
taking somewhere around 3 minutes.
I think that's an acceptable speedup :-)
Rather than having PKGARCHIVE set to
#PKGARCHIVE="${PARENT_WS}/packages/${MACH}/nightly"; export PKGARCHIVE
where PARENT_WS was an nfs path, I set it to the workspace:
PKGARCHIVE="${CODEMGR_WS}/packages/${MACH}/nightly"; export PKGARCHIVE
and added this post-nightly hook:
post_nightly() {
date
mkdir -p ${PARENT_WS}/packages/${MACH}/nightly
mkdir -p ${PARENT_WS}/packages/${MACH}/nightly-nd
rsync -aHz ${CODEMGR_WS}/packages/sparc/* \
remotehost:/path/to/build/packages/${MACH}
}
POST_NIGHTLY=post_nightly; export POST_NIGHTLY
Incidentally, rsync to an nfs path was faster than the rcp,
but still took about 70 minutes for the nightly build.
cheers!
James
--
Senior Software Engineer, Solaris
Sun Microsystems
http://www.jmcp.homeunix.com/blog
