Re: nbld-bootstrap fails to build iso
Hi Tim, Sorry I didn't respond sooner. I like the changes you made to remove the code duplication. The ncurses package builds much like before except now uses amd64 directory so that looks good to me. The same two problems I mentioned in my other reply still exist though. I'll try to look into it some more and let you know if I figure any more out. Do you think we need a ld shadow wrapper for the second issue like I speculated? Thanks, Jason On Sat, Feb 7, 2009 at 5:04 PM, Tim Spriggs t...@tajinc.org wrote: Hi Jason, The code duplication throughout the scripts was driving me batty. Before it got any worse I refactored the code to use a base class (which provides the new dir-name semantics.) There will be a little more refactoring to come but the changes should be very minor (comparatively.) I've only done some high level testing and I found a few bugs that may have kept things from working fully. It's possible I introduced some new ones too though :-/ I have to run off for tonight so I committed the changes to the repository so you can grab them. If you could test the shadow_env on the packages you previously mentioned I would appreciate it to know if all problems still exist before diving in again. Cheers, -Tim Jason Upton wrote: Tim- Thanks for the feedback. Yeah, as I substituted amd64 for 64 to create that patch I was thinking to myself man this is probably not what we really want - it was just more of an idea and to see if that was really the root cause of the bootstrap failures. So I re-wrote to do the uname check like you mentioned and uploaded a new diff file called shadow_bin.svndiff2 in the same place. It seems to work but I don't know very much python so it may not be optimal. Hopefully it gets the idea across. I think I also uncovered another side-effect to using 64 instead of amd64 - there are actually some collisions for some files between libncurses5-dev and sunwcsl: sunwcsl: /usr/lib/amd64/libform.so sunwcsl: /usr/lib/amd64/libpanel.so sunwcsl: /usr/lib/amd64/libmenu.so But when libncurses5-dev used /usr/lib/64 it blindly over-wrote the files (because dpkg/apt don't seem to recognize the file through the symlink). For example- # dpkg -S /usr/lib/amd64/libmenu.so sunwcsl: /usr/lib/amd64/libmenu.so # dpkg -S /usr/lib/64/libmenu.so dpkg: /usr/lib/64/libmenu.so not found. So the newly built packages needed a dpkg-divert to rename the three files above. BTW - this does fix the ISO building problem after I populated my local /tank/ncp2 repo with all the .debs from the libiconv and ncurses shadow builds with amd64 as the 64bit directory. Let me know if you want me to upload them. Jason On Fri, Feb 6, 2009 at 1:33 PM, Tim Spriggs t...@tajinc.org wrote: The only problem with the diff (and the reason I wanted to just use 64) is that the script won't work on sparc once we have a port to it. This is a real PITA. On the other hand having to call uname every time one of the wrapper commands is called is a real bummer too. It may not be as expensive as I think since python has a uname module which should just be internal env calls. I'll keep the patch in mind but I'm not ultimately sure how I want to deal with this yet. As for the package version, adding -1 is appropriate. Thanks, -Tim Jason Upton wrote: Tim- FYI... I put a svn diff of what I think needs to change for the shadow build scripts. It is on gnusolaris.org in: ~wavejumper/shadow_bin.svndiff I have rebuilt libiconv with the changes and it seems to work (files are in amd64 now). I'm rebuilding ncurses now and I'll put the debs I built into my local /tank/ncp2 repo and see if that fixes the iso building problems and report back later today. I can upload them after my iso testing if you want (and if you agree with my proposal). I changed the version suffix from +64 to +64-1. Wasn't sure what you'd expect/want for version updates on this +64 suffix. Jason On Fri, Feb 6, 2009 at 11:50 AM, Jason Upton wavejumpe...@gmail.com wrote: I think I have found the problem. The sunwcsl package wants to create a symlink for /usr/lib/64 to /usr/lib/amd64 but the new shadow-built packages with combined 32/64 libs (ex libiconv, libncurses) has created a real directory for /usr/lib/64 already. So debootstrap fails while unpacking sunwcsl with: tar: ./usr/lib/64: Cannot create symlink to `amd64': File exists tar: Error exit delayed from previous errors In the sunwcsl package you can see that it wants /usr/lib/amd64 to be a real directory and /usr/lib/64 to be a symlink: r...@upton3:/tmp# dpkg -c /tank/ncp2/dists/hardy-unstable/main/binary-solaris-i386/base/sunwcsl_5.11.104-5_solaris-i386.deb |grep /usr/lib/64 lrwxrwxrwx root/root 0 2009-02-04 14:45:45 ./usr/lib/64 - amd64 r...@upton3:/tmp# dpkg -c /tank/ncp2/dists/hardy-unstable/main/binary-solaris-i386/base/sunwcsl_5.11.104-5_solaris-i386.deb |grep
Re: nbld-bootstrap fails to build iso
Tim-
Thanks for the feedback.
Yeah, as I substituted amd64 for 64 to create that patch I was
thinking to myself man this is probably not what we really want - it
was just more of an idea and to see if that was really the root cause
of the bootstrap failures. So I re-wrote to do the uname check like
you mentioned and uploaded a new diff file called shadow_bin.svndiff2
in the same place. It seems to work but I don't know very much python
so it may not be optimal. Hopefully it gets the idea across.
I think I also uncovered another side-effect to using 64 instead of
amd64 - there are actually some collisions for some files between
libncurses5-dev and sunwcsl:
sunwcsl: /usr/lib/amd64/libform.so
sunwcsl: /usr/lib/amd64/libpanel.so
sunwcsl: /usr/lib/amd64/libmenu.so
But when libncurses5-dev used /usr/lib/64 it blindly over-wrote the
files (because dpkg/apt don't seem to recognize the file through the
symlink). For example-
# dpkg -S /usr/lib/amd64/libmenu.so
sunwcsl: /usr/lib/amd64/libmenu.so
# dpkg -S /usr/lib/64/libmenu.so
dpkg: /usr/lib/64/libmenu.so not found.
So the newly built packages needed a dpkg-divert to rename the three
files above.
BTW - this does fix the ISO building problem after I populated my
local /tank/ncp2 repo with all the .debs from the libiconv and ncurses
shadow builds with amd64 as the 64bit directory. Let me know if you
want me to upload them.
Jason
On Fri, Feb 6, 2009 at 1:33 PM, Tim Spriggs t...@tajinc.org wrote:
The only problem with the diff (and the reason I wanted to just use 64) is
that the script won't work on sparc once we have a port to it. This is a
real PITA. On the other hand having to call uname every time one of the
wrapper commands is called is a real bummer too. It may not be as expensive
as I think since python has a uname module which should just be internal env
calls. I'll keep the patch in mind but I'm not ultimately sure how I want to
deal with this yet.
As for the package version, adding -1 is appropriate.
Thanks,
-Tim
Jason Upton wrote:
Tim-
FYI...
I put a svn diff of what I think needs to change for the shadow build
scripts. It is on gnusolaris.org in:
~wavejumper/shadow_bin.svndiff
I have rebuilt libiconv with the changes and it seems to work (files
are in amd64 now). I'm rebuilding ncurses now and I'll put the debs I
built into my local /tank/ncp2 repo and see if that fixes the iso
building problems and report back later today.
I can upload them after my iso testing if you want (and if you agree
with my proposal). I changed the version suffix from +64 to
+64-1. Wasn't sure what you'd expect/want for version updates on
this +64 suffix.
Jason
On Fri, Feb 6, 2009 at 11:50 AM, Jason Upton wavejumpe...@gmail.com
wrote:
I think I have found the problem. The sunwcsl package wants to create
a symlink for /usr/lib/64 to /usr/lib/amd64 but the new shadow-built
packages with combined 32/64 libs (ex libiconv, libncurses) has
created a real directory for /usr/lib/64 already. So debootstrap
fails while unpacking sunwcsl with:
tar: ./usr/lib/64: Cannot create symlink to `amd64': File exists
tar: Error exit delayed from previous errors
In the sunwcsl package you can see that it wants /usr/lib/amd64 to be
a real directory and /usr/lib/64 to be a symlink:
r...@upton3:/tmp# dpkg -c
/tank/ncp2/dists/hardy-unstable/main/binary-solaris-i386/base/sunwcsl_5.11.104-5_solaris-i386.deb
|grep /usr/lib/64
lrwxrwxrwx root/root 0 2009-02-04 14:45:45 ./usr/lib/64 - amd64
r...@upton3:/tmp# dpkg -c
/tank/ncp2/dists/hardy-unstable/main/binary-solaris-i386/base/sunwcsl_5.11.104-5_solaris-i386.deb
|grep /usr/lib/amd64/$
drwxr-xr-x root/bin 0 2009-02-04 14:45:25 ./usr/lib/amd64/
And on a system without updates to include the new shadow-built 32/64
packages, you can see that indeed all 64 directories are symlinks to
real amd64 directories under /usr/lib:
r...@upton3:/tmp# find /usr/lib -name 64 -exec ls -ald {} \;
lrwxrwxrwx 1 root root 5 Dec 29 22:15 /usr/lib/link_audit/64 - amd64
lrwxrwxrwx 1 root root 5 Dec 29 22:15 /usr/lib/64 - amd64
drwxr-xr-x 2 root bin 3 Dec 29 22:16 /usr/lib/dtrace/64
lrwxrwxrwx 1 root root 5 Dec 29 22:15 /usr/lib/security/64 - amd64
lrwxrwxrwx 1 root root 5 Dec 29 22:15 /usr/lib/secure/64 - amd64
lrwxrwxrwx 1 root root 5 Dec 29 22:15 /usr/lib/lwp/64 - amd64
r...@upton3:/tmp# find /usr/lib -name amd64 -exec ls -ald {} \;
drwxr-xr-x 4 root bin 390 Jan 28 11:54 /usr/lib/amd64
drwxr-xr-x 2 root bin 6 Dec 29 22:17 /usr/lib/sasl/amd64
drwxr-xr-x 2 root bin 4 Dec 29 22:17 /usr/lib/krb5/amd64
drwxr-xr-x 2 root sys 3 Dec 29 22:19 /usr/lib/adb/amd64
drwxr-xr-x 2 root bin 5 Dec 29 22:17 /usr/lib/link_audit/amd64
drwxr-xr-x 2 root bin 5 Dec 29 22:17 /usr/lib/ld/amd64
drwxr-xr-x 2 root bin 58 Jan 28 11:54 /usr/lib/security/amd64
drwxr-xr-x 2 root bin 2 Dec 16 12:13 /usr/lib/secure/amd64
drwxr-xr-x 2 root sys 3 Jan 28 11:54 /usr/lib/fs/smbfs/amd64
drwxr-xr-x 2
Re: nbld-bootstrap fails to build iso
Hi Tim, Looking more at the ncurses package I built, it appears that there are two other problems I see. 1. Some of the 32 bit /usr/lib/*.so symlinks are not updated to the new package. For example, libform.so; r...@devzone4:/usr/src/build/ncurses# ll /usr/lib/libform.so* lrwxrwxrwx 1 root root14 Dec 30 00:41 /usr/lib/libform.so - ./libform.so.1* -rwxr-xr-x 1 root bin 72548 Dec 18 00:37 /usr/lib/libform.so.1* lrwxrwxrwx 1 root root14 Feb 7 09:50 /usr/lib/libform.so.5 - libform.so.5.5 -rw-r--r-- 1 root root 41996 Feb 7 09:04 /usr/lib/libform.so.5.5 r...@devzone4:/usr/src/build/ncurses# dpkg -S /usr/lib/libform.so* sunwcsl: /usr/lib/libform.so sunwcsl: /usr/lib/libform.so.1 libncurses5: /usr/lib/libform.so.5 libncurses5: /usr/lib/libform.so.5.5 So the /usr/lib/libform.so symlinks still points to the sunwcsl version. Not sure if this is a problem with the shadow build environment or the ncurses package itself. I'll keep looking into it and let you know if I find anything. I was wondering why I only needed the dpkg-divert for the amd64 versions as I previously noted in my last reply. Looks like we should end up having diverts for 32 and 64 once this is resolved. 2. some 64 bit libs are still not quite correct. For example, libform again. In 32 bit there is a symlink to a file that exists (like in issue 1 above): r...@devzone4:/usr/src/build/ncurses# ll /usr/lib/libform.so.5* lrwxrwxrwx 1 root root14 Feb 7 09:50 /usr/lib/libform.so.5 - libform.so.5.5 -rw-r--r-- 1 root root 41996 Feb 7 09:04 /usr/lib/libform.so.5.5 r...@devzone4:/usr/src/build/ncurses# dpkg -S /usr/lib/libform.so.5* libncurses5: /usr/lib/libform.so.5 libncurses5: /usr/lib/libform.so.5.5 So libform.so.5.5 is real and from the libncurses5 package. but in 64 bit there is a symlink, but no real file: r...@devzone4:/usr/src/build/ncurses# ll /usr/lib/amd64/libform.so* llrwxrwxrwx 1 root root12 Feb 7 09:50 /usr/lib/amd64/libform.so - libform.so.5 -rwxr-xr-x 1 root bin 99760 Dec 18 00:36 /usr/lib/amd64/libform.so.1* rwxrwxrwx 1 root root 14 Feb 7 09:50 /usr/lib/amd64/libform.so.5 - libform.so.5.5 r...@devzone4:/usr/src/build/ncurses# dpkg -S /usr/lib/amd64/libform.so* libncurses5-dev: /usr/lib/amd64/libform.so sunwcsl: /usr/lib/amd64/libform.so.1 libncurses5: /usr/lib/amd64/libform.so.5 There is no libform.so.5.5 file for 64bit from the libncurses package. But 64 bit doesn't have the problem from issue#1 above... So looking through the log of my build and searching for libform.so, I see that there are ld -dy -G -h calls which I think create the 32 bit version. So I think we may also need a ld shadow wrapper to create the 64 bit versions. Any thoughts/comments? Thanks, Jason On Sat, Feb 7, 2009 at 10:07 AM, Jason Upton wavejumpe...@gmail.com wrote: Tim- Thanks for the feedback. Yeah, as I substituted amd64 for 64 to create that patch I was thinking to myself man this is probably not what we really want - it was just more of an idea and to see if that was really the root cause of the bootstrap failures. So I re-wrote to do the uname check like you mentioned and uploaded a new diff file called shadow_bin.svndiff2 in the same place. It seems to work but I don't know very much python so it may not be optimal. Hopefully it gets the idea across. I think I also uncovered another side-effect to using 64 instead of amd64 - there are actually some collisions for some files between libncurses5-dev and sunwcsl: sunwcsl: /usr/lib/amd64/libform.so sunwcsl: /usr/lib/amd64/libpanel.so sunwcsl: /usr/lib/amd64/libmenu.so But when libncurses5-dev used /usr/lib/64 it blindly over-wrote the files (because dpkg/apt don't seem to recognize the file through the symlink). For example- # dpkg -S /usr/lib/amd64/libmenu.so sunwcsl: /usr/lib/amd64/libmenu.so # dpkg -S /usr/lib/64/libmenu.so dpkg: /usr/lib/64/libmenu.so not found. So the newly built packages needed a dpkg-divert to rename the three files above. BTW - this does fix the ISO building problem after I populated my local /tank/ncp2 repo with all the .debs from the libiconv and ncurses shadow builds with amd64 as the 64bit directory. Let me know if you want me to upload them. Jason On Fri, Feb 6, 2009 at 1:33 PM, Tim Spriggs t...@tajinc.org wrote: The only problem with the diff (and the reason I wanted to just use 64) is that the script won't work on sparc once we have a port to it. This is a real PITA. On the other hand having to call uname every time one of the wrapper commands is called is a real bummer too. It may not be as expensive as I think since python has a uname module which should just be internal env calls. I'll keep the patch in mind but I'm not ultimately sure how I want to deal with this yet. As for the package version, adding -1 is appropriate. Thanks, -Tim Jason Upton wrote: Tim- FYI... I put a svn diff of what I think needs to change for the shadow build scripts. It is on gnusolaris.org
