Re: 32 bit ports on an AMD64 system
On Tue, Sep 1, 2009 at 1:39 PM, b. f.bf1...@googlemail.com wrote: You've given some of your reasons for using amd64 -- but are your reasons for using 32-bit binaries on amd64 strong enough to make all of this worthwhile? Why not just use 64-bit binaries for all but the 32-bit-only ports? Sure, some 32-bit applications will actually run faster (the opposite is also often true) or use fewer resources, but is it worth the hassle? Mostly that's what I want to do. However I'd like to do some Python QT development with PyQT4, and would like to test the performance using py-psycho (i386 bit only). In that situation, I'm not sure if I'll have to install QT4 as 32 bit or not, likewise for X... It's a bit messy, but we'll see what happens. Thanks for all the advice and help, -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
First, I hope that you have a good reason for doing this, because it is going to be a PITA, and prone to all sorts of problems. [...] Unfortunately I do. The 32 bit stuff is *would be really nice, but not necessary*, but the ability to use extra memory *and* dynamically load kernel modules is a bit more important to me. If you are just talking about unshadowed ports, you could try adding something like the following to /etc/make.conf: .ifdef(32BIT) PREFIX=/usr/local/32 LOCALBASE=/usr/local/32 LDCONFIG=/sbin/ldconfig -32 CFLAGS+=-m32 .endif and then wrapping your portupgrade, portmaster, or make commands with: env 32BIT=yes [insert rest of command here] If there is shadowing, you have to consider how to manage the environment PATH so that you can actually run the 32-bit binary executables without typing in fully-qualified pathnames every time, and maybe also using LD_RUN_PATH or LD_LIBRARY_PATH so that the right libraries are used --- you can see how problems crop up at every turn? I was thinking of leaving the standard paths alone, and setting up one or more [ba|c|]sh32 alias/scripts which would start up the appropriate shell, with 32-bit directories at the front of the path. I know I'll end up having both 32- and 64-bit of Python on my system (many of the main apps I use require python - but I also want to be able to use py-psyco), as well as a few copies of the various GUI toolkits in 32-bit. Thanks for the info, -Jim Stapleton -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
Also note that it is possible to have an i386 port-building jail on an amd64 system. So, make/build/run a normal jail using the 64 bit os world, add -m32 to the make.conf CFLAGS and CXXFLAGS, build stuff in the jail, and copy it to main (non-jailed) system, and run ldconfig on the library directories? I'd probably also change PREFIX/LOCALBASE to prevent the files from the ports from colling with the base system files when I copy them over. Thanks, -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
Well, this would certainly help with building the ports safely. But I think we -- at least I was -- were thinking that you would actually leave them in the jail, and run them from the jailed environment, so there would be fewer run-time problems, and no work to transfer them over. Remember that you've got to ensure that there is no problem with run-time linking of shared libraries, some of which (in your current scheme) will have both 32-bit and 64-bit versions with the same soname. You can probably work around this problem as well, but it seems easier to leave them in the jail. With only console stuff, that'd probably be fine, a jail wouldn't be much more tedious than the environment shuffling I'd need to run the 32 bit stuff, however I'll want to do some X11 stuff.. I know you can access X between different users on a machine, but can a jail'ed shell open a window on an X server running from the main machine? I'm not even sure what terms I would use for searching on how to get that working. Thanks, -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
On Tue, Sep 01, 2009 at 09:44:10AM -0400, Jim wrote: First, I hope that you have a good reason for doing this, because it is going to be a PITA, and prone to all sorts of problems. [...] Unfortunately I do. The 32 bit stuff is *would be really nice, but not necessary*, but the ability to use extra memory *and* dynamically load kernel modules is a bit more important to me. All FreeBSD supported platforms can dynamically load native kernel modules, so why should that be a factor in choosing between i386 and amd64? Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgptfLLWcw7kA.pgp Description: PGP signature
Re: 32 bit ports on an AMD64 system
[...] but the ability to use extra memory *and* dynamically load kernel modules is a bit more important to me. All FreeBSD supported platforms can dynamically load native kernel modules, so why should that be a factor in choosing between i386 and amd64? Roland I didn't specify just loading modules, but extra memory as well (the beyond 4GB addressable space). Using the options in i386 that allow you to access memory beyond 4GB, also eliminates the ability to dynamically load kernel modules. -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
On Tue, Sep 01, 2009 at 09:49:47AM -0400, Jim wrote: Also note that it is possible to have an i386 port-building jail on an amd64 system. So, make/build/run a normal jail using the 64 bit os world, add -m32 to the make.conf CFLAGS and CXXFLAGS, build stuff in the jail, and copy it to main (non-jailed) system, and run ldconfig on the library directories? I'd probably also change PREFIX/LOCALBASE to prevent the files from the ports from colling with the base system files when I copy them over. The following post gives some more explanation. Essentially you're doing a cross-build: http://en.jnlin.org/2008/06/07/12/ Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgpAAoL3GNFom.pgp Description: PGP signature
Re: 32 bit ports on an AMD64 system
You've given some of your reasons for using amd64 -- but are your reasons for using 32-bit binaries on amd64 strong enough to make all of this worthwhile? Why not just use 64-bit binaries for all but the 32-bit-only ports? Sure, some 32-bit applications will actually run faster (the opposite is also often true) or use fewer resources, but is it worth the hassle? As for your earlier question, I haven't used multiple instances of X myself, either in or out of jails, but I have seen reports of others doing so, so I think it is possible, except perhaps in a few cases where hardware balks because the graphics driver isn't good enough. I guess you'll have to make some experiments. If you don't make provisions for running a 32-bit X, then is there much point to having, for example, 32-bit window managers, windowing toolkits, or GUIs? If you use a different LOCALBASE for 32-bit ports, you are going to have to use 32-bit versions of most trunk and branch ports. I still think a jail is your best bet -- after all, a thin jail, which reuses those portions of your system that don't need to be different inside the jail, is just a more thorough version of what you had hoped to accomplish with your 32-bit shell scripts. If you don't use a jail ... well, I have not tried to install a large number of 32-bit and 64-bit ports in parallel, so I am not sure if the default setup for our loader will make the appropriate distinctions between 32-bit and 64-bit versions of the same libraries depending upon the executables or libraries that need them, but I think that there is a good chance that it will not, and that you will have to do some extra work to make sure that it does. In the case that it does not, your only alternative is to either patch a large number of ports (very time-consuming and error-prone), or to add loader environment variables to your 32-bit shell scripts to make the loader look in the 32-bit library directories first, or to write a custom loader script to only load the appropriate libraries depending upon whether the executable or library that needs them is 32-bit or 64-bit. It would be nice to have this flexibility, but given the current state of the base system and infrastructure, it just seems like more trouble than it is worth. b. On 9/1/09, Jim stapleton...@gmail.com wrote: [...] but the ability to use extra memory *and* dynamically load kernel modules is a bit more important to me. All FreeBSD supported platforms can dynamically load native kernel modules, so why should that be a factor in choosing between i386 and amd64? Roland I didn't specify just loading modules, but extra memory as well (the beyond 4GB addressable space). Using the options in i386 that allow you to access memory beyond 4GB, also eliminates the ability to dynamically load kernel modules. -Jim Stapleton ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
On 9/1/09, b. f. bf1...@googlemail.com wrote: . If you don't use a jail ... well, I have not tried to install a large number of 32-bit and 64-bit ports in parallel, so I am not sure if the default setup for our loader will make the appropriate distinctions between 32-bit and 64-bit versions of the same libraries depending upon the executables or libraries that need them, but I think that there is a good chance that it will not, and that you will have to do some extra work to make sure that it does. In the case that it does not, your only alternative is to either patch a large number of ports (very time-consuming and error-prone), or to add loader environment variables to your 32-bit shell scripts to make the loader look in the 32-bit library directories first, or to write a custom loader script to only load the appropriate libraries depending upon whether the executable or library that needs them is 32-bit or 64-bit. It would Ok, so maybe I was a bit too pessimistic here, and the framework that is already in place to handle the 32-bit-only ports will suffice to deal with the case of parallel installs of 32-bit and 64-bit ports libraries as well, provided that ldconfig is invoked with the proper options when registering the libraries, so that the right hints are available, and the ports are built properly. If you are content to use 32-bit binaries on top of a 64-bit base system, try it out on a test case first -- some of the diagnostic environment variables listed in rtld(1) may help, along with ldd(1). It still seems like more trouble than it is worth to me, though. b. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
I want to compile some 32 bit ports on an AMD64 system. I know the GCC has to receive the -m32 flag to compile the ports as 32 bit, but I also want to change the install directory with 32 bit ports, I was wondering which would be the most appropriate root given unix themes and standard FreeBSD directory structure. /usr/local/[same-as-before]-32 (i.e. [...]/bin32, [...]/lib32,[...]/libexec32, etc) /usr/32/ /usr/32bit/ /usr/FreeBSD32/ /usr/compat/32bit/ /usr/compat/FreeBSD32/ Also, would I be able to pass the appropriate flags through port upgrade (or use environmental variables), or would I have to make duplicates of the port directories and modify the makefiles? Thanks, -Jim Stapleton First, I hope that you have a good reason for doing this, because it is going to be a PITA, and prone to all sorts of problems. The Ports collection is full of sloppy linking statements, and not all ports respect custom CFLAGS, PREFIX, and LOCALBASE settings, although they are supposed to do so. Other ports use a compiler that may not be able to produce 32-bit binaries on amd64. If you mean to build a large number of ports this way, you are better off either installing i386, or at least building and installing these ports in a jail, where you can more effectively isolate them from your 64-bit ports. Mixing and matching a large number of 32-bit and 64-bit ports will be a headache, especially if you have both 32- and 64-bit versions of some ports installed in parallel. If you are just talking about unshadowed ports, you could try adding something like the following to /etc/make.conf: .ifdef(32BIT) PREFIX=/usr/local/32 LOCALBASE=/usr/local/32 LDCONFIG=/sbin/ldconfig -32 CFLAGS+=-m32 .endif and then wrapping your portupgrade, portmaster, or make commands with: env 32BIT=yes [insert rest of command here] If there is shadowing, you have to consider how to manage the environment PATH so that you can actually run the 32-bit binary executables without typing in fully-qualified pathnames every time, and maybe also using LD_RUN_PATH or LD_LIBRARY_PATH so that the right libraries are used --- you can see how problems crop up at every turn? b. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
On Mon, Aug 31, 2009 at 09:28:44AM -0400, Jim wrote: I want to compile some 32 bit ports on an AMD64 system. I know the GCC has to receive the -m32 flag to compile the ports as 32 bit, but I also want to change the install directory with 32 bit ports, I was wondering which would be the most appropriate root given unix themes and standard FreeBSD directory structure. /usr/local/[same-as-before]-32 (i.e. [...]/bin32, [...]/lib32, [...]/libexec32, etc) The one above sounds most logical. The base system puts 32 bit libraries in /usr/lib32. Also, would I be able to pass the appropriate flags through port upgrade (or use environmental variables), or would I have to make duplicates of the port directories and modify the makefiles? Currently the ports system is not set up to handle multiple versions of the same port, AFAIK. You could ask on the ports list. You'd need a separate /var/db/pkg32 to store information about installed 32-bit ports. And /var/db/ports32 for options for 32-bit ports. You would also need changed port makefiles (different /ve/db paths, -m32 flag etc.) for this to work. It should be possible, but I don't think anybody has volunteered to do the work yet. :-) The consensus seems to be that if you _really_ need 32-bit apps that badly, you should use the i386 architecture. A lot of people run amd64 because the hardware supports it, not because they need* it. (* 'need' in this case means that you are regularly running out of address space on i386.) Roland -- R.F.Smith http://www.xs4all.nl/~rsmith/ [plain text _non-HTML_ PGP/GnuPG encrypted/signed email much appreciated] pgp: 1A2B 477F 9970 BA3C 2914 B7CE 1277 EFB0 C321 A725 (KeyID: C321A725) pgphgkG43tiI0.pgp Description: PGP signature
Re: 32 bit ports on an AMD64 system
Roland Smith rsm...@xs4all.nl writes: On Mon, Aug 31, 2009 at 09:28:44AM -0400, Jim wrote: I want to compile some 32 bit ports on an AMD64 system. I know the GCC has to receive the -m32 flag to compile the ports as 32 bit, but I also want to change the install directory with 32 bit ports, I was wondering which would be the most appropriate root given unix themes and standard FreeBSD directory structure. /usr/local/[same-as-before]-32 (i.e. [...]/bin32, [...]/lib32, [...]/libexec32, etc) The one above sounds most logical. The base system puts 32 bit libraries in /usr/lib32. Also, would I be able to pass the appropriate flags through port upgrade (or use environmental variables), or would I have to make duplicates of the port directories and modify the makefiles? Currently the ports system is not set up to handle multiple versions of the same port, AFAIK. You could ask on the ports list. You'd need a separate /var/db/pkg32 to store information about installed 32-bit ports. And /var/db/ports32 for options for 32-bit ports. You would also need changed port makefiles (different /ve/db paths, -m32 flag etc.) for this to work. It should be possible, but I don't think anybody has volunteered to do the work yet. :-) The consensus seems to be that if you _really_ need 32-bit apps that badly, you should use the i386 architecture. A lot of people run amd64 because the hardware supports it, not because they need* it. (* 'need' in this case means that you are regularly running out of address space on i386.) Also note that it is possible to have an i386 port-building jail on an amd64 system. -- Lowell Gilbert, embedded/networking software engineer, Boston area http://be-well.ilk.org/~lowell/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
Re: 32 bit ports on an AMD64 system
/usr/local/[same-as-before]-32 (i.e. [...]/bin32, [...]/lib32, [...]/libexec32, etc) The one above sounds most logical. The base system puts 32 bit libraries in /usr/lib32. It's too much trouble to append a 32 to every subdirectory of /usr/local/ -- I'd still recommend something like PREFIX=/usr/local/32, with all relative paths the same. Currently the ports system is not set up to handle multiple versions of the same port, AFAIK. You could ask on the ports list. Multiple versions of the same port with different PKGNAME s can be handled. Otherwise, no. You'd need a separate /var/db/pkg32 to store information about installed 32-bit ports. And /var/db/ports32 for options for 32-bit ports. You would also need changed port makefiles (different /ve/db paths, -m32 flag etc.) for this to work. It should be possible, but I don't think anybody has volunteered to do the work yet. :-) In many cases, the same port Makefile would suffice, using the make.conf additions I mentioned in my earlier message. In some cases, you may have to add patches to the port or change the port Makefile. The point about the package registration is a good one. It would certainly be easier to maintain two separate package databases. To do so, and to also have different OPTIONS-handling for the 32-bit ports via separate ports databases, then you could simply add: PKG_DBDIR=/var/db/pkg32 PORT_DBDIR=/var/db/ports32 to the make.conf block in my last message. However, I think it would be better to instead maintain unified package and port databases and somehow differentiate between the entries for 32-bit and 64-bit packages, by either tinkering with the fake-pkg target in bsd.port.mk, or, preferably, by changing the PKGNAME and UNIQUENAME. The simplest way to do this would be to add a block like .ifdef(32BIT) PKGNAMEPREFIX+=32 .endif just before the definitions of PKGNAME and UNIQUENAME in bsd.port.mk. ... (* 'need' in this case means that you are regularly running out of address space on i386.) Or if you need improved performance from a 64-bit app, etc. As the I and the others have mentioned, a jail or a separate 32-bit installation would probably be better. b. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to freebsd-questions-unsubscr...@freebsd.org
