RE: [opendx-dev] compile on 2cpu amd_64; rhel3
Dear All, Unfortunately, a reinstall is not an option for our machine, so taking Lloyd's findings to heart in that a wait for a new NVidia driver is necessary we are currently using the quantian version of the Knoppix line DVD (http://dirk.eddelbuettel.com/quantian.html) to dual boot the machine. DX on the latest release of quantian (0.6.9.1) is working fine, but of course has the 32 bit speed reductions Lloyd mentions. Of course, it reads and writes to the hard drives OK. So once loaded and having DX running isn't to slow. Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Lloyd A Treinish Sent: 25 October 2004 05:15 To: opendx2-dev@lists.berlios.de Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 Here's a further update, for everyone's reference. The 64-bit version of the nvidia driver is definitely broken. But there are some workarounds for now. As a test, I installed the 32-bit version of RHEL-WS3.0 with the latest version of the standard x86 version of the nvidia driver. The RH9 4.3.2 rpm on the web site installed right away and everything works. Compared to the version I built from the July 30 4.3.3 tarball for the 64-bit version of RHEL, various modules run anywhere from 10% to 50% slower with the 32-bit RHEL. Nevertheless, the machine is still pretty fast. For anyone else that ends of doing this as a workaround, there is a known 32-bit RedHat issue on some Opteron machines if they have more than 4GB of RAM because the area of memory reserved for caching the AGP aperture ends up above that space, so AGPGART ends up with no memory and video performance disappears. The workaround is restricting the memory with 'mem=4095M' as a kernel option in the grub menu. Obviously, any additional memory is wasted, but the 32-bit kernel wouldn't be addressing it anyway... |-+-- | | Lloyd A| | | Treinish/Watson/[EMAIL PROTECTED] | | | Sent by: | | | [EMAIL PROTECTED]| | | son.ibm.com| | | | | | | | | 10/19/2004 02:26 PM| | | Please respond to | | | opendx-dev | | | | |-+-- --- -| | | | To: opendx2-dev@lists.berlios.de | | cc: | | Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 | | | --- -| Here's an update, for everyone's reference. I have been able to successfully make a few different builds with the July 30 4.3.3 tarball for the Opteron RHEL-WS3.0 system. The first was with large-arena and SMP support: ./configure --with-netcdf --with-large-arenas --enable-dependency-tracking --enable-smp-linux --includedir=/usr/include:/usr/local/include --libdir=/usr/lib64:/usr/local/lib --x-libraries=/usr/X11R6/lib64 --x-includes=/usr/X11R6/include The second was with small arenas and no SMP support. In both cases, I built a version of the libnetcdf for this platform. ./configure --with-netcdf --enable-dependency-tracking --includedir=/usr/include:/usr/local/include --libdir=/usr/lib64:/usr/local/lib --x-libraries=/usr/X11R6/lib64 --x-includes=/usr/X11R6/include Unfortunately, dxexec from the SMP build hangs when two cpus are requested. With p1, it works fine. In addition, requests for memory beyond 2GB remained clipped at 2 GB. Both versions appear to perform similarly when running with a single cpu and less than 2GB of memory (pretty fast). The hardware rendering problem is the same in both versions, which suggests problems from nvidia. I tried building with and without pointing specifically to the gl includes from nvidia, although it didn't seem to matter. I have also sent an e-mail to nvidia's support group, but I have not received a response yet. Lloyd A Treinish/Watson/[EMAIL PROTECTED] To: Sent by:opendx2-dev@lists.berlios.de [EMAIL PROTECTED] cc: ibm.com Subject:Re: [opendx-dev] compile on 2cpu amd_64; rhel3 10/17/2004 07:07 PM Please respond to opendx-dev I figured that was the situation. I have not had the chance to look into it in much more detail. I just booted the machine for the first time on Friday and tried the few things that I cited in my posting. Unfortunately, I don't have the time to put a lot of effort into this. So, I'll do
Re: [opendx-dev] compile on 2cpu amd_64; rhel3
Here's a further update, for everyone's reference. The 64-bit version of the nvidia driver is definitely broken. But there are some workarounds for now. As a test, I installed the 32-bit version of RHEL-WS3.0 with the latest version of the standard x86 version of the nvidia driver. The RH9 4.3.2 rpm on the web site installed right away and everything works. Compared to the version I built from the July 30 4.3.3 tarball for the 64-bit version of RHEL, various modules run anywhere from 10% to 50% slower with the 32-bit RHEL. Nevertheless, the machine is still pretty fast. For anyone else that ends of doing this as a workaround, there is a known 32-bit RedHat issue on some Opteron machines if they have more than 4GB of RAM because the area of memory reserved for caching the AGP aperture ends up above that space, so AGPGART ends up with no memory and video performance disappears. The workaround is restricting the memory with 'mem=4095M' as a kernel option in the grub menu. Obviously, any additional memory is wasted, but the 32-bit kernel wouldn't be addressing it anyway... |-+-- | | Lloyd A| | | Treinish/Watson/[EMAIL PROTECTED] | | | Sent by: | | | [EMAIL PROTECTED]| | | son.ibm.com| | | | | | | | | 10/19/2004 02:26 PM| | | Please respond to | | | opendx-dev | | | | |-+-- | | | | To: opendx2-dev@lists.berlios.de | | cc: | | Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 | | | | Here's an update, for everyone's reference. I have been able to successfully make a few different builds with the July 30 4.3.3 tarball for the Opteron RHEL-WS3.0 system. The first was with large-arena and SMP support: ./configure --with-netcdf --with-large-arenas --enable-dependency-tracking --enable-smp-linux --includedir=/usr/include:/usr/local/include --libdir=/usr/lib64:/usr/local/lib --x-libraries=/usr/X11R6/lib64 --x-includes=/usr/X11R6/include The second was with small arenas and no SMP support. In both cases, I built a version of the libnetcdf for this platform. ./configure --with-netcdf --enable-dependency-tracking --includedir=/usr/include:/usr/local/include --libdir=/usr/lib64:/usr/local/lib --x-libraries=/usr/X11R6/lib64 --x-includes=/usr/X11R6/include Unfortunately, dxexec from the SMP build hangs when two cpus are requested. With p1, it works fine. In addition, requests for memory beyond 2GB remained clipped at 2 GB. Both versions appear to perform similarly when running with a single cpu and less than 2GB of memory (pretty fast). The hardware rendering problem is the same in both versions, which suggests problems from nvidia. I tried building with and without pointing specifically to the gl includes from nvidia, although it didn't seem to matter. I have also sent an e-mail to nvidia's support group, but I have not received a response yet. Lloyd A Treinish/Watson/[EMAIL PROTECTED] To: Sent by:opendx2-dev@lists.berlios.de [EMAIL PROTECTED] cc: ibm.com Subject:Re: [opendx-dev] compile on 2cpu amd_64; rhel3 10/17/2004 07:07 PM Please respond to opendx-dev I figured that was the situation. I have not had the chance to look into it in much more detail. I just booted the machine for the first time on Friday and tried the few things that I cited in my posting. Unfortunately, I don't have the time to put
Re: [opendx-dev] compile on 2cpu amd_64; rhel3
Here's an update, for everyone's reference. I have been able to successfully make a few different builds with the July 30 4.3.3 tarball for the Opteron RHEL-WS3.0 system. The first was with large-arena and SMP support: ./configure --with-netcdf --with-large-arenas --enable-dependency-tracking --enable-smp-linux --includedir=/usr/include:/usr/local/include --libdir=/usr/lib64:/usr/local/lib --x-libraries=/usr/X11R6/lib64 --x-includes=/usr/X11R6/include The second was with small arenas and no SMP support. In both cases, I built a version of the libnetcdf for this platform. ./configure --with-netcdf --enable-dependency-tracking --includedir=/usr/include:/usr/local/include --libdir=/usr/lib64:/usr/local/lib --x-libraries=/usr/X11R6/lib64 --x-includes=/usr/X11R6/include Unfortunately, dxexec from the SMP build hangs when two cpus are requested. With p1, it works fine. In addition, requests for memory beyond 2GB remained clipped at 2 GB. Both versions appear to perform similarly when running with a single cpu and less than 2GB of memory (pretty fast). The hardware rendering problem is the same in both versions, which suggests problems from nvidia. I tried building with and without pointing specifically to the gl includes from nvidia, although it didn't seem to matter. I have also sent an e-mail to nvidia's support group, but I have not received a response yet. Lloyd A Treinish/Watson/[EMAIL PROTECTED] Sent by: [EMAIL PROTECTED] 10/17/2004 07:07 PM Please respond to opendx-dev To: opendx2-dev@lists.berlios.de cc: Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 I figured that was the situation. I have not had the chance to look into it in much more detail. I just booted the machine for the first time on Friday and tried the few things that I cited in my posting. Unfortunately, I don't have the time to put a lot of effort into this. So, I'll do some more experimentation and report any useful findings. However, I see the most serious problem as being with the OpenGL drivers, for which I don't expect an immediate solution. Admittedly, I don't have a lot of experience on 64-bit platforms. I have found that under AIX I didn't have a lot of these problems. On the other hand, I only use those systems for production visualization with software rendering. |-+-- | | David Thompson | | | [EMAIL PROTECTED]| | | Sent by: | | | [EMAIL PROTECTED]| | | son.ibm.com | | | | | | | | | 10/17/2004 05:40 PM| | | Please respond to | | | opendx-dev | | | | |-+-- ---| | | |To:opendx2-dev@lists.berlios.de | |cc: | |Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 | | | ---| With linux and shared libraries, you need to use ldconfig to help the os find them. As for the library incompatibility, this is going to be a 64 bit OS library versus a 32 bit OS library. The libtool guys have been discussing how to handle some of this--but they still have not come to any kind of consensus. As of now, no one has come forward to start working on any of the 64 bit os issues beyond the SGI. So unless someone is willing to undertake this or pay someone (like myself) to undertake it--I wish you luck. David Coincidently, I just got a similar box as a loaner for a couple of weeks and I am seeing similar problems. It's a dual 2.2GHz AMD Opteron system running RHEL-WS3.0 for x86_64 with an SMP kernel (2.4.21-20.ELsmp). It has gcc v3.2.3 (from Red Hat Linux 3.2.3-42). The system has an nVidia Quadro FX3000 with the latest driver from nvidia.com for this architecture. I started by using the latest tarball (July 30, 2004) for OpenDX 4.3.3. My initial attempt was to make a build for large arena and SMP enabled as well as with netCDF. I had to build a netCDF library for this platform, but it seems to work ok. Anyway, the dx build creates an executable but reports similar loader problems: /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXpm.so when searching for -lXpm /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXpm.a when searching for -lXpm /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXp.so when searching for -lXp /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXp.a when searching for -lXp /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXmu.so when searching
Re: [opendx-dev] compile on 2cpu amd_64; rhel3
I figured that was the situation. I have not had the chance to look into it in much more detail. I just booted the machine for the first time on Friday and tried the few things that I cited in my posting. Unfortunately, I don't have the time to put a lot of effort into this. So, I'll do some more experimentation and report any useful findings. However, I see the most serious problem as being with the OpenGL drivers, for which I don't expect an immediate solution. Admittedly, I don't have a lot of experience on 64-bit platforms. I have found that under AIX I didn't have a lot of these problems. On the other hand, I only use those systems for production visualization with software rendering. |-+-- | | David Thompson | | | [EMAIL PROTECTED]| | | Sent by: | | | [EMAIL PROTECTED]| | | son.ibm.com| | | | | | | | | 10/17/2004 05:40 PM| | | Please respond to | | | opendx-dev | | | | |-+-- ---| | | | To: opendx2-dev@lists.berlios.de | | cc: | | Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 | | | ---| With linux and shared libraries, you need to use ldconfig to help the os find them. As for the library incompatibility, this is going to be a 64 bit OS library versus a 32 bit OS library. The libtool guys have been discussing how to handle some of this--but they still have not come to any kind of consensus. As of now, no one has come forward to start working on any of the 64 bit os issues beyond the SGI. So unless someone is willing to undertake this or pay someone (like myself) to undertake it--I wish you luck. David Coincidently, I just got a similar box as a loaner for a couple of weeks and I am seeing similar problems. It's a dual 2.2GHz AMD Opteron system running RHEL-WS3.0 for x86_64 with an SMP kernel (2.4.21-20.ELsmp). It has gcc v3.2.3 (from Red Hat Linux 3.2.3-42). The system has an nVidia Quadro FX3000 with the latest driver from nvidia.com for this architecture. I started by using the latest tarball (July 30, 2004) for OpenDX 4.3.3. My initial attempt was to make a build for large arena and SMP enabled as well as with netCDF. I had to build a netCDF library for this platform, but it seems to work ok. Anyway, the dx build creates an executable but reports similar loader problems: /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXpm.so when searching for -lXpm /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXpm.a when searching for -lXpm /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXp.so when searching for -lXp /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXp.a when searching for -lXp /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXmu.so when searching for -lXmu /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXmu.a when searching for -lXmu /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXext.so when searching for -lXext /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXext.a when searching for -lXext /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXt.so when searching for -lXt /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXt.a when searching for -lXt /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libX11.so when searching for -lX11 /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libX11.a when searching for -lX11 /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libSM.so when searching for -lSM /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libSM.a when searching for -lSM /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libICE.so when searching for -lICE /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libICE.a when searching for -lICE I've noticed that this occurs whether it's pointing to /usr/X11R6/lib or /usr/X11R6/lib64. I was going to experiment with other building options
Re: [opendx-dev] compile on 2cpu amd_64; rhel3
Coincidently, I just got a similar box as a loaner for a couple of weeks and I am seeing similar problems. It's a dual 2.2GHz AMD Opteron system running RHEL-WS3.0 for x86_64 with an SMP kernel (2.4.21-20.ELsmp). It has gcc v3.2.3 (from Red Hat Linux 3.2.3-42). The system has an nVidia Quadro FX3000 with the latest driver from nvidia.com for this architecture. I started by using the latest tarball (July 30, 2004) for OpenDX 4.3.3. My initial attempt was to make a build for large arena and SMP enabled as well as with netCDF. I had to build a netCDF library for this platform, but it seems to work ok. Anyway, the dx build creates an executable but reports similar loader problems: /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXpm.so when searching for -lXpm /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXpm.a when searching for -lXpm /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXp.so when searching for -lXp /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXp.a when searching for -lXp /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXmu.so when searching for -lXmu /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXmu.a when searching for -lXmu /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXext.so when searching for -lXext /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXext.a when searching for -lXext /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXt.so when searching for -lXt /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libXt.a when searching for -lXt /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libX11.so when searching for -lX11 /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libX11.a when searching for -lX11 /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libSM.so when searching for -lSM /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libSM.a when searching for -lSM /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libICE.so when searching for -lICE /usr/bin/ld: skipping incompatible /usr/X11R6/lib/libICE.a when searching for -lICE I've noticed that this occurs whether it's pointing to /usr/X11R6/lib or /usr/X11R6/lib64. I was going to experiment with other building options (with and without large arenas, with and without SMP support), although I'll doubt it will matter since it seems to be just a ui problem. I have also seen problems with hardware rendering. A few from the sample set seem to work fine (i.e, changing rendering to hardware mode first or others like TextureMap). When I run something more complex, some screen objects are visible (e.g., Captions) while others are not (e.g., labels on color bars). When I rotate a 3d object, the surfaces will disappear as if an opaque triangle colored with the background color is draw in front, which grows with further rotation. The problem is repeatable. Software rendering seems ok, but I've not done exhaustive testing. My initial guess as to the rendering (not the loader) problem is with the driver from nVidia (see http://www.nvnews.net/vbulletin/showthread.php?s=1a67d4b4573fcbe9d9001c1ef411a486t=34005, for example). As an alternative, I also tried to install the RH9 4.3.3 rpm, which works just fine on RHEL-WS3 Intel boxes. At installation it complains about libxml and libpng being missing, although are there, and LD_LIBRARY_PATH and LIBPATH point to the right place. I can use --force and --nodeps to get beyond that. None of the executables will run, complaining about not being able to fine libpng even with the environment variables set correctly. I am also open to any suggestions on either of these issues as I continue to look into them. Thanks. |-+-- | | Peter Connolly | | | [EMAIL PROTECTED]| | | lsruhe.de | | | Sent by: | | | [EMAIL PROTECTED]| | | son.ibm.com| | | | | | | | | 10/13/2004 04:27 PM| | | Please respond to | | | opendx-dev | | | | |-+-- ---| | | | To: opendx2-dev@lists.berlios.de | | cc: | | Subject: [opendx-dev] compile on 2cpu amd_64; rhel3 | |
RE: [opendx-dev] compile on 2cpu amd_64; rhel3
Hi David That is much better. It now runs, but only with software rendering and I get some consistent, errors with no data type from colormap. I will investigate more next week Thanks for the help Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Thompson Sent: 14 October 2004 00:50 To: opendx2-dev@lists.berlios.de Subject: RE: [opendx-dev] compile on 2cpu amd_64; rhel3 Try setting it to 24 and see if your problem clears up. 16 bit has been problematic (although we do have some fixes that are in CVS for 16 bit). David Hi David It is set to 16 in /etc/X11/XF86Config and that is what /var/log/XFree86.0.log says it loads Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Thompson Sent: 13 October 2004 23:09 To: opendx2-dev@lists.berlios.de Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 What is X's bit depth set to. We have seen issues with bit depth problems. David -- ... . . David L. Thompson Visualization and Imagery Solutions, Inc. mailto:[EMAIL PROTECTED]5515 Skyway Drive, Missoula, MT 59804 Phone : (406)756-7472 -- . David L. Thompson Visualization and Imagery Solutions, Inc. mailto:[EMAIL PROTECTED]5515 Skyway Drive, Missoula, MT 59804 Phone : (406)756-7472
Re: [opendx-dev] compile on 2cpu amd_64; rhel3
What is X's bit depth set to. We have seen issues with bit depth problems. David -- . David L. Thompson Visualization and Imagery Solutions, Inc. mailto:[EMAIL PROTECTED]5515 Skyway Drive, Missoula, MT 59804 Phone : (406)756-7472
RE: [opendx-dev] compile on 2cpu amd_64; rhel3
Hi David It is set to 16 in /etc/X11/XF86Config and that is what /var/log/XFree86.0.log says it loads Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Thompson Sent: 13 October 2004 23:09 To: opendx2-dev@lists.berlios.de Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 What is X's bit depth set to. We have seen issues with bit depth problems. David -- . David L. Thompson Visualization and Imagery Solutions, Inc. mailto:[EMAIL PROTECTED]5515 Skyway Drive, Missoula, MT 59804 Phone : (406)756-7472
RE: [opendx-dev] compile on 2cpu amd_64; rhel3
Try setting it to 24 and see if your problem clears up. 16 bit has been problematic (although we do have some fixes that are in CVS for 16 bit). David Hi David It is set to 16 in /etc/X11/XF86Config and that is what /var/log/XFree86.0.log says it loads Peter -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Thompson Sent: 13 October 2004 23:09 To: opendx2-dev@lists.berlios.de Subject: Re: [opendx-dev] compile on 2cpu amd_64; rhel3 What is X's bit depth set to. We have seen issues with bit depth problems. David -- . David L. Thompson Visualization and Imagery Solutions, Inc. mailto:[EMAIL PROTECTED]5515 Skyway Drive, Missoula, MT 59804 Phone : (406)756-7472 -- . David L. Thompson Visualization and Imagery Solutions, Inc. mailto:[EMAIL PROTECTED]5515 Skyway Drive, Missoula, MT 59804 Phone : (406)756-7472
