Re: Various performance problems
guy keren wrote: On Mon, 15 Dec 2003, Shachar Shemesh wrote: I have a (couple of) client(s) that have performance problems across linux distirbutions. That is - they try an operation on a given platform with a given (redhat both cases) distribution, and get a certain performance. Then they try the same operation on the same hardware with a newer distribution, and get considerably lower performance. In one case, the fast distro is RedHat 7.2, and the slow one is 9. In another I'm not sure what the fast one is, but the slow on is AS 3. I suspect that RedHat screwed something up with their newer kernels (perhaps something incorrectly backported from 2.6?). people have such short memories: http://www.mail-archive.com/[EMAIL PROTECTED]/msg30749.html (summary - nadav harel, checks why hspell on redhat 9 runs several times slower then on redhat 7.something, and finds that changing the LD_LIBRARY_PATH to use the non-tls libraries solves the problem). That's a very interesting thread I'm wondering -- would I gain a performance boost by making my RH9 always use /lib/i686 instead of /lib/tls? And another question, for the uninitiated (that's me) -- when I try to output $LD_LIBRARY_PATH I get nothing. How can I find out what are the default libraries used by the system? Alon. = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
Alon Weinstein [EMAIL PROTECTED] writes: And another question, for the uninitiated (that's me) -- when I try to output $LD_LIBRARY_PATH I get nothing. How can I find out what are the default libraries used by the system? cat /etc/ld.so.conf See also man 8 ldconfig man 8 ld.so -- Oleg Goldshmidt | [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, Dec 16, 2003 at 09:21:27AM +0200, Alon Weinstein wrote: That's a very interesting thread I'm wondering -- would I gain a performance boost by making my RH9 always use /lib/i686 instead of /lib/tls? Possibly, due to a bug in RH 9's NPTL implementation, IIRC. And another question, for the uninitiated (that's me) -- when I try to output $LD_LIBRARY_PATH I get nothing. How can I find out what are the default libraries used by the system? cat /etc/ld.so.conf, assuming it hasn't been changed since the last time someone ran ldconfig. Cheers, Muli -- Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/ the nucleus of linux oscillates my world - [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Various performance problems
On Tue, Dec 16, 2003 at 09:21:27AM +0200, Alon Weinstein wrote: [snip] And another question, for the uninitiated (that's me) -- when I try to output $LD_LIBRARY_PATH I get nothing. How can I find out what are the default libraries used by the system? RTFM - man ld.so . Basically /etc/ld.so.conf, but somethings (ld.so, libc? I don't know) have other means. E.g. /lib/tls is used but not mentioned either in the file of in the manpage. So I would like more informed people to add (or I will search myself eventually). -- Didi Alon. = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
Muli Ben-Yehuda wrote: On Tue, Dec 16, 2003 at 09:21:27AM +0200, Alon Weinstein wrote: cat /etc/ld.so.conf, assuming it hasn't been changed since the last time someone ran ldconfig. Looking at /etc/ld.so.conf: /usr/kerberos/lib /usr/X11R6/lib /usr/lib/qt-3.1/lib /opt/opengroupware.org/Libraries/ix86/linux-gnu/gnu-fd-nil /opt/skyrix/system/Libraries/ix86/linux-gnu/gnu-fd-nil /usr/lib/sane /usr/lib/mysql No tls or i686... greping for either 'tls' or 'i686' in etc/sysconfig or etc/rc.d (I'm just reaching here, not really sure where or if I should be looking) got me nothing. And my questions are (drum roll) : How can I tell which is the default library used? (or is it fixed that RH9 uses tls?) How can I change that default? (perhaps using an ugly hack -- ln -s /lib/i686 /lib/tls ? ;) ) Alon. = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, Dec 16, 2003, Alon Weinstein wrote about Re: Various performance problems: http://www.mail-archive.com/[EMAIL PROTECTED]/msg30749.html (summary - nadav harel, checks why hspell on redhat 9 runs several times slower then on redhat 7.something, and finds that changing the LD_LIBRARY_PATH to use the non-tls libraries solves the problem). That's a very interesting thread I'm wondering -- would I gain a performance boost by making my RH9 always use /lib/i686 instead of /lib/tls? I am not sure. Some programs I measured did not exhibit any speedup by using /lib/i686. Apparently (but this is just a guess because I unfortunately did not have the time to follow this up) there is a specific library call or several specific library calls that are extraordinary slow in the tls library, and this slows down programs that use them, but doesn't affect programs which only use these functions rarely. Also, I'm guessing that TLS (thread local storage, NOT transport layer security) is not utter garbage, and is used for *something* - maybe it actually speeds up, not slows down, certain types of programs - presumably programs which use threads in some manner (don't ask me how or why...). And another question, for the uninitiated (that's me) -- when I try to output $LD_LIBRARY_PATH I get nothing. How can I find out what are the default libraries used by the system? Check out /etc/ld.so.conf. Unfortunately, this doesn't tell the whole story, because the tls, i686, i386 variant directory don't appear them. That sort of black-magic (choosing the correct variant) is done somehow by the dynamic linker, in a manner I never fully understood (and never saw documented). -- Nadav Har'El| Tuesday, Dec 16 2003, 21 Kislev 5764 [EMAIL PROTECTED] |- Phone: +972-53-790466, ICQ 13349191 |echo '[q]sa[ln0=aln256%Pln256/snlbx] http://nadav.harel.org.il |sb3135071790101768542287578439snlbxq'|dc = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
guy keren [EMAIL PROTECTED] writes: people have such short memories: http://www.mail-archive.com/[EMAIL PROTECTED]/msg30749.html From memory (thanks for reminding, Guy), someone (Mulix?) found this (I just found it on my own, but it does ring a bell) http://kerneltrap.org/comment/reply/1574 Note that Ulrich Drepper says there that Fedora Core 1 and RHEL3 should not have the problem. Shachar says that RHAS3 is slow - question is, whether or not that is the same kernel as RHEL3 uses, and which version Ulrich actually meant (he raises the point of kernel version numbers, then provides benchmarks without any indication of versions...). So in the 7.2 vs 9 case, can the client grab the kernel RPM from Fedora Core 1, rpm -ivh it reboot, and compare again? -- Oleg Goldshmidt | [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
Perhaps you can try http://www.linuxtested.com/linux_tools.html -EK - Original Message - From: Oleg Goldshmidt [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Tuesday, December 16, 2003 1:26 PM Subject: Re: Various performance problems Oleg Goldshmidt [EMAIL PROTECTED] writes: http://kerneltrap.org/comment/reply/1574 Note that Ulrich Drepper says there that Fedora Core 1 and RHEL3 should not have the problem. Shachar says that RHAS3 is slow - question is, whether or not that is the same kernel as RHEL3 uses, and which version Ulrich actually meant (he raises the point of kernel version numbers, then provides benchmarks without any indication of versions...). So in the 7.2 vs 9 case, can the client grab the kernel RPM from Fedora Core 1, rpm -ivh it reboot, and compare again? Eh, it might be more involved. Look at the last posting from Jakub Jelinek in the URL above: Ulrich meant glibc CVS HEAD. Makes sense: NPTL is split between the kernel and glibc. Jakub is, IIRC, the glibc maintainer at Red Hat. The posting is from Nov 5, the latest glibc erratum is from Nov 13, https://rhn.redhat.com/errata/RHSA-2003-325.html It is not clear from the description if this includes the locking patch. I don't know how dangerous it is to upgrade glibc on RH9 to Fedora's version. I would not do it on a mission-critical machine. -- Oleg Goldshmidt | [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On 16 Dec 2003, Oleg Goldshmidt wrote: guy keren [EMAIL PROTECTED] writes: people have such short memories: http://www.mail-archive.com/[EMAIL PROTECTED]/msg30749.html From memory (thanks for reminding, Guy), someone (Mulix?) found this (I just found it on my own, but it does ring a bell) http://kerneltrap.org/comment/reply/1574 Note that Ulrich Drepper says there that Fedora Core 1 and RHEL3 should not have the problem. that's not what he was writing. he said that the benchmark should be perfrmed on one of those systems. he didn't say these systems contained the NPTL code from CVS head. get yourself one of those systems, and test. Shachar says that RHAS3 is slow - question is, whether or not that is the same kernel as RHEL3 uses, and which version Ulrich actually meant (he raises the point of kernel version numbers, then provides benchmarks without any indication of versions...). So in the 7.2 vs 9 case, can the client grab the kernel RPM from Fedora Core 1, rpm -ivh it reboot, and compare again? it's not a kernel issue - it's a libraries issue. do read that post again, to see the light ;) and please DON'T suggest that people take binaries frm one distro and install on another - some people on this list will not notice your question mark, thinkthis is the way to go, and find their systems unable to boot any longer. people tend to read what they want see :0 -- guy For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On 16 Dec 2003, Oleg Goldshmidt wrote: Oleg Goldshmidt [EMAIL PROTECTED] writes: http://kerneltrap.org/comment/reply/1574 Note that Ulrich Drepper says there that Fedora Core 1 and RHEL3 should not have the problem. Shachar says that RHAS3 is slow - question is, whether or not that is the same kernel as RHEL3 uses, and which version Ulrich actually meant (he raises the point of kernel version numbers, then provides benchmarks without any indication of versions...). So in the 7.2 vs 9 case, can the client grab the kernel RPM from Fedora Core 1, rpm -ivh it reboot, and compare again? Eh, it might be more involved. Look at the last posting from Jakub Jelinek in the URL above: Ulrich meant glibc CVS HEAD. Makes sense: NPTL is split between the kernel and glibc. Jakub is, IIRC, the glibc maintainer at Red Hat. The posting is from Nov 5, the latest glibc erratum is from Nov 13, https://rhn.redhat.com/errata/RHSA-2003-325.html It is not clear from the description if this includes the locking patch. I don't know how dangerous it is to upgrade glibc on RH9 to Fedora's version. I would not do it on a mission-critical machine. and i would not do it on _any_ system - unless i am realy bored, and want to see what if at work ;) it's fix-able - but what will the user gain from this excersize? -- guy For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, 16 Dec 2003, Aaron wrote: I in fact noticed in RH 9.0 hspell and aspell were locking up my system. I was running a lowlatency kernel but found that jack and ardour were slow. I am now running Fedora, in kde and gnome window minimize slowly and maximize slowly. Programs lockup and apt-get takes over to the point where my mouse pointer dissapears for a time. I am wondering if this is just a RH issue or Mandrake and friends also have this problem?? did you _read_ the mentioned post(s)? they give you a direction regarding what to try doing. try running the problematic program using: LD_ASSUME_KERNEL=2.4.1 program parameters... read the posts for the exact command - i might be typing it wrong here, and i have no RH9 to test it on. -- guy For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
guy keren wrote: did you _read_ the mentioned post(s)? they give you a direction regarding what to try doing. try running the problematic program using: LD_ASSUME_KERNEL=2.4.1 program parameters... read the posts for the exact command - i might be typing it wrong here, and i have no RH9 to test it on. I think the main part was LD_LIBRARY_PATH=., where . presumably contains the latest compiled CVS of libc. Anyway, it was bleeding-edge developer discussion, not something I'd try with libc on any system unless it was a system I didn't care to reformat and re-install afterwords. --Amos = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
guy keren [EMAIL PROTECTED] writes: that's not what he was writing. he said that the benchmark should be perfrmed on one of those systems. he didn't say these systems contained the NPTL code from CVS head. get yourself one of those systems, and test. Sorry, misattributed. Linus did test Fedora and responded to Ulrich ;-) So in the 7.2 vs 9 case, can the client grab the kernel RPM from Fedora Core 1, rpm -ivh it reboot, and compare again? it's not a kernel issue - it's a libraries issue. do read that post again, to see the light ;) I did correct myself in a followup, didn't I? ;-) and please DON'T suggest that people take binaries frm one distro and install on another - some people on this list will not notice your question mark, thinkthis is the way to go, and find their systems unable to boot any longer. people tend to read what they want see :0 The rpm -ivh kernel*.i686.rpm procedure on redhat usually works for me. There is little danger that the system won't boot any longer - the old kernel stays there. Note (you did, this is for the benefit of some people on this list only) that I merely asked if it was feasible in one case out of Shachar's two: the one that does not involve an enterprize-class server. Glibc is trickier, as I mentioned in a followup to myself. Don't try that at work... -- Oleg Goldshmidt | [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, Dec 16, 2003 at 11:42:49AM +0200, Nadav Har'El wrote: I'm guessing that TLS (thread local storage, NOT transport layer security) Is there any work to remove this name clash? -- If you have an apple and I have an apple and we exchange apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas. -- George Bernard Shaw (sent by shaulk @ actcom . net . il) = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, Dec 16, 2003, Shaul Karl wrote about Re: Various performance problems: On Tue, Dec 16, 2003 at 11:42:49AM +0200, Nadav Har'El wrote: I'm guessing that TLS (thread local storage, NOT transport layer security) Is there any work to remove this name clash? Yes, the Thread-Local-Storage people were annoyed by this clash, and decided to change their name. The new name they came up was Storing Stuff Locally, or SSL for short. An no, if any one was wondering, the previous paragraph was NOT serious. But the question didn't sound very serious either :) -- Nadav Har'El| Tuesday, Dec 16 2003, 22 Kislev 5764 [EMAIL PROTECTED] |- Phone: +972-53-790466, ICQ 13349191 |Strike not only while the iron is hot, http://nadav.harel.org.il |make the iron hot by striking it. = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, 16 Dec 2003 [EMAIL PROTECTED] wrote: guy keren wrote: did you _read_ the mentioned post(s)? they give you a direction regarding what to try doing. try running the problematic program using: LD_ASSUME_KERNEL=2.4.1 program parameters... read the posts for the exact command - i might be typing it wrong here, and i have no RH9 to test it on. I think the main part was LD_LIBRARY_PATH=., where . presumably contains the latest compiled CVS of libc. for those who did not yet get the clue, please look at the following page: http://www.talkaboutprogramming.com/group/comp.programming.threads/messages/39627.html look at this question, and the first immediate answer (http://www.talkaboutprogramming.com/group/comp.programming.threads/messages/39634.html) Anyway, it was bleeding-edge developer discussion, not something I'd try with libc on any system unless it was a system I didn't care to reformat and re-install afterwords. which is precisely why LD_ASSUME_KERNEL=2.4.1 will _avoid_ using the NPTL threading library (= bleeding edge, very bloody), and cause the dynamic-loader to use the linux-threads library (=older, safer, better tested, good for the user). do yourself a favour, and re-read that page. you _will_ see the light ;) -- guy refusal to thoroughly read information, is futile For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, 16 Dec 2003, Oleg Goldshmidt wrote: guy keren [EMAIL PROTECTED] writes: that's not what he was writing. he said that the benchmark should be perfrmed on one of those systems. he didn't say these systems contained the NPTL code from CVS head. get yourself one of those systems, and test. Sorry, misattributed. Linus did test Fedora and responded to Ulrich ;-) what linus did test, was that on the system he was using, _avoiding_ the use of NPTL (via the LD_ASSUME_KERNEL=2.4.1 hack) solved the problem. linus was merely interested in making sure this was NOT a linux kernel problem - beacause the linux kernel is his concern, while the NPTL code is somebody else's concern. after he isolated the problem and showed it was not a kernel issue, his work was done. So in the 7.2 vs 9 case, can the client grab the kernel RPM from Fedora Core 1, rpm -ivh it reboot, and compare again? it's not a kernel issue - it's a libraries issue. do read that post again, to see the light ;) I did correct myself in a followup, didn't I? ;-) indeed, but it was important to make sure people don't try doing what your first post suggested, which is why i sent two replies - one for each letter. and please DON'T suggest that people take binaries frm one distro and install on another - some people on this list will notnotice your question mark, thinkthis is the way to go, and find their systems unable to boot any longer. people tend to read what they want see :0 The rpm -ivh kernel*.i686.rpm procedure on redhat usually works for me. ofcourse, when you take the kernel for the same distro for which it is intended, then it should be ok. but taking a kernel that was meant for a _different_ distro, is a bad idea. There is little danger that the system won't boot any longer - the old kernel stays there. unless the new kernel is so incompatible, that it could break the disk's contents, in which case it could cause unfixable problems (that is, unfixable, unless you make a re-install). Glibc is trickier, as I mentioned in a followup to myself. Don't try that at work... but do try the LD_ASSUME_KERNEL=2.4.1 method. it causes the system to switch to an _older_ version of the threading library, which does work. -- guy For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, 16 Dec 2003, Alon Weinstein wrote: Muli Ben-Yehuda wrote: On Tue, Dec 16, 2003 at 09:21:27AM +0200, Alon Weinstein wrote: cat /etc/ld.so.conf, assuming it hasn't been changed since the last time someone ran ldconfig. Looking at /etc/ld.so.conf: /usr/kerberos/lib /usr/X11R6/lib /usr/lib/qt-3.1/lib /opt/opengroupware.org/Libraries/ix86/linux-gnu/gnu-fd-nil /opt/skyrix/system/Libraries/ix86/linux-gnu/gnu-fd-nil /usr/lib/sane /usr/lib/mysql No tls or i686... greping for either 'tls' or 'i686' in etc/sysconfig or etc/rc.d (I'm just reaching here, not really sure where or if I should be looking) got me nothing. and in case you missed the previous posts for some reason: you can not know. they made some stupid(?!!?) hacks in glibc, makint it imposible for you to know, as a user, which version of the library is actually in use, other then by writing test programs, stracing programs, etc. so do try running your program with: LD_ASSUME_KERNEL=2.4.1 program parameters... and see what you come up with. How can Ichange that default? (perhaps using an ugly hack -- ln -s /lib/i686 /lib/tls ? ;) ) not sure if this will work - it _could_ be that the dynamic loader expects to find certain symbols (=functions, variables) in the tls libraries, which it does not expect to find in the non-tls libraries, causing your system to break fully and completely (until a rescue operation is performed). you will need to set the 'LD_ASSUME_KERNEL' variable very early during system boot - that will make _most_ processes launched with the non-tls libraries. however, the 'init' process, is launched before any of the system's initialization scripts are launched - and the 'init' process is the process that launches the terminal handler processes (getty, mgetty or similar), and those launch your login shells, etc. and if i were you, i wouldn't have messed up with the launching of the 'init' process, unless i had: 1. a very good idea how to restore the original state, in case of problems. 2. a few hours to spare. 3. the will to write a small C program, that will be launched _in place of_ the regular init program, set the LD_ASSUME_KERNEL environment variable, export it, and then launch the original init program. -- guy For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Wed, Dec 17, 2003 at 02:23:45AM +0200, guy keren wrote: you will need to set the 'LD_ASSUME_KERNEL' variable very early during system boot - that will make _most_ processes launched with the non-tls libraries. however, the 'init' process, is launched before any of the system's initialization scripts are launched - and the 'init' process is the process that launches the terminal handler processes (getty, mgetty or similar), and those launch your login shells, etc. and if i were you, i wouldn't have messed up with the launching of the 'init' process, unless i had: 1. a very good idea how to restore the original state, in case of problems. 2. a few hours to spare. 3. the will to write a small C program, that will be launched _in place of_ the regular init program, set the LD_ASSUME_KERNEL environment variable, export it, and then launch the original init program. 1. add the following boot parameter (in lilo.conf: append, or in grub.conf: in the command-line): INIT=/bin/init.test.sh /bin/init.test.sh should be: #!/bin/sh export LD_ASSUME_KERNEL=2.4.1 exec /sbin/init But then again, there is a simpler way: Add the following to your boot parameters: LD_ASSUME_KERNEL=2.4.1 lilo/grub won't catch it. The kernel won't catch it (right?), and thus it will be passed to the environment of the init process. -- Tzafrir Cohen +---+ http://www.technion.ac.il/~tzafrir/ |vim is a mutt's best friend| mailto:[EMAIL PROTECTED] +---+ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Various performance problems
Hi list, I have a (couple of) client(s) that have performance problems across linux distirbutions. That is - they try an operation on a given platform with a given (redhat both cases) distribution, and get a certain performance. Then they try the same operation on the same hardware with a newer distribution, and get considerably lower performance. In one case, the fast distro is RedHat 7.2, and the slow one is 9. In another I'm not sure what the fast one is, but the slow on is AS 3. I suspect that RedHat screwed something up with their newer kernels (perhaps something incorrectly backported from 2.6?). So far, I have seen neither computers myself, and I only had chance to do repetitive try this, what now? with one of them over the phone. The problem is that nothing shows problems! The tasks seem almost unrelated. In one case it's copying stuff over the net to another computer (1:2 performance ratio), in another it's running ./configure (1:4 ratio). In both cases, there does not appear to be any clear-cut curlpit. In one case I asked the client to try out the 7.2 kernel with the 8 distribution, but I don't have the results in yet. The questions: a. Does anyone have a recommended benchmarking tool? I found this page (http://lbs.sourceforge.net/), but I'd really rather not start messing around with each and every one of those until I find the one I like. If anyone here has prior experience, I'd love it if you could share. b. Occasionally, I get a system that responds slow, but aside from a high load average, there seems to be nothing wrong with it. CPU is idle most of the time, etc. Any ideas how I can find out what and why is going on? Many thanks, Shachar -- Shachar Shemesh Open Source integration consulting Home page resume - http://www.shemesh.biz/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Mon, 15 Dec 2003, Shachar Shemesh wrote: b. Occasionally, I get a system thatresponds slow, but aside from a high load average, there seems to be nothing wrong with it. CPU is idle most of the time, etc. Any ideas how I can find out what and why is going on? I'd look for: - I/O bound processes - Inefficient scheduling by the kernel - this is something, which might have been screwed up by kernel patches, as I understand that there was a lot of development work in the kernel scheduler. - Resources, which have been reserved for much longer time than necessary (and hence blocking processes which are waiting for them). Tools? The only relevant tool I know about is vmstat. Suggestions for other tools are very welcome. --- Omer My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Mon, Dec 15, 2003 at 06:39:35PM +0200, Shachar Shemesh wrote: Hi list, Hi Shachar, good questions! I'm curious what others will answer. Here are some comments from me. In one case, the fast distro is RedHat 7.2, and the slow one is 9. In another I'm not sure what the fast one is, but the slow on is AS 3. I suspect that RedHat screwed something up with their newer kernels (perhaps something incorrectly backported from 2.6?). Why do you suspect the kernel? is the operation that's slower kernel bound? for example, there was an issue with RH's use of UTF-8 as default in RH9, IIRC, that caused e.g. grep to be much slower. So far, I have seen neither computers myself, and I only had chance to do repetitive try this, what now? with one of them over the phone. The problem is that nothing shows problems! The tasks seem almost unrelated. In one case it's copying stuff over the net to another computer (1:2 performance ratio), in another it's running ./configure (1:4 ratio). In both cases, there does not appear to be any clear-cut curlpit. In one case I asked the client to try out the 7.2 kernel with the 8 distribution, but I don't have the results in yet. Ok, that's more interesting. For the network case, packet dumps on both sides of the connections would be interesting. For the configure, might it be gcc 2.96 vs. gcc 3.2.x? The questions: a. Does anyone have a recommended benchmarking tool? I found this page (http://lbs.sourceforge.net/), but I'd really rather not start messing around with each and every one of those until I find the one I like. If anyone here has prior experience, I'd love it if you could share. I don't like whole world benchmarks - I prefer benchmarks that allow you to measure one area specifically. For a general benchmark, I use - what else - a kernel compilation. b. Occasionally, I get a system that responds slow, but aside from a high load average, there seems to be nothing wrong with it. CPU is idle most of the time, etc. Any ideas how I can find out what and why is going on? profiling the system with e.g. oprofile is a good start - it will tell you where your kernel + application are spending most of their time, by function. Cheers, Muli -- Muli Ben-Yehuda http://www.mulix.org | http://mulix.livejournal.com/ the nucleus of linux oscillates my world - [EMAIL PROTECTED] signature.asc Description: Digital signature
Re: Various performance problems
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This is not the 1st time that I hear that RedHat9 is slower than earlier ver. I think that I read somewhere that the DMA of the HD is turned off in RedHat9, but maybe i'm just fantasyzing. I tried googling a little (but i don't have a lot of time right now for it), but didn't find intersting stuff. Maybe i'll have time l8er, i'll take a look at it.. Noam Shachar Shemesh wrote: | Hi list, | | I have a (couple of) client(s) that have performance problems | across linux distirbutions. That is - they try an operation on a | given platform with a given (redhat both cases) distribution, and | get a certain performance. Then they try the same operation on the | same hardware with a newer distribution, and get considerably lower | performance. | | In one case, the fast distro is RedHat 7.2, and the slow one is 9. | In another I'm not sure what the fast one is, but the slow on is AS | 3. I suspect that RedHat screwed something up with their newer | kernels (perhaps something incorrectly backported from 2.6?). | | So far, I have seen neither computers myself, and I only had chance | to do repetitive try this, what now? with one of them over the | phone. The problem is that nothing shows problems! The tasks seem | almost unrelated. In one case it's copying stuff over the net to | another computer (1:2 performance ratio), in another it's running | ./configure (1:4 ratio). In both cases, there does not appear to be | any clear-cut curlpit. In one case I asked the client to try out | the 7.2 kernel with the 8 distribution, but I don't have the | results in yet. | | The questions: a. Does anyone have a recommended benchmarking tool? | I found this page (http://lbs.sourceforge.net/), but I'd really | rather not start messing around with each and every one of those | until I find the one I like. If anyone here has prior experience, | I'd love it if you could share. b. Occasionally, I get a system | that responds slow, but aside from a high load average, there | seems to be nothing wrong with it. CPU is idle most of the time, | etc. Any ideas how I can find out what and why is going on? | | Many thanks, | | Shachar | -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQE/3eo9cgxJvekqy4kRAtERAKCVUZZoFjE0pzNULE2+Iujc4YVKlgCdED4m GbY35NlAMMKOu9KdGcbsHI8= =xaUq -END PGP SIGNATURE- = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
Shachar Shemesh [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] [snip] The questions: a. Does anyone have a recommended benchmarking tool? I found this page (http://lbs.sourceforge.net/), but I'd really rather not start messing around with each and every one of those until I find the one I like. If anyone here has prior experience, I'd love it if you could share. [snip] Look at C/C++ Program Perfometer (Tool for measuring comparative performance of the C/C++ code). http://groups.google.com/groups?selm=bm9216%24jvo98%241%40ID-79865.news.uni-berlin.de Perhaps it is not exactly what you are looking for, but maybe it might help you. = Alex Vinokur mailto:[EMAIL PROTECTED] http://mathforum.org/library/view/10978.html news://news.gmane.org/gmane.comp.lang.c++.perfometer = = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
Shachar Shemesh [EMAIL PROTECTED] writes: a. Does anyone have a recommended benchmarking tool? I found this page (http://lbs.sourceforge.net/), but I'd really rather not start messing around with each and every one of those until I find the one I like. If anyone here has prior experience, I'd love it if you could share. Red Hat point to this URL from their docs: http://people.redhat.com/alikins/system_tuning.html (among others) Maybe something there will prove useful for monitoring / tuning. My own experience with RH9 is very recent, and mostly on a workstation (laptop). X and whatever is related seems *very* slow compared to 7.3. In particular, X it is really slow to start. Others have complained about similar performance problems as well. The problems your client report seem to be unrelated to graphics, and occur on servers. If too many different things look slow, can it be taken as a sign that the problem lies in the kernel? -- Oleg Goldshmidt | [EMAIL PROTECTED] = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Mon, 15 Dec 2003, Shachar Shemesh wrote: I have a (couple of) client(s) that have performance problems across linux distirbutions. That is - they try an operation on a given platform with a given (redhat both cases) distribution, and get a certain performance. Then they try the same operation on the same hardware with a newer distribution, and get considerably lower performance. In one case, the fast distro is RedHat 7.2, and the slow one is 9. In another I'm not sure what the fast one is, but the slow on is AS 3. I suspect that RedHat screwed something up with their newer kernels (perhaps something incorrectly backported from 2.6?). people have such short memories: http://www.mail-archive.com/[EMAIL PROTECTED]/msg30749.html (summary - nadav harel, checks why hspell on redhat 9 runs several times slower then on redhat 7.something, and finds that changing the LD_LIBRARY_PATH to use the non-tls libraries solves the problem). -- guy good consultants are tested by their speed of memory recall operations ;) For world domination - press 1, or dial 0, and please hold, for the creator. -- nob o. dy = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
I must say, from my own experience with RH9, it is actually much faster than my old distro (7.3). My educated guess is as good as any but it doesn't *feel* like a kernel problem (hehe, yes I am a Zen master :). I might be tempted to look into the standard C library or any other general component of the system that practically everything uses... Eli On Mon, 15 Dec 2003 19:07:10 +0200, Noam Meltzer [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This is not the 1st time that I hear that RedHat9 is slower than earlier ver. I think that I read somewhere that the DMA of the HD is turned off in RedHat9, but maybe i'm just fantasyzing. I tried googling a little (but i don't have a lot of time right now for it), but didn't find intersting stuff. Maybe i'll have time l8er, i'll take a look at it.. Noam Shachar Shemesh wrote: | Hi list, | | I have a (couple of) client(s) that have performance problems | across linux distirbutions. That is - they try an operation on a | given platform with a given (redhat both cases) distribution, and | get a certain performance. Then they try the same operation on the | same hardware with a newer distribution, and get considerably lower | performance. | | In one case, the fast distro is RedHat 7.2, and the slow one is 9. | In another I'm not sure what the fast one is, but the slow on is AS | 3. I suspect that RedHat screwed something up with their newer | kernels (perhaps something incorrectly backported from 2.6?). | | So far, I have seen neither computers myself, and I only had chance | to do repetitive try this, what now? with one of them over the | phone. The problem is that nothing shows problems! The tasks seem | almost unrelated. In one case it's copying stuff over the net to | another computer (1:2 performance ratio), in another it's running | ./configure (1:4 ratio). In both cases, there does not appear to be | any clear-cut curlpit. In one case I asked the client to try out | the 7.2 kernel with the 8 distribution, but I don't have the | results in yet. | | The questions: a. Does anyone have a recommended benchmarking tool? | I found this page (http://lbs.sourceforge.net/), but I'd really | rather not start messing around with each and every one of those | until I find the one I like. If anyone here has prior experience, | I'd love it if you could share. b. Occasionally, I get a system | that responds slow, but aside from a high load average, there | seems to be nothing wrong with it. CPU is idle most of the time, | etc. Any ideas how I can find out what and why is going on? | | Many thanks, | | Shachar | -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQE/3eo9cgxJvekqy4kRAtERAKCVUZZoFjE0pzNULE2+Iujc4YVKlgCdED4m GbY35NlAMMKOu9KdGcbsHI8= =xaUq -END PGP SIGNATURE- = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED] -- Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
I in fact noticed in RH 9.0 hspell and aspell were locking up my system. I was running a lowlatency kernel but found that jack and ardour were slow. I am now running Fedora, in kde and gnome window minimize slowly and maximize slowly. Programs lock up and apt-get takes over to the point where my mouse pointer dissapears for a time. I am wondering if this is just a RH issue or Mandrake and friends also have this problem?? Aaron = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
On Tue, Dec 16, 2003 at 01:47:12AM +0200, Aaron wrote: I in fact noticed in RH 9.0 hspell and aspell were locking up my system. I was running a lowlatency kernel but found that jack and ardour were slow. I am now running Fedora, in kde and gnome window minimize slowly and maximize slowly. Programs lock up and apt-get takes over to the point where my mouse pointer dissapears for a time. Is dma enabled on your system? hdparm -v /dev/hda Or more specifically: hdparm -d /dev/hda -- Tzafrir Cohen +---+ http://www.technion.ac.il/~tzafrir/ |vim is a mutt's best friend| mailto:[EMAIL PROTECTED] +---+ = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
Re: Various performance problems
This is what I used. /sbin/hdparm -c 3 -d 1 -m 16 -A1 /dev/hda Aaron you ever sleep? Aa hdparm -v /dev/hda Or more specifically: hdparm -d /dev/hda = To unsubscribe, send mail to [EMAIL PROTECTED] with the word unsubscribe in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
