Re: [PERFORM] Pg+Linux swap use
Greg Stark wrote: William Yu [EMAIL PROTECTED] writes: Rob Sell wrote: Not being one to hijack threads, but I haven't heard of this performance hit when using HT, I have what should all rights be a pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. Not to get into a big Intel vs AMD argument but 50% sounds about right. Let's first assume that the QS rating for the MP1400 is relatively accurate and convert that to a 1.4GHz Xeon. 2.4/1.4 = +71%. Since processor performance does not increase linearly with clockspeed, 50% is in line with expectations. Hm. You've read 50% as fast as 50% faster. I wonder which the original poster intended. Hyper-threading makes 2 cpus be 4 cpu's, but the 4 cpu's are each only 70% as fast, so HT is taking 2x cpus and making it 4x0.70 cpu's, which gives 2.80 cpu's, and you get that only if you are hammering all four cpu's with a full load. Imagine ifd get two cpu-bound processes on the first die (first 2 cpu's of 4) and the other CPU die is idle, and you can see that HT isn't all that useful unless you are sure to keep all 4 cpu's busy. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Pg+Linux swap use
William Yu [EMAIL PROTECTED] writes: Rob Sell wrote: Not being one to hijack threads, but I haven't heard of this performance hit when using HT, I have what should all rights be a pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. Not to get into a big Intel vs AMD argument but 50% sounds about right. Let's first assume that the QS rating for the MP1400 is relatively accurate and convert that to a 1.4GHz Xeon. 2.4/1.4 = +71%. Since processor performance does not increase linearly with clockspeed, 50% is in line with expectations. Hm. You've read 50% as fast as 50% faster. I wonder which the original poster intended. -- greg ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] Pg+Linux swap use
Rob Sell wrote: Not being one to hijack threads, but I haven't heard of this performance hit when using HT, I have what should all rights be a pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. I have read everything I could find on Pg performance tweaked all the variables that were suggested and nothing. Which is why I subscribed to this list, just been lurking so far but this caught my eye. Not to get into a big Intel vs AMD argument but 50% sounds about right. Let's first assume that the QS rating for the MP1400 is relatively accurate and convert that to a 1.4GHz Xeon. 2.4/1.4 = +71%. Since processor performance does not increase linearly with clockspeed, 50% is in line with expectations. Then you throw in the fact that (1) QS ratings for slower AMD chips are understated (but overstated for the fastest chips), (2) AMD uses a point-to-point CPU/memory interface (much better for SMP) versus the P4/Xeon's shared bus, (3) Athlon architecture is more suited for DB work compared to the P4, I'd say you're lucky to see 50% more performance from a Xeon 2.4. As for HT, I've seen quite a few benchmarks where HT hurts performance. The problem is it's not only app and workload specific but also system and usage specific. As it involves the internal rescheduling of processes, adding more simultaneous processes could help to a point and then start hurting or vice-versa. ---(end of broadcast)--- TIP 6: Have you searched our list archives? http://archives.postgresql.org
Re: [PERFORM] Pg+Linux swap use
On Thu, 30 Oct 2003 17:49:08 -0200 (BRST) alexandre :: aldeia digital [EMAIL PROTECTED] wrote: Both use: Only postgresql on server. Buffers = 8192, effective cache = 10 Well, I'm assuming you meant 1GB of ram, not 1MB :) Check a ps auxw to see what is running. Perhaps X is running gobbling up your precious mem. But still.. with 1GB there should be virtually no swap activity. How busy is the DB? How many connections? and is sort_mem set high? -- Jeff Trout [EMAIL PROTECTED] http://www.jefftrout.com/ http://www.stuarthamm.net/ ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] Pg+Linux swap use
Jeff wrote: On Thu, 30 Oct 2003 17:49:08 -0200 (BRST) alexandre :: aldeia digital [EMAIL PROTECTED] wrote: Both use: Only postgresql on server. Buffers = 8192, effective cache = 10 Well, I'm assuming you meant 1GB of ram, not 1MB :) Check a ps auxw to see what is running. Perhaps X is running gobbling up your precious mem. But still.. with 1GB there should be virtually no swap activity. How busy is the DB? How many connections? and is sort_mem set high? Also are two kernels exactly same? In my experience linux kernel behaves slightly different from version to version w.r.t swap aggressiveness... Shridhar ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
Re: [PERFORM] Pg+Linux swap use
Scott, Jeff and Shridhar: 1 GB RAM :) The stock kernels are not the same, HyperThreading enabled. 80 simultaneous connections. sort_mem = 4096 I will compile my own kernel on this weekend, and I will report to the list after. Thank's all Alexandre Also are two kernels exactly same? In my experience linux kernel behaves slightly different from version to version w.r.t swap aggressiveness... Shridhar ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster ---(end of broadcast)--- TIP 5: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faqs/FAQ.html
Re: [PERFORM] Pg+Linux swap use
On Fri, Oct 31, 2003 at 12:03:59PM -0200, alexandre :: aldeia digital wrote: Scott, Jeff and Shridhar: 1 GB RAM :) The stock kernels are not the same, HyperThreading enabled. 80 Some people have reported that things actually slow down with HT enabled. Have you tried turning it off? A -- Andrew Sullivan 204-4141 Yonge Street Afilias CanadaToronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] Pg+Linux swap use
Not being one to hijack threads, but I haven't heard of this performance hit when using HT, I have what should all rights be a pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. I have read everything I could find on Pg performance tweaked all the variables that were suggested and nothing. Which is why I subscribed to this list, just been lurking so far but this caught my eye. Rob -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Sullivan Sent: Friday, October 31, 2003 8:36 AM To: [EMAIL PROTECTED] Subject: Re: [PERFORM] Pg+Linux swap use On Fri, Oct 31, 2003 at 12:03:59PM -0200, alexandre :: aldeia digital wrote: Scott, Jeff and Shridhar: 1 GB RAM :) The stock kernels are not the same, HyperThreading enabled. 80 Some people have reported that things actually slow down with HT enabled. Have you tried turning it off? A -- Andrew Sullivan 204-4141 Yonge Street Afilias CanadaToronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 8: explain analyze is your friend ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] Pg+Linux swap use
Jeff wrote: On Fri, 31 Oct 2003 09:31:19 -0600 Rob Sell [EMAIL PROTECTED] wrote: Not being one to hijack threads, but I haven't heard of this performance hit when using HT, I have what should all rights be a pretty fast server, dual 2.4 Xeons with HT 205gb raid 5 array, 1 gig of memory. And it is only 50% as fast as my old server which was a dual AMD MP 1400's with a 45gb raid 5 array and 1gb of ram. I have read everything I could find on Pg performance tweaked all the variables that were suggested and nothing. Which is why I subscribed to this list, just been lurking so far but this caught my eye. Rob There's benchmarks around that show in _some_ cases HT is not all it is cracked up to be, somtimes running slower. To use HT effectively on needs. 1. A kernel that understands HT. 2. A task scheduler that understands HT 3. A CPU intensive load. So if you are running a stock RedHat and production postgresql database, turn it off. It won't hurt certainly(Almost certainly) I'm guessing RH is running some useless stuff in the BG.. or maybe he's running a retarded kernel... or.. maybe.. just.. maybe.. little elves are doing it. Too much..:-) I guess Alexandre can tune bdflush to be little less agressive. Comparing bdflush values on two machines might turn up something. His idea of compiling kernel is also good one. He can also try tuning some values in /proc/sys/vm but I don't find any documentation offhand. I usually run slackware and a handcompiled 2.6-test4. None of them use any swap unless true memory starts falling low. This touch-swap-even-if-oodles-of-ram-is-free is something I have't experienced on my desktop for quite a while. Shridhar ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] Pg+Linux swap use
Bill Moran [EMAIL PROTECTED] writes: Just for an additional viewpoint. I'm finishing up a project based on FreeBSD and PostgreSQL. The target server is a Dual 2.4G Intel machine. I have tested the application with hyperthreading enabled and disabled. To all appearances, enabling hyperthreading makes the box act like a quad, with the expected increase in processing capability - _for_this_application_. I have also heard the claims and seen the tests that show hyperthreading occasionally decreasing performance. I think in the end, you just have to test your particular application to see how it reacts. My understanding is that the case where HT hurts is precisely your case. When you have two real processors with HT the kernel will sometimes schedule two jobs on the two virtual processors on the same real processor leaving the two virtual processors on the other real processor idle. As far as I know a single processor machine with HT does not benefit from disabling HT. -- greg ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Pg+Linux swap use
On Fri, 2003-10-31 at 11:37, Greg Stark wrote: My understanding is that the case where HT hurts is precisely your case. When you have two real processors with HT the kernel will sometimes schedule two jobs on the two virtual processors on the same real processor leaving the two virtual processors on the other real processor idle. If you're seeing this behavior, it's sounds like a bug/deficiency in your kernel's scheduler: if it is HT-aware, it should go to some lengths to avoid this kind of processor allocation. -Neil ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
[PERFORM] Pg+Linux swap use
Hi, Old: Post 7.3.2, P4 1.8, 1 MB RAM, 2 x IDE SW RAID 1, RedHat 8 New: Post 7.3.4, Xeon 2.4, 1 MB RAM, 2 x SCSI 15k SW RAID 1, RedHat 9 Both use: Only postgresql on server. Buffers = 8192, effective cache = 10 In old plataform the free and vmstat reports no use of swap. In new, the swap is in constant use (40-100 MB), with a low but constant swap out and swap in. The cache memory ~ 83 and buffers ~ 43000 I try to reduce the buffers to 1024 with no effects. Thanks, Alexandre ---(end of broadcast)--- TIP 4: Don't 'kill -9' the postmaster
