Re: [PERFORM] Dual Xeon + HW RAID question
Mindaugas Riauba wrote: I missed your orig. post, but AFAIK multiprocessing kernels will handle HT CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4 CPUs. This way, I don't think HT would improve any single query (afaik no postgres process uses more than one cpu), but overall multi-query performance has to improve. When you use hyperthreading, each virtual cpu runs at 70% of a full CPU, so hyperthreading could be slower than non-hyperthreading. On a fully loaded dual cpu system, you are looking at 2.8 cpu's (0.70 * 4), while if it isn't loaded, you are looking at slowing down if you are only using 1 or 2 cpu's. Virtual cpus are not running at 70% of real cpus :). Slowdown will happen if scheduler will run 2 processes on the same real cpu. And I read that there are patches for Linux kernel to fix that. Sooner rather than later they will appear in Linus kernel. Right, I simplified it. The big deal is whether the OS favors the second real CPU over one of the virtual CPU's on the same die --- by default, it doesn't. Ever if it did work perfectly, you are talking about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Dual Xeon + HW RAID question
by default -- do you mean there is a way to tell Linux to favor the second real cpu over the HT one? how? G. --- cut here --- - Original Message - From: Bruce Momjian [EMAIL PROTECTED] Sent: Tuesday, July 22, 2003 6:26 PM Subject: Re: [PERFORM] Dual Xeon + HW RAID question Right, I simplified it. The big deal is whether the OS favors the second real CPU over one of the virtual CPU's on the same die --- by default, it doesn't. Ever if it did work perfectly, you are talking about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much. ---(end of broadcast)--- TIP 8: explain analyze is your friend
Re: [PERFORM] Dual Xeon + HW RAID question
SZUCS Gábor wrote: by default -- do you mean there is a way to tell Linux to favor the second real cpu over the HT one? how? Right now there is no way the kernel can tell which virtual cpu's are on each physical cpu's, and that is the problem. Once there is a way, hyperthreading will be more useful, but even then, it doesn't double your CPU throughput, just increases by 40%. Right, I simplified it. The big deal is whether the OS favors the second real CPU over one of the virtual CPU's on the same die --- by default, it doesn't. Ever if it did work perfectly, you are talking about going from 1 to 1.4 or 2 to 2.8, which doesn't seem like much. ---(end of broadcast)--- TIP 8: explain analyze is your friend -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] Dual Xeon + HW RAID question
Jord Tanner wrote: On Tue, 2003-07-22 at 10:39, Bruce Momjian wrote: But CPU affinity isn't realated to hyperthreading, as far as I know. CPU affinity tries to keep processes on the same cpu in case there is still valuable info in the cpu cache. It is true that CPU affinity is designed to prevent the dump of valuable CPU cache. My thought is that if you are trying to prevent CPU contention, you could use CPU affinity to prevent 2 postmaster processes from running simultaneously on the same die. Am I out to lunch here? I've not worked with CPU affinity before, so I'm not familiar with the intimate details. I guess you could but it is the backends that use the cpu. I don't think manually specifying affinity will work for most applications. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ can be your backup.| Newtown Square, Pennsylvania 19073 ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] Dual Xeon + HW RAID question
On Tue, 2003-07-22 at 11:50, Bruce Momjian wrote: Jord Tanner wrote: On Tue, 2003-07-22 at 10:39, Bruce Momjian wrote: But CPU affinity isn't realated to hyperthreading, as far as I know. CPU affinity tries to keep processes on the same cpu in case there is still valuable info in the cpu cache. It is true that CPU affinity is designed to prevent the dump of valuable CPU cache. My thought is that if you are trying to prevent CPU contention, you could use CPU affinity to prevent 2 postmaster processes from running simultaneously on the same die. Am I out to lunch here? I've not worked with CPU affinity before, so I'm not familiar with the intimate details. I guess you could but it is the backends that use the cpu. I don't think manually specifying affinity will work for most applications. This is beating a dead horse, but I'll take one more kick at it. CPU affinity is defined by a bit mask, so multiple processors can be selected. It is also inherited by child processes, so assigning CPU 0 and CPU 2 (which I assume would be on different dies in a dual processor hyper-threading system) to the parent postmaster should prevent CPU contention with respect to the postgres backend. I would be very interested to see if any advantage could be gained by a combination of multiple HT processors and cpu affinity over multiple non-HT processors. Yet Another Performance Testing To Do (YAPTTD)! -- Jord Tanner [EMAIL PROTECTED] ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] Dual Xeon + HW RAID question
SZUCS, In my tests, I don´t a great performance enhacement with HT. I suspect that my problem resides on I/O performance. I will wait for a best moment to resinstall the system with other disk configurations and then I will report here. Thanks for all replys! Alexandre Alexandre, I missed your orig. post, but AFAIK multiprocessing kernels will handle HT CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4 CPUs. This way, I don't think HT would improve any single query (afaik no postgres process uses more than one cpu), but overall multi-query performance has to improve. - Original Message - From: Nikolaus Dilger [EMAIL PROTECTED] Sent: Saturday, July 12, 2003 8:25 PM Alexandre, Since you want the fastest speed I would do the 2 data disks in RAID 0 (striping) not RAID 1 (mirroring). If you would care about not loosing any transactions you would keep all 3 disks in RAID 5. Don't know the answer to the Hyperthreading question. Why don't you run a test to find out? Regards, Nikolaus On Thu, 10 Jul 2003 14:43:25 -0300 (BRT), alexandre arruda paes :: aldeia digital wrote: Hi, I have this machine with a 10 million records: * Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI , Adaptec 2110S, RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0 - 2.4.18 The database is mirrored with contrib/dbmirror in a P4 1 Gb Ram + IDE If a disk failure occurs, I can use the server in the mirror. I will format the main server in this weekend and I have seen in the list some people that recomends a Software RAID instead HW. I think too remove the RAID 5 and turn a RAID 1 for data in 2 HDs. SO, WAL and swap in the thrid HD. My questions: 1) I will see best disk performance changing the disk layout like above 2) HyperThreading really improve a procces basead program, like postgres Thank´s for all Alexandre ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED] ---(end of broadcast)--- TIP 7: don't forget to increase your free space map settings
Re: [PERFORM] Dual Xeon + HW RAID question
Alexandre, I missed your orig. post, but AFAIK multiprocessing kernels will handle HT CPUs as 2 CPUs each. Thus, our dual Xeon 2.4 is recognized as 4 Xeon 2.4 CPUs. This way, I don't think HT would improve any single query (afaik no postgres process uses more than one cpu), but overall multi-query performance has to improve. - Original Message - From: Nikolaus Dilger [EMAIL PROTECTED] Sent: Saturday, July 12, 2003 8:25 PM Alexandre, Since you want the fastest speed I would do the 2 data disks in RAID 0 (striping) not RAID 1 (mirroring). If you would care about not loosing any transactions you would keep all 3 disks in RAID 5. Don't know the answer to the Hyperthreading question. Why don't you run a test to find out? Regards, Nikolaus On Thu, 10 Jul 2003 14:43:25 -0300 (BRT), alexandre arruda paes :: aldeia digital wrote: Hi, I have this machine with a 10 million records: * Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI , Adaptec 2110S, RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0 - 2.4.18 The database is mirrored with contrib/dbmirror in a P4 1 Gb Ram + IDE If a disk failure occurs, I can use the server in the mirror. I will format the main server in this weekend and I have seen in the list some people that recomends a Software RAID instead HW. I think too remove the RAID 5 and turn a RAID 1 for data in 2 HDs. SO, WAL and swap in the thrid HD. My questions: 1) I will see best disk performance changing the disk layout like above 2) HyperThreading really improve a procces basead program, like postgres Thank´s for all Alexandre ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]
Re: [PERFORM] Dual Xeon + HW RAID question
On Sat, Jul 12, 2003 at 11:25:14AM -0700, Nikolaus Dilger wrote: Alexandre, Since you want the fastest speed I would do the 2 data disks in RAID 0 (striping) not RAID 1 (mirroring). Note that RAID 0 buys you nothing at all in redundancy. So if the point is to be able to recover from a disk failure, you need 1 (or some combination of 0 and 1, or 5). A -- Andrew Sullivan 204-4141 Yonge Street Liberty RMS Toronto, Ontario Canada [EMAIL PROTECTED] M2P 2A8 +1 416 646 3304 x110 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] Dual Xeon + HW RAID question
Alexandre, Since you want the fastest speed I would do the 2 data disks in RAID 0 (striping) not RAID 1 (mirroring). If you would care about not loosing any transactions you would keep all 3 disks in RAID 5. Don't know the answer to the Hyperthreading question. Why don't you run a test to find out? Regards, Nikolaus On Thu, 10 Jul 2003 14:43:25 -0300 (BRT), alexandre arruda paes :: aldeia digital wrote: Hi, I have this machine with a 10 million records: * Dual Xeon 2.0 (HyperThreading enabled), 3 7200 SCSI , Adaptec 2110S, RAID 5 - 32k chunk size, 1 GB Ram DDR 266 ECC, RH 8.0 - 2.4.18 The database is mirrored with contrib/dbmirror in a P4 1 Gb Ram + IDE If a disk failure occurs, I can use the server in the mirror. I will format the main server in this weekend and I have seen in the list some people that recomends a Software RAID instead HW. I think too remove the RAID 5 and turn a RAID 1 for data in 2 HDs. SO, WAL and swap in the thrid HD. My questions: 1) I will see best disk performance changing the disk layout like above 2) HyperThreading really improve a procces basead program, like postgres Thank´s for all Alexandre ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly ---(end of broadcast)--- TIP 8: explain analyze is your friend
