Re: [PERFORM] LVM and Postgres
On 06/12/05, Michael Stone ([EMAIL PROTECTED]) wrote: On Tue, Dec 06, 2005 at 07:52:25PM -0500, Alex Turner wrote: I would argue that almost certainly won't by doing that as you will create a new place even further away for the disk head to seek to instead of just another file on the same FS that is probably closer to the current head position. I would argue that you should benchmark it instead of speculating. Is there a good way of benchmarking? We don't have much in the way of test data at present. Regards, Rory -- Rory Campbell-Lange [EMAIL PROTECTED] www.campbell-lange.net ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
[PERFORM] LVM and Postgres
I need to slice up a web server's disk space to provide space for postgres and storing binaries such as images and sound files. I'm thinking of using logical volume management (LVM) to help manage the amount of space I use between postgres and the data volumes. The server has a 250GB RAID10 (LSI 320-I + BBU) volume which I am thinking of slicing up in the following way (Linux 2.6 kernel): / : ext3 : 47GB (root, home etc) /boot : ext3 : 1GB /tmp : ext2 : 2GB /usr : ext3 : 4GB /var : ext3 : 6GB --- 60GB VG:190GB approx --- Initially divided so: /data : ext3 : 90GB /postgres : xfs : 40GB This gives me left over space of roughly 60GB to extend into on the volume group, which I can balance between the /data and /postgres logical volumes as needed. Are there any major pitfalls to this approach? Thanks, Rory -- Rory Campbell-Lange [EMAIL PROTECTED] www.campbell-lange.net ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] LVM and Postgres
Rory Campbell-Lange wrote: The server has a 250GB RAID10 (LSI 320-I + BBU) volume which I am thinking of slicing up in the following way (Linux 2.6 kernel): / : ext3 : 47GB (root, home etc) /boot : ext3 : 1GB /tmp : ext2 : 2GB /usr : ext3 : 4GB /var : ext3 : 6GB --- 60GB VG:190GB approx --- Initially divided so: /data : ext3 : 90GB /postgres : xfs : 40GB This gives me left over space of roughly 60GB to extend into on the volume group, which I can balance between the /data and /postgres logical volumes as needed. Are there any major pitfalls to this approach? Thanks, Rory It looks like you are using fast disks and xfs for filesystem on the /postgresql partition. That's nice. How many disks in the array? One thing you miss is sticking a bunch of sequential log writes on a separate spindle as far as I can see with this? WAL / XFS (i think) both have this pattern. If you've got a fast disk and can do BBU write caching your WAL writes will hustle. Others can probably speak a bit better on any potential speedups. - August ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match
Re: [PERFORM] LVM and Postgres
Hi August. Thanks very much for your mail. On 06/12/05, August Zajonc ([EMAIL PROTECTED]) wrote: Rory Campbell-Lange wrote: The server has a 250GB RAID10 (LSI 320-I + BBU) volume which I am thinking of slicing up in the following way (Linux 2.6 kernel): / : ext3 : 47GB (root, home etc) /boot : ext3 : 1GB /tmp : ext2 : 2GB /usr : ext3 : 4GB /var : ext3 : 6GB --- 60GB VG:190GB approx --- Initially divided so: /data : ext3 : 90GB /postgres : xfs : 40GB This gives me left over space of roughly 60GB to extend into on the volume group, which I can balance between the /data and /postgres logical volumes as needed. It looks like you are using fast disks and xfs for filesystem on the /postgresql partition. That's nice. How many disks in the array? Four. One thing you miss is sticking a bunch of sequential log writes on a separate spindle as far as I can see with this? WAL / XFS (i think) both have this pattern. If you've got a fast disk and can do BBU write caching your WAL writes will hustle. Yes, we don't have any spare disks unfortunately. We have enabled the BBU write, so we are hoping for good performance. I'd be grateful for some advice on dd/bonnie++ tests for checking this. Others can probably speak a bit better on any potential speedups. I'd better test extending the Logical Volumes too! Many thanks Rory ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [PERFORM] LVM and Postgres
On Tue, Dec 06, 2005 at 09:36:23PM +, Rory Campbell-Lange wrote: Yes, we don't have any spare disks unfortunately. We have enabled the BBU write, so we are hoping for good performance. Even if you don't use seperate disks you'll probably get better performance by putting the WAL on a seperate ext2 partition. xfs gives good performance for the table data, but is not particularly good for the WAL. Mike Stone ---(end of broadcast)--- TIP 6: explain analyze is your friend
Re: [PERFORM] LVM and Postgres
I would argue that almost certainly won't by doing that as you will create a new place even further away for the disk head to seek to instead of just another file on the same FS that is probably closer to the current head position. Alex On 12/6/05, Michael Stone [EMAIL PROTECTED] wrote: On Tue, Dec 06, 2005 at 09:36:23PM +, Rory Campbell-Lange wrote: Yes, we don't have any spare disks unfortunately. We have enabled the BBU write, so we are hoping for good performance. Even if you don't use seperate disks you'll probably get better performance by putting the WAL on a seperate ext2 partition. xfs gives good performance for the table data, but is not particularly good for the WAL. Mike Stone ---(end of broadcast)--- TIP 6: explain analyze is your friend ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [PERFORM] LVM and Postgres
On Tue, Dec 06, 2005 at 07:52:25PM -0500, Alex Turner wrote: I would argue that almost certainly won't by doing that as you will create a new place even further away for the disk head to seek to instead of just another file on the same FS that is probably closer to the current head position. I would argue that you should benchmark it instead of speculating. You are perhaps underestimating the effect of the xfs log. (Ordinarily xfs has great performance, but it seems to be fairly lousy at fsync/osync/etc operations in my benchmarks; my wild speculation is that the sync forces a log flush.) At any rate you're going to have a lot of head movement on any reasonably sized filesystem anyway, and I'm not convinced that hoping that your data will happen to land close to your log is a valid, repeatable optimization technique. Note that the WAL will wander around the disk as files are created and deleted, whereas tables are basically updated in place. Mike Stone ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
Re: [PERFORM] LVM and Postgres
Michael Stone wrote: Note that the WAL will wander around the disk as files are created and deleted, whereas tables are basically updated in place. Huh? I was rather under the impression that the WAL files (in pg_xlog, right?) were reused once they'd been created, so their locations on the disk should remain the same, as should their data blocks (roughly, depending on the implementation of the filesystem, of course). -- Kevin Brown [EMAIL PROTECTED] ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
