Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-28 Thread Gudmundsson Martin (mg)
I would also make sure to check that the hypervisor does write to permanent storage before returning to the VM with acknowledgement. In the case of ESX, there is no such concern per http://kb.vmware.com/kb/1008542. Very useful info! As Heikki commented, VMware recently compared

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-28 Thread Dong Ye
There was some earlier comment in this discussion about not using NFS datastores for Postgres VMDK's. Would you think you'd see a difference in scalability behavior or performance in these tests if a NFS datastore would be used instead? Provided the architecture is properly setup for that,

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-28 Thread Scott Marlowe
On Wed, Nov 27, 2013 at 7:58 PM, Dong Ye y...@vmware.com wrote: As Heikki commented, VMware recently compared Postgres performance in an ESX (5.1) VM versus in a comparable native Linux. We saw 1. ESX-level locking causes no vertical scalability degradation, 2. FYI Vmware has an optimized

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-28 Thread Michael Paquier
On Fri, Nov 29, 2013 at 3:40 AM, Scott Marlowe scott.marl...@gmail.com wrote: On Wed, Nov 27, 2013 at 7:58 PM, Dong Ye y...@vmware.com wrote: As Heikki commented, VMware recently compared Postgres performance in an ESX (5.1) VM versus in a comparable native Linux. We saw 1. ESX-level locking

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-27 Thread Dong Ye
On Mon, Nov 25, 2013 at 4:00 PM, Gudmundsson Martin (mg) martin.mg.gudmunds...@volvo.com wrote: I would also make sure to check that the hypervisor does write to permanent storage before returning to the VM with acknowledgement. In the case of ESX, there is no such concern per

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Rafael Martinez
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/25/2013 09:01 PM, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. [] Hello

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Boszormenyi Zoltan
2013-11-25 21:19 keltezéssel, Heikki Linnakangas írta: On 25.11.2013 22:01, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. Here at work we

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Stephen Frost
Zoltan, * Boszormenyi Zoltan (z...@cybertec.at) wrote: When the virtual disk is a disk file on the host machine, we've measured 20% or lower. The host used Fedora 19/x86_64 with IIRC a 3.10.x Linux kernel with EXT4 filesystem (this latter is sure, not IIRC). The effect was observed both under

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Andrew Dunstan
On 11/26/2013 09:26 AM, Craig James wrote: On 25.11.2013 22:01, Lee Nguyen wrote: Why shouldn't we run Postgres in a VM? What are the downsides? Does anyone have any metrics or benchmarks with the latest Postgres? For those of us with small (a

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Andrew Dunstan
On 11/26/2013 08:51 AM, Boszormenyi Zoltan wrote: 2013-11-25 21:19 keltezéssel, Heikki Linnakangas írta: On 25.11.2013 22:01, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread David Boreham
On 11/26/2013 7:26 AM, Craig James wrote: For those of us with small (a few to a dozen servers), we'd like to get out of server maintenance completely. Can anyone with experience on a cloud VM solution comment? Do the VM solutions provided by the major hosting companies have the same good

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Craig James
On Tue, Nov 26, 2013 at 8:29 AM, David Boreham david_l...@boreham.orgwrote: On 11/26/2013 7:26 AM, Craig James wrote: For those of us with small (a few to a dozen servers), we'd like to get out of server maintenance completely. Can anyone with experience on a cloud VM solution comment? ...

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Josh Berkus
On 11/25/2013 12:01 PM, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. This is pretty dated advice. Early VMs had horrible performance

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Merlin Moncure
On Tue, Nov 26, 2013 at 11:31 AM, Josh Berkus j...@agliodbs.com wrote: On 11/25/2013 12:01 PM, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread Craig James
On Tue, Nov 26, 2013 at 10:40 AM, Ben Chobot be...@silentmedia.com wrote: On Nov 26, 2013, at 9:24 AM, Craig James wrote: So far I'm impressed by what I've read about Amazon's Postgres instances. Maybe the reality will be disappointing, but (for example) the idea of setting up streaming

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-26 Thread David Kerr
On Tue, Nov 26, 2013 at 11:18:41AM -0800, Craig James wrote: - On Tue, Nov 26, 2013 at 10:40 AM, Ben Chobot be...@silentmedia.com wrote: - - On Nov 26, 2013, at 9:24 AM, Craig James wrote: - - So far I'm impressed by what I've read about Amazon's Postgres instances. - Maybe the reality will

[PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Lee Nguyen
Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. Here at work we were entertaining the idea of running our Postgres database on our VM farm alongside our

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Gavin Flower
On 26/11/13 09:01, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. Here at work we were entertaining the idea of running our Postgres

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Heikki Linnakangas
On 25.11.2013 22:01, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. Here at work we were entertaining the idea of running our Postgres

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Andrew Dunstan
On 11/25/2013 03:19 PM, Heikki Linnakangas wrote: On 25.11.2013 22:01, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. Here at work we

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Mark Kirkwood
On 26/11/13 09:28, Andrew Dunstan wrote: On 11/25/2013 03:19 PM, Heikki Linnakangas wrote: On 25.11.2013 22:01, Lee Nguyen wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Merlin Moncure
On Mon, Nov 25, 2013 at 2:01 PM, Lee Nguyen leemob...@gmail.com wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to go. Here at work we were entertaining the

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread David Lang
On Mon, 25 Nov 2013, Merlin Moncure wrote: On Mon, Nov 25, 2013 at 2:01 PM, Lee Nguyen leemob...@gmail.com wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that Postgres shouldn't be run inside a VM. That bare metal is the only way to

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Merlin Moncure
On Mon, Nov 25, 2013 at 4:57 PM, David Lang da...@lang.hm wrote: On Mon, 25 Nov 2013, Merlin Moncure wrote: On Mon, Nov 25, 2013 at 2:01 PM, Lee Nguyen leemob...@gmail.com wrote: Hi, Having attended a few PGCons, I've always heard the remark from a few presenters and attendees that

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Xenofon Papadopoulos
We have been running several Postgres databases on VMs for the last 9 months. The largest one currently has a few hundreds of millions of rows (~1.5T of data, ~100G of frequently queried data ) and performs at ~1000 tps. Most of our transactions are part of a 2PC, which effectively results to high

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread David Lang
On Tue, 26 Nov 2013, Xenofon Papadopoulos wrote: We have been running several Postgres databases on VMs for the last 9 months. The largest one currently has a few hundreds of millions of rows (~1.5T of data, ~100G of frequently queried data ) and performs at ~1000 tps. Most of our transactions

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-25 Thread Xenofon Papadopoulos
Which scenario do you have in mind? We don't add servers to scale out, we only scale up in each single project. We use SAN for storage, so if we need to increase disk space we provide more through LVM. There is one case we need to move data around, when we relocate projects over to new storage