Re: [GENERAL] Large Databases redux

2012-03-22 Thread Marti Raudsepp
On Thu, Mar 22, 2012 at 00:20, Martijn van Oosterhout wrote: > That, and a good RAID controller with BBU cache will go a long way to > relieving the pain of fsync. Well a BBU cache RAID is helpful, but fsyncs are a minor problem in data warehouse workloads, since inserts are done in large bulks a

Re: [GENERAL] Large Databases redux

2012-03-21 Thread John R Pierce
On 03/21/12 3:20 PM, Martijn van Oosterhout wrote: That, and a good RAID controller with BBU cache will go a long way to relieving the pain of fsync. even better than BBU cache is the newer 'flash backed caches'. works the same, but uses a supercap rather than a battery, and backs the cache

Re: [GENERAL] Large Databases redux

2012-03-21 Thread Martijn van Oosterhout
On Wed, Mar 21, 2012 at 02:58:43PM -0700, John R Pierce wrote: > On 03/21/12 2:18 PM, Jason Herr wrote: > >I have my own theories based on what I've read and my puttering. > >I think I can get away with a disk for the OS, disk for the WAL, > >disk for the large table (tablespaces) and a disk for th

Re: [GENERAL] Large Databases redux

2012-03-21 Thread John R Pierce
On 03/21/12 2:18 PM, Jason Herr wrote: I have my own theories based on what I've read and my puttering. I think I can get away with a disk for the OS, disk for the WAL, disk for the large table (tablespaces) and a disk for the rest. And when I say disk I mean storage device. I'm thinking RAI

[GENERAL] Large Databases redux

2012-03-21 Thread Jason Herr
Hey, In an attempt to NOT pollute the thread started by Kjetil Nygård, I decided to ask a very similar question with likely different data. I am interested in hearing recommendations on hardware specs in terms of Drives/RAM/shared_buffers/CPUs. I have been doing some research/testing, and am look

Re: [GENERAL] Large Databases

2004-08-31 Thread Cott Lang
On Tue, 2004-08-31 at 20:37, Joe Conway wrote: > > > However, my results changed drastically under the 2.6 kernel, when the > > NFS results stayed about the same as 2.4, but the SAN jumped about 50% > > in transactions per second. > > Very interesting. Whose SAN are you using that supports the 2.

Re: [GENERAL] Large Databases

2004-08-31 Thread Joe Conway
Cott Lang wrote: My best NFS results were within about 15% of the speed of my best SAN results. Good info, and consistent with what I've seen. However, my results changed drastically under the 2.6 kernel, when the NFS results stayed about the same as 2.4, but the SAN jumped about 50% in transaction

Re: [GENERAL] Large Databases

2004-08-31 Thread Cott Lang
On Tue, 2004-08-31 at 15:07, Joe Conway wrote: > I suppose there *may* be some fundamental technical difference that > makes Postgres less reliable than Oracle when using NFS, but I'm not > sure what it would be -- if anyone knows of one, please speak up ;-). > Early testing on NFS mounted NAS

[GENERAL] Large Databases

2004-08-31 Thread elein
What is the linux and/or postgres limitation for very large databases, if any? We are looking at 6T-20T. My understanding is that if the hardware supports it, then it can be done in postgres. But can hardware support that? --elein [EM