Re: [PERFORM] Moving pg_xlog

2005-06-02 Thread Himanshu Baweja
My database has two scsi disks my current configuration has pg_xlog on disk1 and data on disk2 the machine is used for database only now did some logging and came to a conclusion that my disk2(data disk) is getting used around 3 times more than disk1(pg_xlog)   so wht is recommended

Re: [PERFORM] How to avoid database bloat

2005-06-02 Thread Mindaugas Riauba
> > Our database increases in size 2.5 times during the day. > > What to do to avoid this? Autovacuum running with quite > > aggressive settings, FSM settings are high enough. > > First thing I'd suggest is to get a more detailed idea of exactly > what is bloating --- which tables/indexes are th

Re: [PERFORM] SURVEY: who is running postgresql on 8 or more CPUs?

2005-06-02 Thread Dirk Lutzebäck
Hi, I just got one reply for this survey. Is almost nobody using postgresql on 8+ machines? Regards, Dirk Dirk Lutzebäck wrote: Hi, I would like to start a little survey who is running postgresql on an 8way or more machine (Intel, Sparc, AMD no matter). Purpose: find out how postgresql

Re: [PERFORM] Adaptec/LSI/?? RAID

2005-06-02 Thread Andrew McMillan
On Wed, 2005-06-01 at 20:42 -0700, Stacy White wrote: > We're in the process of buying another Opteron server to run Postgres, and > based on the suggestions in this list I've asked our IT director to get an > LSI MegaRaid controller rather than one of the Adaptecs. > > But when we tried to place

Re: [PERFORM] Adaptec/LSI/?? RAID

2005-06-02 Thread Richard Welty
On Wed, 01 Jun 2005 22:00:09 -0700 William Yu <[EMAIL PROTECTED]> wrote: > I've previously stayed away from Adaptec because we used to run Solaris > x86 and the driver was somewhat buggy. For Linux and FreeBSD, I'd be > less worried as open source development of drivers usually lead to > better

Re: [PERFORM] SURVEY: who is running postgresql on 8 or more CPUs?

2005-06-02 Thread Dawid Kuroczko
On 6/2/05, Dirk Lutzebäck <[EMAIL PROTECTED]> wrote: > I just got one reply for this survey. Is almost nobody using postgresql > on 8+ machines? My guess is when someone is using PostgreSQL on 8+ machine, she's in highly competitive (or sensitive) market and either cannot give company's work detai

Re: [PERFORM] SURVEY: who is running postgresql on 8 or more CPUs?

2005-06-02 Thread Dirk Lutzebäck
Hi Dawid, postgresql is open source and we also want it to be used in high performance areas. What's wrong with people telling on which machines they use it? I don't care about business details but techinal details would be quite interesting. In the end it is interesting to know how you need

Re: [PERFORM] postgresql-8.0.1 performance tuning

2005-06-02 Thread Martin Fandel
Hi, hmmm i don't understand which are the best values for shmmax and shmall. I've googled around but every site says something different. I've 2GB of RAM now and set it to: kernel.shmmax=715827882 kernel.shmall=2097152 Is that value ok for 2GB of RAM? I've set the shared_buffers in my postgres

Re: [PERFORM] postgresql-8.0.1 performance tuning

2005-06-02 Thread Martin Fandel
Ups, i'm sorry. i've set the following values: postgresql.conf: shared_buffers = 7 effective_cache_size= 1744762 work_mem= 32768 maintenance_work_mem= 262144 max_fsm_pages = 20 sysctl.conf: vm.swappiness=10

Re: [PERFORM] postgresql-8.0.1 performance tuning

2005-06-02 Thread Martin Fandel
I've forgotten the settings for the pgbench-tests. I use 150 clients with 5 transactions each. Am Donnerstag, den 02.06.2005, 15:10 +0200 schrieb Martin Fandel: > Ups, > i'm sorry. i've set the following values: > > postgresql.conf: > shared_buffers = 7 > effective_cache_siz

Re: [PERFORM] How to avoid database bloat

2005-06-02 Thread Tom Lane
"Mindaugas Riauba" <[EMAIL PROTECTED]> writes: >> First thing I'd suggest is to get a more detailed idea of exactly >> what is bloating --- which tables/indexes are the problem? > I think the most problematic table is this one. After vacuum full/reindex > it was 20MB in size now (after 6 hours)

Re: [PERFORM] How to avoid database bloat

2005-06-02 Thread Mindaugas Riauba
> >> First thing I'd suggest is to get a more detailed idea of exactly > >> what is bloating --- which tables/indexes are the problem? > > > I think the most problematic table is this one. After vacuum full/reindex > > it was 20MB in size now (after 6 hours) it is already 70MB and counting. > >

Re: [PERFORM] Moving pg_xlog

2005-06-02 Thread Tom Lane
Himanshu Baweja <[EMAIL PROTECTED]> writes: > My database has two scsi disks > my current configuration has pg_xlog on disk1 and data on disk2 > the machine is used for database only > now did some logging and came to a conclusion that my disk2(data disk) is > getting used around 3 tim

Re: [PERFORM] Moving pg_xlog

2005-06-02 Thread Himanshu Baweja
Tom Lane <[EMAIL PROTECTED]> wrote: >It seems highly unlikely that putting more stuff on the xlog disk will>improve performance --- at least not if your bottleneck is update speed.   Tom you are right.. i did some testing... 1) default config--- xlog on disk1 and data on disk2=>     27 mins and 22

Re: [PERFORM] Specific query performance problem help requested - postgresql 7.4

2005-06-02 Thread Brad Might
How is it that the index scan has such poor performance? Shouldn't index lookups be quicker? -Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Thursday, May 26, 2005 1:32 PM To: Brad Might Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Specific query performa

[PERFORM] Query plan for very large number of joins

2005-06-02 Thread philb
Hi, I am using PostgreSQL (7.4) with a schema that was generated automatically (using hibernate). The schema consists of about 650 relations. One particular query (also generated automatically) consists of left joining approximately 350 tables. At this stage, most tables are empty and those

Re: [PERFORM] SURVEY: who is running postgresql on 8 or more CPUs?

2005-06-02 Thread ohp
On Tue, 31 May 2005, Dirk Lutzebäck wrote: > Date: Tue, 31 May 2005 15:16:37 +0200 > From: Dirk Lutzebäck <[EMAIL PROTECTED]> > To: pgsql-performance@postgresql.org > Subject: [PERFORM] SURVEY: who is running postgresql on 8 or more CPUs? > > Hi, > > I would like to start a little survey who is ru

Re: [PERFORM] How to avoid database bloat

2005-06-02 Thread Matthew T. O'Connor
Mindaugas Riauba wrote: AFAICT the vacuum is doing what it is supposed to, and the problem has to be just that it's not being done often enough. Which suggests either an autovacuum bug or your autovacuum settings aren't aggressive enough. -D -d 1 -v 1000 -V 0.5 -a 1000 -A 0.1 -s 10 Th

Re: [PERFORM] Query plan for very large number of joins

2005-06-02 Thread Richard Huxton
[EMAIL PROTECTED] wrote: Hi, I am using PostgreSQL (7.4) with a schema that was generated automatically (using hibernate). The schema consists of about 650 relations. One particular query (also generated automatically) consists of left joining approximately 350 tables. May I be the first to of

Re: [PERFORM] How to avoid database bloat

2005-06-02 Thread Tom Lane
"Mindaugas Riauba" <[EMAIL PROTECTED]> writes: > And what in vacuum verbose output suggests that vacuum is not done > often enough? Current output (table is 100MB already) is below. The output shows vacuum cleaning up about a third of the table. Usually people like to keep the overhead down to

Re: [PERFORM] Query plan for very large number of joins

2005-06-02 Thread Tom Lane
Richard Huxton writes: > [EMAIL PROTECTED] wrote: >> I am using PostgreSQL (7.4) with a schema that was generated >> automatically (using hibernate). The schema consists of about 650 >> relations. One particular query (also generated automatically) >> consists of left joining approximately 350 tab

Re: [PERFORM] Adaptec/LSI/?? RAID

2005-06-02 Thread J. Andrew Rogers
I've got a bunch of mission-critical Postgres servers on Opterons, all with no less than 4GB RAM, running Linux + XFS, and most with LSI MegaRAID cards. We've never had a single system crash or failure on our postgres servers, and some of them are well-used and with uptimes in excess of a y

Re: [PERFORM] Adaptec/LSI/?? RAID

2005-06-02 Thread Joshua D. Drake
It may be anecdotal, but LSI MegaRAID cards generally seem to work pretty well with Linux. The only problem I've ever seen was a BIOS problem between the LSI and the motherboard, which was solved by flashing the BIOS on the motherboard with the latest version (it was grossly out of date anyw

Re: [PERFORM] Adaptec/LSI/?? RAID

2005-06-02 Thread Vivek Khera
On Jun 1, 2005, at 11:42 PM, Stacy White wrote: "we find LSI does not work well with 4GB of RAM. Our engineering find that LSI card could cause system crashes. One of our customer ... has found that Adaptec cards works well on PostGres SQL -- they're using it as a preforce server with xfs

Re: [PERFORM] Query plan for very large number of joins

2005-06-02 Thread PFC
I am using PostgreSQL (7.4) with a schema that was generated automatically (using hibernate). The schema consists of about 650 relations. One particular query (also generated automatically) consists of left joining approximately 350 tables. At this Just out of curiosity, what appl

Re: [PERFORM] Adaptec/LSI/?? RAID (what about JBOD?)

2005-06-02 Thread mudfoot
I have a similar question about what to choose (either LSI or Adaptec U320), but plan to use them just for JBOD drivers. I expect to be using either net or freebsd. The system CPU will be Opteron. My impression is that both the ahd and mpt drivers (for U320 Adaptec and LSI, respectively) are qui

Re: [PERFORM] Query plan for very large number of joins

2005-06-02 Thread Sebastian Hennebrueder
Tom Lane schrieb: Richard Huxton writes: [EMAIL PROTECTED] wrote: I am using PostgreSQL (7.4) with a schema that was generated automatically (using hibernate). The schema consists of about 650 relations. One particular query (also generated automatically) consists of left joining ap

Re: [PERFORM] Adaptec/LSI/?? RAID (what about JBOD?)

2005-06-02 Thread Andrew McMillan
On Thu, 2005-06-02 at 14:02 -0700, [EMAIL PROTECTED] wrote: > I have a similar question about what to choose (either LSI or Adaptec U320), > but > plan to use them just for JBOD drivers. I expect to be using either net or > freebsd. The system CPU will be Opteron. My impression is that both the

Re: [PERFORM] Adaptec/LSI/?? RAID (what about JBOD?)

2005-06-02 Thread mudfoot
Thanks, Andrew. I expect to choose between HBAs with no RAID functionality or with the option to completely bypass RAID functionality--meaning that I'll hopefully avoid the situation that you've described. I'm mostly curious as to whether the driver problems described for U320 Adaptec RAID contro

SHMMAX / SHMALL Was (Re: [PERFORM] postgresql-8.0.1 performance tuning)

2005-06-02 Thread Paul McGarry
On 6/1/05, Mark Kirkwood <[EMAIL PROTECTED]> wrote: > Cosimo Streppone wrote: > > # Config > >> /etc/sysctl.conf: > >> kernel.shmall = 786432000 > >> kernel.shmmax = 786432000 > > > > I think you have a problem here. > > kernel.shmmax should *not* be set to an amount of RAM, bu

Re: SHMMAX / SHMALL Was (Re: [PERFORM] postgresql-8.0.1 performance

2005-06-02 Thread Mark Kirkwood
Paul McGarry wrote: Based on the powerpostgresql.com Performance Checklist [1] and Annotated Postgresql.conf [2] I understand that: -I should have less than 1/3 of my total memory as shared_buffers -For my server 15000 is a fairly reasonable starting point for shared_buffers which is ~120MB -I

[PERFORM] Query limitations (size, number of UNIONs ...)

2005-06-02 Thread Marc Mamin
Hello, I've split my data in daily tables to keep them in an acceptable size. Now I have quite complex queries which may be very long if I need to query a large number of daily tables. I've just made a first test wich resulted in a query being 15KB big annd containing 63 UNION. The Query pla