Re: [PERFORM] [ADMIN] syslog slowing the database?

2004-03-15 Thread scott.marlowe
On Thu, 11 Mar 2004, Christopher Kings-Lynne wrote: > > You could also consider not using syslog at all: let the postmaster > > output to its stderr, and pipe that into a log-rotation program. > > I believe some people use Apache's log rotator for this with good > > results. > > Not an option I'm

Re: [PERFORM] [ADMIN] syslog slowing the database?

2004-03-15 Thread scott.marlowe
On Mon, 15 Mar 2004, Bruce Momjian wrote: > scott.marlowe wrote: > > On Thu, 11 Mar 2004, Christopher Kings-Lynne wrote: > > > > > > You could also consider not using syslog at all: let the postmaster > > > > output to its stderr, and pipe that into a log-r

Re: [PERFORM] [ADMIN] Benchmarking postgres on Solaris/Linux

2004-03-23 Thread scott.marlowe
On Tue, 23 Mar 2004, Josh Berkus wrote: > Matt, Stalin, > > > As for the compute intensive side (complex joins & sorts etc), the Dell will > most likely beat the Sun by some distance, although > > what the Sun lacks in CPU power it may make up a bit in memory bandwidth/ > latency. > > Personall

Re: [PERFORM] slow vacuum performance

2004-03-24 Thread scott.marlowe
On Wed, 24 Mar 2004, pginfo wrote: > Hi, > > I am running pg 7.4.1 on linux box. > I have a midle size DB with many updates and after it I try to run > vacuum full analyze. Is there a reason to not use just regular vacuum / analyze (i.e. NOT full)? > It takes about 2 h. Full vacuums, by the

Re: [PERFORM] slow vacuum performance

2004-03-24 Thread scott.marlowe
On Wed, 24 Mar 2004, pginfo wrote: > Hi, > > scott.marlowe wrote: > > > On Wed, 24 Mar 2004, pginfo wrote: > > > > > Hi, > > > > > > I am running pg 7.4.1 on linux box. > > > I have a midle size DB with many updates and after it I try

Re: [PERFORM] postgres eating CPU on HP9000

2004-03-26 Thread scott.marlowe
On Fri, 26 Mar 2004, Fabio Esposito wrote: > > On Fri, 26 Mar 2004, scott.marlowe wrote: > > > > It maintains 48hours of data, so its not a large database; roughly > > > <600mbs. We do this by running a housekeeping program in a cron job. > > > It de

Re: [PERFORM] odd planner choice

2004-03-26 Thread scott.marlowe
On Thu, 25 Mar 2004, Ara Anjargolian wrote: > I've run into this odd planner choice which I don't quite understand. > > I have two tables articles, users and > articles.article_id and users.user_id are primary keys. > > Insides articles there are two optional fields author_id1, author_id2 > whic

Re: [PERFORM] postgres eating CPU on HP9000

2004-03-26 Thread scott.marlowe
On Tue, 23 Mar 2004, Fabio Esposito wrote: > > Hello fellow PostgreSQL users. > > We've been working on this interesting issue for some time now, and we're > hoping that someone can help. > > We've recently integrated postgres into an existing mature app. Its a > time sensitive 24x7 system. I

Re: [PERFORM] atrocious update performance

2004-04-05 Thread scott.marlowe
On Mon, 5 Apr 2004, Kevin Barnard wrote: > On 5 Apr 2004 at 12:05, Rosser Schwarz wrote: > > > Just this morning, however, I created a copy of the target table (all > > 4.7M rows), with absolutely no foreign keys referring to it, and ran > > the update against the copy. That update took 2300 sec

Re: [PERFORM] good pc but bad performance,why?

2004-04-07 Thread scott.marlowe
On Wed, 7 Apr 2004, Andrew McMillan wrote: > On Wed, 2004-04-07 at 20:56, huang yaqin wrote: > > Hello, Richard Huxton, > > > > You said turning fsync off may cause losing data, that's terrible. > > I use SCSI disk, and file system is ext3. I tune postgresql.conf and can't ge

Re: [PERFORM] Postgresql educational sources

2004-04-07 Thread scott.marlowe
On Wed, 7 Apr 2004, Nid wrote: > Rather than ask some general, unanswerable question on how to tune my > database...I thought I ask where I might find an introduction to...or > manual/tutorial for the depths of managing a postgres db. Books? > Websites? Assume a basic to intermediate knowled

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-19 Thread scott.marlowe
On Mon, 19 Apr 2004, Bruce Momjian wrote: > Josh Berkus wrote: > > Tom, > > > > > So in the short term I think we have to tell people that Xeon MP is not > > > the most desirable SMP platform to run Postgres on. (Josh thinks that > > > the specific motherboard chipset being used in these machine

Re: [PERFORM] Setting Shared Buffers , Effective Cache, Sort Mem

2004-04-22 Thread scott.marlowe
On Thu, 22 Apr 2004, Pallav Kalva wrote: > Hi > > We are in the process of building a new machine for our production > database. Below you will see some of the harware specs for the machine. > I need some help with setting these parameters (shared buffers, > effective cache, sort mem) in t

Re: [PERFORM] Looking for ideas on how to speed up warehouse loading

2004-04-23 Thread scott.marlowe
On Thu, 22 Apr 2004, Sean Shanny wrote: > I should have included this as well: > fsync | on > shared_buffers | 4000 > sort_mem | 64000 For purposes of loading only, you can try turning off fsync, assuming this is a virgin load an

Re: [PERFORM] Help with performance problems

2004-04-23 Thread scott.marlowe
On Fri, 23 Apr 2004, Chris Hoover wrote: > DB's on Powervaults 220S using raid 5 (over 6 disks) What controller is this, the adaptec? We've found it to be slower than the LSI megaraid based controller, but YMMV. > Running RH ES 2.1 Are you running the latest kernel for ES 2.1? Early 2.4 kern

Re: [PERFORM] Help with performance problems

2004-04-23 Thread scott.marlowe
On Fri, 23 Apr 2004, Chris Hoover wrote: > On Friday 23 April 2004 13:21, scott.marlowe wrote: > > On Fri, 23 Apr 2004, Chris Hoover wrote: > > > DB's on Powervaults 220S using raid 5 (over 6 disks) > > > > What controller is this, the adaptec? We'v

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Paul Tuckfield wrote: > If you are having a "write storm" or bursty writes that's burying > performance, a scsi raid controler with writeback cache will greatly > improve the situation, but I do believe they run around $1-2k. If > it's write specific problem, the cache ma

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Paul Tuckfield wrote: > Love that froogle. > > It looks like a nice card. One thing I didn't get straight is if > the cache is writethru or write back. > > If the original posters problem is truly a burst write problem (and not > linux caching or virtual memory overcom

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Rob Sell wrote: > > If you don't mind slumming on ebay :-) keep an eye out for PERC III cards, > they are dell branded LSI cards. Perc = Power Edge Raid Controller. There > are models on there dual channel u320 and dell usually sells them with > battery backed cache. That's

Re: [PERFORM] Quad processor options

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Allan Wind wrote: > On 2004-05-11T15:29:46-0600, scott.marlowe wrote: > > The other nice thing about the LSI cards is that you can install >1 and > > the act like one big RAID array. i.e. install two cards with a 20 drive > > RAID0 then make a RAID1

Re: [PERFORM] Configuring PostgreSQL to minimize impact of checkpoints

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004 [EMAIL PROTECTED] wrote: > Quoting Rob Fielding <[EMAIL PROTECTED]>: > > > Assuming you're running with optimal schema and index design (ie you're > > not doing extra work unnecessarily), and your backend has > > better-then-default config options set-up (plenty of tips arou

Re: [PERFORM] Why queries takes too much time to execute?

2004-05-10 Thread scott.marlowe
On Mon, 10 May 2004, Anderson Boechat Lopes wrote: > Hum... now i think i´m beginning to understand. > > The vacuum analyse is recommended to perform at least every day, after > adding or deleting a large number of records, and not vacuum full analyse. > I´ve performed the vacuum full ana

Re: [PERFORM] Help how to tune-up my Database

2004-05-10 Thread scott.marlowe
Sorry about that, I meant kbytes, not megs. My point being it's NOT measured in 8k blocks, like a lot of other settings. sorry for the mixup. On Fri, 7 May 2004, Bricklen wrote: > scott.marlowe wrote: > > sort_mem might do with a small bump, especially if you're only

Re: [PERFORM] Help how to tune-up my Database

2004-05-07 Thread scott.marlowe
On Fri, 7 May 2004, Michael Ryan S. Puncia wrote: > Hi, > > I am a newbie here and just starting to use postgresql. My > problems is how to tune up my server because it its too slow. First, read this: http://www.varlena.com/varlena/GeneralBits/Tidbits/perf.html > This is my PC spec

Re: [PERFORM] cache table

2004-05-04 Thread scott.marlowe
On Mon, 3 May 2004, Joseph Shraibman wrote: > I have a big table with some int fields. I frequently need to do > queries like: > > SELECT if2, count(*) FROM table WHERE if1 = 20 GROUP BY if2; > > The problem is that this is slow and frequently requires a seqscan. I'd > like to cache the resul

Re: [PERFORM] OT: Help with performance problems

2004-04-27 Thread scott.marlowe
On Mon, 26 Apr 2004, Rob Fielding wrote: > scott.marlowe wrote: > > On Fri, 23 Apr 2004, Chris Hoover wrote: > > > > > >>DB's on Powervaults 220S using raid 5 (over 6 disks) > > > > > > What controller is this, the adaptec? We'

Re: [PERFORM] index usage

2004-04-28 Thread scott.marlowe
On Mon, 26 Apr 2004, Stephan Szabo wrote: > > On Fri, 23 Apr 2004 [EMAIL PROTECTED] wrote: > > > I have a query which I think should be using an index all of the time but > > postgres only uses the index part of the time. The index > > (ticket_crm_map_crm_id_suppid) has the where clause column

Re: [PERFORM] [ADMIN] 7.4.2 out of memory

2004-04-28 Thread scott.marlowe
On Wed, 28 Apr 2004, Jie Liang wrote: > All, > After I upgraded postgres from 7.3.4 to 7.4.2, one of my program got following error: > DRROR:out of memory > DETAIL: Fail on request of size 92. > > any idea?? > does memory management have big difference between 7.3.4 and 7.4.2??? > t

Re: [PERFORM] [OFF-TOPIC] - Known maximum size of the PostgreSQL

2004-05-05 Thread scott.marlowe
On Sat, 5 Jun 2004, Carlos Eduardo Smanioto wrote: > Hello all, > > What's the case of bigger database PostgreSQL (so greate and amount of > registers) that they know??? http://www.postgresql.org/docs/faqs/FAQ.html#4.5 ---(end of broadcast)--- TI

Re: [PERFORM] Quad processor options

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Bjoern Metzdorf wrote: > Hi, > > I am curious if there are any real life production quad processor setups > running postgresql out there. Since postgresql lacks a proper > replication/cluster solution, we have to buy a bigger machine. > > Right now we are running on a dual

Re: [PERFORM] Quad processor options

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Bjoern Metzdorf wrote: > scott.marlowe wrote: > > > Well, from what I've read elsewhere on the internet, it would seem the > > Opterons scale better to 4 CPUs than the basic Xeons do. Of course, the > > exception to this is SGI's altix

Re: [PERFORM] Quad processor options

2004-05-11 Thread scott.marlowe
On Tue, 11 May 2004, Bjoern Metzdorf wrote: > scott.marlowe wrote: > > Sure, adaptec makes one, so does lsi megaraid. Dell resells both of > > these, the PERC3DI and the PERC3DC are adaptec, then lsi in that order, I > > believe. We run the lsi megaraid with 64 megs

Re: [ADMIN] [PERFORM] Quad processor options

2004-05-12 Thread scott.marlowe
On Wed, 12 May 2004, Grega Bremec wrote: > ...and on Tue, May 11, 2004 at 03:02:24PM -0600, scott.marlowe used the keyboard: > > > > If you get the LSI megaraid, make sure you're running the latest megaraid > > 2 driver, not the older, slower 1.18 series. If you ar

Re: [PERFORM] Disappointing performance in db migrated from MS SQL

2004-06-03 Thread scott.marlowe
On Fri, 13 Feb 2004 [EMAIL PROTECTED] wrote: > > Josh, the disks in the new system should be substantially faster than > > the old. Both are Ultra160 SCSI RAID 5 arrays, but the new system has > > 15k RPM disks, as opposed to the 10k RPM disks in the old system. > > Spindle speed does not correla

Re: [PERFORM] Tables on multiple disk drives

2004-06-03 Thread scott.marlowe
On Tue, 17 Feb 2004, Craig Thomas wrote: > > On Tue, 17 Feb 2004, Konstantin Tokar wrote: > > > >> Hi! > >> Does PostgreSQL allow to create tables and indices of a single > >> database on multiple disk drives with a purpose of increase > >> performance as Oracle database does? If a symbolic refere

Re: [PERFORM] General performance questions about postgres on Apple

2004-06-05 Thread scott.marlowe
On Fri, 20 Feb 2004, Sean Shanny wrote: > max_connections = 100 > > # - Memory - > > shared_buffers = 16000 # min 16, at least max_connections*2, > 8KB each > sort_mem = 256000 # min 64, size in KB You might wanna drop sort_mem somewhat and just set it during your impor

Re: [PERFORM] Slow query

2004-06-09 Thread scott.marlowe
On Tue, 24 Feb 2004, Ivan Voras wrote: > -> Nested Loop (cost=1.04..788.76 rows=3 width=500) (actual > time=4078.85..20185.89 rows=38999 loops=1) > -> Nested Loop (cost=1.04..771.27 rows=3 width=485) > (actual time=4078.71..14673.27 rows=38999 loops=1) >

Re: [PERFORM] Memory question

2003-06-27 Thread scott.marlowe
This is actually normal. Look at the amount cached: 6257620K. That's 6.2Gig of cache. Linux is using only 6517776k - 6257620k of memory, the rest is just acting as kernel cache. If anything tries to allocate a bit of memory, linux will flush enough cache to give the memory to the applicatio

Re: [PERFORM] Memory question

2003-06-27 Thread scott.marlowe
On Fri, 27 Jun 2003, Matthew Hixson wrote: > On Friday, June 27, 2003, at 01:17 PM, Jord Tanner wrote: > > I've heard anecdotally that Linux has troubles if the swap space is > > less > > than the RAM size. I note that you have 6G of RAM, but only 2G of swap. > > I've heard that too, but it does

Re: [PERFORM] Version 7 question

2003-07-01 Thread scott.marlowe
8192 is only 64 megs of RAM, not much, but a good number. Keep in mind that the kernel tends to be better at buffering huge amounts of disk, while postgresql is better left to use buffers that are large enough for the current working set (i.e. not your whole database, just the largest amount o

Re: FW: [PERFORM] Version 7 question

2003-07-01 Thread scott.marlowe
uld take > into account in determining the value? > > > > > -Original Message----- > > From: scott.marlowe [SMTP:[EMAIL PROTECTED] > > Sent: 01 July 2003 02:56 > > To: Michael Mattox > > Cc: Hilary Forbes; [EMAIL PROTECTED] > > Subjec

Re: [PERFORM] Version 7 question

2003-07-01 Thread scott.marlowe
On Tue, 1 Jul 2003, Michael Mattox wrote: > My understanding is to use as much effect cache as possible, so figure out > how much ram you need for your other applications & OS and then give the > rest to postgres as effective cache. > > What I learned to day is the shared_buffers 25% of RAM guide

Re: [PERFORM] Effective Cache Size

2003-07-01 Thread scott.marlowe
kernel's disk > cache that will be used for PostgreSQL data files ). > > What then will be the effect of setting this too high? > > And too low? > > How does it impact on other applications eg Java ? > > > > -Original Message- > > From:

Re: [PERFORM] Version 7 question

2003-07-01 Thread scott.marlowe
ve_cache_size > > on Postgres 7.3.2, assuming say 1.5 GB of RAM and > > shared_buffers set to 8192, and shmmax set to 750mb? > > > > And what are the most important factors one should take > > into account in determining the value? > > > > > > >

Re: [PERFORM] Moving postgresql.conf tunables into 2003...

2003-07-03 Thread scott.marlowe
On Thu, 3 Jul 2003, Sean Chittenden wrote: > What are the odds of going through and revamping some of the tunables > in postgresql.conf for the 7.4 release? I was just working with > someone on IRC and on their 7800 RPM IDE drives, their > random_page_cost was ideally suited to be 0.32: a far cry

Re: [PERFORM] Extreme high load averages

2003-07-07 Thread scott.marlowe
On Sun, 6 Jul 2003, Martin Foster wrote: > Shridhar Daithankar wrote: > > > > It gives hint to psotgresql how much file system cache is available in the > > system. > > > > You have 1GB memory and your application requirement does not exceed 400MB. So > > OS can use roughly 600MB for file sys

Re: [PERFORM] PostgreSQL vs. MySQL

2003-07-07 Thread scott.marlowe
On Mon, 7 Jul 2003, Brian Tarbox wrote: > Oddly enough, the particular application in question will have an extremely > small user base...perhaps a few simultainous users at most. > > As to the testing, I neglected to say early in this thread that my manager > instructed me _not_ to do further pe

Re: [PERFORM] PostgreSQL vs. MySQL

2003-07-07 Thread scott.marlowe
On Fri, 4 Jul 2003, Brian Tarbox wrote: > I'm actually leaving this list but I can answer this question. Our results > were with a single user and we were running Inodb. We were running on > RedHat 8.0 / 9.0 with vanilla linux settings. Hi Brian, I just wanted to add that if you aren't testing

Re: [PERFORM] [pgsql-advocacy] About the default performance

2003-07-07 Thread scott.marlowe
I'm willing to help too. I'm basically a DBA / developer type, with mild C hacking skills (I develop in PHP, so my C coding is quite rusty nowadays.) If nothing else testing on different equipment / OSes. On Fri, 4 Jul 2003, Josh Berkus wrote: > Kaarel: > > (cross-posted back to Performance

Re: [PERFORM] Extreme high load averages

2003-07-08 Thread scott.marlowe
On Mon, 7 Jul 2003, Matthew Nuzum wrote: > > A common problem is a table like this: > > > > create table test (info text, id int8 primary key); > > insert into test values ('ted',1); > > .. a few thousand more inserts; > > vacuum full; > > analyze; > > select * from test where id=1; > > > > will

Re: [PERFORM] Moving postgresql.conf tunables into 2003...

2003-07-09 Thread scott.marlowe
On Wed, 9 Jul 2003, Kaarel wrote: > >>Are you willing to say that the PostgreSQL database system should only be > >>used by DBAs? I believe that Postgres is such a good and useful tool that > >>anyone should be able to start using it with little or no barrier to entry. > > > > > > I quite agree

Re: [PERFORM] index / sequential scan problem

2003-07-18 Thread scott.marlowe
On Fri, 18 Jul 2003, Tom Lane wrote: > =?ISO-8859-1?Q?Dennis_Bj=F6rklund?= <[EMAIL PROTECTED]> writes: > > On Fri, 18 Jul 2003, Fabian Kreitner wrote: > >> Adjusting the cpu_tuple_cost to 0.042 got the planner to choose the index. > > > Doesn't sound very good and it will most likely make other q

Re: [PERFORM] Sanity check requested

2003-07-18 Thread scott.marlowe
On Fri, 18 Jul 2003, Ang Chin Han wrote: > Shridhar Daithankar wrote: > > On 17 Jul 2003 at 10:41, Nick Fankhauser wrote: > > > >>I'm using ext2. For now, I'll leave this and the OS version alone. If I > > > > > > I appreciate your approach but it almost proven that ext2 is not the best and >

Re: [PERFORM] Commenting postgresql.conf

2003-07-21 Thread scott.marlowe
On Mon, 21 Jul 2003, Josh Berkus wrote: > Folks, > > There was a general consensus (I think) on this list that we want more verbose > comments in postgresql.conf for 7.4. Is anyone available to do the work? > We'll need the patch this week ... I'll help. this is probably the kind of thing

Re: [PERFORM] Tuning PostgreSQL

2003-07-22 Thread scott.marlowe
On Tue, 22 Jul 2003, Jim C. Nasby wrote: > On Tue, Jul 22, 2003 at 03:27:20PM +0200, Alexander Priem wrote: > > Wow, I never figured how many different RAID configurations one could think > > of :) > > > > After reading lots of material, forums and of course, this mailing-list, I > > think I am

Re: [PERFORM] Mapping Database completly into memory

2003-07-28 Thread scott.marlowe
On Sun, 27 Jul 2003, Daniel Migowski wrote: > Hallo pgsql-performance, > > I just wondered if there is a possibility to map my database running > on a linux system completly into memory and to only use disk > accesses for writes. > > I got a nice machine around with 2 gigs of ram, and my databas

Re: [PERFORM] Tuning PostgreSQL

2003-07-29 Thread scott.marlowe
On 29 Jul 2003, Ron Johnson wrote: > On Tue, 2003-07-29 at 10:14, Vivek Khera wrote: > > >>>>> "GS" == Greg Stark <[EMAIL PROTECTED]> writes: > > > > GS> "scott.marlowe" <[EMAIL PROTECTED]> writes: > > > > GS&

Re: [PERFORM] Tuning PostgreSQL

2003-07-29 Thread scott.marlowe
On 29 Jul 2003, Ron Johnson wrote: > On Tue, 2003-07-29 at 11:18, scott.marlowe wrote: > > On 29 Jul 2003, Ron Johnson wrote: > > > > > On Tue, 2003-07-29 at 10:14, Vivek Khera wrote: > > > > >>>>> "GS" == Greg Stark <[EMAIL

Re: [PERFORM] Tuning PostgreSQL

2003-07-29 Thread scott.marlowe
On 29 Jul 2003, Ron Johnson wrote: > On Tue, 2003-07-29 at 14:00, scott.marlowe wrote: > > On 29 Jul 2003, Ron Johnson wrote: > > > > > On Tue, 2003-07-29 at 11:18, scott.marlowe wrote: > > > > On 29 Jul 2003, Ron Johnson wrote: > > > > > >

[PERFORM] postgresql.conf

2003-07-30 Thread scott.marlowe
I'm looking at doing the example postgresql.conf files for the 7.4 release. So far, the catagories we have would be a matrix of: -- Large Machine -- Small Machine Webserver OLAP OLTP Workstation But likely only one entry for workstation. anyone have any advice on what they use in wh

Re: [PERFORM] postgresql.conf

2003-07-30 Thread scott.marlowe
On Wed, 30 Jul 2003, Josh Berkus wrote: > Ron, > > > I don't know if this is representative of other postgresql installs, but > > I would also put in my vote for the differentiation added, as these are > > not small machines but are multi-server boxes. > > But how is the Multi-purpose configur

Re: [PERFORM] OSDL Database Test Suite 3 is available on PostgreSQL

2003-08-04 Thread scott.marlowe
On Mon, 4 Aug 2003, Manfred Koizar wrote: > On 01 Aug 2003 11:04:10 -0700, Jenny Zhang <[EMAIL PROTECTED]> wrote: > >A sample OSDL-DBT3 test result report can be found at: > >http://khack.osdl.org/stp/276912/ > > > >Your comments are welcome, > > | effective_cache_size | 1000 > > With

Re: [PERFORM] OSDL Database Test Suite 3 is available on PostgreSQL

2003-08-04 Thread scott.marlowe
On Mon, 4 Aug 2003, Manfred Koizar wrote: > On 01 Aug 2003 11:04:10 -0700, Jenny Zhang <[EMAIL PROTECTED]> wrote: > >A sample OSDL-DBT3 test result report can be found at: > >http://khack.osdl.org/stp/276912/ > > > >Your comments are welcome, > > | effective_cache_size | 1000 > > With

Re: [osdldbt-general] Re: [PERFORM] OSDL Database Test Suite 3 is

2003-08-04 Thread scott.marlowe
On 4 Aug 2003, Jenny Zhang wrote: > On Mon, 2003-08-04 at 06:33, Manfred Koizar wrote: > > | effective_cache_size | 1000 > > > > With 4GB of memory this is definitely too low and *can* (note that I > > don't say *must*) lead the planner to wrong decisions. > > > I changed the default t

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-11 Thread scott.marlowe
On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > Hi All! > > > Richard Huxton wrote: > > >>>On Wednesday 06 August 2003 08:34, Yaroslav Mazurak wrote: > > sort_mem = 131072 > > >>>This sort_mem value is *very* large - that's 131MB for *each sort* that > > It's not TOO la

Re: [PERFORM] ext3 block size

2003-08-11 Thread scott.marlowe
On Wed, 6 Aug 2003, Wilson A. Galafassi Jr. wrote: > hello. > my database size is 5GB. what is the block size recommend? Well, the biggest block size currently supported by stock linux distros is 4k, so I'd go with that. Postgresql's default block size of 8k is fine also. Note that linux page

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Fri, 8 Aug 2003, mixo wrote: > I have just installed redhat linux 9 which ships with Pg > 7.3.2. Pg has to be setup so that data inserts (blobs) should > be able to handle at least 8M at a time. Nothing has to be done to tune postgresql to handle this, 8 Meg blobs are no problem as far as I k

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread scott.marlowe
On Wed, 13 Aug 2003, ingrid martinez wrote: > Floes table looks like this > > Table "flows" > Column| Type | Modifiers > --+--+--- > flidload | bigint | n

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On 11 Aug 2003, Ron Johnson wrote: > On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote: > > > Well, yeah. But given the Linux propensity for introducing major > > > features in "minor" releases (and thereby introducing all the > > > attendant bugs), I'd think twice about using _any_ Linu

Re: [PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread scott.marlowe
On Wed, 6 Aug 2003, Tom Lane wrote: > "scott.marlowe" <[EMAIL PROTECTED]> writes: > > On Wed, 6 Aug 2003, Tom Lane wrote: > >> One obvious question is whether you have your foreign keys set up > >> efficiently in the first place. As a rule, the referenc

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Fri, 8 Aug 2003, Andrew Sullivan wrote: > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote: > > > > Redhat puts ext3 on by default. Consider switching to a non-journaling FS > > (ext2?) with the partition that holds your data and WAL. > > I would give you exactly the opposite

Re: [PERFORM] partitioning for postgresql

2003-08-14 Thread scott.marlowe
On Wed, 6 Aug 2003, Wilson A. Galafassi Jr. wrote: > hello!!! > what is suggested partitioning schema for postgresql?? > the size of my db is 5BG and i have 36GB scsi disk! The first recommendation is to run Postgresql on a RAID set for reliability. I'm assuming you're building a machine and

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread scott.marlowe
More than likely you are suffering from an affliction known as type mismatch. This is listed as tip 9 here on the performance list (funny, it was sent at the bottom of your reply :-) What happens is that when you do: select * from some_table where id=123; where id is a bigint the query planne

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread scott.marlowe
On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > Hi All! > > Shridhar Daithankar wrote: > > > On 7 Aug 2003 at 10:05, Yaroslav Mazurak wrote: > > >>>It needs to reflect how much cache the system is using - try the "free" > >>>command to see figures. > > >>I'm not found "free" ut

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread scott.marlowe
On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > scott.marlowe wrote: > > > On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > > >>Shridhar Daithankar wrote: > > > That's a nice theory, but it doesn't work out that way. About every two > > months s

Re: [PERFORM] On Linux Filesystems

2003-08-14 Thread scott.marlowe
On Tue, 12 Aug 2003, Andrew Sullivan wrote: > On Mon, Aug 11, 2003 at 10:58:18PM -0400, Christopher Browne wrote: > > 1. Nobody has gone through any formal proofs, and there are few > > systems _anywhere_ that are 100% reliable. > > I think the problem is that ext2 is known to be not perfectly

Re: [PERFORM] How Many Inserts Per Transactions

2003-08-14 Thread scott.marlowe
On Tue, 5 Aug 2003, Trevor Astrope wrote: > I was wondering if anyone found a sweet spot regarding how many inserts to > do in a single transaction to get the best performance? Is there an > approximate number where there isn't any more performance to be had or > performance may drop off? > >

Re: [PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread scott.marlowe
On Wed, 6 Aug 2003, Tom Lane wrote: > Sebastien Lemieux <[EMAIL PROTECTED]> writes: > > All the time is taken at the commit of both transaction. > > Sounds like the culprit is foreign-key checks. > > One obvious question is whether you have your foreign keys set up > efficiently in the first pla

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Tue, 12 Aug 2003, Neil Conway wrote: > On Tue, Aug 12, 2003 at 12:52:46AM -0400, Bruce Momjian wrote: > I don't use Linux and was just repeating what I had heard from others, > > and read in postings. I don't have any first-hand experience with ext2 > > (except for a laptop I borrowed that wo

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Tue, 12 Aug 2003, Christopher Kings-Lynne wrote: > > Well, yeah. But given the Linux propensity for introducing major > > features in "minor" releases (and thereby introducing all the > > attendant bugs), I'd think twice about using _any_ Linux feature > > until it's been through a major versi

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Mon, 11 Aug 2003, Bruce Momjian wrote: > scott.marlowe wrote: > > On Fri, 8 Aug 2003, Andrew Sullivan wrote: > > > > > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote: > > > > > > > > Redhat puts ext3 on by default. Consider

Re: [PERFORM] Tests

2003-08-25 Thread scott.marlowe
On Fri, 22 Aug 2003, Josh Berkus wrote: > Also another test I'd really like to see is hardware raid (Adaptec, LSI) > against Linux software raid, and 5-disk RAID 5 against 4-disk RAID 1+0. It would be nice to cross those, so you have RAID5 sw vs RAID5 hw, vs RAID1+0 sw vs RAID1+0 hw. Now if on

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread scott.marlowe
On 27 Aug 2003, matt wrote: > I'm wondering if the good people out there could perhaps give me some > pointers on suitable hardware to solve an upcoming performance issue. > I've never really dealt with these kinds of loads before, so any > experience you guys have would be invaluable. Apologies

Re: [PERFORM] Best tweak for fast results.. ?

2003-08-28 Thread scott.marlowe
On Tue, 26 Aug 2003, JM wrote: > need input on parameter values on confs... > > our database is getting 1000 transactions/sec on peak periods.. > > sitting on RH 7.3 > 2.4.7-10smp > RAM: 1028400 > SWAP: 2040244 1: Upgrade your kernel. 2.4.7 on RH3 was updated to 2.4.18-24 in March, and the

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread scott.marlowe
On 27 Aug 2003, matt wrote: > > You probably, more than anything, should look at some kind of > > superfast, external storage array > > Yeah, I think that's going to be a given. Low end EMC FibreChannel > boxes can do around 20,000 IOs/sec, which is probably close to good > enough. > > You men

Re: [PERFORM] opinion on RAID choice

2003-08-28 Thread scott.marlowe
On Thu, 28 Aug 2003, Vivek Khera wrote: > I just ran a handful of tests on a 14-disk array on a SCSI hardware > RAID card. SNIP > Has anyone else done similar tests of different RAID levels? What > were your conclusions? Yes I have. I had a 6 disk array plus 2 disks inside my machine (this wa

Re: [PERFORM] The results of my PostgreSQL/filesystem performance

2003-08-28 Thread scott.marlowe
On Thu, 28 Aug 2003, Sean Chittenden wrote: > > What it still leaves quite open is just what happens when the OS has > > more than one disk drive or CPU to play with. It's not clear what > > happens in such cases, whether FreeBSD would catch up, or be "left > > further in the dust." The traditio

Re: [PERFORM] Queries sometimes take 1000 times the normal time

2003-08-28 Thread scott.marlowe
Just to add to the clutch here, also check your bdflush settings (if you're on linux) or equivalent (if you're not.) Many times the swapping algo in linux can be quite bursty if you have it set to move too many pages at a time during cleanup / flush. ---(end of broadca

Re: [PERFORM] opinion on RAID choice

2003-08-29 Thread scott.marlowe
On Thu, 28 Aug 2003, Vivek Khera wrote: > > "sm" == scott marlowe writes: > > sm> My experience has been that once you get past 6 disks, RAID5 is faster > sm> than RAID1+0. > > Any opinion on stripe size for the RAID? That's more determined by what kind of data you're gonna be handling.

Re: [PERFORM] Selecting random rows efficiently

2003-09-03 Thread scott.marlowe
Can you just create an extra serial column and make sure that one is always in order and no holes in it? (i.e. a nightly process, etc...)??? If so, then something like this truly flies: select * from accounts where aid = (select cast(floor(random()*10)+1 as int)); My times on it on a 100,0

Re: [PERFORM] FreeBSD page size (was Re: The results of my

2003-09-04 Thread scott.marlowe
On Wed, 3 Sep 2003, Vivek Khera wrote: > > "SC" == Sean Chittenden <[EMAIL PROTECTED]> writes: > > >> I need to step in and do 2 things: > SC> Thanks for posting that. Let me know if you have any questions while > SC> doing your testing. I've found that using 16K blocks on FreeBSD > SC> res

Re: [PERFORM] SELECT's take a long time compared to other DBMS

2003-09-04 Thread scott.marlowe
On Wed, 3 Sep 2003, Relaxin wrote: > I have a table with 102,384 records in it, each record is 934 bytes. > > Using the follow select statement: > SELECT * from > > PG Info: version 7.3.4 under cygwin on Windows 2000 > ODBC: version 7.3.100 > > Machine: 500 Mhz/ 512MB RAM / IDE HDD > > > U

Re: [PERFORM] PostgreSQL Reliability when fsync = false on Linux-XFS

2003-09-04 Thread scott.marlowe
On Thu, 4 Sep 2003, Federico Sevilla III wrote: > (Please follow Mail-Followup-To, I'm not on the pgsql-performance > mailing list but am on the Linux-XFS mailing list. My apologies too for > the cross-post. I'm cc'ing the Linux-XFS mailing list in case people > there will be interested in this, t

Re: [PERFORM] SELECT's take a long time compared to other DBMS

2003-09-04 Thread scott.marlowe
You would "get" all rows, but they'd be stored server side until your client asked for them. I.e. a cursor would level the field here, since you say that the other test cases stored the entire result set on the server. Or did I misunderstand what you meant there? On Thu, 4 Sep 2003, Relaxin w

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread scott.marlowe
On Sun, 7 Sep 2003, Pailloncy Jean-Gérard wrote: Asking a question about why max(id) is so much slower than select id order by id desc limit 1, Pailloncy said: > I ask for the same thing. > That's better ! This is a Frequently asked question about something that isn't likely to change any time

Re: [PERFORM] slow plan for min/max

2003-09-08 Thread scott.marlowe
On Mon, 8 Sep 2003, Neil Conway wrote: > On Mon, 2003-09-08 at 11:56, scott.marlowe wrote: > > Basically, Postgresql uses an MVCC locking system that makes massively > > parallel operation possible, but costs in certain areas, and one of those > > areas is aggregate perfor

Re: [PERFORM] Need advice about triggers

2003-09-09 Thread scott.marlowe
On Tue, 9 Sep 2003, Mindaugas Riauba wrote: > > Hello, > > I have small table (up to 1 rows) and every row will be updated > once per minute. Table also has "before update on each row" trigger > written in plpgsql. But trigger 99.99% of the time will do nothing > to the database. It will

Re: [PERFORM] [GENERAL] how to get accurate values in pg_statistic

2003-09-11 Thread scott.marlowe
On Thu, 11 Sep 2003, Tom Lane wrote: > Christopher Browne <[EMAIL PROTECTED]> writes: > > The "right answer" for most use seems likely to involve: > > a) Getting an appropriate number of bins (I suspect 10 is a bit > > small, but I can't justify that mathematically), and > > I suspect that a

Re: [PERFORM] [GENERAL] how to get accurate values in pg_statistic

2003-09-12 Thread scott.marlowe
On Thu, 11 Sep 2003, Christopher Browne wrote: > [EMAIL PROTECTED] ("scott.marlowe") writes: > > On Thu, 11 Sep 2003, Tom Lane wrote: > > > >> Christopher Browne <[EMAIL PROTECTED]> writes: > >> > The "right answer" for most use seems

Re: [PERFORM] Inconsistent performance

2003-09-15 Thread scott.marlowe
On Mon, 15 Sep 2003, Joseph Bove wrote: > Stephan, > > I've run explain analyze a number of times and have gotten results between > 5.5 and 7.5 seconds > > Attached is a typical output > > QUERY PLAN > - > Aggregate (cost=9993.92..9993.92 rows=1 width=0

  1   2   >