Re: [PERFORM]

2006-08-03 Thread Ian Westmacott
? On 8/3/06, Ian Westmacott [EMAIL PROTECTED] wrote: is that all? psql -l | grep 'rows)' (2016 rows) On Thu, 2006-08-03 at 21:15 +0200, Steinar H. Gunderson wrote: On Thu, Aug 03, 2006 at 01:33:35PM -0500, Jim Nasby wrote

Re: [PERFORM] Getting even more insert performance (250m+rows/day)

2006-05-24 Thread Ian Westmacott
We were able to achieve 2B (small) rows per day sustained with very little latency. It is beefy hardware, but things that did help include WAL on its own I/O channel, XFS, binary copy, and tuning bgwriter and checkpoint settings for the application and hardware. Things that didn't help much were

[PERFORM] mergehashloop

2006-04-14 Thread Ian Westmacott
) (actual time=0.013..0.015 rows=1 loops=1741) Index Cond: (outer.sva = objects33c3.sva) Filter: ((finish - start) = 8738) Total runtime: 200.719 ms (12 rows) -- Ian Westmacott [EMAIL PROTECTED] IntelliVid Corp. ---(end of broadcast

Re: [PERFORM] mergehashloop

2006-04-14 Thread Ian Westmacott
On Fri, 2006-04-14 at 12:13 -0400, Tom Lane wrote: Ian Westmacott [EMAIL PROTECTED] writes: I have this query, where PG (8.1.2) prefers Merge Join over Hash Join over Nested Loop. However, this order turns out to increase in performance. I was hoping someone might be able to shed some

Re: [PERFORM] improving write performance for logging

2006-01-06 Thread Ian Westmacott
On Thu, 2006-01-05 at 19:08 -0600, Jim C. Nasby wrote: On Wed, Jan 04, 2006 at 11:00:38AM -0500, Ian Westmacott wrote: The WAL is a 2-spindle (SATA) RAID0 with its own controller (ext3). The tables are on a 10-spindle (SCSI) RAID50 with dual U320 controllers (XFS). This is overkill

Re: [PERFORM] improving write performance for logging

2006-01-06 Thread Ian Westmacott
of raid5 for WAL. Or just use raid10. If the WAL is lost, can you lose more than the data since the last checkpoint? -- Ian Westmacott [EMAIL PROTECTED] Intellivid Corp. ---(end of broadcast)--- TIP 5: don't forget to increase your free space map

Re: [PERFORM] improving write performance for logging application

2006-01-04 Thread Ian Westmacott
INSERT commands in each PQexec() call, and it isn’t clear that COPY is worth the effort in our application. Thanks. ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster -- Ian Westmacott [EMAIL PROTECTED] Intellivid Corp

Re: [PERFORM] improving write performance for logging

2006-01-04 Thread Ian Westmacott
ANALYZE and VACUUM in the background without interrupting the inserts (the app is 24x7). The databases are 4TB, so these operations can be lengthy. -- Ian Westmacott [EMAIL PROTECTED] Intellivid Corp. ---(end of broadcast)--- TIP 3: Have you checked our

Re: [PERFORM] wal_buffers

2005-10-06 Thread Ian Westmacott
On Thu, 2005-10-06 at 02:39, Thomas F. O'Connell wrote: The WAL Configuration chapter (25.2) has a pretty good discussion of how wal_buffers is used: http://www.postgresql.org/docs/8.0/static/wal-configuration.html You might also take a look at Josh Berkus' recent testing on this

[PERFORM] wal_buffers

2005-10-05 Thread Ian Westmacott
Can anyone tell me what precisely a WAL buffer contains, so that I can compute an appropriate setting for wal_buffers (in 8.0.3)? I know the documentation suggests there is little evidence that supports increasing wal_buffers, but we are inserting a large amount of data that, I believe, easily

Re: [PERFORM] Performance considerations for very heavy INSERT

2005-09-13 Thread Ian Westmacott
On Tue, 2005-09-13 at 11:30, Brandon Black wrote: I started looking closer at my options for partitioning (inheritance, union all), and at Bizgres today. Bizgres partitioning appears to be basically the same kind of inheritance partitioning one can do in mainline PostgreSQL. Am I correct in

Re: [PERFORM] Planner doesn't look at LIMIT?

2005-08-11 Thread Ian Westmacott
On Wed, 2005-08-10 at 18:55, Tom Lane wrote: Ian Westmacott [EMAIL PROTECTED] writes: In a nutshell, I have a LIMIT query where the planner seems to favor a merge join over a nested loop. The planner is already estimating only one row out of the join, and so the LIMIT doesn't affect its

Re: [PERFORM] Planner doesn't look at LIMIT?

2005-08-10 Thread Ian Westmacott
I have a case that I though was an example of this issue, and that this patch would correct. I applied this patch to an 8.0.3 source distribution, but it didn't seem to solve my problem. In a nutshell, I have a LIMIT query where the planner seems to favor a merge join over a nested loop. I've

Re: [PERFORM] Why hash join instead of nested loop?

2005-08-09 Thread Ian Westmacott
Yes, sorry, two totally different machines. The 7.4.8 run was on a dual P4 3.2GHz, and the 7.4.2 run was on a dual hyperthreaded Xeon 2.4GHz. --Ian On Tue, 2005-08-09 at 10:33, Tom Lane wrote: Ian Westmacott [EMAIL PROTECTED] writes: On Mon, 2005-08-08 at 20:58, Tom Lane wrote

Re: [PERFORM] cost-based vacuum

2005-07-13 Thread Ian Westmacott
On Wed, 2005-07-13 at 11:55, Simon Riggs wrote: On Tue, 2005-07-12 at 13:50 -0400, Ian Westmacott wrote: It appears not to matter whether it is one of the tables being written to that is ANALYZEd. I can ANALYZE an old, quiescent table, or a system table and see this effect. Can you

Re: [PERFORM] cost-based vacuum

2005-07-13 Thread Ian Westmacott
I can at least report that the problem does not seem to occur with Postgres 8.0.1 running on a dual Opteron. --Ian On Wed, 2005-07-13 at 16:39, Simon Riggs wrote: On Wed, 2005-07-13 at 14:58 -0400, Tom Lane wrote: Ian Westmacott [EMAIL PROTECTED] writes: On Wed, 2005-07-13 at 11

Re: [PERFORM] cost-based vacuum

2005-07-12 Thread Ian Westmacott
On Tue, 2005-07-12 at 03:45, Simon Riggs wrote: Since vacuum_cost_delay is a userset parameter, you should be able to SET this solely for the analyze_thread. That way we will know with more certainty that it is the analyze_thread that is interfering. That is what I have been doing. In fact, I

[PERFORM] cost-based vacuum

2005-07-08 Thread Ian Westmacott
I am beginning to look at Postgres 8, and am particularly interested in cost-based vacuum/analyze. I'm hoping someone can shed some light on the behavior I am seeing. Suppose there are three threads: writer_thread every 1/15 second do BEGIN TRANSACTION COPY table1 FROM stdin