Re: [HACKERS] Any conclusion on the Xeon context-switching issue?

2006-02-27 Thread Jeffrey W. Baker
On Mon, 2006-02-27 at 16:54 +, Richard Huxton wrote: Tom Lane wrote: Richard Huxton dev@archonet.com writes: Subject says it all really. I've got a new client who seems to be suffering from it, and I'm not sure if any conclusion was reached. What's he using? 8.1 seems to have

Re: [HACKERS] [GENERAL] New project launched : PostgreSQL GUI

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 03:50 -0400, Marc G. Fournier wrote: On Tue, 31 Jan 2006, Tino Wildenhain wrote: Devrim GUNDUZ schrieb: Hi, ... Are you going to work with the underlying system's package manager, or put everything in /usr/local? We'll work with the package manager --

Re: [HACKERS] Configuration WAS: New project launched : PostgreSQL

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 12:11 -0800, Josh Berkus wrote: Jeffery, PostgreSQL *desperately* needs a better means of dealing with configuration (though I guess I shouldn't be pushing too hard for this since the current state of affairs brings me business). Any improvement in this area

Re: [HACKERS] Configuration WAS: New project launched : PostgreSQL

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 17:06 -0600, Jim C. Nasby wrote: On Tue, Jan 31, 2006 at 12:36:31PM -0800, Jeffrey W. Baker wrote: Random page cost - should possible to determine this at runtime Before worrying about random_page_cost at all it makes a lot more sense to look at the query cost

Re: [HACKERS] New project launched : PostgreSQL GUI Installer for

2006-01-31 Thread Jeffrey W. Baker
On Tue, 2006-01-31 at 13:02 -0600, Jim C. Nasby wrote: On Tue, Jan 31, 2006 at 11:46:03AM +, Andreas Pflug wrote: Tino Wildenhain wrote: Figuring out the correct values for some of the buffers and costs is still a bit tough. Otoh, I guess there is no easy way to predict all

Re: [HACKERS] New project launched : PostgreSQL GUI Installer for

2006-01-30 Thread Jeffrey W. Baker
On Mon, 2006-01-30 at 19:52 -0800, Joshua D. Drake wrote: On my Debian systems, I can install PostgreSQL quite readily via the command apt-get install postgresql-8.1, which can get GUIed at least somewhat if I run aptitude, synaptic, or such... Yes Christopher, you can... I can, and

Re: [HACKERS] New project launched : PostgreSQL GUI Installer for

2006-01-30 Thread Jeffrey W. Baker
On Mon, 2006-01-30 at 20:53 -0800, Joshua D. Drake wrote: Oracle's graphical installer is a material impediment to Oracle adoption. The installer only works on systems where particular versions of Java and Motif libraries are available. On 64-bit Opteron systems it only works with the

Re: [HACKERS] ice-broker scan thread

2005-11-29 Thread Jeffrey W. Baker
On Tue, 2005-11-29 at 09:45 -0500, Pollard, Mike wrote: Anyway, what I did was the following. When doing a sequential scan, we were starting at the beginning of the table and scanning forward. If I threw up some threads to read ahead, then my user thread and my read ahead threads would

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-05 Thread Jeffrey W. Baker
On Wed, 2005-10-05 at 12:14 -0400, Ron Peacetree wrote: I've now gotten verification from multiple working DBA's that DB2, Oracle, and SQL Server can achieve ~250MBps ASTR (with as much as ~500MBps ASTR in setups akin to Oracle RAC) when attached to a decent (not outrageous, but decent) HD

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-03 Thread Jeffrey W. Baker
On Mon, 2005-10-03 at 13:34 -0700, Josh Berkus wrote: Michael, Realistically, you can't do better than about 25MB/s on a single-threaded I/O on current Linux machines, What on earth gives you that idea? Did you drop a zero? Nope, LOTS of testing, at OSDL, GreenPlum and Sun. For

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-03 Thread Jeffrey W. Baker
On Mon, 2005-10-03 at 14:16 -0700, Josh Berkus wrote: Jeff, Nope, LOTS of testing, at OSDL, GreenPlum and Sun. For comparison, A Big-Name Proprietary Database doesn't get much more than that either. I find this claim very suspicious. I get single-threaded reads in excess of

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-29 Thread Jeffrey W. Baker
On Thu, 2005-09-29 at 10:06 -0700, Luke Lonergan wrote: Josh, On 9/29/05 9:54 AM, Josh Berkus josh@agliodbs.com wrote: Following an index creation, we see that 95% of the time required is the external sort, which averages 2mb/s. This is with seperate drives for the WAL, the pg_tmp,

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-29 Thread Jeffrey W. Baker
On Thu, 2005-09-29 at 11:03 -0700, Josh Berkus wrote: Jeff, Josh, do you happen to know how many passes are needed in the multiphase merge on your 60GB table? No, any idea how to test that? I would just run it under the profiler and see how many times beginmerge() is called. -jwb

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-28 Thread Jeffrey W. Baker
On Wed, 2005-09-28 at 12:03 -0400, Ron Peacetree wrote: From: Jeffrey W. Baker [EMAIL PROTECTED] Sent: Sep 27, 2005 1:26 PM To: Ron Peacetree [EMAIL PROTECTED] Subject: Re: [HACKERS] [PERFORM] A Better External Sort? On Tue, 2005-09-27 at 13:15 -0400, Ron Peacetree wrote: That Btree can

Re: [HACKERS] Database file compatability

2005-09-27 Thread Jeffrey W. Baker
On Mon, 2005-09-26 at 17:27 -0500, Jim C. Nasby wrote: If a database is created with a 64 bit version of initdb, would a 32bit backend be able to talk to it? Likewise, would a backend compiled by a different compiler be able to? If there was some kind of incompatability, would the backend

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-09-27 Thread Jeffrey W. Baker
On Tue, 2005-09-27 at 13:15 -0400, Ron Peacetree wrote: That Btree can be used to generate a physical reordering of the data in one pass, but that's the weakest use for it. The more powerful uses involve allowing the Btree to persist and using it for more efficient re-searches or combining

Re: [HACKERS] data on devel code perf dip

2005-08-21 Thread Jeffrey W. Baker
On Sun, 2005-08-21 at 20:37 -0400, Tom Lane wrote: The whole thing's pretty bizarre. I hate to sound obvious, but does the missing performance return if you back the patch out? It seemed to have been decided on Tue, 16 Aug 2005 15:45:30 -0700 that the performance was the same before and after.

Re: [HACKERS] Simplifying wal_sync_method

2005-08-08 Thread Jeffrey W. Baker
On Mon, 2005-08-08 at 17:03 -0400, Tom Lane wrote: That's a decision that hasn't got a shred of evidence to justify imposing it on every platform. This option has its uses on Linux, however. In my testing it's good for a large speedup (20%) on a 10-client pgbench, and a minor improvement

Re: [HACKERS] ORDER BY field not in return list

2005-07-25 Thread Jeffrey W. Baker
On Mon, 2005-07-25 at 19:08 -0300, Marc G. Fournier wrote: On Mon, 25 Jul 2005, Jeffrey W. Baker wrote: On Mon, 2005-07-25 at 18:11 -0300, Marc G. Fournier wrote: Just curious as to whether or not a warning or something should be issued in a case like: SELECT c.* FROM

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-07-14 Thread Jeffrey W. Baker
On Fri, 2005-06-24 at 09:37 -0400, Tom Lane wrote: ITAGAKI Takahiro [EMAIL PROTECTED] writes: ... So I'll post the new results: checkpoint_ | writeback | segments| cache | open_sync | fsync=false | O_DIRECT only | fsync_direct | open_direct

Re: [HACKERS] [PATCHES] O_DIRECT for WAL writes

2005-07-14 Thread Jeffrey W. Baker
On Fri, 2005-06-24 at 10:19 -0500, Jim C. Nasby wrote: On Fri, Jun 24, 2005 at 09:37:23AM -0400, Tom Lane wrote: ITAGAKI Takahiro [EMAIL PROTECTED] writes: ... So I'll post the new results: checkpoint_ | writeback | segments| cache | open_sync | fsync=false | O_DIRECT

Re: [HACKERS] Consumer-grade vs enterprise-grade disk drives

2005-05-31 Thread Jeffrey W. Baker
On Mon, 2005-05-30 at 21:38 -0700, Luke Lonergan wrote: Tom, This is a story that is evolving. Anyone else use StorageReview? Great comprehensive drive benchmarks: http://www.storagereview.com/ Check the comparisons between 15K RPM SCSI drives and the 2004 Western Digital 10K RPM

Re: [HACKERS] Bitmap scan cost model (was Re: bitmap scans, btree

2005-05-18 Thread Jeffrey W. Baker
On Wed, 2005-05-18 at 11:27 -0400, Tom Lane wrote: Jeffrey W. Baker [EMAIL PROTECTED] writes: Obviously in this case sequential scan was (would have been) a huge win. Incrementing random_page_cost from 4 (the default) to 5 causes the planner to make a better decision. But to get

[HACKERS] Bitmap scan cost model (was Re: bitmap scans, btree scans, and tid order)

2005-05-17 Thread Jeffrey W. Baker
On Mon, 2005-05-16 at 14:35 -0400, Tom Lane wrote: Jeffrey W. Baker [EMAIL PROTECTED] writes: On Mon, 2005-05-16 at 09:53 -0400, Tom Lane wrote: This is a fallacy, and I think your concern is largely mistaken. Have you experimented with the cases you are worried about? Perhaps I have

Re: [HACKERS] bitmap scans, btree scans, and tid order

2005-05-16 Thread Jeffrey W. Baker
On Mon, 2005-05-16 at 09:53 -0400, Tom Lane wrote: Jeffrey Baker [EMAIL PROTECTED] writes: Change the planner/executor to use the bitmap scan in all cases where index order is unimportant. From my reading of the current code, the bitmap scan is only used in case of a join. This is a

Re: [HACKERS] bitmap scans, btree scans, and tid order

2005-05-16 Thread Jeffrey W. Baker
On Mon, 2005-05-16 at 14:35 -0400, Tom Lane wrote: Jeffrey W. Baker [EMAIL PROTECTED] writes: It's also possible that changing the btree scan to work in groups of tuples instead of single tuples would make more sense, which is why I ventured two different solution to the problem. My

[HACKERS] bitmap scans, btree scans, and tid order

2005-05-15 Thread Jeffrey W. Baker
About this time last year I was holding forth on the value of visiting the heap in TID order, even when index scan tuples are randomly ordered. Today I decided to start working on the problem stated in this TODO item: Fetch heap pages matching index entries in sequential order

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-20 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 12:55, Tom Lane wrote: Glen Parker [EMAIL PROTECTED] writes: What am I missing? Why is a performance bottle neck of this magnitude not on the same list of priorities as PITR, replication, and Win32? It's higher on my personal to-do list than most of those ;-). But

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-19 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 07:58, Tom Lane wrote: Jeffrey W. Baker [EMAIL PROTECTED] writes: We have noticed a way to make a major improvement in Pg's performance on our workload, and I would like to get your thoughts before I go off to work up a patch. For starters, read the previous

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-19 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 11:56, Tom Lane wrote: Jeffrey W. Baker [EMAIL PROTECTED] writes: Are you saying that index scan results are sorted by something other than the index key? Because in my scheme they would still be sorted by index key. Not unless you add yet another sort step after

Re: [HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-19 Thread Jeffrey W. Baker
On Wed, 2004-05-19 at 13:10, Tom Lane wrote: Glen Parker [EMAIL PROTECTED] writes: Not unless you add yet another sort step after the fetch step, that is the idea becomes 1. read index to get candidate TIDs 2. sort TIDs into physical order 3. read heap in physical order, check row

[HACKERS] proposal: be smarter about i/o patterns in index scan

2004-05-17 Thread Jeffrey W. Baker
Greetings all, We have noticed a way to make a major improvement in Pg's performance on our workload, and I would like to get your thoughts before I go off to work up a patch. The basic problem is that Pg seeks far too much when performing an index scan. I have saved an strace of a backend

Re: [HACKERS] Schemas: status report, call for developers

2002-05-02 Thread Jeffrey W. Baker
On Tue, Apr 30, 2002 at 09:41:47PM +0300, Oleg Bartunov wrote: I think DBD::Pg driver very much depends on system tables. Hope, Jeffrey (current maintainer) is online. These changes may break DBD::Pg. What is the expected time of this release? I will review my code for impact. Thanks for

Re: [HACKERS] Schemas: status report, call for developers

2002-05-02 Thread Jeffrey W. Baker
On Thu, May 02, 2002 at 05:28:36PM +0300, Oleg Bartunov wrote: On Wed, 1 May 2002, Jeffrey W. Baker wrote: On Tue, Apr 30, 2002 at 09:41:47PM +0300, Oleg Bartunov wrote: I think DBD::Pg driver very much depends on system tables. Hope, Jeffrey (current maintainer) is online