Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Simon Riggs
On Sun, 2008-08-03 at 22:57 -0400, Robert Treat wrote: I still think it is worth revisiting what problems people are trying to solve, and see if there are better tools they can be given to solve them. Barring that, I suppose a crude solution is better than nothing, though I fear people

[HACKERS] Initial Unsigned Integer data type performance test results.

2008-08-04 Thread Ryan Bradetich
Hello All, I wanted to pass some performance data on to the group regarding the unsigned integer data types I am working on. I tested on two systems running Ubuntu Hardy. The first system is an 8 x 2.66GHz x86-64 processor system. The second system is a 2 x 533 celeron i386 system. For this

Re: [HACKERS] unnecessary code in_bt_split

2008-08-04 Thread Simon Riggs
On Sun, 2008-08-03 at 19:44 -0400, Tom Lane wrote: Zdenek Kotala [EMAIL PROTECTED] writes: I found that _bt_split function calls PageGetTempPage, but next call is _bt_page_init which clear all contents anyway. Is there any reason to call PageGetTempPage instead of palloc? Not

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: ISTR that what ended up killing the enthusiasm for this was that most people realized that this GUC was just a poor tool to take a stab at solving other problems (ie. rate limiting cpu for queries). I'm not concerned with that,

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Gregory Stark
Josh Berkus [EMAIL PROTECTED] writes: Tom, Wasn't this exact proposal discussed and rejected awhile back? We rejected Greenplum's much more invasive resource manager, because it created a large performance penalty on small queries whether or not it was turned on. However, I don't

Re: [HACKERS] unnecessary code in_bt_split

2008-08-04 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala [EMAIL PROTECTED] writes: I found that _bt_split function calls PageGetTempPage, but next call is _bt_page_init which clear all contents anyway. Is there any reason to call PageGetTempPage instead of palloc? Not violating a perfectly good abstraction? OK.

Re: [HACKERS] Location for pgstat.stat

2008-08-04 Thread Magnus Hagander
Tom Lane wrote: Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: It doesn't seem to me that it'd be hard to support two locations for the stats file --- it'd just take another parameter to the read and write routines. pgstat.c already knows the difference between a normal write and

[HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Gregory Stark
It seems there's something wrong with CheckOtherDBBackends() but I haven't exactly figured out what. There are no other sessions but drop database keeps saying regression is being accessed by other users. I do see Autovacuum touching tables in regression but CheckOtherDBBackends() is supposed to

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Alvaro Herrera
Gregory Stark wrote: It seems there's something wrong with CheckOtherDBBackends() but I haven't exactly figured out what. There are no other sessions but drop database keeps saying regression is being accessed by other users. I do see Autovacuum touching tables in regression but

Re: [HACKERS] pg_regress inputdir

2008-08-04 Thread Jorgen Austvik - Sun Norway
Alvaro Herrera wrote: In my opinion, the need for running tests outside the test dir is not very strong (or we would have heard complaints before), and thus the solution is to remove --inputdir and --outputdir. Attached is a patch that removes --inputdir and --outputdir. I still prefere the

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Michael Fuhr
On Mon, Aug 04, 2008 at 11:51:35AM +0100, Gregory Stark wrote: It seems there's something wrong with CheckOtherDBBackends() but I haven't exactly figured out what. There are no other sessions but drop database keeps saying regression is being accessed by other users. Are any prepared

Re: [HACKERS] unnecessary code in_bt_split

2008-08-04 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: Tom Lane napsal(a): Not violating a perfectly good abstraction? By my opinion It would be better to have three functions: PageCreateTempPage - only allocate memory and call pageinit PageCloneSpecial - copy special section from source page

Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies

2008-08-04 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes: Alvaro Herrera wrote: (I think it's better to reuse the same postmaster executable, because that way it's easier to have the same parsing routines.) Change that to pg_ctl and you have a deal :) Did you not understand Alvaro's point? Putting this

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Gregory Stark
Michael Fuhr [EMAIL PROTECTED] writes: On Mon, Aug 04, 2008 at 11:51:35AM +0100, Gregory Stark wrote: It seems there's something wrong with CheckOtherDBBackends() but I haven't exactly figured out what. There are no other sessions but drop database keeps saying regression is being accessed by

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Gregory Stark
Alvaro Herrera [EMAIL PROTECTED] writes: The buildfarm would be all red if this wasn't something local to your installation, I think. Maybe you should get gdb on the backend and set a breakpoint on errfinish, or maybe step into CheckOtherDBBackends to see why it isn't working. Michael Fuhr

Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies

2008-08-04 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Tom Lane wrote: Seems a lot better to me to just train people to run the check-config code by hand before pulling the trigger to load the settings for real. I think it'd be reasonable to refuse starting if the config is *known broken* (such as

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Alvaro Herrera
Gregory Stark wrote: Alvaro Herrera [EMAIL PROTECTED] writes: The buildfarm would be all red if this wasn't something local to your installation, I think. Maybe you should get gdb on the backend and set a breakpoint on errfinish, or maybe step into CheckOtherDBBackends to see why it

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Andrew Dunstan
Gregory Stark wrote: Alvaro Herrera [EMAIL PROTECTED] writes: The buildfarm would be all red if this wasn't something local to your installation, I think. Maybe you should get gdb on the backend and set a breakpoint on errfinish, or maybe step into CheckOtherDBBackends to see why it

Re: [HACKERS] Location for pgstat.stat

2008-08-04 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes: Attached is a patch that implements this. I went with the option of just storing it in a temporary directory that can be symlinked, and not bothering with a GUC for it. Comments? (documentation updates are also needed, but I'll wait with those until I

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Gregory Stark wrote: Michael Fuhr solved it so this is academic but, the buildfarm runs make installcheck? I thought it just ran make check It runs both. It also runs contrib installcheck, which will most definitely exercise DROP DATABASE.

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Michael Fuhr [EMAIL PROTECTED] writes: Are any prepared transactions still open? Uh, yes, I did notice that but didn't put two and two together. That does make sense now that you mention it. I've been bit by that too, and so have other people. Maybe

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Heikki Linnakangas
Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: Michael Fuhr [EMAIL PROTECTED] writes: Are any prepared transactions still open? Uh, yes, I did notice that but didn't put two and two together. That does make sense now that you mention it. I've been bit by that too, and so have

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Tom Lane wrote: I've been bit by that too, and so have other people. Maybe it'd be worth the trouble to improve the message so that it explicitly tells you when there are prepared transactions blocking the DROP. Yes, that should be easy enough.

[HACKERS] PL/PythonU

2008-08-04 Thread David Blewett
Hi All: This is an off-shoot of the Do we really want to migrate plproxy and citext into PG core distribution? thread. On the way home from PyOhio, I had a conversation with a few people that use Zope a lot. I happened to mention that Postgres doesn't have an untrusted version of pl/python and

[HACKERS] Strawberry Perl?

2008-08-04 Thread David E. Wheeler
Howdy, I noticed this in the weekly news: Magnus Hagander committed: - In pgsql/doc/src/sgml/install-win32.sgml, document which versions of ActivePerl and ActiveTcl are required for building on MSVC, and that the free distribution is enough (no need for the enterprise version). Per gripe

Re: [HACKERS] [PATCH] \ef function in psql

2008-08-04 Thread David E. Wheeler
On Jul 31, 2008, at 00:07, Abhijit Menon-Sen wrote: I have attached two patches: - funcdef.diff implements pg_get_functiondef() - edit.diff implements \ef function in psql based on (1). Comments appreciated. +1 I like! The ability to easily edit a function on the fly in psql will be very

Re: [HACKERS] Type Categories for User-Defined Types

2008-08-04 Thread David E. Wheeler
On Jul 31, 2008, at 10:42, David E. Wheeler wrote: Good point --- so new members of STRING category aren't going to be that common, except for domains which apparently aren't bothering people anyway. I'll go ahead and make the change. (I think it's just a trivial change in

Re: [HACKERS] Type Categories for User-Defined Types

2008-08-04 Thread Alvaro Herrera
David E. Wheeler wrote: On Jul 31, 2008, at 10:42, David E. Wheeler wrote: Wow. Really nice, Tom. Thanks! The attached patch has all the tests I added to my svn version against 8.3, and for which I had to write 60 additional cast functions. Ping! Just wanted to make sure this wasn't lost

Re: [HACKERS] PL/PythonU

2008-08-04 Thread Hannu Krosing
On Mon, 2008-08-04 at 13:08 -0400, David Blewett wrote: Hi All: This is an off-shoot of the Do we really want to migrate plproxy and citext into PG core distribution? thread. On the way home from PyOhio, I had a conversation with a few people that use Zope a lot. I happened to mention

Re: [HACKERS] Type Categories for User-Defined Types

2008-08-04 Thread David E. Wheeler
On Aug 4, 2008, at 11:02, Alvaro Herrera wrote: Ping! Just wanted to make sure this wasn't lost in the shuffle… Please add it here: http://wiki.postgresql.org/wiki/CommitFest:2008-09 Sure, although it's a simple refinement (read: tests, mainly) of an accepted July patch, submitted before

Re: [HACKERS] Type Categories for User-Defined Types

2008-08-04 Thread Andrew Dunstan
David E. Wheeler wrote: On Aug 4, 2008, at 11:02, Alvaro Herrera wrote: Ping! Just wanted to make sure this wasn't lost in the shuffle… Please add it here: http://wiki.postgresql.org/wiki/CommitFest:2008-09 Sure, although it's a simple refinement (read: tests, mainly) of an accepted

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Robert Treat
On Monday 04 August 2008 03:50:40 daveg wrote: On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: ISTR that what ended up killing the enthusiasm for this was that most people realized that this GUC was just a poor tool to take a stab at solving other problems (ie. rate limiting

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Josh Berkus
Greg, Well that's going to depend on the application But I suppose there's nothing wrong with having options which aren't always a good idea to use. The real question I guess is whether there's ever a situation where it would be a good idea to use this. I'm not 100% sure. I can think of

Re: [HACKERS] PL/Python

2008-08-04 Thread David Blewett
On Mon, Aug 4, 2008 at 1:56 PM, Hannu Krosing [EMAIL PROTECTED] wrote: Hannu: You had mentioned bringing pl/python up to the level of some of the other pl's. Have you thought any more about pl/pythonu? Obviously, I meant pl/python. Subject line fixed to. Sorry for the noise. David Blewett --

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Greg Smith
On Mon, 4 Aug 2008, daveg wrote: On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: Not such a great argument. Cost models on development servers can and often are quite different from those on production, so you might be putting an artifical limit on top of your developers. We

Re: [HACKERS] PL/Python

2008-08-04 Thread Hannu Krosing
On Mon, 2008-08-04 at 15:02 -0400, David Blewett wrote: On Mon, Aug 4, 2008 at 1:56 PM, Hannu Krosing [EMAIL PROTECTED] wrote: Hannu: You had mentioned bringing pl/python up to the level of some of the other pl's. Have you thought any more about pl/pythonu? Obviously, I meant pl/python.

[HACKERS] CommitFest July Over

2008-08-04 Thread Josh Berkus
Hackers, Well, after a month the July CommitFest is officially closed. At this point, we're operating with the defacto rule that commitfests shouldn't last more than a month. Because some patches are still being discussed, they've been moved over automatically to the September commitfest.

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 03:09:34PM -0400, Greg Smith wrote: On Mon, 4 Aug 2008, daveg wrote: We load the production dumps into our dev environment, which are the same hardware spec, so the costs should be identical. Not identical, just close. ANALYZE samples data from your table randomly.

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 02:35:07PM -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: That's great for you, I am talking in the scope of a general solution. (Note I'd also bet that even given the same hardware, different production loads can produce different

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Hannu Krosing
On Mon, 2008-08-04 at 14:35 -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote: ... I still think it is worth revisiting what problems people are trying to solve, and see if there are better tools they can

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Gregory Stark
Josh Berkus [EMAIL PROTECTED] writes: In such a production application, it is better to have false positives and reject otherwise-OK queries becuase their costing is wrong, than to let a single cartesian join bog down an application serving 5000 simultaneous users. Further, with a SQL error,

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 11:59:03AM -0700, Josh Berkus wrote: Greg, Well that's going to depend on the application But I suppose there's nothing wrong with having options which aren't always a good idea to use. The real question I guess is whether there's ever a situation where it would

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Kevin Grittner
Hannu Krosing [EMAIL PROTECTED] wrote: I was once told about a company, who claimed to have produced a positively fool-proof lawn-mower, only to find out, that a university professor had tried to use it to trim a hedge and cut off his toes. Odd. Seriously, about 45 years ago I lived next

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Simon Riggs
On Mon, 2008-08-04 at 14:35 -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: And you'll note, I specifically said that a crude tool is better than nothing. But your completely ignoring that a crude tool can often end-up as a foot-gun once relased into the wild. The

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Josh Berkus
Greg, For such an application this would be a major foot-gun which would give a false sense of security simultaneously causing random outages and not providing even the protection you're counting on. Hmmm. That sounds like a call for some testing. While our cost estimation has some issues,

[HACKERS] Automatic Client Failover

2008-08-04 Thread Simon Riggs
When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. We might want to specify that centrally and then send the redirection address to the client when it connects. Sounds like lots of work though. Seems fairly

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Robert Treat
On Monday 04 August 2008 16:49:43 Simon Riggs wrote: On Mon, 2008-08-04 at 14:35 -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: And you'll note, I specifically said that a crude tool is better than nothing. But your completely ignoring that a crude tool can

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Jonah H. Harris
On Mon, Aug 4, 2008 at 5:08 PM, Simon Riggs [EMAIL PROTECTED] wrote: When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. This would be a nice feature which many people I've talked to have asked for. In

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread Robert Treat
On Monday 04 August 2008 15:56:25 daveg wrote: On Mon, Aug 04, 2008 at 02:35:07PM -0400, Robert Treat wrote: On Monday 04 August 2008 03:50:40 daveg wrote: That's great for you, I am talking in the scope of a general solution. (Note I'd also bet that even given the same hardware,

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Josh Berkus
On Monday 04 August 2008 14:08, Simon Riggs wrote: When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. We might want to specify that centrally and then send the redirection address to the client when it

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Jonah H. Harris
On Mon, Aug 4, 2008 at 5:39 PM, Josh Berkus [EMAIL PROTECTED] wrote: Well, it's less simple, but you can already do this with pgPool on the client machine. Yeah, but if you have tens or hundreds of clients, you wouldn't want to be installing/managing a pgpool on each. Similarly, I think an

Re: [HACKERS] IN vs EXISTS equivalence

2008-08-04 Thread Kevin Grittner
On Mon, Oct 22, 2007 at 1:30 PM, Simon Riggs wrote: On Mon, 2007-10-22 at 09:31 -0500, Kevin Grittner wrote: I've requested this before without response, but I'm asking again because it just caused me pain again: could we get a TODO added to have the planner recognize equivalent IN and

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Tom Lane
Jonah H. Harris [EMAIL PROTECTED] writes: On Mon, Aug 4, 2008 at 5:39 PM, Josh Berkus [EMAIL PROTECTED] wrote: Well, it's less simple, but you can already do this with pgPool on the client machine. Yeah, but if you have tens or hundreds of clients, you wouldn't want to be installing/managing

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Hannu Krosing
On Mon, 2008-08-04 at 22:08 +0100, Simon Riggs wrote: When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. We might want to specify that centrally and then send the redirection address to the client when

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Dimitri Fontaine
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, Le 5 août 08 à 01:13, Tom Lane a écrit : There is one really bad consequence of the oversimplified failover design that Simon proposes, which is that clients might try to fail over for reasons other than a primary server failure. (Think

Re: [HACKERS] IN vs EXISTS equivalence

2008-08-04 Thread Tom Lane
Kevin Grittner [EMAIL PROTECTED] writes: I'm adding some NOT EXISTS examples to the thread for completeness of what someone might want to address while working on it. For two queries which can easily be shown (to a human viewer, anyway) to return identical results, I see performance

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Tom Lane
Dimitri Fontaine [EMAIL PROTECTED] writes: Le 5 août 08 à 01:13, Tom Lane a écrit : There is one really bad consequence of the oversimplified failover design that Simon proposes, which is that clients might try to fail over for reasons other than a primary server failure. (Think network

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Josh Berkus
Tom, Failover that actually works is not something we can provide with trivial changes to Postgres. I think the proposal was for an extremely simple works 75% of the time failover solution. While I can see the attraction of that, the consequences of having failover *not* work are pretty

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Jens-Wolfhard Schicke
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tom Lane wrote: ERROR: database %s is being accessed by other users DETAIL: There are %d session(s) and %d prepared transaction(s) using the database. I'm aware that this phrasing might not translate very nicely ... anyone have a suggestion

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:17:59PM -0400, Jonah H. Harris wrote: On Mon, Aug 4, 2008 at 5:08 PM, Simon Riggs [EMAIL PROTECTED] wrote: When primary server fails, it would be good if the clients connected to the primary knew to reconnect to the standby servers automatically. This would be a

Re: [HACKERS] Mini improvement: statement_cost_limit

2008-08-04 Thread daveg
On Mon, Aug 04, 2008 at 05:19:50PM -0400, Robert Treat wrote: See, this is what we ended up talking about before. Someone will say I'd like to prevent my devs from accidentally doing queries with cartesian products and they will use this to do it... but that will only work in some cases, so

Re: [HACKERS] CommitFest July Over

2008-08-04 Thread Robert Treat
On Monday 04 August 2008 15:38:35 Josh Berkus wrote: Hackers, Well, after a month the July CommitFest is officially closed. At this point, we're operating with the defacto rule that commitfests shouldn't last more than a month. Because some patches are still being discussed, they've been

Re: [HACKERS] Automatic Client Failover

2008-08-04 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: I think the proposal was for an extremely simple works 75% of the time failover solution. While I can see the attraction of that, the consequences of having failover *not* work are pretty severe. Exactly. The point of failover (or any other HA feature)

Re: [HACKERS] DROP DATABASE always seeing database in use

2008-08-04 Thread Tom Lane
Jens-Wolfhard Schicke [EMAIL PROTECTED] writes: Tom Lane wrote: ERROR: database %s is being accessed by other users DETAIL: There are %d session(s) and %d prepared transaction(s) using the database. I'm aware that this phrasing might not translate very nicely ... anyone have a suggestion