Re: [HACKERS] slow IN() clause for many cases

2005-12-02 Thread Simon Riggs
On Wed, 2005-11-30 at 07:18 +0100, Martijn van Oosterhout wrote: And finally, why can't: Select * From Sales where month IN ( select month from time_dimension where FinYear = 2005 and Quarter = 3) Be written as: Select sales.* From Sales, time_dimension where month =

Re: [HACKERS] Using multi-row technique with COPY

2005-12-02 Thread Simon Riggs
On Wed, 2005-11-30 at 02:10 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: [...context omitted...] We would need to flush all the blocks in the table out of cache at commit time, for that table only. This seems striking close to the Old Concept of temp tables, which we got

Re: [HACKERS] slow IN() clause for many cases

2005-12-02 Thread Martijn van Oosterhout
On Fri, Dec 02, 2005 at 08:18:44AM +, Simon Riggs wrote: It can, of course, but there must be value in that optimization. If you consider how IN () would be transformed into =ANY(ARRAY(subselect)) you'll see that the subselect values would be treated as constants that could result in a

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Csaba Nagy
On Thu, 2005-12-01 at 22:01, Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: On the other hand the type I would prefer to see are hints that feed directly into filling in information the planner lacks. This only requires that the user understand his own data and still lets the

Re: [HACKERS] Shared locking in slru.c

2005-12-02 Thread Manfred Koizar
On Wed, 30 Nov 2005 13:53:13 -0500, Tom Lane [EMAIL PROTECTED] wrote: The way the attached patch attacks this is for the shared-lock access case to simply set the page's LRU counter to zero, without bumping up the LRU counters of the other pages as the normal adjustment would do. If you still

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Martijn van Oosterhout
On Fri, Dec 02, 2005 at 11:07:06AM +0100, Csaba Nagy wrote: So for me the hint mechanism is good for telling the server that I'm not interested at all in the BEST plan but which risks getting very bad on occasions, but in a good enough plan which is safe. I'm wondering if long term another

[HACKERS] Graphics in postgress using GTK

2005-12-02 Thread Anuj Tripathi
Hi We are trying to implement a progress estiamator for long queries. We are trying to make the display graphical using GTK but we get an error saying Xlib : Connection to :0.0 refused by server xlib : no protocol specified GTK-warning ** : Cannot open display Can someone suggest a method to

Re: [HACKERS] Graphics in postgress using GTK

2005-12-02 Thread Martijn van Oosterhout
On Fri, Dec 02, 2005 at 04:15:55PM +0530, Anuj Tripathi wrote: Hi We are trying to implement a progress estiamator for long queries. We are trying to make the display graphical using GTK but we get an error saying Xlib : Connection to :0.0 refused by server xlib : no protocol specified

Re: [HACKERS] Graphics in postgress using GTK

2005-12-02 Thread Richard Huxton
Anuj Tripathi wrote: Hi We are trying to implement a progress estiamator for long queries. We are trying to make the display graphical using GTK but we get an error saying Xlib : Connection to :0.0 refused by server xlib : no protocol specified GTK-warning ** : Cannot open display Can

Re: [HACKERS] Buildfarm: Bear, Branch 2?

2005-12-02 Thread Andrew Dunstan
Michael Glaesemann said: Out of curiosity, what is this beast? http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=beardt=2005-11-13% 2012:01:08 pilot error. I'll clean it up. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to

Re: [HACKERS] Buildfarm: Bear, Branch 2?

2005-12-02 Thread Michael Glaesemann
On Dec 2, 2005, at 20:54 , Andrew Dunstan wrote: Michael Glaesemann said: Out of curiosity, what is this beast? http://pgbuildfarm.org/cgi-bin/show_log.pl?nm=beardt=2005-11-13% 2012:01:08 pilot error. I'll clean it up. Shucks. I was hoping it was a secret new port that reduced count(*)

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Pollard, Mike
Greg Stark [EMAIL PROTECTED] writes: You more or less missed my entire point. Only because I am still getting used to how powerful and flexible Postgres is; but I am working on expanding my horizons. In the extreme, no amount of added intelligence in the optimizer is going to help it come up

Re: [HACKERS] Shared locking in slru.c

2005-12-02 Thread Tom Lane
Manfred Koizar [EMAIL PROTECTED] writes: On Wed, 30 Nov 2005 13:53:13 -0500, Tom Lane [EMAIL PROTECTED] wrote: The way the attached patch attacks this is for the shared-lock access case to simply set the page's LRU counter to zero, without bumping up the LRU counters of the other pages as the

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Simon Riggs
On Fri, 2005-12-02 at 02:14 -0500, Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: It was a *major* new feature that many people were waiting for when Oracle finally implemented live CREATE INDEX and REINDEX. The ability to run create an index without blocking any operations on a

[HACKERS] Optimizer oddness, possibly compounded in 8.1

2005-12-02 Thread Philip Warner
The optimizer seems to want to use sequential scans on inherited tables when crossed with another table, as the following seems to demonstrate: Create Table base(f1 bigserial); create table inh1(f2 bigint) inherits (base); create table inh2(f2 bigint) inherits (base); create table inh3(f2

Re: [HACKERS] Graphics in postgress using GTK

2005-12-02 Thread Cristian Prieto
What are you doing? Trying to run the GTK application in the same server as the database? Or are you trying to do a graphical interface in a server without an X server?. I guess there is something very bad with your approach, I guess your graphical interface should be running in the client side

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Rod Taylor
In the extreme, no amount of added intelligence in the optimizer is going to help it come up with any sane selectivity estimate for something like WHERE radius_authenticate(user) = 'OK' Why not? The missing capability in this case is to be able to provide or generate (self learning?)

Re: [HACKERS] Graphics in postgress using GTK

2005-12-02 Thread Jaime Casanova
On 12/2/05, Anuj Tripathi [EMAIL PROTECTED] wrote: Hi We are trying to implement a progress estiamator for long queries. We are trying to make the display graphical using GTK but we get an error saying Xlib : Connection to :0.0 refused by server xlib : no protocol specified GTK-warning **

Re: [HACKERS] Optimizer oddness, possibly compounded in 8.1

2005-12-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: The optimizer seems to want to use sequential scans on inherited tables when crossed with another table, as the following seems to demonstrate: Is it intentional that your test case omits an analyze on t2? Coz when I add that, I get the same plan you

[HACKERS] postgres questions (semi-joins, algebraic space)

2005-12-02 Thread [EMAIL PROTECTED]
Good Morning, I'm an italian student and I'm working on a project involving postgres. I'm sorry for my very bad english. I've some questions: 1)Does postgres upport semi-joins? 2)What about projections or selections in query tree writing? Does Postgres support algebraic space (in other word

[HACKERS]

2005-12-02 Thread [EMAIL PROTECTED]
Good Morning, I'm an italian student and I'm working on a project involving postgres. I'm sorry for my very bad english. I've some questions: 1)Does postgres upport semi-joins? 2)What about projections or selections in query tree writing? Does Postgres support algebraic space (in other word

Re: [HACKERS] What`s wrong with the lists?

2005-12-02 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Is there something weird going on with the lists? I ask because lately there has been spam showing up rather frequently, I'm not seeing this here. Can you point to a specific message on hackers? The only list I see that has spam is pg-press.

Re: [HACKERS] What`s wrong with the lists?

2005-12-02 Thread Alvaro Herrera
Greg Sabino Mullane wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Is there something weird going on with the lists? I ask because lately there has been spam showing up rather frequently, I'm not seeing this here. Can you point to a specific message on hackers? The only

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Greg Stark
Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2005-12-02 at 02:14 -0500, Tom Lane wrote: Greg Stark [EMAIL PROTECTED] writes: It was a *major* new feature that many people were waiting for when Oracle finally implemented live CREATE INDEX and REINDEX. The ability to run create an

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Greg Stark
Rod Taylor [EMAIL PROTECTED] writes: In the extreme, no amount of added intelligence in the optimizer is going to help it come up with any sane selectivity estimate for something like WHERE radius_authenticate(user) = 'OK' Why not? The missing capability in this case is to be

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Gregory Maxwell
On 02 Dec 2005 15:25:58 -0500, Greg Stark [EMAIL PROTECTED] wrote: I suspect this comes out of a very different storage model from Postgres's. Postgres would have no trouble building an index of the existing data using only shared locks. The problem is that any newly inserted (or updated)

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Gregory Maxwell
On 02 Dec 2005 15:49:02 -0500, Greg Stark [EMAIL PROTECTED] wrote: Rod Taylor [EMAIL PROTECTED] writes: The missing capability in this case is to be able to provide or generate (self learning?) statistics for a function that describe a typical result and the cost of getting that result.

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Alvaro Herrera
Gregory Maxwell wrote: On 02 Dec 2005 15:25:58 -0500, Greg Stark [EMAIL PROTECTED] wrote: I suspect this comes out of a very different storage model from Postgres's. Postgres would have no trouble building an index of the existing data using only shared locks. The problem is that any

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Simon Riggs
On Fri, 2005-12-02 at 19:04 -0300, Alvaro Herrera wrote: Gregory Maxwell wrote: On 02 Dec 2005 15:25:58 -0500, Greg Stark [EMAIL PROTECTED] wrote: I suspect this comes out of a very different storage model from Postgres's. Postgres would have no trouble building an index of the

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Jochem van Dieten
On 12/2/05, Alvaro Herrera wrote: Gregory Maxwell wrote: After you're mostly caught up, change locking behavior to block further updates while the final catchup happens. This could be driven by a hurestic that says make up to N attempts to catch up without blocking, after that just take a

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: CREATE INDEX uses SnapshotAny, so the scan that feeds the build could easily include rows added after the CREATE INDEX started. When the scan was exhausted we could mark that last TID and return to it after the sort/build. And do what? This has nothing

Re: [HACKERS] Optional postgres database not so optional in 8.1

2005-12-02 Thread Andrew Dunstan
I never saw a followup to this. Is someone working on a ping protocol extension, or should we revert pg_ctl to using template1 on the ground that it does a poor man's ping anyway? cheers andrew Andrew Dunstan wrote: Tom Lane wrote: Andrew Dunstan [EMAIL PROTECTED] writes: I now

[HACKERS] Spam 508

2005-12-02 Thread Simon Riggs
It appears that a bunch of spammers have invaded -hackers, claiming they can avoid the heat death of the universe in exchange for 2 bytes. Or something like that. :-) Even my Krull-powered Spam filters can't cope. Best Regards, Simon Riggs ---(end of

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Simon Riggs
On Fri, 2005-12-02 at 11:53 +0100, Martijn van Oosterhout wrote: On Fri, Dec 02, 2005 at 11:07:06AM +0100, Csaba Nagy wrote: So for me the hint mechanism is good for telling the server that I'm not interested at all in the BEST plan but which risks getting very bad on occasions, but in a

Re: [HACKERS] generalizing the planner knobs

2005-12-02 Thread Trent Shipley
Is it possible to submit a hand written or arbitrary execution plan to the retrieval engine? (That is, can one bypass the SQL parser and planner or optimizer and just provide instructions to nested loop join table a to table b ...) ---(end of

Re: [HACKERS] Spam 508

2005-12-02 Thread Marc G. Fournier
I haven't received any yet, that I can tell ... sure its coming through the lists, and not around them? On Fri, 2 Dec 2005, Simon Riggs wrote: It appears that a bunch of spammers have invaded -hackers, claiming they can avoid the heat death of the universe in exchange for 2 bytes. Or

Re: [HACKERS] Reducing relation locking overhead

2005-12-02 Thread Tom Lane
Jochem van Dieten [EMAIL PROTECTED] writes: How about the following sceanrio for building a new index: - create an empty index - flag it as incomplete - commit it so it becomes visible to new transactions - new transactions will update the index when inserting / updating - the planner will

[HACKERS] strange behavior (corruption?) of large production database

2005-12-02 Thread Joe Conway
We have very strange behavior from an internal production database. There are multiple symptoms, all pointing to a problem with clusterwide tables. For example: [EMAIL PROTECTED]:~ psql -U postgres -p 5433 cyspec Welcome to psql 7.4.8, the PostgreSQL interactive terminal. Type: \copyright

Re: [HACKERS] Spam 508

2005-12-02 Thread Tom Lane
Marc G. Fournier [EMAIL PROTECTED] writes: On Fri, 2 Dec 2005, Simon Riggs wrote: Even my Krull-powered Spam filters can't cope. I haven't received any yet, that I can tell ... sure its coming through the lists, and not around them? I think Simon is complaining about the length of the

Re: [HACKERS] strange behavior (corruption?) of large production database

2005-12-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: We have very strange behavior from an internal production database. There are multiple symptoms, all pointing to a problem with clusterwide tables. For example: psql -l really should produce the same results as doing \l in the template1 database. Does it?

Re: [HACKERS] Spam 508

2005-12-02 Thread Jan Wieck
On 12/2/2005 6:19 PM, Marc G. Fournier wrote: I haven't received any yet, that I can tell ... sure its coming through the lists, and not around them? Some Tom Lane guy and a bunch of other well known addresses sent it. Could be forged From fields though ;-) Jan On Fri, 2 Dec 2005,

Re: [HACKERS] Spam 508

2005-12-02 Thread Marc G. Fournier
*roll eyes* Its not even April Fools yet ... On Fri, 2 Dec 2005, Jan Wieck wrote: On 12/2/2005 6:19 PM, Marc G. Fournier wrote: I haven't received any yet, that I can tell ... sure its coming through the lists, and not around them? Some Tom Lane guy and a bunch of other well known

Re: [HACKERS] strange behavior (corruption?) of large production database

2005-12-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: So they agree in template1 and cyspec databases. OK, in that case I'd wonder about whether you've suffered XID wraparound in pg_database and/or pg_shadow. The typical symptom of this is that entries are valid from the system's point of view but not visible

Re: [HACKERS] strange behavior (corruption?) of large production

2005-12-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: So they agree in template1 and cyspec databases. OK, in that case I'd wonder about whether you've suffered XID wraparound in pg_database and/or pg_shadow. The typical symptom of this is that entries are valid from the system's point of

Re: [HACKERS] strange behavior (corruption?) of large production database

2005-12-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Tom Lane wrote: You could try a VACUUM FREEZE on pg_database though. Since this is a production machine, putting pg_filedump on it may be problematic -- if I grovel through the bits by hand, can you give me a hint about what to look for? How about you

Re: [HACKERS] Optimizer oddness, possibly compounded in 8.1

2005-12-02 Thread Philip Warner
Is it intentional that your test case omits an analyze on t2? No; my mistake. (The larger point that joins of inheritance unions aren't well-planned is true, but it's always been true...) It also seems to have a probkem with unions in views. Is there anything that can be done about this --

Re: [HACKERS] Optimizer oddness, possibly compounded in 8.1

2005-12-02 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: (The larger point that joins of inheritance unions aren't well-planned is true, but it's always been true...) It also seems to have a probkem with unions in views. Is there anything that can be done about this -- workarounds etc? Any plans to address

Re: [HACKERS] strange behavior (corruption?) of large production database

2005-12-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: Since this database has many large, but static tables (i.e. new data is loaded each day, but the tables are partitioned into year-month tables), I'm thinking we can run VACUUM FREEZE on the whole database once, and then run VACUUM FREEZE periodically on

Re: [HACKERS] strange behavior (corruption?) of large production

2005-12-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: Since this database has many large, but static tables (i.e. new data is loaded each day, but the tables are partitioned into year-month tables), I'm thinking we can run VACUUM FREEZE on the whole database once, and then run VACUUM FREEZE

[HACKERS] Building Windows Server Extensions Using VC++ 2005

2005-12-02 Thread Charles F. I. Savage
Hi everyone, I've been able to successfully build server extension using Visual Studio 2005 for Windows Postgresql 8.1. However, it took a few tweaks which I thought I should document (maybe these issues could be fixed in future postgresql versions?): 1. There is no lib file for VC++ to

Re: [HACKERS] strange behavior (corruption?) of large production database

2005-12-02 Thread Tom Lane
Joe Conway [EMAIL PROTECTED] writes: cyspec=# vacuum freeze pg_catalog.pg_class; ERROR: failed to re-find parent key in pg_class_relname_nsp_index It seems that we cannot vacuum pg_class, because vacuum itself fails. Any suggestions on how to bootstrap the fixing of pg_class? REINDEX?

Re: [HACKERS] strange behavior (corruption?) of large production

2005-12-02 Thread Joe Conway
Tom Lane wrote: Joe Conway [EMAIL PROTECTED] writes: cyspec=# vacuum freeze pg_catalog.pg_class; ERROR: failed to re-find parent key in pg_class_relname_nsp_index It seems that we cannot vacuum pg_class, because vacuum itself fails. Any suggestions on how to bootstrap the fixing of

Re: [HACKERS] Optimizer oddness, possibly compounded in 8.1

2005-12-02 Thread Philip Warner
Tom Lane wrote: It's something that's on the ever-growing TODO list ... I dunno if anyone has any near-term plans to work on it. It'd definitely be nice to teach the planner to do joins-over-unions well, and then make inheritance just invoke that behavior instead of being a crocky special case.