Re: [HACKERS] About method of PostgreSQL's Optimizer

2005-09-14 Thread Jonah H. Harris
Pryscila, For research reference, you may want to look at the work done on the Columbia Query Optimization Framework.  As I recall, I think it (or its predecessors) had both cost and rule-based optimization.  If you need the code to it, I can dig it up on one of my old systems. Albeit dated, anot

Re: [HACKERS] About method of PostgreSQL's Optimizer

2005-09-14 Thread Jonah H. Harris
Tom, I agree.  There have been several occasions where GEQO has performed poorly for me.  I'll search the archives for the past discussions. sorry for sending this to you twice Tom... forgot to hit reply all :(On 9/14/05, Tom Lane <[EMAIL PROTECTED] > wrote:"Jonah H. Harris" <[EMAIL PROTECTED]

Re: [HACKERS] Per-table freeze limit proposal

2005-09-14 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > In fact this seems pretty easy to do. Add a field to pg_class, tell > VACUUM to update it using the determined freezeLimit, and that's it. I think that it'd be worth fixing things so that the recorded value is not the freeze cutoff value (as now), but

Re: [HACKERS] inverse OR distributive law?

2005-09-14 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: > * process_duplicate_ors > * Given a list of exprs which are ORed together, try to apply > * the inverse OR distributive law. > Anybody enlighten what "inverse OR distributive law" is? Well, it's defined right above that: * The following co

Re: [HACKERS] Constraint Type Coercion issue?

2005-09-14 Thread Tom Lane
Martijn van Oosterhout writes: > Well yes, but given the number of possible locales, creating one class > for each seems excessive. And each class would have to create 5 > operators (with underlying functions) and 1 comparitor function. Unless > you could shortcut something like: > CREATE OPERATO

[HACKERS] Per-table freeze limit proposal

2005-09-14 Thread Alvaro Herrera
Hackers, As you've probably heard too many times already, I'm thinking in improving vacuum, so we can keep track of the freeze Xid on a table level, rather than database level. Hopefully this will eliminate the need for database-wide vacuums. In fact this seems pretty easy to do. Add a field to

Re: [HACKERS] inverse OR distributive law?

2005-09-14 Thread Dann Corbit
To find out about boolean logic, take a look here: http://www.laynetworks.com/Boolean%20Algebra.htm Where I work, we took the SIS toolkit from Berkeley and did a simplification of the where clause as if it was a Boolean integrated circuit. Of course, you may get answers that you do not expect if

[HACKERS] inverse OR distributive law?

2005-09-14 Thread Tatsuo Ishii
Hi, I have been looking around optimizer's code and found a comment: /* * process_duplicate_ors *Given a list of exprs which are ORed together, try to apply *the inverse OR distributive law. Anybody enlighten what "inverse OR distributive law" is? -- SRA OSS, Inc. Japan Tatsuo

[HACKERS] Bug with cursor declaration in PL/pgSQL in CVS tip?

2005-09-14 Thread Michael Paesold
I have used to declare cursors in the DECLARE section of a PL/pgSQL function. The example here seems to be broken in CVS tip: CREATE FUNCTION test () RETURNS void AS ' DECLARE credit_cursor CURSOR (p_account integer, p_reference integer) FOR SELECT * FROM booking WHERE account_id=p_acc

Re: [HACKERS] parameterized fetch

2005-09-14 Thread Oliver Jowett
Merlin Moncure wrote: > I've noticed that trying to parameterize a fetch statement via > ExecParams returns a syntax error: > > fetch $1 from my_cursor; > > This is not really a big deal, but maybe it should be documented which > statements can be parameterized and which can't Currently the docu

Re: [HACKERS] Constraint Type Coercion issue?

2005-09-14 Thread Martijn van Oosterhout
On Wed, Sep 14, 2005 at 05:28:42PM -0400, Tom Lane wrote: > > To some extent, collate > > implies a sort of parameterised operator class... > > Hmm. But an index couldn't support more than one collation order > AFAICS. It'd probably make more sense to create operators and an > operator class for

Re: [HACKERS] Constraint Type Coercion issue?

2005-09-14 Thread Tom Lane
Martijn van Oosterhout writes: > How much discussion has there been on this? None yet; I had a few half-baked ideas but nothing worth presenting to the list. > To some extent, collate > implies a sort of parameterised operator class... Hmm. But an index couldn't support more than one collation

Re: [HACKERS] Constraint Type Coercion issue?

2005-09-14 Thread Martijn van Oosterhout
On Wed, Sep 14, 2005 at 02:23:29PM -0400, Tom Lane wrote: > I've been thinking about this off and on, and would like to solve it > in the 8.2 time frame, but it's not happening for 8.1. At a minimum > it'll require some significant changes in our concept of what an > operator class is. The half-j

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Michael Paesold
Tom Lane wrote: I wrote: We could ameliorate this if there were a way to acquire ownership of the cache line without necessarily winning the spinlock. I'm imagining that we insert a "dummy" locked instruction just ahead of the xchgb, which touches the spinlock in such a way as to not change i

Re: [HACKERS] Constraint Type Coercion issue?

2005-09-14 Thread Tom Lane
Josh Berkus writes: > So, is this a real bug in constraints or does the problem lie somewhere > else? Is it fixable? Not readily. The problem is here: * We must find a btree opclass that contains both operators, else the * implication can't be determined. Also, the pred_op has to

Re: [HACKERS] About method of PostgreSQL's Optimizer

2005-09-14 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > As for using both in the same optimizer, I could only see an algorithm such > as a customized-A* being used to planning *some* large queries. The reason I > say this is because the cost calculation, which would still need to be > breadth-first, cou

[HACKERS] Constraint Type Coercion issue?

2005-09-14 Thread Josh Berkus
Folks, Bob Ippolito found this while testing Bizgres. It *seems* like our smarter type coercion rules do not apply when constraints are being generated. That is, the types of constants in constraints, if not coerced, still default to the old "dumb" casting where the type of the comparing colu

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Tom Lane
I wrote: > Another thought came to mind: maybe the current data layout for LWLocks > is bad. Right now, the spinlock that protects each LWLock data struct > is itself part of the struct, and since the structs aren't large (circa > 20 bytes), the whole thing is usually all in the same cache line. .

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Tom Lane
I wrote: > We could ameliorate this if there were a way to acquire ownership of the > cache line without necessarily winning the spinlock. I'm imagining > that we insert a "dummy" locked instruction just ahead of the xchgb, > which touches the spinlock in such a way as to not change its state. I

Re: [HACKERS] About method of PostgreSQL's Optimizer

2005-09-14 Thread Jonah H. Harris
Pryscila, Step 2 is basically where you find the difference between a cost-based optimizer (CBO) and a rule-based optimizer (RBO).  A CBO is based on the computed execution cost of the query whereas an RBO uses more generalized heuristics. Let's get an example of what you're proposing and see if

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Stephen Frost
Tom, et al., Updated, with full recompiles between everything and the new modification: N, runtime: Tip:1 31s 2 37s 4 86s 8 159s no-cmpb:1 32s 2 43s 4 83s 8 168s spin: 1 32s 2 51s 4 84s 8 160s spin+mod: 1 32s

Re: [HACKERS] About method of PostgreSQL's Optimizer

2005-09-14 Thread Pryscila B Guttoski
Hi Jonah, Thank's for your email, I really appreciate your opinions. Is it interesting to use both techniques? For example: Given a query, an optimizer: 1. Generates one of the possible execution plans. 2. Does transformations on the original plan, based on rules and heuristics, resulting in new

Re: [HACKERS] About method of PostgreSQL's Optimizer

2005-09-14 Thread Josh Berkus
Pryscila, > > There are other methods for query optimization, one of them is based on > > plan transformations (for example, using A-Star algorithm) instead of > > plan constructions used by PostgreSQL. We do certainly need a specific optimization for large star-schema joins. I'm not certain th

Re: [HACKERS] 8.1 system info / admin functions

2005-09-14 Thread Andreas Pflug
Stephen Frost wrote: * Tom Lane ([EMAIL PROTECTED]) wrote: Neil Conway <[EMAIL PROTECTED]> writes: While we're on the subject, the units used by pg_size_pretty() are incorrect, at least according to the IEC: for example, "MB" is strictly-speaking one million bytes, not 1024^2 bytes. 1024^2 b

[HACKERS] parameterized fetch

2005-09-14 Thread Merlin Moncure
I've noticed that trying to parameterize a fetch statement via ExecParams returns a syntax error: fetch $1 from my_cursor; This is not really a big deal, but maybe it should be documented which statements can be parameterized and which can't (I take it that any statement that can be prepared can

Re: [HACKERS] Bug with cursor declaration in plpgsql? (Repost)

2005-09-14 Thread Tom Lane
"Michael Paesold" <[EMAIL PROTECTED]> writes: > I get: > ERROR: syntax error at or near "," at character 237 > LINE 9: credit_cursor CURSOR (p_account integer, p_reference integ... > The same function works perfectly well in 7.4.8 and 8.0.3. > A bug? Yeah, looks like Neil accidentally dropped

Re: [HACKERS] 8.1 system info / admin functions

2005-09-14 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: > Neil Conway <[EMAIL PROTECTED]> writes: > > While we're on the subject, the units used by pg_size_pretty() are > > incorrect, at least according to the IEC: for example, "MB" is > > strictly-speaking one million bytes, not 1024^2 bytes. 1024^2 bytes is 1 >

Re: [HACKERS] VACUUM VERBOSE 8.1dev

2005-09-14 Thread Dave Cramer
I'm not even sure that the new output does tell me the same thing. I certainly prefer the previous output. I think this will be very confusing to users who aren't familiar with the internals of postgres. Dave On 13-Sep-05, at 11:44 PM, Joshua D. Drake wrote: Hello, It seems the new VACUUM

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Martijn van Oosterhout
On Tue, Sep 13, 2005 at 08:14:47PM -0400, Stephen Frost wrote: > I suppose another option would be to provide seperate packages... Could > this be done as a shared library so it's more 'plug-and-play' to switch > between the two? I dunno, just trying to think about how to deal with > this without

Re: [HACKERS] 8.1 system info / admin functions

2005-09-14 Thread Andreas Pflug
Tom Lane wrote: Neil Conway <[EMAIL PROTECTED]> writes: (2) pg_cancel_backend(), pg_reload_conf(), and pg_rotate_logfile() all return an int indicating success (1) or failure (0). Why shouldn't these functions return a boolean? I would have used boolean as return code for success and failure

Re: [HACKERS] postgresql CVS callgraph data from dbt2

2005-09-14 Thread Michael Paesold
Mark Wong wrote: Hi everyone, For those of you watching the the daily results generated from STP (http://developer.osdl.org/markw/postgrescvs/dbt2/) I have callgraph data from oprofile collected starting from the Sept 9 results. Here is an example of what it looks like: http://www.testing.osd

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Michael Paesold
Tom Lane wrote: "Michael Paesold" <[EMAIL PROTECTED]> writes: To have other data, I have retested the patches on a single-cpu Intel P4 3GHz w/ HT (i.e. 2 virtual cpus), no EM64T. Comparing to the 2,4 dual-Xeon results it's clear that this is in reality only one cpu. While the runtime for N=1

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Greg Stark
Stephen Frost <[EMAIL PROTECTED]> writes: > * Tom Lane ([EMAIL PROTECTED]) wrote: > > I'm starting to think that we might have to succumb to having a compile > > option "optimize for multiprocessor" or "optimize for single processor". > > It's pretty hard to see how we'd alter a data structure dec

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Dave Page
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Tom Lane > Sent: 13 September 2005 23:03 > To: Marko Kreen > Cc: pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Spinlocks, yet again: analysis and > proposed patches > > I'm starting to thi

Re: [HACKERS] Spinlocks, yet again: analysis and proposed patches

2005-09-14 Thread Michael Paesold
Tom Lane wrote: But the cmpb instruction in the 8.0 version of TAS would have done that, and I think we've already established that the cmpb is a loss on most machines (except maybe single-physical-CPU Xeons). Note that this was a regular Pentium 4 system, not a Xeon. Best Regards, Michael Pa