Re: [HACKERS] PL/Python warnings in CVS HEAD

2007-03-06 Thread Neil Conway
On Tue, 2007-03-06 at 00:18 -0500, Tom Lane wrote: Sounds like #ifdef time to me --- but it seems a bit strange; wouldn't the Python guys have taken a bit more care for compatibility of user-supplied code? Yeah, I was a bit surprised as well. I won't claim to have any familiarity with the

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Simon Riggs
On Tue, 2007-03-06 at 00:54 +0100, Florian G. Pflug wrote: Simon Riggs wrote: But it would break the idea of letting a second seqscan follow in the first's hot cache trail, no? No, but it would make it somewhat harder to achieve without direct synchronization between scans. It could still

[HACKERS] user-defined tree methods in GIST

2007-03-06 Thread sharath kumar
I need some help in building new index structure using GIST. I have to include a procedure for restructuring the tree immediately after i have created the index using gist. How do i do it? Actually I am trying to implement the slim-tree index using gist. To increase the performance, a slim-down

Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-03-06 Thread Peter Eisentraut
Mike Rylander wrote: The patch adds support for default XML namespaces in xml2 by providing a mechanism for supplying a prefix to a named namespace URI. How does it support multiple namespaces in one document? -- Peter Eisentraut http://developer.postgresql.org/~petere/

Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-03-06 Thread Nikolay Samokhvalov
On 3/6/07, Mike Rylander [EMAIL PROTECTED] wrote: Attatched you'll find a patch that I've been kicking around for a while that I'd like to propose for inclusion in 8.3. I attempted to submit this through the original xml2 author (as far back as the 7.4 days) but got no response. It's really

Re: [HACKERS] Aggressive freezing in lazy-vacuum

2007-03-06 Thread ITAGAKI Takahiro
Tom Lane [EMAIL PROTECTED] wrote: This is a stand-alone patch for aggressive freezing. I'll propose to use OldestXmin instead of FreezeLimit as the freeze threshold in the circumstances below: I think it's a really bad idea to freeze that aggressively under any circumstances except

Re: [HACKERS] PL/Python warnings in CVS HEAD

2007-03-06 Thread Marko Kreen
On 3/6/07, Neil Conway [EMAIL PROTECTED] wrote: On Tue, 2007-03-06 at 00:18 -0500, Tom Lane wrote: Perhaps they provide a compatibility hack that you didn't spot? Quite possibly. Anyone have any suggestions? Python guys discuss the situation here: http://www.python.org/dev/peps/pep-0353/

[HACKERS] A Survey on Defect Management Practices in Free/Open Source Software

2007-03-06 Thread Anu Gupta DCSA
Dear PostgreSQL Contributors, I seek help from designers, developers, testers,defect fixers,project managers or playing any other key role in Free/Open Source software development or maintenence in carrying out a study on practices and problems of defect management in various Free/Open Source

Re: [HACKERS] Aggressive freezing in lazy-vacuum

2007-03-06 Thread Gregory Stark
ITAGAKI Takahiro [EMAIL PROTECTED] writes: I don't think we can supply such a historical database functionality here, because we can guarantee it just only for INSERTed tuples even if we pay attention. We've already enabled autovacuum as default, so that we cannot predict when the next

Re: [HACKERS] user-defined tree methods in GIST

2007-03-06 Thread Martijn van Oosterhout
On Tue, Mar 06, 2007 at 01:50:34PM +0530, sharath kumar wrote: I need some help in building new index structure using GIST. I have to include a procedure for restructuring the tree immediately after i have created the index using gist. How do i do it? Actually I am trying to implement the

[HACKERS] Auto creation of Partitions

2007-03-06 Thread NikhilS
Hi, This is to get feedback to meet the following TODO: - Simplify ability to create partitioned tables This would allow creation of partitioned tables without requiring creation of rules for INSERT/UPDATE/DELETE, and constraints for rapid partition selection. Options could include

Re: [HACKERS] Latest plans for Utilities with HOT

2007-03-06 Thread Simon Riggs
On Mon, 2007-03-05 at 22:25 +0530, Pavan Deolasee wrote: Simon Riggs wrote: On Mon, 2007-03-05 at 21:39 +0530, Pavan Deolasee wrote: Currently each tuple is moved individually. You'd need to inspect the whole HOT chain on a page, calculate space for that and then try to move them all

Re: [HACKERS] Updated propsoal for read-only queries on PITR slaves (SoC 2007)

2007-03-06 Thread Florian G. Pflug
Jim Nasby wrote: On Mar 1, 2007, at 8:45 AM, Florian G. Pflug wrote: Another possibility would be to move this setting into the recovery.conf. The problems with this approach is that the recovery.conf file is deleted after the information it contains is incorporated into pg_control. Thus, the

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Peter Eisentraut
NikhilS wrote: iv) Based on the PRIMARY, UNIQUE, REFERENCES information specified, pass it on to the children tables. How will you maintain a primary key in such a table, considering that indexes can't span multiple tables? 1) Whether we should use triggers/rules for step number (iii) above.

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Teodor Sigaev [EMAIL PROTECTED] writes: A closer reading, however, shows that at least for cases like intarray, btree_gist, etc., the detoasting of an index value is being done in the gist decompress function, so the value seen via GISTENTRY in the other functions should already have been

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread NikhilS
Hi, On 3/6/07, Peter Eisentraut [EMAIL PROTECTED] wrote: NikhilS wrote: iv) Based on the PRIMARY, UNIQUE, REFERENCES information specified, pass it on to the children tables. How will you maintain a primary key in such a table, considering that indexes can't span multiple tables? We will

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Gregory Stark
NikhilS [EMAIL PROTECTED] writes: the intention is to use this information from the parent and make it a property of the child table. This will avoid the step for the user having to manually specify CREATE INDEX and the likes on all the children tables one-by-one. Missed the start of this

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread NikhilS
On 3/6/07, NikhilS [EMAIL PROTECTED] wrote: Hi, On 3/6/07, Gregory Stark [EMAIL PROTECTED] wrote: NikhilS [EMAIL PROTECTED] writes: the intention is to use this information from the parent and make it a property of the child table. This will avoid the step for the user having to

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread NikhilS
Hi, On 3/6/07, Gregory Stark [EMAIL PROTECTED] wrote: NikhilS [EMAIL PROTECTED] writes: the intention is to use this information from the parent and make it a property of the child table. This will avoid the step for the user having to manually specify CREATE INDEX and the likes on all the

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Andrew - Supernews
On 2007-03-06, Gregory Stark [EMAIL PROTECTED] wrote: Teodor Sigaev [EMAIL PROTECTED] writes: A closer reading, however, shows that at least for cases like intarray, btree_gist, etc., the detoasting of an index value is being done in the gist decompress function, so the value seen via

Re: [HACKERS] Aggressive freezing in lazy-vacuum

2007-03-06 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] wrote: I think it's a really bad idea to freeze that aggressively under any circumstances except being told to (ie, VACUUM FREEZE). When you freeze, you lose history information that might be needed later --- for forensic

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Teodor Sigaev
The problem is that this is the only place in the code where we make wholesale assumptions that a datum that comes from a tuple (heap tuple or index tuple) isn't toasted. There are other places but they're always flagged with big comments explaining *why* the datum can't be toasted and they're

Re: [HACKERS] user-defined tree methods in GIST

2007-03-06 Thread Teodor Sigaev
I need some help in building new index structure using GIST. I have to include a procedure for restructuring the tree immediately after i have created the index using gist. How do i do it? IMHO, you don't have a chance to do it without patching postgres or developing your own low-level

Re: [HACKERS] Log levels for checkpoint/bgwriter monitoring

2007-03-06 Thread Jim Nasby
On Mar 5, 2007, at 8:34 PM, Greg Smith wrote: On Thu, 22 Feb 2007, Jim C. Nasby wrote: It would also be extremely useful to make checkpoint stats visible somewhere in the database (presumably via the existing stats mechanism)... I'm thinking just tracking how many pages had to be flushed

[HACKERS] Calculated view fields (8.1 != 8.2)

2007-03-06 Thread Gaetano Mendola
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi all, I'm observing this behavior in 8.2: kalman=# create table test ( a integer, b integer ); CREATE TABLE kalman=# kalman=# CREATE OR REPLACE FUNCTION sp_test ( INTEGER ) kalman-# RETURNS INTEGER AS' kalman'# DECLARE kalman'#a_id_contract

Re: [HACKERS] Trivial HugeTLB Benchmark

2007-03-06 Thread Ryan Cumming
On Sun, 2007-03-04 at 10:14 -0800, Tom Lane wrote: If you did this only once, the results are not really trustworthy; you need to average several similar runs before you can have much confidence. pgbench's inter-run variation is usually upwards of 10%, so trying to draw conclusions about

Re: [HACKERS] ERROR: operator does not exist: integer !=- integer

2007-03-06 Thread William ZHANG
I get it. scan.l converts '!=' to '': 644 /* Convert != operator to for compatibility */ 645 if (strcmp(yytext, !=) == 0) 646 yylval.str = pstrdup(); 647 else 648

Re: [HACKERS] NTA access on Solaris

2007-03-06 Thread Sherry Moore
With copyout_max_cached being 8K: ^^^ 4K Working set 16K 32K 64K 128K256K512K1M 2M 128M Seconds 4.8

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Andrew - Supernews [EMAIL PROTECTED] writes: The places in the intarray code that you tried to fix in your patch at the start of this thread are not dealing with data that came from a tuple, but from data that came from a decompress method. It's expected that the decompress method does the

Re: [HACKERS] NTA access on Solaris

2007-03-06 Thread Sherry Moore
On a 1P system system with 512K L2, it is more obvious why we shouldn't bypass L2 for small reads: The same readtest as my previous mail invoked as following: ./readtest -s working-set-size -f /platform/i86pc/boot_archive -n 100 With copyout_max_cached being 128K: Working set 16K

[HACKERS] NTA access on Solaris

2007-03-06 Thread Sherry Moore
- Forwarded message from Sherry Moore [EMAIL PROTECTED] - Date: Mon, 5 Mar 2007 21:34:19 -0800 From: Sherry Moore [EMAIL PROTECTED] To: Tom Lane [EMAIL PROTECTED] Cc: Luke Lonergan [EMAIL PROTECTED], Mark Kirkwood [EMAIL PROTECTED], Pavan Deolasee [EMAIL PROTECTED],

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Sherry Moore
Hi Tom, Sorry about the delay. I have been away from computers all day. In the current Solaris release in development (Code name Nevada, available for download at http://opensolaris.org), I have implemented non-temporal access (NTA) which bypasses L2 for most writes, and reads larger than

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Shane Ambler
NikhilS wrote: Hi, On 3/6/07, Peter Eisentraut [EMAIL PROTECTED] wrote: NikhilS wrote: iv) Based on the PRIMARY, UNIQUE, REFERENCES information specified, pass it on to the children tables. How will you maintain a primary key in such a table, considering that indexes can't span multiple

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Teodor Sigaev [EMAIL PROTECTED] writes: I'm afraid that we have some lack of understanding. Flow of operation with indexed tuple in gist is: - read tuple - get n-th attribute with a help of index_getattr - call user-defined decompress method which should, at least, detoast value - result

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread David Fetter
On Mon, Mar 05, 2007 at 09:04:46PM -0600, Jim Nasby wrote: FWIW, this sounds like a subset of the Query By Example stuff that someone is working on. I don't have a URL handy since I'm on a plane, but I think google can find it. It's now called ObelisQ http://pgfoundry.org/projects/qbe

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Gregory Stark [EMAIL PROTECTED] writes: Andrew - Supernews [EMAIL PROTECTED] writes: So I think you've mis-analyzed the problem. That's especially true since you are claiming that the existing code is already buggy when in fact no such bugs have been reported (and clearly intarray has been

Re: [HACKERS] Synchronized Scan update

2007-03-06 Thread Josh Berkus
JD, Don't get me wrong, I want things to be easily understandable as well but the reason you site above pretty much makes us need to remove most of the postgresql.conf, including all bgwriter, vacuum cost delay, and autovac settings. Not to mention commit delay and others ;). Wouldn't that

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Teodor Sigaev
So when does index_form_tuple get called? The single call of index_form_tuple in GiST core is in gistFormTuple which initially compress any indexed value with a help of their compress methods. Only tuples formed by gistFormTuple could be inserted in index. Does every data type define a

Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-03-06 Thread Mike Rylander
On 3/6/07, Peter Eisentraut [EMAIL PROTECTED] wrote: Mike Rylander wrote: The patch adds support for default XML namespaces in xml2 by providing a mechanism for supplying a prefix to a named namespace URI. How does it support multiple namespaces in one document? It supports one default

Re: [HACKERS] [PATCHES] xml2 contrib patch supporting default XML namespaces

2007-03-06 Thread Mike Rylander
On 3/6/07, Nikolay Samokhvalov [EMAIL PROTECTED] wrote: On 3/6/07, Mike Rylander [EMAIL PROTECTED] wrote: Attatched you'll find a patch that I've been kicking around for a while that I'd like to propose for inclusion in 8.3. I attempted to submit this through the original xml2 author (as far

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Teodor Sigaev
And it's not clear _int_gist.c has been running with toasted array values for years because it's limited to arrays of 100 integers (or perhaps 200 integers, there's a factor of 2 in the test). That's not enough to trigger toasting unless there are other large columns in the same table. That's was

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Does every data type define a compress/decompress method? Even if it's not a data type that normally gets very large? Yes, any GiST opclass should have such methods. In trivial case it just returns input value. As I remember, only R-Tree emulation over boxes, contrib/seg and contrib/cube

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread Chris Browne
[EMAIL PROTECTED] (Jim Nasby) writes: FWIW, this sounds like a subset of the Query By Example stuff that someone is working on. I don't have a URL handy since I'm on a plane, but I think google can find it. The pgFoundry project is here... http://pgfoundry.org/projects/qbe And yes, indeed,

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Chris Browne
[EMAIL PROTECTED] (Peter Eisentraut) writes: NikhilS wrote: iv) Based on the PRIMARY, UNIQUE, REFERENCES information specified, pass it on to the children tables. How will you maintain a primary key in such a table, considering that indexes can't span multiple tables? On the one hand, I

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Teodor Sigaev
input value. As I remember, only R-Tree emulation over boxes, contrib/seg and contrib/cube have simple compress method. Hm, if they just return the original datum without detoasting it then it could be an issue. I'll check. seg and box aren't a varlena types, but cube is and it seems broken :(.

[HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread Tom Lane
[ cc'd to pgsql-jdbc which seems the group most likely to be affected by any protocol change ] So I've been working on a plan cache module per my earlier proposal, and I've run up against a problem with getting exec_parse_message to use it. The problem is that the current rather hackish

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread Josh Berkus
And yes, indeed, this sounds quite a lot like what Meredith Patterson presented at the Toronto conference. This would be good to have, though, since Meredith's work has some problematic IP encumbrances. Question, though: is the SKYLINE syntax part of a standard anywhere? -- Josh Berkus

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Martijn van Oosterhout
On Wed, Mar 07, 2007 at 02:43:39AM +1030, Shane Ambler wrote: I think a way can be devised to maintain the primary key and unique constraints. If a search is done on the parent table, the planner knows to rewrite the query as a union (or union all) of all child tables that relate to the

Re: [HACKERS] Calculated view fields (8.1 != 8.2)

2007-03-06 Thread Tom Lane
Gaetano Mendola [EMAIL PROTECTED] writes: [ 8.2 evaluates volatile functions in the targetlist of a view ] If I mark the function as STABLE or IMMUTABLE then even with version 8.2 the function is not evaluated. Is this the intended behavior? Yes; people complained that we needed to be more

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Teodor Sigaev [EMAIL PROTECTED] writes: And it's not clear _int_gist.c has been running with toasted array values for years because it's limited to arrays of 100 integers (or perhaps 200 integers, there's a factor of 2 in the test). That's not enough to trigger toasting unless there are

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jeff Davis
On Mon, 2007-03-05 at 21:02 -0700, Jim Nasby wrote: On Mar 5, 2007, at 2:03 PM, Heikki Linnakangas wrote: Another approach I proposed back in December is to not have a variable like that at all, but scan the buffer cache for pages belonging to the table you're scanning to initialize the

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: If I were to implement this idea, I think Heikki's bitmap of pages already read is the way to go. I think that's a good way to guarantee that you'll not finish in time for 8.3. Heikki's idea is just at the handwaving stage at this point, and I'm not even

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread Tom Lane
Josh Berkus josh@agliodbs.com writes: Question, though: is the SKYLINE syntax part of a standard anywhere? There's certainly not anything like that in SQL2003. I'm also kind of wondering if the main use-cases couldn't be met with suitable multi-input custom aggregates, which is something we

Re: [HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: B. Don't store the unnamed statement in the plan cache. To make sure it's not used anymore when the plan might be stale, forcibly discard the unnamed statement after execution. This would get rid of a lot of overhead but would mean a significant change in

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jeff Davis
On Tue, 2007-03-06 at 12:59 -0500, Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: If I were to implement this idea, I think Heikki's bitmap of pages already read is the way to go. I think that's a good way to guarantee that you'll not finish in time for 8.3. Heikki's idea is just

Re: [HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Can we forcibly discard it if *any* messages are received that might invalidate a plan? So basically it would work fine unless anyone in the system does any DDL at all? I guess that has the downside of introducing random unpredictable failures. Ugh :-(

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Florian G. Pflug
Martijn van Oosterhout wrote: On Wed, Mar 07, 2007 at 02:43:39AM +1030, Shane Ambler wrote: I think a way can be devised to maintain the primary key and unique constraints. If a search is done on the parent table, the planner knows to rewrite the query as a union (or union all) of all child

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Heikki Linnakangas
Jeff Davis wrote: On Mon, 2007-03-05 at 21:02 -0700, Jim Nasby wrote: On Mar 5, 2007, at 2:03 PM, Heikki Linnakangas wrote: Another approach I proposed back in December is to not have a variable like that at all, but scan the buffer cache for pages belonging to the table you're scanning to

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Peter Eisentraut
NikhilS wrote: We will not (I know its a hard thing to do :) ), the intention is to use this information from the parent and make it a property of the child table. This will avoid the step for the user having to manually specify CREATE INDEX and the likes on all the children tables

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Josh Berkus
Florian, This sounds like what is really needed is a way to lock a certain condition, namely the existance or non-existance of a record with certain values in certain fields. This would not only help this case, it would also help RI triggers, because those wouldn't have to acquire a share

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread Josh Berkus
Tom, I'm also kind of wondering if the main use-cases couldn't be met with suitable multi-input custom aggregates, which is something we already have as of 8.2. Actually, given that skyline of is *only* for aggregate sorting (as far as I can tell) it doesn't present the complications which

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: But when I say CREATE TABLE ( a int PRIMARY KEY, ... ) PARTITION blah ... then I expect that the primary key will be enforced across all partitions. We currently sidestep that issue by not offering seemingly transparent partitioning. But if you

Re: [JDBC] [HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread Dave Cramer
I think C is how the JDBC driver is written. We name the statements if they have been used more than prepareThreshold times. So we have a mechanism by which to allow statements to be cached, or not. Dave On 6-Mar-07, at 1:14 PM, Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes:

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Teodor Sigaev
It doesn't actually crash, it just fails CHECKARRVALID. I added an assertion in there to cause it to generate a core dump. Wow, catch that, see attached patch. g_int_decompress doesn't returns detoasted array in case it was empty. Previously it was safe because empty array never has been

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Heikki Linnakangas
Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: If I were to implement this idea, I think Heikki's bitmap of pages already read is the way to go. I think that's a good way to guarantee that you'll not finish in time for 8.3. Heikki's idea is just at the handwaving stage at this point,

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Josh Berkus
Agreed, it needs to Just Work. I think it'd still be useful though if we only support auto-partitioning on the primary key, and that restriction avoids the indexing problem. +1 -- Josh Berkus PostgreSQL @ Sun San Francisco ---(end of

Re: [HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread andyk
Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: Can we forcibly discard it if *any* messages are received that might invalidate a plan? So basically it would work fine unless anyone in the system does any DDL at all? I guess that has the downside of introducing random unpredictable

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread Alvaro Herrera
Josh Berkus wrote: Tom, I'm also kind of wondering if the main use-cases couldn't be met with suitable multi-input custom aggregates, which is something we already have as of 8.2. Actually, given that skyline of is *only* for aggregate sorting (as far as I can tell) it doesn't

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Andrew Dunstan
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: But when I say CREATE TABLE ( a int PRIMARY KEY, ... ) PARTITION blah ... then I expect that the primary key will be enforced across all partitions. We currently sidestep that issue by not offering seemingly transparent

Re: [HACKERS] Arrays of Complex Types

2007-03-06 Thread David Fetter
On Fri, Mar 02, 2007 at 06:59:50PM -0500, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: 1. In src/backend/commands/tablecmds.c, change DefineRelation as follows: * After the first call to heap_create_with_catalog, construct and do another call to for the array type.

Re: [HACKERS] Arrays of Complex Types

2007-03-06 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: I noticed something in src/backend/commands/tablecmds.c which worries me, namely that it ignores functions and views. What? regards, tom lane ---(end of broadcast)--- TIP 3: Have you

Re: [HACKERS] Arrays of Complex Types

2007-03-06 Thread David Fetter
On Tue, Mar 06, 2007 at 04:14:07PM -0500, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: I noticed something in src/backend/commands/tablecmds.c which worries me, namely that it ignores functions and views. What? The it in question is, find_composite_type_dependencies() Cheers, D

Re: [HACKERS] Arrays of Complex Types

2007-03-06 Thread Tom Lane
David Fetter [EMAIL PROTECTED] writes: On Tue, Mar 06, 2007 at 04:14:07PM -0500, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: I noticed something in src/backend/commands/tablecmds.c which worries me, namely that it ignores functions and views. What? The it in question is,

Re: [HACKERS] Arrays of Complex Types

2007-03-06 Thread David Fetter
On Tue, Mar 06, 2007 at 04:24:36PM -0500, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: On Tue, Mar 06, 2007 at 04:14:07PM -0500, Tom Lane wrote: David Fetter [EMAIL PROTECTED] writes: I noticed something in src/backend/commands/tablecmds.c which worries me, namely that it

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Joris Dobbelsteen
-Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Josh Berkus Sent: dinsdag 6 maart 2007 19:45 To: pgsql-hackers@postgresql.org Cc: Florian G. Pflug; Martijn van Oosterhout; Shane Ambler; NikhilS; Peter Eisentraut Subject: Re: [HACKERS] Auto creation of

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Simon Riggs
On Mon, 2007-03-05 at 21:34 -0800, Sherry Moore wrote: - Based on a lot of the benchmarks and workloads I traced, the target buffer of read operations are typically accessed again shortly after the read, while writes are usually not. Therefore, the default operation

Re: [HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread Simon Riggs
On Tue, 2007-03-06 at 12:22 -0500, Tom Lane wrote: A. Just accept the extra overhead, thereby preserving the current behavior of unnamed statements, and gaining the benefit that plan invalidation will work correctly in the few cases where an unnamed statement's plan lasts long enough to need

Re: [HACKERS] Plan invalidation vs. unnamed prepared statements

2007-03-06 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Tue, 2007-03-06 at 12:22 -0500, Tom Lane wrote: A. Just accept the extra overhead, thereby preserving the current behavior of unnamed statements, and gaining the benefit that plan invalidation will work correctly in the few cases where an unnamed

Re: [HACKERS] GIST and TOAST

2007-03-06 Thread Gregory Stark
Teodor Sigaev [EMAIL PROTECTED] writes: It doesn't actually crash, it just fails CHECKARRVALID. I added an assertion in there to cause it to generate a core dump. Wow, catch that, see attached patch. g_int_decompress doesn't returns detoasted array in case it was empty. Previously it was

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jeff Davis
On Tue, 2007-03-06 at 18:47 +, Heikki Linnakangas wrote: Tom Lane wrote: Jeff Davis [EMAIL PROTECTED] writes: If I were to implement this idea, I think Heikki's bitmap of pages already read is the way to go. I think that's a good way to guarantee that you'll not finish in time

Re: [HACKERS] PostgreSQL - 'SKYLINE OF' clause added!

2007-03-06 Thread Gavin Sherry
On Tue, 6 Mar 2007, Alvaro Herrera wrote: Also, keep in mind that there were plenty of changes in the executor. This stuff is not likely to be very easy to implement efficiently using our extant executor machinery; note that Ranbeer mentioned implementation of block nested loop and other

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jim Nasby
On Mar 6, 2007, at 12:17 AM, Tom Lane wrote: Jim Nasby [EMAIL PROTECTED] writes: An idea I've been thinking about would be to have the bgwriter or some other background process actually try and keep the free list populated, The bgwriter already tries to keep pages just in front of the clock

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jim Nasby
On Mar 6, 2007, at 10:56 AM, Jeff Davis wrote: We also don't need an exact count, either. Perhaps there's some way we could keep a counter or something... Exact count of what? The pages already in cache? Yes. The idea being if you see there's 10k pages in cache, you can likely start 9k

Re: [HACKERS] Aggressive freezing in lazy-vacuum

2007-03-06 Thread ITAGAKI Takahiro
Gregory Stark [EMAIL PROTECTED] wrote: The hoped for gain here is that vacuum finds fewer pages with tuples that exceed vacuum_freeze_min_age? That seems useful though vacuum is still going to have to read every page and I suspect most of the writes pertain to dead tuples, not freezing

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jeff Davis
On Tue, 2007-03-06 at 17:43 -0700, Jim Nasby wrote: On Mar 6, 2007, at 10:56 AM, Jeff Davis wrote: We also don't need an exact count, either. Perhaps there's some way we could keep a counter or something... Exact count of what? The pages already in cache? Yes. The idea being if you see

Re: [HACKERS] Aggressive freezing in lazy-vacuum

2007-03-06 Thread ITAGAKI Takahiro
Tom Lane [EMAIL PROTECTED] wrote: I said nothing about expired tuples. The point of not freezing is to preserve information about the insertion time of live tuples. I don't know what good it will do -- for debugging? Why don't you use CURRENT_TIMESTAMP? And your test case is

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Jeff Davis
On Tue, 2007-03-06 at 18:29 +, Heikki Linnakangas wrote: Jeff Davis wrote: On Mon, 2007-03-05 at 21:02 -0700, Jim Nasby wrote: On Mar 5, 2007, at 2:03 PM, Heikki Linnakangas wrote: Another approach I proposed back in December is to not have a variable like that at all, but scan the

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Luke Lonergan
Incidentally, we tried triggering NTA (L2 cache bypass) unconditionally and in various patterns and did not see the substantial gain as with reducing the working set size. My conclusion: Fixing the OS is not sufficient to alleviate the issue. We see a 2x penalty (1700MB/s versus 3500MB/s) at

Re: [HACKERS] Log levels for checkpoint/bgwriter monitoring

2007-03-06 Thread Greg Smith
On Tue, 6 Mar 2007, Jim Nasby wrote: The flipside is that it's much easier to machine-parse a table rather than trying to scrape the logs. Now you might realize why I've been so vocal on the SQL log export implementation details. And I don't think we'll generally care about each individual

Re: [HACKERS] Aggressive freezing in lazy-vacuum

2007-03-06 Thread Tom Lane
ITAGAKI Takahiro [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] wrote: I said nothing about expired tuples. The point of not freezing is to preserve information about the insertion time of live tuples. I don't know what good it will do -- for debugging? Exactly. As an example, I've

Re: [HACKERS] Log levels for checkpoint/bgwriter monitoring

2007-03-06 Thread ITAGAKI Takahiro
Greg Smith [EMAIL PROTECTED] wrote: After a few months of staring at this data, I've found averages like that misleading. The real problem areas correlate with the peak pages written at any one checkpoint. Lowering that value is really the end-game for optimizing the background writer,

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread NikhilS
Hi Shane, Maybe I'm looking at auto-maintenance which is beyond any current planning? Many of your suggestions are useful, but auto-maintenance will be beyond the current plan. Regards, Nikhils EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread NikhilS
Hi, On 3/7/07, Tom Lane [EMAIL PROTECTED] wrote: Peter Eisentraut [EMAIL PROTECTED] writes: But when I say CREATE TABLE ( a int PRIMARY KEY, ... ) PARTITION blah ... then I expect that the primary key will be enforced across all partitions. We currently sidestep that issue by not offering

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Stefan Kaltenbrunner
Andrew Dunstan wrote: Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: But when I say CREATE TABLE ( a int PRIMARY KEY, ... ) PARTITION blah ... then I expect that the primary key will be enforced across all partitions. We currently sidestep that issue by not offering seemingly

Re: [HACKERS] Bug: Buffer cache is not scan resistant

2007-03-06 Thread Hannu Krosing
Ühel kenal päeval, T, 2007-03-06 kell 18:28, kirjutas Jeff Davis: On Tue, 2007-03-06 at 18:29 +, Heikki Linnakangas wrote: Jeff Davis wrote: On Mon, 2007-03-05 at 21:02 -0700, Jim Nasby wrote: On Mar 5, 2007, at 2:03 PM, Heikki Linnakangas wrote: Another approach I proposed back in

Re: [HACKERS] Auto creation of Partitions

2007-03-06 Thread Joshua D. Drake
Maybe. The most obvious use for automatic partitioning that I can think of would be based in the value of a timestamptz field rather than any PK. Of course I tend to work more in the OLTP field than in DW type apps, where other considerations might apply. I second that - partitioning on