Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Heikki Linnakangas
David Fetter wrote: On Tue, Sep 16, 2008 at 08:59:08PM -0400, Alvaro Herrera wrote: Simon Riggs wrote: Disabling autovacuum can have catastrophic effects, since it disables the ANALYZing of tables. Can we have a mode where we disable autoVACUUM yet enable autoANALYZE? You mean something like

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 10:09 +0300, Heikki Linnakangas wrote: > David Fetter wrote: > > On Tue, Sep 16, 2008 at 08:59:08PM -0400, Alvaro Herrera wrote: > >> Simon Riggs wrote: > >>> Disabling autovacuum can have catastrophic effects, since it disables > >>> the ANALYZing of tables. > >>> > >>> Can

Re: [HACKERS] EXEC_BACKEND

2008-09-17 Thread Simon Riggs
On Tue, 2008-09-16 at 15:53 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > We keep talking about EXEC_BACKEND mode, though until recently I had > > misunderstood what that meant. I also realised that I have more than > > once neglected to take it into account when writing a p

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Pavel Stehule
2008/9/17 Tom Lane <[EMAIL PROTECTED]>: > Tatsuo Ishii <[EMAIL PROTECTED]> writes: >>> Do we really have to make RECURSIVE a fully reserved keyword? > >> According to the standard, RECURSIVE is a reserved keyword, I believe. > > Sure, but our general rule is to make keywords no more reserved than >

Re: [HACKERS] text search patch status update?

2008-09-17 Thread Heikki Linnakangas
Sushant Sinha wrote: Patch #2. I think this is a straigt forward bug fix. Yes, I think you're right. In hlCover(), *q is 0 when the only match is the first item in the text, and we shouldn't bail out with "return false" in that case. But there seems to be something else going on here as wel

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
> Now, it's questionable how tense we need to be about that as long as > event proc failure aborts calling of later event procs. That means > that procs have to be robust against getting DESTROY with no CREATE > calls in any case. Should we try to make that less uncertain? > > I attached a patc

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Andrew Chernow wrote: > Now, it's questionable how tense we need to be about that as long as > event proc failure aborts calling of later event procs. That means > that procs have to be robust against getting DESTROY with no CREATE > calls in any case. Should we try to make that less uncert

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Robert Haas
> I am not sure, if these rule is good. Somebody who develop on > postgresql should have a problems when they will be port to other > databases in future. Reserved words in standards should be respected. I disagree. I have never ported an app written for PostgreSQL to another database system, and

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Tom Lane
Andrew Chernow <[EMAIL PROTECTED]> writes: >>> Now, it's questionable how tense we need to be about that as long as >>> event proc failure aborts calling of later event procs. That means >>> that procs have to be robust against getting DESTROY with no CREATE >>> calls in any case. Should we try t

Re: [HACKERS] New FSM patch

2008-09-17 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Heikki Linnakangas wrote: Let me describe this test case first: - The test program calls RecordAndGetPageWithFreeSpace in a tight loop, with random values. There's no activity to the heap. In normal usage, the time spent in RecordAndGetWithFreeSpace is minusc

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tom Lane
"Robert Haas" <[EMAIL PROTECTED]> writes: >> I am not sure, if these rule is good. Somebody who develop on >> postgresql should have a problems when they will be port to other >> databases in future. Reserved words in standards should be respected. > I disagree. I have never ported an app written

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Stephen R. van den Berg
Tom Lane wrote: >Ron Mayer <[EMAIL PROTECTED]> writes: >> Tom Lane wrote: >> If I read SQL 200N's spec correctly >> select interval '-1 1:00:00'; >> should mean"-1 days -1 hours", >> yet 8.3 sees it as "-1 days +1 hours". >I think we are kind of stuck on this one. If we change it, the

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Tom Lane
"Stephen R. van den Berg" <[EMAIL PROTECTED]> writes: > Intervals are a scalar, not an addition of assorted values, alternating signs > between fields would be wrong. Sorry, you're the one who's wrong on that. We've treated intervals as three independent fields for years now (and before that it w

Re: [HACKERS] per-table autovacuum configuration

2008-09-17 Thread Martin Pihlak
Alvaro Herrera wrote: >> any new thoughts on the matter? Perhaps someone is already working >> on it? > > It is still a wanted feature, but a couple of people have offered > patches and I'm waiting for them ... > Aha, good. I was considering taking a stab at it, but under the circumstances will

Re: [HACKERS] Proposal of SE-PostgreSQL patches (for CommitFest:Sep)

2008-09-17 Thread KaiGai Kohei
KaiGai Kohei wrote: Peter, thanks for your comments. > Let's review: > > *) System-wide consistency in access controls could be nice to have in > some cases. But is it really achievable? In the typical three-tier web > application scenario, do you really have system-wide consistency? Can

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Tom Lane wrote: Some thought would need to be given to how that interacts with RESULTCOPY. Presumably on the destination side, RESULTCOPY is the equivalent of RESULTCREATE, ie, don't DESTROY if you didn't get COPY. But what of the source side? Arguably you shouldn't invoke COPY on events that

Re: [HACKERS] NDirectFileRead and Write

2008-09-17 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > - NDirectFile{Read|Write} are renamed to BufFile{Read|Write}Count. > - They are still declared in execdebug.h and buffile.c includes it. After some thought I concluded that the least ugly way to do this was to declare and define the variables in the s

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Tom Lane
Andrew Chernow <[EMAIL PROTECTED]> writes: > Tom Lane wrote: >> Some thought would need to be given to how that interacts with >> RESULTCOPY. Presumably on the destination side, RESULTCOPY is the >> equivalent of RESULTCREATE, ie, don't DESTROY if you didn't get COPY. >> But what of the source sid

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Kevin Grittner
>>> Tom Lane <[EMAIL PROTECTED]> wrote: > "Robert Haas" <[EMAIL PROTECTED]> writes: >>> I am not sure, if these rule is good. Somebody who develop on >>> postgresql should have a problems when they will be port to other >>> databases in future. Reserved words in standards should be respected. > >

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Ron Mayer
Tom Lane wrote: "Stephen R. van den Berg" <[EMAIL PROTECTED]> writes: Intervals are a scalar, not an addition of assorted values, alternating signs between fields would be wrong. Sorry, you're the one who's wrong on that. We've treated intervals as three independent fields for years now (and

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Alvaro Herrera
Simon Riggs wrote: > > On Wed, 2008-09-17 at 10:09 +0300, Heikki Linnakangas wrote: > > Isn't autoanalyze a waste of time during a bulk load? Seems better to > > run ANALYZE manually at the end. > > Its not a waste of time because it catches tables immediately they have > been loaded, not just

Re: [HACKERS] New FSM patch

2008-09-17 Thread Heikki Linnakangas
Zdenek Kotala wrote: I tested it with DTrace on Solaris 10 and 8CPUs SPARC machine. I got similar result as you. Main problem in your new implementation is locking. On small tables where FSM fits on one page clients spend about 3/4 time to waiting on page lock. That in itself is not that surp

Re: Column level privileges was:(Re: [HACKERS] Extending grant insert on tables to sequences)

2008-09-17 Thread Stephen Frost
Jaime, * Stephen Frost ([EMAIL PROTECTED]) wrote: > * Jaime Casanova ([EMAIL PROTECTED]) wrote: > > On 7/25/08, Stephen Frost <[EMAIL PROTECTED]> wrote: > > > Yes, I'm working on it > > > > hi, any work on it? may i help? > > If you look at the commitfest, I've posted my WIP so far there. Most

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Heikki Linnakangas
Alvaro Herrera wrote: Simon Riggs wrote: On Wed, 2008-09-17 at 10:09 +0300, Heikki Linnakangas wrote: Isn't autoanalyze a waste of time during a bulk load? Seems better to run ANALYZE manually at the end. Its not a waste of time because it catches tables immediately they have been loaded, not

Re: [HACKERS] [PATCHES] libpq events patch (with sgml docs)

2008-09-17 Thread Andrew Chernow
Andrew Chernow wrote: New patch following our discussion with updated docs. few logical errors). I don't think it makes sense to do it otherwise, it would be like calling free after a malloc failure. I can live with that definition, but please document it. To build on this analogy, PGEVT

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tatsuo Ishii
> Is physical_tlist optimization sensible for RecursiveScan? We seem > to use it for every other Scan node type. To enable physical_tlist optimization, it seems build_physical_tlist, use_physical_tlist and disuse_physical_tlist need to be changed. build_physical_tlist and use_physical_tlist have

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Tom Lane
Heikki Linnakangas <[EMAIL PROTECTED]> writes: > Alvaro Herrera wrote: >> Why doesn't this new request conflict with that one? > The problem back then was that a CREATE INDEX was waiting on the > autoanalyze to finish, and the autoanalyze took a long time to finish > because of vacuum_cost_delay

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tom Lane
Tatsuo Ishii <[EMAIL PROTECTED]> writes: >> Is physical_tlist optimization sensible for RecursiveScan? We seem >> to use it for every other Scan node type. > To enable physical_tlist optimization, it seems build_physical_tlist, > use_physical_tlist and disuse_physical_tlist need to be > changed.

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tatsuo Ishii
> > To enable physical_tlist optimization, it seems build_physical_tlist, > > use_physical_tlist and disuse_physical_tlist need to be > > changed. build_physical_tlist and use_physical_tlist have been already > > patched and only disuse_physical_tlist needs to be patched. Any other > > place I miss

Re: [HACKERS] Patch for SQL-standard negative valued year-month literals

2008-09-17 Thread Stephen R. van den Berg
Tom Lane wrote: >"Stephen R. van den Berg" <[EMAIL PROTECTED]> writes: >> Intervals are a scalar, not an addition of assorted values, alternating signs >> between fields would be wrong. >Sorry, you're the one who's wrong on that. We've treated intervals as >three independent fields for years now

Re: Column level privileges was:(Re: [HACKERS] Extending grant insert on tables to sequences)

2008-09-17 Thread Jaime Casanova
On 9/17/08, Stephen Frost <[EMAIL PROTECTED]> wrote: > > > A piece which can be broken off pretty easily is adding support to track > > the columns used through to the executor so we can check the permissions > > in the right place. > > Jamie, have you had a chance to work on this? It's next on my

Re: [HACKERS] text search patch status update?

2008-09-17 Thread Teodor Sigaev
I remember about that but right now I havn't time to make final review. Sorry. Will return soon. Sushant Sinha wrote: Any status updates on the following patches? 1. Fragments in tsearch2 headlines: http://archives.postgresql.org/pgsql-hackers/2008-08/msg00043.php 2. Bug in hlCover: http://ar

[HACKERS] Patch for SQL-Standard Interval output and decoupling DateStyle from IntervalStyle

2008-09-17 Thread Ron Mayer
Tom Lane wrote: In fact, given that we are now somewhat SQL-compliant on interval input, a GUC that selected PG traditional, SQL-standard, or ISO 8601 interval output format seems like it could be a good idea. Short summary: The attached patch (1) adds a new GUC called "IntervalStyle"

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 10:52 -0400, Tom Lane wrote: > Heikki Linnakangas <[EMAIL PROTECTED]> writes: > > Alvaro Herrera wrote: > >> Why doesn't this new request conflict with that one? > > > The problem back then was that a CREATE INDEX was waiting on the > > autoanalyze to finish, and the autoana

Re: [HACKERS] Autovacuum and Autoanalyze

2008-09-17 Thread Robert Haas
> I tend to agree with Alvaro that there's not very much of a use case for > an analyze-only autovacuum mode. Assuming that we get to the point of > having a parallelizing pg_restore, it would be interesting to give it an > option to include ANALYZE for each table it's loaded among the tasks > tha

[HACKERS] proposal - function's default parametes

2008-09-17 Thread Pavel Stehule
Hello, before any implementation of named params we have to implement parameters with default values. I don't see any sense of named params without it. It's ortogonal to variadic functions (variadic functions are called when we put more parameters than functions has, dafault parameters are used wh

[HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Hi, We've been profiling a large system (8 CPUs, 64 GB of memory, some dozens of disks) which seems rather more swamped than it should. Part of the problem seems to come from CleanupTempFiles, the second entry in oprofile output. This is the top 3 symbol report for a 2 minute oprofile run: $ op

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote: > We've been profiling a large system (8 CPUs, 64 GB of memory, some > dozens of disks) which seems rather more swamped than it should. Part > of the problem seems to come from CleanupTempFiles, the second entry in > oprofile output. I'm

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Simon Riggs wrote: > > On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote: > > > We've been profiling a large system (8 CPUs, 64 GB of memory, some > > dozens of disks) which seems rather more swamped than it should. Part > > of the problem seems to come from CleanupTempFiles, the second en

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Alvaro Herrera wrote: > Simon Riggs wrote: > > Simple solution is to have a state variable so you can see whether a > > backend has created an temp files in this transaction. Most don't, so I > > think the above two solutions are overkill. If we created any, scan for > > them, if not, don't. Just

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Ah -- like this? +1, but there are two kinds of temp files in that module, and only one of them is relevant here. Call it something like have_xact_temporary_files to make things clearer. I concur that the explicit test on SizeVfdCache > 0 is a waste o

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 17:34 -0400, Alvaro Herrera wrote: > Simon Riggs wrote: > > > > On Wed, 2008-09-17 at 16:25 -0400, Alvaro Herrera wrote: > > > > > We've been profiling a large system (8 CPUs, 64 GB of memory, some > > > dozens of disks) which seems rather more swamped than it should. Part

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Simon Riggs
On Wed, 2008-09-17 at 17:34 -0400, Alvaro Herrera wrote: > Ah -- like this? if test should include || isProcExit so you don't skip non-transactional temp files at proc exit when there weren't any created in the last transaction. -- Simon Riggs www.2ndQuadrant.com PostgreS

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > Ah -- like this? > > +1, but there are two kinds of temp files in that module, and only > one of them is relevant here. Call it something like > have_xact_temporary_files to make things clearer. Ok, so that's what I called it. Sim

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Alvaro Herrera
BTW in testing this patch I was surprised by the fact that temp tables files are removed at checkpoint time, rather than when the transaction ends (at first I thought I had broken the removal of temp files). Is this a recent feature? (I verified that this continues to work fine for WITH HOLD cur

Re: [HACKERS] New FSM patch

2008-09-17 Thread Decibel!
On Sep 17, 2008, at 9:30 AM, Heikki Linnakangas wrote: I think we'd still need to WAL log operations that decrease the amount of free space on page. Otherwise, after we have partial vacuum, we might never revisit a page, and update the FSM, even though there's usable space on the page, leavi

[HACKERS] 0x1A in control file on Windows

2008-09-17 Thread ITAGAKI Takahiro
I found a bug that pg_controldata ends with error if control files contain 0x1A (Ctrl+Z) on Windows. We probably need to add PG_BINARY when we open control files because 0x1A is an end-of-file marker on Windows. This fix needs to be applied in back versions (8.2, 8.3 and HEAD). Index: src/bin/pg

[HACKERS] Where is Aggregation Attribute

2008-09-17 Thread Zhe He
I want to get the attribute from an Aggregation node, where is it? typedef struct Agg { Plan plan; AggStrategy aggstrategy; int numCols; /* number of grouping columns */ AttrNumber *grpColIdx; /* their indexes in the target list */ Oid *grpOperators; /* equality operators to compare with */ long

Re: [HACKERS] Common Table Expressions (WITH RECURSIVE) patch

2008-09-17 Thread Tatsuo Ishii
Tom, thanks for the review. Here is an in-progress report. Patches against CVS HEAD attached. (uncommented items are work-in-progress). -- Tatsuo Ishii SRA OSS, Inc. Japan > Tatsuo Ishii <[EMAIL PROTECTED]> writes: > > Included is the latest patches against CVS HEAD. > > I spent some time readin

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > BTW in testing this patch I was surprised by the fact that temp tables > files are removed at checkpoint time, [ blink... ] Doesn't look like that should happen. What is your test case? regards, tom lane -- Sent via pgsql-ha

Re: [HACKERS] 0x1A in control file on Windows

2008-09-17 Thread Tom Lane
ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > I found a bug that pg_controldata ends with error if control files > contain 0x1A (Ctrl+Z) on Windows. > We probably need to add PG_BINARY when we open control files > because 0x1A is an end-of-file marker on Windows. Well, why is that a bug? If the

Re: [HACKERS] optimizing CleanupTempFiles

2008-09-17 Thread Heikki Linnakangas
Tom Lane wrote: Alvaro Herrera <[EMAIL PROTECTED]> writes: BTW in testing this patch I was surprised by the fact that temp tables files are removed at checkpoint time, [ blink... ] Doesn't look like that should happen. What is your test case? Hmph, must be because of the patch from last wi

Re: [HACKERS] 0x1A in control file on Windows

2008-09-17 Thread ITAGAKI Takahiro
Tom Lane <[EMAIL PROTECTED]> wrote: > ITAGAKI Takahiro <[EMAIL PROTECTED]> writes: > > We probably need to add PG_BINARY when we open control files > > because 0x1A is an end-of-file marker on Windows. > > Well, why is that a bug? If the platform is so silly as to define text > files that way,