Re: [HACKERS] Why do we want to %Remove behavior of postmaster -o

2006-06-06 Thread Peter Eisentraut
Am Samstag, 3. Juni 2006 04:27 schrieb Tom Lane: Actually, the TODO item is very badly worded. The idea is to get rid of the spelling differences between postmaster and postgres options, and then there will be no need for '-o' because you'll just say what you want --- that is, -o -foo and

[HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE constraint

2006-06-06 Thread Travis Cross
I have a table that I am using to store email token data for DSPAM. I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. The server currently processes a few thousand emails per day, and this

[HACKERS] AIX check in datetime.h

2006-06-06 Thread Joachim Wieland
Can someone please explain why in include/utils/datetime.h (struct datetkn) there is a check for _AIX that either initializes a char* pointer or a char array? Is there any advantage of a char-array except for warnings of some compilers if the initilization string is too long? Apart from that I

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread Simon Riggs
On Tue, 2006-06-06 at 11:45 +0900, ITAGAKI Takahiro wrote: Hi Hackers, I'm rewriting fillfactor patch, per the following discussion, http://archives.postgresql.org/pgsql-hackers/2006-03/msg00287.php Now fillfactor can be set using WITH syntax: - CREATE INDEX index ON table USING btree

[HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Mark Woodward
OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. We want to keep about a years worth of data at any specific time. We have triggers that fire is something interesting

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Hannu Krosing
Ühel kenal päeval, E, 2006-06-05 kell 14:10, kirjutas Tom Lane: Neil Conway [EMAIL PROTECTED] writes: On Mon, 2006-06-05 at 19:17 +0200, Zoltan Boszormenyi wrote: The general case cannot be applied for all particular cases. E.g. you cannot use cursors from shell scripts This could be

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Heikki Linnakangas
On Tue, 6 Jun 2006, Mark Woodward wrote: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. I would suggest doing all the inserts of one frame in one transaction.

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-06-06 kell 08:42, kirjutas Mark Woodward: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. We want to keep about a years worth of data at

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Alvaro Herrera
Mark Woodward wrote: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. Maybe if you grouped multiple operations on bigger transactions, the I/O savings could be

Re: [HACKERS] Why do we want to %Remove behavior of postmaster -o

2006-06-06 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: Am Samstag, 3. Juni 2006 04:27 schrieb Tom Lane: Actually, the TODO item is very badly worded. The idea is to get rid of the spelling differences between postmaster and postgres options, and then there will be no need for '-o' because you'll just say

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Tom Lane
Hannu Krosing [EMAIL PROTECTED] writes: Ühel kenal päeval, E, 2006-06-05 kell 14:10, kirjutas Tom Lane: Note of course that such a thing would push the incomplete-result problem further upstream. For instance in (hypothetical --cursor switch) psql --cursor -c select ... | myprogram

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Martijn van Oosterhout
On Mon, Jun 05, 2006 at 11:02:33PM -0400, Tom Lane wrote: Just got this rather surprising result: snip bogus explain output The Total runtime is correct AFAICT, which puts the top node's actual time rather far out in left field. This is pretty repeatable on my old slow HPPA machine. A new

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE constraint

2006-06-06 Thread Tom Lane
Travis Cross [EMAIL PROTECTED] writes: I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. Corrupt index, looks like ... you might try reindexing the index. I don't believe that the PANIC you

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Why not implement an array of option parameters on pg_class, so both heaps and indexes can be given additional parameters? That way you wouldn't need a specific relfillfactor attribute. That would allow us to keep CREATE TABLE free of additional keywords

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 09:48:43AM -0400, Tom Lane wrote: Hannu Krosing [EMAIL PROTECTED] writes: ??hel kenal p??eval, E, 2006-06-05 kell 14:10, kirjutas Tom Lane: Note of course that such a thing would push the incomplete-result problem further upstream. For instance in (hypothetical

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: On Tue, Jun 06, 2006 at 09:48:43AM -0400, Tom Lane wrote: psql --cursor -c select ... | myprogram there would be no very good way for myprogram to find out that it'd been sent an incomplete result due to error partway through the SELECT. So if an error

Re: [HACKERS] COPY (query) TO file

2006-06-06 Thread Harald Fuchs
In article [EMAIL PROTECTED], Andrew Dunstan [EMAIL PROTECTED] writes: Mark Woodward wrote: Tom had posted a question about file compression with copy. I thought about it, and I want to through this out and see if anyone things it is a good idea. Currently, the COPY command only copies a

Re: [HACKERS] COPY (query) TO file

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 04:47:40PM +0200, Harald Fuchs wrote: In article [EMAIL PROTECTED], Andrew Dunstan [EMAIL PROTECTED] writes: Mark Woodward wrote: Tom had posted a question about file compression with copy. I thought about it, and I want to through this out and see if anyone

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE constraint

2006-06-06 Thread Ian Barwick
On 6/6/06, Tom Lane [EMAIL PROTECTED] wrote: Travis Cross [EMAIL PROTECTED] writes: I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. Corrupt index, looks like ... you might try reindexing

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread Simon Riggs
On Tue, 2006-06-06 at 10:27 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Why not implement an array of option parameters on pg_class, so both heaps and indexes can be given additional parameters? That way you wouldn't need a specific relfillfactor attribute. That would allow

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Travis Cross
Kenneth Marshall wrote: We have used postgresql 7.4, 8.0, and 8.1 with DSPAM and have never had a single problem like you are describing. In the past on this mailing list, these sorts of issues have been caused by hardware problems on the DB server in some cases. Good luck with tracking it down.

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Bruce Momjian
Wow, that is strange. We could remove it for 8.2 and see how testing goes. --- Joachim Wieland wrote: Can someone please explain why in include/utils/datetime.h (struct datetkn) there is a check for _AIX that either

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes: Joachim Wieland wrote: Can someone please explain why in include/utils/datetime.h (struct datetkn) there is a check for _AIX that either initializes a char* pointer or a char array? Wow, that is strange. We could remove it for 8.2 and see how

Re: [HACKERS] adding new field

2006-06-06 Thread ranbeer makin
hey, let me clarify something. 1. Whenever you run a query, its result or some other thing gets cached. I want to flush this cache? Also let me know when this cache gets flushed automatically. 2. After adding a new field in Resdom structure and making necessary changes in outfuncs.c,

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Tom Lane
I wrote: It looks like a workaround for some ancient compiler problem. [ digs for awhile... ] Very ancient: we inherited that hack from Berkeley, In fact, now that I know where to look, I find the same thing in the postgres-v4r2 tarball, which means the hack is pre-1994. I don't have

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Joachim Wieland wrote: Can someone please explain why in include/utils/datetime.h (struct datetkn) there is a check for _AIX that either initializes a char* pointer or a char array? Wow, that is strange. We could remove it

[HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Roberto Rezende de Assis
Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. I wish to develop more statistical aggregate functions, and I prefer to use C than to write then in the PL/R. Thanks

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Travis Cross
Tom Lane wrote: Travis Cross [EMAIL PROTECTED] writes: I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. Corrupt index, looks like ... you might try reindexing the index. I probably should

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Travis Cross
Ian Barwick wrote: On 6/6/06, Tom Lane [EMAIL PROTECTED] wrote: Travis Cross [EMAIL PROTECTED] writes: I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. Corrupt index, looks like ... you

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Heikki Linnakangas
On Tue, 6 Jun 2006, Roberto Rezende de Assis wrote: Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. They're in src/backend/utils/adt/numeric.c I wish to develop more statistical aggregate functions, and I prefer

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Joshua D. Drake
Roberto Rezende de Assis wrote: Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. I wish to develop more statistical aggregate functions, and I prefer to use C than to write then in the PL/R.

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Christopher Browne
In an attempt to throw the authorities off his trail, pgman@candle.pha.pa.us (Bruce Momjian) transmitted: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Joachim Wieland wrote: Can someone please explain why in include/utils/datetime.h (struct datetkn) there is a check for

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Christopher Browne
Clinging to sanity, [EMAIL PROTECTED] (Hannu Krosing) mumbled into her beard: Ühel kenal päeval, T, 2006-06-06 kell 08:42, kirjutas Mark Woodward: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few

Re: [HACKERS] Connection Broken with Custom Dicts for TSearch2

2006-06-06 Thread John Jawed
Since we are on the topic, is there a timeline/plans for openfts being brought into core? If not, I'll continue my work on bringing it into Gentoo Portage. John On 6/5/06, Oleg Bartunov oleg@sai.msu.su wrote: On Mon, 5 Jun 2006, Teodor Sigaev wrote: Teodor Sigaev wrote: Sorry, it isn't

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Rod Taylor
On Tue, 2006-06-06 at 13:53 -0400, Christopher Browne wrote: Clinging to sanity, [EMAIL PROTECTED] (Hannu Krosing) mumbled into her beard: Ühel kenal päeval, T, 2006-06-06 kell 08:42, kirjutas Mark Woodward: OK, here's my problem, I have a nature study where we have about 10 video cameras

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Alvaro Herrera
Rod Taylor wrote: With one potential snafu -- it blocks new SELECTs against the parent table while truncate runs on the child (happens with constraint exclusion as well). If your transactions are short then it won't be an issue. If you have mixed length transactions (many short which the

Re: [HACKERS] adding new field

2006-06-06 Thread Martijn van Oosterhout
On Tue, Jun 06, 2006 at 09:44:04PM +0530, ranbeer makin wrote: hey, let me clarify something. 1. Whenever you run a query, its result or some other thing gets cached. I want to flush this cache? Also let me know when this cache gets flushed automatically. What cache? Query results are

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Andrew Dunstan
Christopher Browne wrote: I haven't been monitoring CVS HEAD, but you can be sure this will get tried out when 8.2 gets anywhere vaguely close to relese... The whole point of having a buildfarm is that we shouldn't have to wait, we should be able to see very quickly if we have broken

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Martijn van Oosterhout kleptog@svana.org writes: Wierd. Can you get the output of *instr in each call of InstrEndLoop? Preferably after it does the calculation but before it clears the values. So we get an idea of number of samples and what it guesses. SampleOverhead would be good too. The

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 12:58:10PM -0400, Travis Cross wrote: Do you *know* that the disk drive will not lie about write complete? Know is such a strong word ;) Honestly, I have very little idea. I understand the nature of the problem this presents, as I've read the very fine PostgreSQL

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
I wrote: The problem looks to be an underestimation of SampleOverhead, and on reflection it's clear why: what CalculateSampleOverhead is measuring isn't the total overhead, but the time between the two gettimeofday calls. Which is probably about half the true overhead. On further thought, I

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: The whole point of having a buildfarm is that we shouldn't have to wait, we should be able to see very quickly if we have broken something. And in fact kookaburra seems happy ... regards, tom lane

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 04:50:28PM -0400, Tom Lane wrote: I have a theory about this, and it's not pleasant at all. What I think is that we have a Heisenberg problem here: the act of invoking gettimeofday() actually changes what is measured. That is, the runtime of the second part of

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Jim C. Nasby [EMAIL PROTECTED] writes: If that's the case, then maybe a more sopdisticated method of measuring the overhead would work. I think you missed the point: the time spent in gettimeofday() itself is not the major overhead of EXPLAIN ANALYZE. At least it appears that this is the case

[HACKERS] SERIAL problems?

2006-06-06 Thread Zoltan Boszormenyi
Hi, I just saw these in the TODO list: o %Disallow changing DEFAULT expression of a SERIAL column? This should be done only if the existing SERIAL problems cannot be fixed. o %Disallow ALTER SEQUENCE changes for SERIAL sequences because pg_dump does not dump the changes What are the

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Martijn van Oosterhout
On Tue, Jun 06, 2006 at 04:50:28PM -0400, Tom Lane wrote: But that still leaves me with a problem because my machine is clearly overestimating the correction needed. I added some printouts and got raw totaltime = 0.370937 per_iter = 0.000156913, SampleOverhead = 3.28e-06 adj totaltime =

Re: [HACKERS] SERIAL problems?

2006-06-06 Thread Tom Lane
Zoltan Boszormenyi [EMAIL PROTECTED] writes: What are the existing problems? Please read the archives; this has been discussed recently. There's a lot of disagreement about what ALTER should allow and what pg_dump should do with an altered sequence. I am asking because I am experimenting to

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Martijn van Oosterhout kleptog@svana.org writes: This doesn't make any sense at all. How can a sampling run that only sampled 7% of the actual tuples, end up with a actual measured time that's more than 50% of the actual final runtime? AFAICS, the only conclusion is that the sampled executions

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: And two, that upper plan nodes seem much more affected than lower ones. That makes sense because the execution cycle of an upper node will involve touching more userspace data than a lower node, and therefore more of the flushed TLB entries will need to be

Re: [HACKERS] More thoughts about planner's cost estimates

2006-06-06 Thread Ron Mayer
Tom Lane wrote: One objection to this is that after moving off the gold standard of 1.0 = one page fetch, there is no longer any clear meaning to the cost estimate units; you're faced with the fact that they're just an arbitrary scale. I'm not sure that's such a bad thing, though. It seems

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Christopher Browne
Martha Stewart called it a Good Thing when [EMAIL PROTECTED] (Andrew Dunstan) wrote: Christopher Browne wrote: I haven't been monitoring CVS HEAD, but you can be sure this will get tried out when 8.2 gets anywhere vaguely close to relese... The whole point of having a buildfarm is that we

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: And two, that upper plan nodes seem much more affected than lower ones. That makes sense because the execution cycle of an upper node will involve touching more userspace data than a lower node, and therefore more of the

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Mark Woodward
Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. I wish to develop more statistical aggregate functions, and I prefer to use C than to write then in the PL/R. There is a library in contrib called intagg. I wrote it

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Mark Woodward
Mark Woodward wrote: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. Maybe if you grouped multiple operations on bigger transactions, the I/O savings could be

[HACKERS] DROP INHERITS

2006-06-06 Thread Greg Stark
I have a question about pg_depends, namely how do you identify the dependency tied to a given relationship. Specifically to handle DROP INHERITS. Is there any other reason there might be a dependency between two tables other than inheritance? If there was how would you tell the dependencies

Re: [HACKERS] DROP INHERITS

2006-06-06 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: I have a question about pg_depends, namely how do you identify the dependency tied to a given relationship. Specifically to handle DROP INHERITS. Is there any other reason there might be a dependency between two tables other than inheritance? If there was

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Koichi Suzuki
I've once proposed a patch for 64bit transaction ID, but this causes some overhead to each tuple (XMIN and XMAX). Pgbench with 64bit transaction ID has to pay about a couple of percent of performance. If 64bit transaction ID is a reasonable fix, I've already posted this patch. Anyone

Re: [HACKERS] DROP INHERITS

2006-06-06 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: I suppose it makes more sense to optimize this on the basis of what's used in the planner and executor rather than ALTER TABLE commands though. No, definitely not. Syscaches only exist to support hard-wired lookups in the backend C code. Indexes on