Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Simon Riggs
On Wed, 2007-09-12 at 10:48 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: The following bug fix has not yet been applied to CVS http://archives.postgresql.org/pgsql-patches/2007-06/msg00100.php Frankly, this looks much more like it creates a bug than fixes one. I have not

Re: [HACKERS] Patch queue triage

2007-09-13 Thread Simon Riggs
On Wed, 2007-09-12 at 17:47 -0400, Bruce Momjian wrote: For those who have forgotten the progress we have made toward 8.3, here are the open patches we had for 8.3 as of May 1, 2006: Could you please issue a list of open items for 8.3? I want to check whether you are waiting on me for anything

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Simon Riggs
On Wed, 2007-09-12 at 10:48 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: The following bug fix has not yet been applied to CVS http://archives.postgresql.org/pgsql-patches/2007-06/msg00100.php Frankly, this looks much more like it creates a bug than fixes one. I have not

[HACKERS] Limitation on number of positions (tsearch)

2007-09-13 Thread Heikki Linnakangas
Why is there a limitation of 256 positions per lexeme in a tsvector? There doesn't seem to be a technical reason for that. WordEntryPosVector uses a uint16 to store the number of positions, so it go up to 65535. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com

Re: [HACKERS] Limitation on number of positions (tsearch)

2007-09-13 Thread Teodor Sigaev
Why is there a limitation of 256 positions per lexeme in a tsvector? There doesn't seem to be a technical reason for that. WordEntryPosVector uses a uint16 to store the number of positions, so it go up to 65535. For two reasons: - Ranking might become very slow if number of position is big -

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Magnus Hagander
On Thu, Sep 13, 2007 at 12:12:28AM +0200, Guillaume Lelarge wrote: Tom Lane a écrit : Dave Page [EMAIL PROTECTED] writes: Tom Lane wrote: Peter usually does it --- in theory any committer could, but he actually knows what to do and the rest of us would have to study ;-) Study or

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Guillaume Lelarge
Magnus Hagander a écrit : On Thu, Sep 13, 2007 at 12:12:28AM +0200, Guillaume Lelarge wrote: [...] It seems pretty straightforward now. Perhaps it can be used with cron. No. Doing that with cron is a really bad idea, imho. We do *not* want any automated commits going into the tree. If we

[HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
We know that HOT can cause line pointer bloat because of redirect dead line pointers. In the worst case there could be MaxHeapTuplesPerPage redirect-dead line pointers in a page. VACUUM can reclaim these line pointers and mark them ~LP_USED (what is now called LP_UNUSED). But we don't reclaim the

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Tom Lane
Pavan Deolasee [EMAIL PROTECTED] writes: How about passing a boolean to PageRepairFragmentation to command it to reclaim unused line pointers ? The difficulty with this is having to be 100% confident that noplace in the system tries to dereference a TID without checking that the line number

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
On 9/13/07, Tom Lane [EMAIL PROTECTED] wrote: The difficulty with this is having to be 100% confident that noplace in the system tries to dereference a TID without checking that the line number (offset) is within range. At one time that was demonstrably not so. I think we've cleaned up

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Zeugswetter Andreas ADI SD
CREATE TABLE test (a int, b char(200)); CREATE UNIQUE INDEX testindx ON test(a); INSERT INTO test VALUES (1, 'foo'); Now, if we repeatedly update the tuple so that each update is a COLD update, we would bloat the page with redirect-dead line pointers. Um, sorry for not understanding, but

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
On 9/13/07, Zeugswetter Andreas ADI SD [EMAIL PROTECTED] wrote: CREATE TABLE test (a int, b char(200)); CREATE UNIQUE INDEX testindx ON test(a); INSERT INTO test VALUES (1, 'foo'); Now, if we repeatedly update the tuple so that each update is a COLD update, we would bloat the page

Re: [HACKERS] Patch queue triage

2007-09-13 Thread Bruce Momjian
Pavan Deolasee wrote: On 9/13/07, Bruce Momjian [EMAIL PROTECTED] wrote: For those who have forgotten the progress we have made toward 8.3, here are the open patches we had for 8.3 as of May 1, 2006: You mean May 1, 2007 ;-) Yea, sorry. -- Bruce Momjian [EMAIL PROTECTED]

Re: [HACKERS] Patch queue triage

2007-09-13 Thread Bruce Momjian
Simon Riggs wrote: On Wed, 2007-09-12 at 17:47 -0400, Bruce Momjian wrote: For those who have forgotten the progress we have made toward 8.3, here are the open patches we had for 8.3 as of May 1, 2006: Could you please issue a list of open items for 8.3? I want to check whether you are

[HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Alvaro Herrera
Hi, Darcy Buskermolen noticed that when one has many databases, the autovac launcher starts eating too much CPU. I tried it here with 200 databases and indeed it does seem to eat its share. Even with the default naptime, which I wouldn't have thought that was too high (it does make the launcher

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Bruce Momjian
Josh, is any of this happening for 8.3? --- Josh Berkus wrote: All, I'm working on cleaning up postgresql.conf and pg_settings for the release. Attached is a sample WIP. It's not in patch form because I'm not done

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Momjian wrote: Josh, is any of this happening for 8.3? I would be willing to take a stab at this and submit by Monday. Joshua D. Drake --- Josh Berkus wrote:

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Zeugswetter Andreas ADI SD
The COLD updated (old) tuple would be pruned to dead line pointer once the tuple becomes DEAD. Normally that would let us reuse the tuple storage for other purposes. We do the same for DELETEd tuples. Oh, I thought only pruned tuples from HOT chains can produce a redirect dead line pointer.

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
On 9/13/07, Zeugswetter Andreas ADI SD [EMAIL PROTECTED] wrote: The COLD updated (old) tuple would be pruned to dead line pointer once the tuple becomes DEAD. Normally that would let us reuse the tuple storage for other purposes. We do the same for DELETEd tuples. Oh, I thought only

Re: [HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Alvaro Herrera
Alvaro Herrera wrote: Darcy Buskermolen noticed that when one has many databases, the autovac launcher starts eating too much CPU. I tried it here with 200 databases and indeed it does seem to eat its share. Even with the default naptime, which I wouldn't have thought that was too high

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Momjian wrote: Josh, is any of this happening for 8.3? Should I run with this or let it lay? --- Josh Berkus wrote: All, I'm working on cleaning up

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Josh Berkus
Bruce, Josh, is any of this happening for 8.3? Hmmm, just the format cleanup. I haven't heard any objections, but I haven't heard any comments on the underlying broken functionality either (like seq_scan_cost), which are beyond me to fix. Patch next week. -- Josh Berkus PostgreSQL @ Sun

[HACKERS] Problem

2007-09-13 Thread Pedro Belmino
Hello, I am a developer, I am working in the creation of hypothetical index in the data base postgresql 8.2.4. I have a problem, the data base init normally but when I try to connect to the database it shows the following error: TRAP: FailedAssertion(!(!isnull), File: relcache.c, Line: 959) . You

Re: [HACKERS] Problem

2007-09-13 Thread Heikki Linnakangas
Pedro Belmino wrote: Hello, I am a developer, I am working in the creation of hypothetical index in the data base postgresql 8.2.4. I have a problem, the data base init normally but when I try to connect to the database it shows the following error: TRAP: FailedAssertion(!(!isnull), File:

Re: [HACKERS] RETURNING and DO INSTEAD ... Intentional or not?

2007-09-13 Thread Josh Berkus
All, I'll note that we currently prevent adding RETURNING to a *conditional* DO INSTEAD rule. This means that if we have a conditional DO INSTEAD rule which inserts into a different table than the final unconditional rule, we'll be RETURNING wrong or empty values. Mind you, that's a pretty

[HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Tom Lane
AFAICS, the whole indcreatexid and validForTxn business is a waste of code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, surely any transactions that could see the broken HOT chains are gone. There might have been some reason for this contraption before we had plan

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: AFAICS, the whole indcreatexid and validForTxn business is a waste of code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, surely any transactions that could see the broken HOT chains are gone. There might have been some reason for this

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: AFAICS, the whole indcreatexid and validForTxn business is a waste of code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, surely any transactions that could see the broken HOT chains are gone. There might have been some reason for this

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Gregory Stark
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: AFAICS, the whole indcreatexid and validForTxn business is a waste of code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, surely any transactions that could see the broken HOT chains are gone.

Re: [HACKERS] plpgsql and qualified variable names

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Tom Lane wrote: I have just absorbed the significance of some code that has been in plpgsql since day one, but has

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: David Strong presented some excellent results of his SMP scalability testing at Ottawa in May.

[HACKERS] tsearch2 documentation done

2007-09-13 Thread Bruce Momjian
I am now satisified with the tsearch2 documentation in SGML and have marked the item as complete for 8.3. -- Bruce Momjian [EMAIL PROTECTED] http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: AFAICS, the whole indcreatexid and validForTxn business is a waste of code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, surely any transactions that could see the broken HOT chains are gone. In

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-09-13 Thread Andrew Hammond
On 9/13/07, Bruce Momjian [EMAIL PROTECTED] wrote: Alvaro Herrera wrote: Bruce Momjian wrote: Is this item closed? No, it isn't. Please add a TODO item about it: * Prevent long-lived temp tables from causing frozen-Xid advancement starvation Sorry, I don't understand this.

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-09-13 Thread Bruce Momjian
Alvaro Herrera wrote: Bruce Momjian wrote: Is this item closed? No, it isn't. Please add a TODO item about it: * Prevent long-lived temp tables from causing frozen-Xid advancement starvation Sorry, I don't understand this. Can you give me more text? Thanks. -- Bruce Momjian

Re: [HACKERS] terms for database replication: synchronous vs eager

2007-09-13 Thread Jan Wieck
On 9/7/2007 11:01 AM, Markus Schiltknecht wrote: Hi, I'm asking for advice and hints regarding terms in database replication, especially WRT Postgres-R. (Sorry for crossposting, but I fear not reaching enough people on the Postgres-R ML alone) I'm struggling on how to classify the

Re: [HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Darcy Buskermolen
On Thursday 13 September 2007 09:16:52 Alvaro Herrera wrote: Hi, Darcy Buskermolen noticed that when one has many databases, the autovac launcher starts eating too much CPU. Don't forget the memory leak as well. after 3 or 4 days of running I end up with a 2GB+ AVL.. I tried it here with

Re: [HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Alvaro Herrera
Darcy Buskermolen wrote: On Thursday 13 September 2007 09:16:52 Alvaro Herrera wrote: Hi, Darcy Buskermolen noticed that when one has many databases, the autovac launcher starts eating too much CPU. Don't forget the memory leak as well. after 3 or 4 days of running I end up with a

Re: [HACKERS] [GENERAL] ascii() for utf8

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Alvaro Herrera wrote: Decibel! wrote: Moving to -hackers. On Jul 27, 2007, at 1:22 PM, Stuart wrote: Does

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: Jignesh Shah's scalability testing on Solaris has revealed further tuning opportunities

Re: [HACKERS] clog_buffers to 64 in 8.3?

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Greg Smith wrote: On Thu, 2 Aug 2007, Tom Lane wrote: I find it entirely likely that simply changing the

Re: [HACKERS] What is happening on buildfarm member dugong

2007-09-13 Thread Tom Lane
Sergey E. Koposov [EMAIL PROTECTED] writes: It turned out that the offending assert is Assert(BgWriterShmem != NULL); in bgwriter.c:990 After commenting it out everything works. That's simply bizarre ... Also, I tried to add 'volatile' to the declaration of BgWriterShmem. After that the

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Alvaro Herrera
Bruce Momjian wrote: This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold I think the work on VIDs and latestCompletedXid make this completely obsolete. --- Simon

Re: [HACKERS] PGparam extension version 0.4

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Andrew Chernow wrote: Version 0.4 of libpq param put and PGresult get functions. Added support for inet and

Re: [HACKERS] [EMAIL PROTECTED]: Re: [GENERAL] array_to_set functions]

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Pavel Stehule wrote: 2007/8/15, Merlin Moncure [EMAIL PROTECTED]: On 8/14/07, Bruce Momjian [EMAIL PROTECTED]

Re: [HACKERS] What is happening on buildfarm member dugong

2007-09-13 Thread Sergey E. Koposov
On Tue, 11 Sep 2007, Tom Lane wrote: Well, the first thing I'd suggest is trying to localize which Assert makes it fail. From the bug's behavior I think it is highly probable that the problem is in fsync signalling, which puts it either in bgwriter.c or md.c. Try recompiling those modules

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Alvaro Herrera wrote: I think the work on VIDs and latestCompletedXid make this completely obsolete. Please confirm, all of Simon's issues? Not sure --- the area is certainly still worth looking at, but the recent patches have changed things enough

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Alvaro Herrera
Bruce Momjian wrote: Alvaro Herrera wrote: Bruce Momjian wrote: This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold I think the work on VIDs and latestCompletedXid make this completely obsolete. Please confirm, all of Simon's

Re: [HACKERS] Problem

2007-09-13 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: Pedro Belmino wrote: I am a developer, I am working in the creation of hypothetical index in the data base postgresql 8.2.4. I have a problem, the data base init normally but when I try to connect to the database it shows the following error: TRAP:

Re: [HACKERS] SQL feature requests

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Andrew Dunstan wrote: Chuck McDevitt wrote: PostgreSQL already has a huge amount of non-standard syntax and

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Bruce Momjian
Alvaro Herrera wrote: Bruce Momjian wrote: This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold I think the work on VIDs and latestCompletedXid make this completely obsolete. Please confirm, all of Simon's issues?

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Tom Lane
Zeugswetter Andreas ADI SD [EMAIL PROTECTED] writes: ...This looks like a problem, since we might end up with a page filled with LP_DEAD slots, that all have no visibility info and can thus not be cleaned by vacuum. No, it's the other way round: an LP_DEAD item pointer can *always* be cleaned

Re: [HACKERS] tsearch2 documentation done

2007-09-13 Thread Oleg Bartunov
On Thu, 13 Sep 2007, Bruce Momjian wrote: I am now satisified with the tsearch2 documentation in SGML and have marked the item as complete for 8.3. what's about examples ? They certainly need updating. For example,

Re: [HACKERS] tsearch2 documentation done

2007-09-13 Thread Tom Lane
Oleg Bartunov [EMAIL PROTECTED] writes: On Thu, 13 Sep 2007, Bruce Momjian wrote: I am now satisified with the tsearch2 documentation in SGML and have marked the item as complete for 8.3. what's about examples ? They certainly need updating. Bruce might be satisfied, but I'm not ... feel

Re: [HACKERS] RETURNING and DO INSTEAD ... Intentional or not?

2007-09-13 Thread Tom Lane
Josh Berkus [EMAIL PROTECTED] writes: FYI, after some tinkering around, I've found that RETURNING is 100% incompatible with any table which has conditional DO INSTEAD rules; there's just no way to make it work and return any intelligible data. This would be a completely corner case, except