Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Pavan Deolasee
On 9/14/07, Tom Lane <[EMAIL PROTECTED]> wrote: > > > I would still desperately like to get rid of indcreatexid, though, > because the patch's existing mechanism for clearing it is junk. > There's no guarantee that it will get cleared before it wraps around, > because the clearing is attached to va

Re: [HACKERS] tsearch2 documentation done

2007-09-13 Thread Tom Lane
Oleg Bartunov <[EMAIL PROTECTED]> writes: > On Thu, 13 Sep 2007, Bruce Momjian wrote: >> I am now satisified with the tsearch2 documentation in SGML and have >> marked the item as complete for 8.3. > what's about examples ? They certainly need updating. Bruce might be satisfied, but I'm not ... f

Re: [HACKERS] tsearch2 documentation done

2007-09-13 Thread Oleg Bartunov
On Thu, 13 Sep 2007, Bruce Momjian wrote: I am now satisified with the tsearch2 documentation in SGML and have marked the item as complete for 8.3. what's about examples ? They certainly need updating. For example, http://momjian.us/main/writings/pgsql/sgml/textsearch-rule-dictionary-example.

Re: [HACKERS] RETURNING and DO INSTEAD ... Intentional or not?

2007-09-13 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > FYI, after some tinkering around, I've found that RETURNING is 100% > incompatible with any table which has conditional DO INSTEAD rules; there's > just no way to make it work and return any intelligible data. This would be > a completely corner case, e

Re: [HACKERS] Problem

2007-09-13 Thread Tom Lane
"Heikki Linnakangas" <[EMAIL PROTECTED]> writes: > Pedro Belmino wrote: >> I am a developer, I am working in the creation of hypothetical index in the >> data base postgresql 8.2.4. I have a problem, the data base init normally >> but when I try to connect to the database it shows the following err

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Tom Lane
"Zeugswetter Andreas ADI SD" <[EMAIL PROTECTED]> writes: > ...This looks like a problem, since we might end up with a page filled with > LP_DEAD slots, that all have no visibility info and can thus not be > cleaned by vacuum. No, it's the other way round: an LP_DEAD item pointer can *always* be cl

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Alvaro Herrera
Bruce Momjian wrote: > Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > > This has been saved for the 8.4 release: > > > > > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold > > > > I think the work on VIDs and latestCompletedXid make this completely > > obsolete. > > Please con

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Alvaro Herrera wrote: >> I think the work on VIDs and latestCompletedXid make this completely >> obsolete. > Please confirm, all of Simon's issues? Not sure --- the area is certainly still worth looking at, but the recent patches have changed things eno

Re: [HACKERS] SQL feature requests

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Andrew Dunstan wrote: > > > Chuck McDevitt wrote: > > PostgreSQL already has a huge amount of > > "non-standard" sy

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Bruce Momjian
Alvaro Herrera wrote: > Bruce Momjian wrote: > > > > This has been saved for the 8.4 release: > > > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold > > I think the work on VIDs and latestCompletedXid make this completely > obsolete. Please confirm, all of Simon's issues? htt

Re: [HACKERS] PGparam extension version 0.4

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Andrew Chernow wrote: > Version 0.4 of libpq param put and PGresult get functions. > > Added support for inet and ci

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Alvaro Herrera
Bruce Momjian wrote: > > This has been saved for the 8.4 release: > > http://momjian.postgresql.org/cgi-bin/pgpatches_hold I think the work on VIDs and latestCompletedXid make this completely obsolete. > --- > > Simo

Re: [HACKERS] What is happening on buildfarm member dugong

2007-09-13 Thread Tom Lane
"Sergey E. Koposov" <[EMAIL PROTECTED]> writes: > It turned out that the offending assert is > Assert(BgWriterShmem != NULL); in bgwriter.c:990 > After commenting it out everything works. That's simply bizarre ... > Also, I tried to add 'volatile' to the declaration of BgWriterShmem. After > tha

Re: [HACKERS] [EMAIL PROTECTED]: Re: [GENERAL] array_to_set functions]

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Pavel Stehule wrote: > 2007/8/15, Merlin Moncure <[EMAIL PROTECTED]>: > > On 8/14/07, Bruce Momjian <[EMAIL PROTECTED

Re: [HACKERS] What is happening on buildfarm member dugong

2007-09-13 Thread Sergey E. Koposov
On Tue, 11 Sep 2007, Tom Lane wrote: Well, the first thing I'd suggest is trying to localize which Assert makes it fail. From the bug's behavior I think it is highly probable that the problem is in fsync signalling, which puts it either in bgwriter.c or md.c. Try recompiling those modules sepa

Re: [HACKERS] clog_buffers to 64 in 8.3?

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Greg Smith wrote: > On Thu, 2 Aug 2007, Tom Lane wrote: > > > I find it entirely likely that simply changing the [NU

Re: [HACKERS] [GENERAL] ascii() for utf8

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Alvaro Herrera wrote: > Decibel! wrote: > > Moving to -hackers. > > > > On Jul 27, 2007, at 1:22 PM, Stuart wrote: >

Re: [HACKERS] Reducing Transaction Start/End Contention

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: > Jignesh Shah's scalability testing on Solaris has revealed further > tuning opportunities surrou

Re: [HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Alvaro Herrera
Darcy Buskermolen wrote: > On Thursday 13 September 2007 09:16:52 Alvaro Herrera wrote: > > Hi, > > > > Darcy Buskermolen noticed that when one has many databases, the autovac > > launcher starts eating too much CPU. > > Don't forget the memory leak as well. after 3 or 4 days of running I end up

Re: [HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Darcy Buskermolen
On Thursday 13 September 2007 09:16:52 Alvaro Herrera wrote: > Hi, > > Darcy Buskermolen noticed that when one has many databases, the autovac > launcher starts eating too much CPU. Don't forget the memory leak as well. after 3 or 4 days of running I end up with a 2GB+ AVL.. > > I tried it here

Re: [HACKERS] terms for database replication: synchronous vs eager

2007-09-13 Thread Jan Wieck
On 9/7/2007 11:01 AM, Markus Schiltknecht wrote: Hi, I'm asking for advice and hints regarding terms in database replication, especially WRT Postgres-R. (Sorry for crossposting, but I fear not reaching enough people on the Postgres-R ML alone) I'm struggling on how to classify the Postgres-R

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-09-13 Thread Andrew Hammond
On 9/13/07, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > Alvaro Herrera wrote: > > Bruce Momjian wrote: > > > > > > Is this item closed? > > > > No, it isn't. Please add a TODO item about it: > > * Prevent long-lived temp tables from causing frozen-Xid advancement > >starvation > > Sorry, I d

Re: [HACKERS] [GENERAL] AutoVacuum Behaviour Question

2007-09-13 Thread Bruce Momjian
Alvaro Herrera wrote: > Bruce Momjian wrote: > > > > Is this item closed? > > No, it isn't. Please add a TODO item about it: > * Prevent long-lived temp tables from causing frozen-Xid advancement >starvation Sorry, I don't understand this. Can you give me more text? Thanks. -- Bruce

[HACKERS] tsearch2 documentation done

2007-09-13 Thread Bruce Momjian
I am now satisified with the tsearch2 documentation in SGML and have marked the item as complete for 8.3. -- Bruce Momjian <[EMAIL PROTECTED]> http://momjian.us EnterpriseDB http://www.enterprisedb.com + If your life is a hard drive, Christ can be yo

Re: [HACKERS] Straightforward changes for increased SMP scalability

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: > David Strong presented some excellent results of his SMP scalability > testing at Ottawa in May.

Re: [HACKERS] plpgsql and qualified variable names

2007-09-13 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Tom Lane wrote: > I have just absorbed the significance of some code that has been in > plpgsql since day one, but ha

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Tom Lane
Gregory Stark <[EMAIL PROTECTED]> writes: >> "Tom Lane" <[EMAIL PROTECTED]> writes: >>> AFAICS, the whole indcreatexid and validForTxn business is a waste of >>> code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, >>> surely any transactions that could see the broken HOT chains

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Gregory Stark
"Gregory Stark" <[EMAIL PROTECTED]> writes: > "Tom Lane" <[EMAIL PROTECTED]> writes: > >> AFAICS, the whole indcreatexid and validForTxn business is a waste of >> code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, >> surely any transactions that could see the broken HOT chain

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > AFAICS, the whole indcreatexid and validForTxn business is a waste of > code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, > surely any transactions that could see the broken HOT chains are gone. > There might have been some reason fo

Re: [HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > AFAICS, the whole indcreatexid and validForTxn business is a waste of > code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, > surely any transactions that could see the broken HOT chains are gone. > There might have been some reason fo

[HACKERS] Another HOT thought: why do we need indcreatexid at all?

2007-09-13 Thread Tom Lane
AFAICS, the whole indcreatexid and validForTxn business is a waste of code. By the time CREATE INDEX CONCURRENTLY is ready to set indisvalid, surely any transactions that could see the broken HOT chains are gone. There might have been some reason for this contraption before we had plan invalidatio

Re: [HACKERS] RETURNING and DO INSTEAD ... Intentional or not?

2007-09-13 Thread Josh Berkus
All, > I'll note that we currently prevent adding RETURNING to a *conditional* DO > INSTEAD rule. This means that if we have a conditional DO INSTEAD rule > which inserts into a different table than the final unconditional rule, > we'll be RETURNING wrong or empty values. Mind you, that's a pret

Re: [HACKERS] Problem

2007-09-13 Thread Heikki Linnakangas
Pedro Belmino wrote: > Hello, > I am a developer, I am working in the creation of hypothetical index in the > data base postgresql 8.2.4. I have a problem, the data base init normally > but when I try to connect to the database it shows the following error: > TRAP: FailedAssertion("!(!isnull)", Fil

[HACKERS] Problem

2007-09-13 Thread Pedro Belmino
Hello, I am a developer, I am working in the creation of hypothetical index in the data base postgresql 8.2.4. I have a problem, the data base init normally but when I try to connect to the database it shows the following error: TRAP: FailedAssertion("!(!isnull)", File: "relcache.c", Line: 959) . Y

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Josh Berkus
Bruce, > Josh, is any of this happening for 8.3? Hmmm, just the format cleanup. I haven't heard any objections, but I haven't heard any comments on the underlying broken functionality either (like seq_scan_cost), which are beyond me to fix. Patch next week. -- Josh Berkus PostgreSQL @ Sun S

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Momjian wrote: > Josh, is any of this happening for 8.3? Should I run with this or let it lay? > > --- > > Josh Berkus wrote: >> All, >> >> I'm working on cleaning up pos

Re: [HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Alvaro Herrera
Alvaro Herrera wrote: > Darcy Buskermolen noticed that when one has many databases, the autovac > launcher starts eating too much CPU. > > I tried it here with 200 databases and indeed it does seem to eat its > share. Even with the default naptime, which I wouldn't have thought > that was too hi

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
On 9/13/07, Zeugswetter Andreas ADI SD <[EMAIL PROTECTED]> wrote: > > > > The COLD updated (old) tuple would be pruned to dead line pointer > > once the tuple becomes DEAD. Normally that would let us reuse the > > tuple storage for other purposes. We do the same for DELETEd tuples. > > Oh, I though

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Zeugswetter Andreas ADI SD
> The COLD updated (old) tuple would be pruned to dead line pointer > once the tuple becomes DEAD. Normally that would let us reuse the > tuple storage for other purposes. We do the same for DELETEd tuples. Oh, I thought only pruned tuples from HOT chains can produce a "redirect dead" line point

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Momjian wrote: > Josh, is any of this happening for 8.3? I would be willing to take a stab at this and submit by Monday. Joshua D. Drake > > --- > > Josh Berkus wrote:

Re: [HACKERS] Postgresql.conf cleanup

2007-09-13 Thread Bruce Momjian
Josh, is any of this happening for 8.3? --- Josh Berkus wrote: > All, > > I'm working on cleaning up postgresql.conf and pg_settings for the > release. Attached is a sample WIP. It's not in patch form because I'm > not

[HACKERS] autovacuum launcher eating too much CPU

2007-09-13 Thread Alvaro Herrera
Hi, Darcy Buskermolen noticed that when one has many databases, the autovac launcher starts eating too much CPU. I tried it here with 200 databases and indeed it does seem to eat its share. Even with the default naptime, which I wouldn't have thought that was too high (it does make the launcher

Re: [HACKERS] Patch queue triage

2007-09-13 Thread Bruce Momjian
Simon Riggs wrote: > On Wed, 2007-09-12 at 17:47 -0400, Bruce Momjian wrote: > > For those who have forgotten the progress we have made toward 8.3, here > > are the open patches we had for 8.3 as of May 1, 2006: > > Could you please issue a list of open items for 8.3? > > I want to check whether

Re: [HACKERS] Patch queue triage

2007-09-13 Thread Bruce Momjian
Pavan Deolasee wrote: > On 9/13/07, Bruce Momjian <[EMAIL PROTECTED]> wrote: > > > > > > For those who have forgotten the progress we have made toward 8.3, here > > are the open patches we had for 8.3 as of May 1, 2006: > > > > > You mean May 1, 2007 ;-) Yea, sorry. -- Bruce Momjian <[EMAIL P

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
On 9/13/07, Zeugswetter Andreas ADI SD <[EMAIL PROTECTED]> wrote: > > > > CREATE TABLE test (a int, b char(200)); > > CREATE UNIQUE INDEX testindx ON test(a); > > INSERT INTO test VALUES (1, 'foo'); > > > > Now, if we repeatedly update the tuple so that each update is a > > COLD update, we would bl

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Zeugswetter Andreas ADI SD
> CREATE TABLE test (a int, b char(200)); > CREATE UNIQUE INDEX testindx ON test(a); > INSERT INTO test VALUES (1, 'foo'); > > Now, if we repeatedly update the tuple so that each update is a > COLD update, we would bloat the page with redirect-dead line pointers. Um, sorry for not understanding,

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
On 9/13/07, Tom Lane <[EMAIL PROTECTED]> wrote: > > > > The difficulty with this is having to be 100% confident that noplace in > the system tries to dereference a TID without checking that the line > number (offset) is within range. At one time that was demonstrably > not so. I think we've clean

Re: [HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Tom Lane
"Pavan Deolasee" <[EMAIL PROTECTED]> writes: > How about passing a boolean to PageRepairFragmentation to > command it to reclaim unused line pointers ? The difficulty with this is having to be 100% confident that noplace in the system tries to dereference a TID without checking that the line numbe

[HACKERS] HOT line pointer bloat and PageRepairFragmentation

2007-09-13 Thread Pavan Deolasee
We know that HOT can cause line pointer bloat because of redirect dead line pointers. In the worst case there could be MaxHeapTuplesPerPage redirect-dead line pointers in a page. VACUUM can reclaim these line pointers and mark them ~LP_USED (what is now called LP_UNUSED). But we don't reclaim the

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Guillaume Lelarge
Magnus Hagander a écrit : > On Thu, Sep 13, 2007 at 12:12:28AM +0200, Guillaume Lelarge wrote: >> [...] >> It seems pretty straightforward now. Perhaps it can be used with cron. > > No. Doing that with cron is a really bad idea, imho. We do *not* want any > automated commits going into the tree. I

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Magnus Hagander
On Thu, Sep 13, 2007 at 12:12:28AM +0200, Guillaume Lelarge wrote: > Tom Lane a écrit : > > Dave Page <[EMAIL PROTECTED]> writes: > >> Tom Lane wrote: > >>> Peter usually does it --- in theory any committer could, but he actually > >>> knows what to do and the rest of us would have to study ;-) > >

Re: [HACKERS] Limitation on number of positions (tsearch)

2007-09-13 Thread Teodor Sigaev
Why is there a limitation of 256 positions per lexeme in a tsvector? There doesn't seem to be a technical reason for that. WordEntryPosVector uses a uint16 to store the number of positions, so it go up to 65535. For two reasons: - Ranking might become very slow if number of position is big - Fro

[HACKERS] Limitation on number of positions (tsearch)

2007-09-13 Thread Heikki Linnakangas
Why is there a limitation of 256 positions per lexeme in a tsvector? There doesn't seem to be a technical reason for that. WordEntryPosVector uses a uint16 to store the number of positions, so it go up to 65535. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com ---

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Simon Riggs
On Wed, 2007-09-12 at 10:48 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > The following bug fix has not yet been applied to CVS > > http://archives.postgresql.org/pgsql-patches/2007-06/msg00100.php > > Frankly, this looks much more like it creates a bug than fixes one. > I h

Re: [HACKERS] Patch queue triage

2007-09-13 Thread Simon Riggs
On Wed, 2007-09-12 at 17:47 -0400, Bruce Momjian wrote: > For those who have forgotten the progress we have made toward 8.3, here > are the open patches we had for 8.3 as of May 1, 2006: Could you please issue a list of open items for 8.3? I want to check whether you are waiting on me for anythin

Re: [HACKERS] Preparation for PostgreSQL releases 8.2.5, 8.1.10, 8.0.14, 7.4.18, 7.3.20

2007-09-13 Thread Simon Riggs
On Wed, 2007-09-12 at 10:48 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > The following bug fix has not yet been applied to CVS > > http://archives.postgresql.org/pgsql-patches/2007-06/msg00100.php > > Frankly, this looks much more like it creates a bug than fixes one. > I h