Re: [HACKERS] Index Tuple Compression Approach?

2007-08-16 Thread Heikki Linnakangas
Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: That general approach of storing a common part leading part just once is called prefix compression. Yeah, it helps a lot on long text fields. Tree structures like file paths in particular. You kind of want to do avoid both

Re: [HACKERS] Index Tuple Compression Approach?

2007-08-16 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: That general approach of storing a common part leading part just once is called prefix compression. Yeah, it helps a lot on long text fields. Tree structures like file paths in

[HACKERS] build farm failures

2007-08-16 Thread Michael Meskes
Hi, we have two build farm members failing to make since I committed teh ecpg changes: echidna and herring. It looks like they are still using an old preproc.c although they checked out the new preproc.y. I have no idea how this is supposed to work so could someone please enlighten me? Michael

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Michael Paesold
Bruce Momjian wrote: Uh, how are we going to prevent the auto-casting to tsvector from using the default GUC config, e.g. CREATE INDEX i ON x USING GIN(col)? This is where I started to see the need for education and error-prone nature of the default GUC just wasn't worth having it, though I

Re: [HACKERS] build farm failures

2007-08-16 Thread Andrew Dunstan
Michael Meskes wrote: Hi, we have two build farm members failing to make since I committed teh ecpg changes: echidna and herring. It looks like they are still using an old preproc.c although they checked out the new preproc.y. I have no idea how this is supposed to work so could someone

Re: [HACKERS] XID wraparound and busy databases

2007-08-16 Thread Simon Riggs
On Wed, 2007-08-15 at 12:49 -0400, Tom Lane wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: Maybe we can do something to reduce the xid consumption? For example, reuse xids for read-only queries. Hmm, that's an idea. That would be a very useful idea for additional scalability and it

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Tom Lane
Michael Paesold [EMAIL PROTECTED] writes: Teodor Sigaev wrote: 2) added operator class for text and varchar CREATE INDEX idxname ON tblname USING GIN ( textcolumn ); So just remove the operator class or don't specify it as default operator class for GIN, and the thing is gone. Perhaps there

Re: [HACKERS] XID wraparound and busy databases

2007-08-16 Thread Florian G. Pflug
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Is enlarging the xid field something we should consider for 8.4? No. We just got the tuple header down to 24 bytes, we are not going to give that back and then some. If you are processing 6K transactions per second, you can afford to

Re: [HACKERS] Index Tuple Compression Approach?

2007-08-16 Thread Chris Browne
[EMAIL PROTECTED] (Gregory Stark) writes: Heikki Linnakangas [EMAIL PROTECTED] writes: Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: That general approach of storing a common part leading part just once is called prefix compression. Yeah, it helps a lot on long text

[HACKERS] Deferred Constraints

2007-08-16 Thread Rodrigo Hjort
Dear Hackers, Could you point me the first version PostgreSQL started to support Deferred Constraints (ie: DEFERRABLE keyword on foreign keys creation)? I guess it is earlier than 7.0, right? -- Regards, Rodrigo Hjort http://icewall.org/~hjort ---(end of

Re: [HACKERS] Deferred Constraints

2007-08-16 Thread Decibel!
On Thu, Aug 16, 2007 at 12:45:23PM -0300, Rodrigo Hjort wrote: Dear Hackers, Could you point me the first version PostgreSQL started to support Deferred Constraints (ie: DEFERRABLE keyword on foreign keys creation)? I guess it is earlier than 7.0, right? You could look through the release

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Yeah, unfortunately we overlooked the implications of the conversion to tsvector being environment-dependent. Those opclasses will have to go away again. AFAICS the only safe way to build an index directly on a text column is CREATE INDEX idxname ON

[HACKERS] cvsweb busted (was Re: [COMMITTERS] pgsql: Repair problems occurring when multiple RI updates have to be)

2007-08-16 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: There seems to be something wrong with this commit notification. This file pgsql/src/test/regress/expected: foreign_key.out (r1.44 - r1.45) (http://developer.postgresql.org/cvsweb.cgi/pgsql/src/test/regress/expected/foreign_key.out?r1=1.44r2=1.45)

Re: [HACKERS] build farm failures

2007-08-16 Thread Andrew Dunstan
Darcy Buskermolen wrote: This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall doing, however it's possible. though this does make me ask why

Re: [HACKERS] build farm failures

2007-08-16 Thread Darcy Buskermolen
On Thursday 16 August 2007 04:29:41 Andrew Dunstan wrote: Michael Meskes wrote: Hi, we have two build farm members failing to make since I committed teh ecpg changes: echidna and herring. It looks like they are still using an old preproc.c although they checked out the new

[HACKERS] Re: cvsweb busted (was Re: [COMMITTERS] pgsql: Repair problems occurring when multiple RI updates have to be)

2007-08-16 Thread Magnus Hagander
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: There seems to be something wrong with this commit notification. This file pgsql/src/test/regress/expected: foreign_key.out (r1.44 - r1.45)

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Is there a null configuration which could be the default for the casts? If we did that then there would be no point in having a GUC variable, because it wouldn't control anything. While that is certainly an alternative solution, I think it's riskier than

Re: [HACKERS] build farm failures

2007-08-16 Thread Alvaro Herrera
Andrew Dunstan wrote: Darcy Buskermolen wrote: This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall doing, however it's possible. though

Re: [HACKERS] build farm failures

2007-08-16 Thread Andrew Dunstan
Alvaro Herrera wrote: Andrew Dunstan wrote: Darcy Buskermolen wrote: This sort of thing is usually a symptom of somebody having run a build in the repo directly, a thing that buildfarm owners have been repeatedly advised not to do. This is something I do not recall

Re: [HACKERS] build farm failures

2007-08-16 Thread Michael Meskes
On Thu, Aug 16, 2007 at 08:24:14AM -0700, Darcy Buskermolen wrote: This is something I do not recall doing, however it's possible. though this does make me ask why are the build dependencies in the Makefile are not properly setup to tell that the .y needs to be rebuilt (which I would assume

Re: [HACKERS] build farm failures

2007-08-16 Thread Andrew Dunstan
Michael Meskes wrote: The backend has: $(srcdir)/gram.c: $(srcdir)/parse.h ; $(srcdir)/parse.h: gram.y So except for the different naming it's the same. However, we haven't had that problem with the backend so far, or did we? What do I fail to see? We have had problems in the past.

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Josh Berkus
All, First off, I'll assert that backup/restore is a serious issue and while the folks who want Tsearch in core now are dismissing it, we'll be fielding the complaints later. Any solution which involves setting a GUC at restore time *which could vary per table or even column* isn't

[HACKERS] building gist index on int[] field is terrible slow. Is it bug?

2007-08-16 Thread Pavel Stehule
Hello I tested contrib package intarray and gist support from this package. I was surpirised. Index building needed lot of time. 10K fields .. 106sec 20K f ..265 sec 30Kf .. 450 sec 50Kf .. 1283sec building gin index for 50K fields needed 0.5sec Regards Pavel Stehule

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Oleg Bartunov
On Thu, 16 Aug 2007, Josh Berkus wrote: All, First off, I'll assert that backup/restore is a serious issue and while the folks who want Tsearch in core now are dismissing it, we'll be fielding the complaints later. Any solution which involves setting a GUC at restore time *which could vary

Re: [HACKERS] building gist index on int[] field is terrible slow. Is it bug?

2007-08-16 Thread Oleg Bartunov
On Thu, 16 Aug 2007, Pavel Stehule wrote: Hello I tested contrib package intarray and gist support from this package. I was surpirised. Index building needed lot of time. 10K fields .. 106sec 20K f ..265 sec 30Kf .. 450 sec 50Kf .. 1283sec building gin index for 50K

Re: [HACKERS] XID wraparound and busy databases

2007-08-16 Thread Bruce Momjian
Added to TODO: * Reduce XID consumption of read-only queries http://archives.postgresql.org/pgsql-hackers/2007-08/msg00516.php --- Florian G. Pflug wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes:

Re: [HACKERS] Another idea for index-only scans

2007-08-16 Thread Jeff Davis
On Wed, 2007-08-15 at 11:54 -0400, Bruce Momjian wrote: I have added another idea for index-only scans to the TODO list: A third idea would be for a heap scan to check if all rows are visible and if so set a per-table flag which can be checked by index scans. Any change to the table

Re: [HACKERS] tsearch2 in PostgreSQL 8.3?

2007-08-16 Thread Bruce Momjian
Josh Berkus wrote: All, First off, I'll assert that backup/restore is a serious issue and while the folks who want Tsearch in core now are dismissing it, we'll be fielding the complaints later. Any solution which involves setting a GUC at restore time *which could vary per table or even

Re: [HACKERS] Another idea for index-only scans

2007-08-16 Thread Alvaro Herrera
Jeff Davis wrote: On Wed, 2007-08-15 at 11:54 -0400, Bruce Momjian wrote: I have added another idea for index-only scans to the TODO list: A third idea would be for a heap scan to check if all rows are visible and if so set a per-table flag which can be checked by index scans.

Re: [HACKERS] build farm failures

2007-08-16 Thread Alvaro Herrera
Andrew Dunstan wrote: Michael Meskes wrote: The backend has: $(srcdir)/gram.c: $(srcdir)/parse.h ; $(srcdir)/parse.h: gram.y So except for the different naming it's the same. However, we haven't had that problem with the backend so far, or did we? What do I fail to see? We have had

Re: [HACKERS] Another idea for index-only scans

2007-08-16 Thread Decibel!
On Thu, Aug 16, 2007 at 04:06:35PM -0400, Alvaro Herrera wrote: Jeff Davis wrote: On Wed, 2007-08-15 at 11:54 -0400, Bruce Momjian wrote: I have added another idea for index-only scans to the TODO list: A third idea would be for a heap scan to check if all rows are visible

Re: [HACKERS] build farm failures

2007-08-16 Thread Andrew Dunstan
Alvaro Herrera wrote: How do you create the copy of the repo to build? One idea would be to explicitely skip files that appear on .cvsignore (and maybe croak about them). We are supposed to croak - see

Re: [HACKERS] Re: cvsweb busted (was Re: [COMMITTERS] pgsql: Repair problems occurring when multiple RI updates have to be)

2007-08-16 Thread Marc G. Fournier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, August 16, 2007 18:29:22 +0200 Magnus Hagander [EMAIL PROTECTED] wrote: Marc - how much work to update that one on the old server? I know you have a new one coming up, but as a stop-gap fix on the old one? The new one is just

Re: [HACKERS] building gist index on int[] field is terrible slow. Is it bug?

2007-08-16 Thread Pavel Stehule
2007/8/16, Oleg Bartunov [EMAIL PROTECTED]: On Thu, 16 Aug 2007, Pavel Stehule wrote: Hello I tested contrib package intarray and gist support from this package. I was surpirised. Index building needed lot of time. 10K fields .. 106sec 20K f ..265 sec 30Kf .. 450

Re: [HACKERS] XID wraparound and busy databases

2007-08-16 Thread Paul Lindner
On Thu, Aug 16, 2007 at 04:13:55PM +0200, Florian G. Pflug wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Is enlarging the xid field something we should consider for 8.4? No. We just got the tuple header down to 24 bytes, we are not going to give that back and then some.

Re: [HACKERS] Re: cvsweb busted (was Re: [COMMITTERS] pgsql: Repair problems occurring when multiple RI updates have to be)

2007-08-16 Thread Magnus Hagander
Marc G. Fournier wrote: Marc - how much work to update that one on the old server? I know you have a new one coming up, but as a stop-gap fix on the old one? The new one is just waiting on your time scheduale ... its there ready for you though :) No, that's still waiting on your

Re: [HACKERS] Another idea for index-only scans

2007-08-16 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: On Wed, 2007-08-15 at 11:54 -0400, Bruce Momjian wrote: A third idea would be for a heap scan to check if all rows are visible and if so set a per-table flag which can be checked by index scans. I think it's too coarse-grained to be really useful. If

Re: [HACKERS] Re: cvsweb busted (was Re: [COMMITTERS] pgsql: Repair problems occurring when multiple RI updates have to be)

2007-08-16 Thread Marc G. Fournier
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 - --On Thursday, August 16, 2007 23:16:09 +0200 Magnus Hagander [EMAIL PROTECTED] wrote: But my question still stands - how much work to stop-gap fix it on the old one? rsync should be upgraded now ... - Marc G. Fournier

[HACKERS] tsearch patch and namespace pollution

2007-08-16 Thread Tom Lane
I find the following additions to pg_proc in the current tsearch2 patch: proc | prorettype --+ pg_ts_parser_is_visible(oid) | boolean pg_ts_dict_is_visible(oid) | boolean

[HACKERS] tsvector_update_trigger() is utterly insecure

2007-08-16 Thread Tom Lane
We can't put tsvector_update_trigger() into core in anything like its current form. As is, it will take an unqualified function name, look it up, and call it. The prospects for subversion by search path manipulation are obvious, and even if you aren't concerned about malicious attacks, the

Re: [HACKERS] GIT patch

2007-08-16 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: Tom Lane wrote: So instead of pressing to try to get something into 8.3, I would rather we stand back and think about it some more. I understand why you are saying hold for 8.4, but this issue came up in the middle of the 8.3

Re: [HACKERS] tsvector_update_trigger() is utterly insecure

2007-08-16 Thread Bruce Momjian
Tom Lane wrote: We can't put tsvector_update_trigger() into core in anything like its current form. As is, it will take an unqualified function name, look it up, and call it. The prospects for subversion by search path manipulation are obvious, and even if you aren't concerned about

Re: [HACKERS] GIT patch

2007-08-16 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian [EMAIL PROTECTED] writes: I somehow feel that if HOT wasn't being considered for 8.3 we might have gotten GIT, but with limited resources I think there was more focus on HOT, perhaps rightly so.

Re: [HACKERS] tsearch patch and namespace pollution

2007-08-16 Thread Bruce Momjian
Tom Lane wrote: I find the following additions to pg_proc in the current tsearch2 patch: It seems a lot of these are useless and just bloat. I will mark a few: proc | prorettype --+

Re: [HACKERS] GIT patch

2007-08-16 Thread Bruce Momjian
Joshua D. Drake wrote: These patches will be held for 8.4: o Grouped Index Tuples (GIT) o Bitmap scan changes o Stream bitmaps (API change for Group Index Tuples) o Maintaining cluster order on insert I believe Heikki is in agreement on this. That is

Re: [HACKERS] GIT patch

2007-08-16 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Bruce Momjian wrote: Joshua D. Drake wrote: These patches will be held for 8.4: o Grouped Index Tuples (GIT) o Bitmap scan changes o Stream bitmaps (API change for Group Index Tuples) o Maintaining cluster order on insert

[HACKERS] text search vs schemas

2007-08-16 Thread Tom Lane
I wrote: We can't put tsvector_update_trigger() into core in anything like its current form. As is, it will take an unqualified function name, look it up, and call it. The prospects for subversion by search path manipulation are obvious, and even if you aren't concerned about malicious

Re: [HACKERS] text search vs schemas

2007-08-16 Thread Trevor Talbot
On 8/16/07, Tom Lane [EMAIL PROTECTED] wrote: Actually ... I'm suddenly not happy about the choice to put text search configurations etc. into schemas at all. We've been sitting here and assuming that to_tsvector('english', my_text_col) has a well defined meaning --- but as the patch stands,

Re: [HACKERS] text search vs schemas

2007-08-16 Thread Tom Lane
Trevor Talbot [EMAIL PROTECTED] writes: Wouldn't treating them as actual objects remove this whole issue? Uh, no. Function names for example are subject to search-path confusion. regards, tom lane ---(end of broadcast)---

Re: [HACKERS] text search vs schemas

2007-08-16 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Trevor Talbot [EMAIL PROTECTED] writes: Wouldn't treating them as actual objects remove this whole issue? Uh, no. Function names for example are subject to search-path confusion. Wait, are they? They are in PL languages but only because most languages

Re: [HACKERS] text search vs schemas

2007-08-16 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Uh, no. Function names for example are subject to search-path confusion. Wait, are they? They are in PL languages but only because most languages store their source code as text just as is happening here. Hmmm ...

Re: [HACKERS] text search vs schemas

2007-08-16 Thread Oleg Bartunov
On Thu, 16 Aug 2007, Tom Lane wrote: I wrote: We can't put tsvector_update_trigger() into core in anything like its current form. As is, it will take an unqualified function name, look it up, and call it. The prospects for subversion by search path manipulation are obvious, and even if you