Re: [HACKERS] strange 'vacuum verbose analyze' behaviour

2005-01-29 Thread Oleg Bartunov
Tom, After almost 20 hours running vacuum I see postmaster grew a little bit: PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 20458 postgres 15 0 2136m 553m 204m D 33.2 54.7 198:18.36 postmaster It's strange that I see no output since starting vacuumdb -v -z -f wsdb

Re: [HACKERS] some linker troubles with rc5 on sun studio 9 ...

2005-01-29 Thread Hans-Jürgen Schönig
Tom Lane wrote: =?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= [EMAIL PROTECTED] writes: tuptoaster.c, line 966: member can not have variably modified type: data We've seen that before. Apparently there are some versions of Sun's compiler that are too stupid to reduce this constant expression to a

Re: [HACKERS] Group-count estimation statistics

2005-01-29 Thread Mischa
From: Sailesh Krishnamurthy [EMAIL PROTECTED] Tom == Tom Lane [EMAIL PROTECTED] writes: Tom The only real solution, of course, is to acquire cross-column Tom statistics, but I don't see that happening in the near Tom future. Another approach is a hybrid hashing scheme where

[HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Victor Y. Yegorov
Hello. I'd like to implement bitmap indexes and want your comments. Here is an essence of what I've found regarding bitmaps for the last month. Consider the following table So, the bitmap for attribute A will be the with 1 attribute A(int2): following: # | A Val |

Re: [HACKERS] [pgsql-hackers] Allow GRANT/REVOKE permissions to be applied to all schema

2005-01-29 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: Or just make the user enter two commands for this case. Aside from syntactic simplicity, that might be a good idea anyway. The NEW TABLES case is *fundamentally* different from every other form of GRANT, in that it causes future actions. So it might be

[HACKERS] Allowing VACUUM to time out when waiting for locks?

2005-01-29 Thread Philip Warner
We have a frequently updated (peak 5/sec) table with about 1000 rows. We run VACCUM FULL on this table every 5 minutes. The regular updates are not long in duration, and the vacuum is fast, so they do not produce noticeable delays. When we run a pg_dump on the database: - the dump takes a long

Re: [HACKERS] Allowing VACUUM to time out when waiting for locks?

2005-01-29 Thread Bruno Wolff III
On Sun, Jan 30, 2005 at 01:23:11 +1100, Philip Warner [EMAIL PROTECTED] wrote: We have a frequently updated (peak 5/sec) table with about 1000 rows. We run VACCUM FULL on this table every 5 minutes. Why not just use plain VACUUM? The table will reach a steady state size. You should only

Re: [HACKERS] [pgsql-hackers] Patent issues and 8.1

2005-01-29 Thread Robert Treat
On Friday 28 January 2005 12:36, Josh Berkus wrote: Robert, Read the law... willful vs. unknown infringement are two different things. We're not infringing anything, yet. That's a *pending* patent. *sigh* Thats understood. But you were using the counter-argument that we might be

Re: [HACKERS] Allowing VACUUM to time out when waiting for locks?

2005-01-29 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: We have a frequently updated (peak 5/sec) table with about 1000 rows. We run VACCUM FULL on this table every 5 minutes. I agree with Bruno's comment that you shouldn't be doing that in the first place. Plain vacuum (perhaps executed even more often,

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Tom Lane
Victor Y. Yegorov [EMAIL PROTECTED] writes: I remember, Tom Lane suggested on-disk bitmaps I have suggested no such thing, and in fact believe that the sort of index structure you are proposing would be of very little use. What I've been hoping to look into is *in memory* bitmaps used as an

Re: [HACKERS] [pgsql-hackers] Patent issues and 8.1

2005-01-29 Thread Tom Lane
Robert Treat [EMAIL PROTECTED] writes: I'm not mischarecterizing, I just feel that putting out an lru based 8.0.x release is such a bad idea that I'd rather do (1) than gamble on (2). I don't understand why you think it's such a bad idea. We do have the problem of getting adequate testing,

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Victor Y. Yegorov
* Tom Lane [EMAIL PROTECTED] [29.01.2005 18:24]: Victor Y. Yegorov [EMAIL PROTECTED] writes: I remember, Tom Lane suggested on-disk bitmaps I have suggested no such thing, and in fact believe that the sort of index structure you are proposing would be of very little use. Why? I thought

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Victor Y. Yegorov
* Pawe Niewiadomski [EMAIL PROTECTED] [29.01.2005 17:45]: I'd like to implement bitmap indexes and want your comments. Here is an essence of what I've found regarding bitmaps for the last month. Do you think it would be possible to work on it as a team? Yes, why not. But everything

[HACKERS] Security bugs in tsearch2

2005-01-29 Thread Tom Lane
I just noticed that all of tsearch2's dict init routines are declared like this: CREATE FUNCTION dex_init(text) returns internal as 'MODULE_PATHNAME' language 'C'; This is really unacceptable, because it breaks the type safety of the internal pseudotype. I quote from

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Mike Rylander
On Sat, 29 Jan 2005 19:41:20 +0200, Victor Y. Yegorov [EMAIL PROTECTED] wrote: * Pawe Niewiadomski [EMAIL PROTECTED] [29.01.2005 17:45]: I'd like to implement bitmap indexes and want your comments. Here is an essence of what I've found regarding bitmaps for the last month. Do you think

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Dawid Kuroczko
On Sat, 29 Jan 2005 18:46:44 +, Mike Rylander [EMAIL PROTECTED] wrote: As a side note, wouldn't the in-memory bitmaps pretty much kill the need for multicolumn indexes? It seems that they would be able to join index scans on the same table, and then there would be no need for industrial

Re: [HACKERS] [pgsql-hackers] Patent issues and 8.1

2005-01-29 Thread Robert Treat
On Saturday 29 January 2005 11:33, Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: I'm not mischarecterizing, I just feel that putting out an lru based 8.0.x release is such a bad idea that I'd rather do (1) than gamble on (2). I don't understand why you think it's such a bad idea.

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Dawid Kuroczko
On Sat, 29 Jan 2005 21:54:39 +0200, Victor Yegorov [EMAIL PROTECTED] wrote: * Dawid Kuroczko [EMAIL PROTECTED] [29.01.2005 21:25]: With in-memory bitmap, the search would start with index a, all matching rows would form the bitmap; then the second search would go through b index, forming

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Victor Yegorov
* Dawid Kuroczko [EMAIL PROTECTED] [29.01.2005 21:25]: With in-memory bitmap, the search would start with index a, all matching rows would form the bitmap; then the second search would go through b index, forming another bitmap. Which then would be ANDed with previous bitmap. Not only

Re: [HACKERS] [pgsql-hackers] Allow GRANT/REVOKE permissions to be applied to all schema

2005-01-29 Thread Robert Treat
On Saturday 29 January 2005 09:14, Stephen Frost wrote: * Tom Lane ([EMAIL PROTECTED]) wrote: Or just make the user enter two commands for this case. Aside from syntactic simplicity, that might be a good idea anyway. The NEW TABLES case is *fundamentally* different from every other form

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Tom Lane
Mike Rylander [EMAIL PROTECTED] writes: As a side note, wouldn't the in-memory bitmaps pretty much kill the need for multicolumn indexes? It seems that they would be able to join index scans on the same table, and then there would be no need for industrial strength cross-column correlation

Re: [HACKERS] IBM patent

2005-01-29 Thread Christopher Browne
Oops! [EMAIL PROTECTED] (Tommi Maekitalo) was seen spray-painting on a wall: Hi, I just read about this IBM-patent-issue at www.heise.de. IBM grants this patens to all projects, which follow one of the licenses, which are approved by the open-source-initiative. And the BSD-license is as far

Re: [HACKERS] IBM patent

2005-01-29 Thread Marc G. Fournier
On Wed, 26 Jan 2005, Christopher Browne wrote: Actually, the latter isn't so. If Mammoth or Pervasive or such release their own release of PostgreSQL, nothing has historically mandated that they make that release available under the BSD license. Presumably acceptance of the patent would change

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Neil Conway
Mike Rylander wrote: For on-disk bitmap indexes, yes. I don't see any reason this couldn't be done with GiST It might be possible to do it with GiST, but GiST is designed for implementing tree-structured indexes; I don't think it's the right tool for the job. -Neil

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Mike Rylander
On Sun, 30 Jan 2005 11:07:59 +1100, Neil Conway [EMAIL PROTECTED] wrote: Mike Rylander wrote: For on-disk bitmap indexes, yes. I don't see any reason this couldn't be done with GiST It might be possible to do it with GiST, but GiST is designed for implementing tree-structured indexes; I

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Neil Conway
Mike Rylander wrote: For the initial example where the index is implemented as a set of unique keys from the table and a bitmap for each key this would look a unique index, but with an extra datum at at each index node to hold the bitmap for that key. If implemented that way an augmented B-Tree

Re: [HACKERS] Implementing Bitmap Indexes

2005-01-29 Thread Mike Rylander
On Sun, 30 Jan 2005 12:15:20 +1100, Neil Conway [EMAIL PROTECTED] wrote: It might _work_, I just don't see the point. Given an attribute of a heap relation that has N distinct values and T tuples, you need to store - N bitmaps, each of T bits (before compression) - T ctids - a way to map

[HACKERS] Bug in create operator and/or initdb

2005-01-29 Thread John Hansen
The following seems to me a bug in either initdb or create operator: CREATE FUNCTION my_func (inet, inet) as '$libdir/my_func.so' LANGUAGE 'C' IMMUTABLE STRICT; CREATE OPERATOR ( PROCEDURE = my_func, LEFTARG = cidr, RIGHTARG = cidr, RESTRICT = contsel,

Re: [HACKERS] [BUGS] Bug in create operator and/or initdb

2005-01-29 Thread John Hansen
My opinion is that this is a very bogus shortcut in the network datatype code. There are no cases outside the inet/cidr group where an operator doesn't exactly match its underlying function. (The whole business of inet and cidr being almost but not quite the same type is maldesigned

Re: [HACKERS] [BUGS] Bug in create operator and/or initdb

2005-01-29 Thread Tom Lane
John Hansen [EMAIL PROTECTED] writes: CREATE FUNCTION my_func (inet, inet) as '$libdir/my_func.so' LANGUAGE 'C' IMMUTABLE STRICT; CREATE OPERATOR ( PROCEDURE = my_func, LEFTARG = cidr, RIGHTARG = cidr, RESTRICT = contsel, JOIN = contjoinsel ); ERROR:

Re: [HACKERS] [BUGS] Bug in create operator and/or initdb

2005-01-29 Thread Tom Lane
John Hansen [EMAIL PROTECTED] writes: In that case may I suggest fixing the catalog so network_* functions exists for both datatypes! Redesigning the inet/cidr distinction is on the to-do list (though I'm afraid not very high on the list). ISTM it should either be one type with a

Re: [HACKERS] [BUGS] Bug in create operator and/or initdb

2005-01-29 Thread Steve Atkins
On Sat, Jan 29, 2005 at 10:07:30PM -0500, Tom Lane wrote: John Hansen [EMAIL PROTECTED] writes: In that case may I suggest fixing the catalog so network_* functions exists for both datatypes! Redesigning the inet/cidr distinction is on the to-do list (though I'm afraid not very high on

Re: [HACKERS] IBM patent

2005-01-29 Thread Kevin Brown
Marc G. Fournier wrote: On Wed, 26 Jan 2005, Christopher Browne wrote: Actually, the latter isn't so. If Mammoth or Pervasive or such release their own release of PostgreSQL, nothing has historically mandated that they make that release available under the BSD license. Presumably

[HACKERS] Huge memory consumption during vacuum (v.8.0)

2005-01-29 Thread Oleg Bartunov
Hi there, seems I have a serious problem with vacuuming of rather big table (500,000,000 rows) on dual Intel(R) Xeon(TM) CPU 2.40GHz, 1Gb RAM, running Linux 2.6.7. I have PostgreSQL 8.0 release installed with slightly changed postgresql.conf: shared_buffers = 24576 # min 16, at least

Re: [HACKERS] [BUGS] Bug in create operator and/or initdb

2005-01-29 Thread Tom Lane
Steve Atkins [EMAIL PROTECTED] writes: For a replacement type, how important is it that it be completely compatible with the existing inet/cidr types? Is anyone actually using inet types with a non-cidr mask? If you check the archives you'll discover that our current inet/cidr types were

Re: [HACKERS] Huge memory consumption during vacuum (v.8.0)

2005-01-29 Thread Tom Lane
Oleg Bartunov oleg@sai.msu.su writes: I tried run 'vacuumdb -v -z -f wsdb vacuum-wsdb.log 21' I'm confused. The log trace you showed us before appeared to be from a non-FULL vacuum, but here you're saying it's VACUUM FULL. Which is it ... or did you change? regards,

Re: [HACKERS] Allowing VACUUM to time out when waiting for

2005-01-29 Thread Philip Warner
At 02:53 AM 30/01/2005, Tom Lane wrote: Philip Warner [EMAIL PROTECTED] writes: We have a frequently updated (peak 5/sec) table with about 1000 rows. We run VACCUM FULL on this table every 5 minutes. Plain vacuum (perhaps executed even more often, like once a minute) will cause fewer locking

Re: [HACKERS] Huge memory consumption during vacuum (v.8.0)

2005-01-29 Thread Oleg Bartunov
On Sun, 30 Jan 2005, Tom Lane wrote: Oleg Bartunov oleg@sai.msu.su writes: I tried run 'vacuumdb -v -z -f wsdb vacuum-wsdb.log 21' I'm confused. The log trace you showed us before appeared to be from a non-FULL vacuum, but here you're saying it's VACUUM FULL. Which is it ... or did you change?

Re: [HACKERS] Huge memory consumption during vacuum (v.8.0)

2005-01-29 Thread Tom Lane
Oleg Bartunov oleg@sai.msu.su writes: On Sun, 30 Jan 2005, Tom Lane wrote: I'm confused. The log trace you showed us before appeared to be from a non-FULL vacuum, but here you're saying it's VACUUM FULL. Which is it ... or did you change? Yes, first time I tried vacuum from withing psql,

Re: [HACKERS] Allowing VACUUM to time out when waiting for locks?

2005-01-29 Thread Tom Lane
Philip Warner [EMAIL PROTECTED] writes: Am I correct in saying that the FSM now tracks the entire table, and that the FSM parameters just determine how much is stored in memory? No. Any free space that can't be remembered in FSM is lost to use. (Not completely --- an update of a row on the

Re: [HACKERS] Huge memory consumption during vacuum (v.8.0)

2005-01-29 Thread Oleg Bartunov
On Sun, 30 Jan 2005, Tom Lane wrote: Oleg Bartunov oleg@sai.msu.su writes: On Sun, 30 Jan 2005, Tom Lane wrote: I'm confused. The log trace you showed us before appeared to be from a non-FULL vacuum, but here you're saying it's VACUUM FULL. Which is it ... or did you change? Yes, first time I

Re: [HACKERS] Huge memory consumption during vacuum (v.8.0)

2005-01-29 Thread Oleg Bartunov
On Sun, 30 Jan 2005, Oleg Bartunov wrote: On Sun, 30 Jan 2005, Tom Lane wrote: Oleg Bartunov oleg@sai.msu.su writes: On Sun, 30 Jan 2005, Tom Lane wrote: I'm confused. The log trace you showed us before appeared to be from a non-FULL vacuum, but here you're saying it's VACUUM FULL. Which is it