Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Mark Wong
On Wed, Jan 12, 2005 at 09:17:33PM -0500, Tom Lane wrote: > I notice that the backend seems to have been using some nonstandard C > code: > > Error while reading shared library symbols: > /home/markw/dbt2/storedproc/pgsql/c/../../../storedproc/pgsql/c/funcs.so: No > such file or directory. > > W

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Tom Lane
Bruce Momjian writes: >> Ah, right, I missed the connection. Hmm ... that's sort of the inverse >> of the "killed tuple" optimization we put in a release or two back, >> where an index tuple is marked as definitely dead once it's committed >> dead and the deletion is older than all active transac

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> Even if you could track the tuple's committed-good status reliably, > >> that isn't enough under MVCC. > > > I mentioned that: > > >> (Oh, and you could only update the bit when all active transactions > >> are newer than the creat

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> Even if you could track the tuple's committed-good status reliably, >> that isn't enough under MVCC. > I mentioned that: >> (Oh, and you could only update the bit when all active transactions >> are newer than the creation transaction so we know they sh

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > My basic idea was to keep a status bit on each index entry telling it if > > a previous backend looked at the heap and determined it was valid. > > Even if you could track the tuple's committed-good status reliably, > that isn't enough under MVCC. The

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Tom Lane
Bruce Momjian writes: > My basic idea was to keep a status bit on each index entry telling it if > a previous backend looked at the heap and determined it was valid. Even if you could track the tuple's committed-good status reliably, that isn't enough under MVCC. The tuple might be committed goo

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Bruce Momjian
Jonah H. Harris wrote: > 1. Is there any answer to Bruce?s last statement in the thread, ?Re: > [PERFORM] COUNT(*) again (was Re: Index/Function organized? > (http://archives.postgresql.org/pgsql-hackers/2003-10/msg00245.php) Let me give you my ideas in the above URL and why they are probably w

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > I thought toast only handled having individual large columns. So if I have a > 2kb text column it'll pull that out of the table for me. But if I have 20 > columns each of which have 100 bytes will it still help me? Will it kick in if > I define a single colu

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Greg Stark
Jeff Davis <[EMAIL PROTECTED]> writes: > But of course, we all love toast. Everyone needs to make those wide > tables once in a while, and toast does a great job of taking those > worries away in an efficient way. I am just saying that hopefully we > don't have to seqscan a table with wide tuples

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jeff Davis
> That mechanism exists right now, and it's called TOAST, dubbed the best > thing since sliced bread. We even have documentation for it, new as of > our latest RC: > > http://developer.postgresql.org/docs/postgres/storage-toast.html > Thanks for the link. It looks like it breaks it up into chun

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Tom Lane
Mark Wong <[EMAIL PROTECTED]> writes: > Ok, well I got a core dump with 8.0rc4, but I'm not sure if it's > exactly the same problem. I have the postgres binary and the core > here: > http://developer.osdl.org/markw/pgsql/core/2files.tar.bz2 > But it's for ia64, if you got one. Poking around

Re: [HACKERS] pg_autovacuum w/ dbt2

2005-01-12 Thread Mark Wong
On Tue, Dec 21, 2004 at 05:56:47PM -0500, Tom Lane wrote: > If you want to track it yourself, please change those elog(ERROR)s to > elog(PANIC) so that they'll generate core dumps, then build with > --enable-debug if you didn't already (--enable-cassert would be good too) > and get a debugger stack

[HACKERS] looking for rh9 rpms for pgadmin v 1.2

2005-01-12 Thread Dave Cramer
Hi, Is there any intent to build these rpm's ? Who is responsible for this ? Dave -- Dave Cramer http://www.postgresintl.com 519 939 0336 ICQ#14675561 ---(end of broadcast)--- TIP 9: the planner will ignore your desire to choose an index scan if your

[HACKERS] sparse (static analyzer) report

2005-01-12 Thread Mark Wong
Hi, Just wondering if anyone finds spare's analysis useful. I ran it against 8.0-rc5: http://developer.osdl.org/markw/pgsql/sparse/pg-8.0rc5.txt Sparse can be downloaded http://www.codemonkey.org.uk/projects/bitkeeper/sparse/ or bk://sparse.bkbits.net/sparse

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Rod Taylor
> >The index could then make sensible the reasonably common practice of > >using a covered index - i.e. putting additional columns into the index > >to satisfy the whole query just from the index. > I am willing to take it on and I understand that the workload is mine. > As long as everyone give

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Simon Riggs wrote: Jonah, People's objections are: - this shouldn't be the system default, so would need to be implemented as a non-default option on a b-tree index - its a lot of code and if you want it, you gotta do it Remember you'll need to - agree all changes via the list and accept that redes

Re: [HACKERS] PANIC: right sibling's left-link doesn't match

2005-01-12 Thread Jim Buttafuoco
It did print the query right after the PANIC message, so I do have the table name. I just completed the reindex. Thanks -- Original Message --- From: Tom Lane <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Cc: "pgsql-hackers" Sent: Wed, 12 Jan 2005 16:45:11 -0500 Subject: Re: [HAC

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Bruno Wolff III wrote: On Wed, Jan 12, 2005 at 14:09:07 -0700, "Jonah H. Harris" <[EMAIL PROTECTED]> wrote: Please keep stuff posted to the list so that other people can contribute and learn from the discussion unless there is a particular reason to limited who is involved in the discussion. no

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Simon Riggs
On Wed, 2005-01-12 at 15:09 -0500, Rod Taylor wrote: > On Wed, 2005-01-12 at 12:52 -0700, Jonah H. Harris wrote: > > Tom Lane wrote: > > > > >The fundamental problem is that you can't do it without adding at least > > >16 bytes, probably 20, to the size of an index tuple header. That would > > >d

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Bruno Wolff III
On Wed, Jan 12, 2005 at 14:09:07 -0700, "Jonah H. Harris" <[EMAIL PROTECTED]> wrote: Please keep stuff posted to the list so that other people can contribute and learn from the discussion unless there is a particular reason to limited who is involved in the discussion. > Bruno, > > Thanks for

Re: [HACKERS] PANIC: right sibling's left-link doesn't match

2005-01-12 Thread Tom Lane
"Jim Buttafuoco" <[EMAIL PROTECTED]> writes: > Postgres on one of my big database servers just crashed with the following > message > PANIC: right sibling's left-link doesn't match > Does any one have any idea's what might cause this. Corrupted btree index. REINDEX should help, though I'm af

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Alvaro Herrera
On Wed, Jan 12, 2005 at 12:41:38PM -0800, Jeff Davis wrote: > Except then the two heaps would have to be joined somehow for every > operation. It makes sense some times to (if you have a very wide table) > split off the rarely-accessed attributes into a seperate table to be > joined one-to-one whe

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Marek Mosiewicz
I agree with last statement. count(*) is not most important. Most nice thing with index only scan is when it contains more than one column. When there is join among many tables where from each table only one or few columns are taken it take boost query incredibly. For exmaple on when you have cust

Re: [HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Tom Lane
Reinhard Max writes: > On Wed, 12 Jan 2005 at 14:59, Tom Lane wrote: >> That looks like a reasonable fix, but isn't it needed in >> backend/libpq/auth.c as well? > Yes, indeed: > auth.c: In function `pg_krb5_init': > auth.c:202: warning: implicit declaration of function `com_err' OK, patch appl

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Jeff Davis wrote: Does someone know exactly what oracle actually does? some old info resides here, http://www.orsweb.com/techniques/fastfull.html I'll try and find something more recent. ---(end of broadcast)--- TIP 8: explain analyze is your frien

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jeff Davis
> > > We seem to be in agreement. I'm looking for faster/smarter access to > data, not the monetary cost of doing so. Isn't it faster/smarter to > satisfy a query with the index rather than sequentially scanning an > entire relation if it is possible? > You have to scan every tuple's visibil

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Bruno Wolff III
On Wed, Jan 12, 2005 at 13:42:58 -0700, "Jonah H. Harris" <[EMAIL PROTECTED]> wrote: > We seem to be in agreement. I'm looking for faster/smarter access to > data, not the monetary cost of doing so. Isn't it faster/smarter to > satisfy a query with the index rather than sequentially scanning

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Jon Jensen wrote: If you're willing to do the work, and have the motivation, probably the best thing to do is just do it. Then you can use empirical measurements of the effect on disk space, speed of various operations, etc. to discuss the merits/demerits of your particular implementation. Then

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jon Jensen
On Wed, 12 Jan 2005, Jonah H. Harris wrote: Andrew Dunstan wrote: Monetary cost is not the issue - cost in time is the issue. We seem to be in agreement. I'm looking for faster/smarter access to data, not the monetary cost of doing so. Isn't it faster/smarter to satisfy a query with the index r

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jeff Davis
> > > I recognize the added cost of implementing index only scans. As storage > is relatively cheap these days, everyone I know is more concerned about > faster access to data. Similarly, it would still be faster to scan the > indexes than to perform a sequential scan over the entire relation

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Rod Taylor wrote: grow by about 40GB if this was done. Storage isn't that cheap when you include the hot-backup master, various slaves, RAM for caching of this additional index space, backup storage unit on the SAN, tape backups, additional spindles required to maintain same performance due to incr

[HACKERS] PANIC: right sibling's left-link doesn't match

2005-01-12 Thread Jim Buttafuoco
Postgres on one of my big database servers just crashed with the following message PANIC: right sibling's left-link doesn't match Does any one have any idea's what might cause this. Some background. This is a Debian Sarge system running PG 7.4.5 on i386 dual XEON system with 4G of memory.

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Andrew Dunstan wrote: Monetary cost is not the issue - cost in time is the issue. cheers andrew We seem to be in agreement. I'm looking for faster/smarter access to data, not the monetary cost of doing so. Isn't it faster/smarter to satisfy a query with the index rather than sequentially sca

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jeff Davis
> No, sequential scans require slightly more i/o than index scans. More > importantly they require random access i/o instead of sequential i/o which is > much slower. > Just to clear it up, I think what you meant was the index requires random i/o, not the table. And the word "slightly" depends

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Greg Stark
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > You are correct, I am proposing to add visibility to the indexes. Then I think the only way you'll get any support is if it's an option. Since it would incur a performance penalty on updates and deletes. > As for unqualified counts, I believe that

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Andrew Dunstan
Jonah H. Harris said: > Tom Lane wrote: > >>The fundamental problem is that you can't do it without adding at least >>16 bytes, probably 20, to the size of an index tuple header. That >>would double the physical size of an index on a simple column (eg an >>integer or timestamp). The extra I/O cos

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Rod Taylor
On Wed, 2005-01-12 at 12:52 -0700, Jonah H. Harris wrote: > Tom Lane wrote: > > >The fundamental problem is that you can't do it without adding at least > >16 bytes, probably 20, to the size of an index tuple header. That would > >double the physical size of an index on a simple column (eg an int

Re: [HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Reinhard Max
On Wed, 12 Jan 2005 at 14:59, Tom Lane wrote: > That looks like a reasonable fix, but isn't it needed in > backend/libpq/auth.c as well? Yes, indeed: auth.c: In function `pg_krb5_init': auth.c:202: warning: implicit declaration of function `com_err' cu Reinhard ---

Re: [HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Tom Lane
Reinhard Max writes: > --- src/interfaces/libpq/fe-auth.c > +++ src/interfaces/libpq/fe-auth.c > @@ -244,6 +244,11 @@ > > #include > > +#if !defined(__COM_ERR_H) && !defined(__COM_ERR_H__) > +/* if krb5.h didn't include it already */ > +#include > +#endif > + > /* > * pg_an_to_ln -- retu

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Dann Corbit
A notion for indices that are not unique... (won't help much on select count(*) but might be helpful for other types of query optimization) Put a count in the index for each distinct type. In the worst case, the index is actually unique and you have 8 wasted bytes per index entry and all the entri

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Tom Lane wrote: The fundamental problem is that you can't do it without adding at least 16 bytes, probably 20, to the size of an index tuple header. That would double the physical size of an index on a simple column (eg an integer or timestamp). The extra I/O costs and extra maintenance costs are

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Merlin Moncure
> Greg Stark wrote: > > >I think part of the problem is that there's a bunch of features related > to > >these types of queries and the lines between them blur. > > > >You seem to be talking about putting visibility information inside > indexes for > >so index-only plans can be performed. But you'

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > My thinking is that we may be able to implement index usage for not only > unqualified counts, but also on any query that can be satisfied by the > index itself. The fundamental problem is that you can't do it without adding at least 16 bytes, prob

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Greg Stark wrote: I think part of the problem is that there's a bunch of features related to these types of queries and the lines between them blur. You seem to be talking about putting visibility information inside indexes for so index-only plans can be performed. But you're also talking about q

Re: [HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Reinhard Max
On Wed, 12 Jan 2005 at 20:28, Kurt Roeckx wrote: > This is because the proper prototype is: > extern char const *error_message (long); > > And C automaticly generates a prototype with in int instead. > > On 32 bit platforms this ussualy isn't a problem since both int and > long are ussualy both

Re: [HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Kurt Roeckx
On Wed, Jan 12, 2005 at 07:36:52PM +0100, Reinhard Max wrote: > > The problem is, that the heimdal implementation of kerberos5 used on > sles8 needs an extra include statement for com_err.h in > src/interfaces/libpq/fe-auth.c to get the prototype for > error_message(), while on newer SUSE-relea

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Greg Stark
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > Looking at the message boards, there is significant interest in the COUNT(*) > aspect. However, rather than solely address the COUNT(*) TODO item, why not > fix > it and add additional functionality found in commercial databases as well? I > believe

Re: [HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Reinhard Max
Sorry for following up to myself once more... On Wed, 12 Jan 2005 at 19:36, Reinhard Max wrote: > The problem is, that the heimdal implementation of kerberos5 used on > sles8 needs an extra include statement for com_err.h in > src/interfaces/libpq/fe-auth.c to get the prototype for > error_me

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Tom, Thank you for your prompt response and I understand your statement completely. My thinking is that we may be able to implement index usage for not only unqualified counts, but also on any query that can be satisfied by the index itself. Index usage seems to be a feature that could speed up

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Merlin Moncure
> Tom, Bruce, and others involved in this recurring TODO discussion... > > First, let me start by saying that I understand this has been discussed > many times before; however, I'd like to see what the current state of > affairs is regarding the possibility of using a unique index scan to > speed

[HACKERS] segfault caused by heimdal (was: SUSE port)

2005-01-12 Thread Reinhard Max
On Wed, 12 Jan 2005 at 18:20, Reinhard Max wrote: > I am still not sure whether the kerberos library, glibc, or > PostgreSQL is to blame, or if it's a combination of bugs in these > components that triggers the segfault. The problem is, that the heimdal implementation of kerberos5 used on sles

Re: [HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Tom Lane
"Jonah H. Harris" <[EMAIL PROTECTED]> writes: > Tom, Bruce, and others involved in this recurring TODO discussion… > First, let me start by saying that I understand this has been discussed > many times before; however, I’d like to see what the current state of > affairs is regarding the possibili

[HACKERS] Much Ado About COUNT(*)

2005-01-12 Thread Jonah H. Harris
Tom, Bruce, and others involved in this recurring TODO discussion… First, let me start by saying that I understand this has been discussed many times before; however, I’d like to see what the current state of affairs is regarding the possibility of using a unique index scan to speed up the COUNT

Re: [HACKERS] SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release

2005-01-12 Thread Reinhard Max
On Wed, 12 Jan 2005 at 17:29, Reinhard Max wrote: > The only failure I have to report is sles8-x86_64, where I am > getting segfaults from psql during the regression tests. The segfault in a call to snprintf somewhere in libpq's kerberos5 code. So when I leave out --with-krb5 it compiles and pa

Re: [HACKERS] SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release

2005-01-12 Thread Reinhard Max
On Wed, 12 Jan 2005 at 16:29, Peter Eisentraut wrote: > In the meantime I have received confirmation from Reinhard Max that > his test methods match our requirements, so the list will be > completed with the other platforms he reported in due time. Today I've updated the RPMs on the FTP server

Re: [HACKERS] port report: Linux SuSE 9.1

2005-01-12 Thread Mikhail Terekhov
Sorry for the follow-up. That was with postgresql-8.0rc5.tar.bz2 Mikhail Terekhov wrote: Hello, ./configure --prefix=/tmp/pgsql --enable-thread-safety --with-pam --with-openssl --with-tcl --with-python --with-perl == All 96 tests passed. == Linux usenterekh

[HACKERS] port report: Linux SuSE 9.1

2005-01-12 Thread Mikhail Terekhov
Hello, ./configure --prefix=/tmp/pgsql --enable-thread-safety --with-pam --with-openssl --with-tcl --with-python --with-perl == All 96 tests passed. == Linux usenterekhovx2l 2.6.5-7.111-smp #1 SMP Wed Oct 13 15:45:13 UTC 2004 i686 i686 i386 GNU/Linux SuSE

Re: [HACKERS] Re: Re: SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release Candidate 5)

2005-01-12 Thread Peter Eisentraut
[EMAIL PROTECTED] wrote: > I should note here also that SGI have replied "No" to my request for > help (or access) with porting PostgreSQL onto the latest version of > IRIX... Another year, some result... > Anyone got access to an HP/UX development system? Check out HP's testdrive program. Many

Re: [HACKERS] SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release Candidate 5)

2005-01-12 Thread Peter Eisentraut
Simon Riggs wrote: > Not sure what is going on here: why is SUSE not listed on the > supported platforms list? (still) RC5 contains: SUSE Linux x86 8.0.0 Peter Eisentraut (<[EMAIL PROTECTED]>), 2005-01-10 9.1 In the meantime I have received confirmation from Reinhard Max that his test metho

Re: Re: [HACKERS] SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release Candidate 5)

2005-01-12 Thread simon
Reinhard Max wrote on 12.01.2005, 11:38:55: > On Wed, 12 Jan 2005 at 03:53, Tom Lane wrote: > > > > ...or is it because his postings to ANNOUNCE that the port to SUSE > > > have gone unnoticed by those that compile the supported platforms > > > list? > > > > If he insists on posting such rout

[HACKERS] Re: Re: SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release Candidate 5)

2005-01-12 Thread simon
Reinhard Max wrote on 12.01.2005, 11:17:52: > On Wed, 12 Jan 2005 at 08:23, Simon Riggs wrote: > > > Not sure what is going on here: why is SUSE not listed on the supported > > platforms list? (still) > > > > ...is it because Reinhard seems resistant > > why do you think so? > > > (after priv

Re: [HACKERS] SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release Candidate 5)

2005-01-12 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Not sure what is going on here: why is SUSE not listed on the supported > platforms list? (still) I haven't seen any reports of passes on SUSE. I have zero doubt that PG works on SUSE, since it's pretty much exactly like every other Linux, but there's bee

[HACKERS] SUSE port (was [ANNOUNCE] PostgreSQL 8.0.0 Release Candidate 5)

2005-01-12 Thread Simon Riggs
Not sure what is going on here: why is SUSE not listed on the supported platforms list? (still) ...is it because Reinhard seems resistant (after private conversation) to the idea of submitting a formal port report via HACKERS, like everybody else? ...or is it because his postings to ANNOUNCE tha