Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Gokulakannan Somasundaram
Just one more thought on the same. This implementation also assumes that there won't be any update chains across pages, which is the current stage. Heikki, Is it planned as a optional feature? (I support the optional feature model) Thanks, Gokul. On Nov 6, 2007 12:20 PM, Gregory Stark

Re: [HACKERS] Hash index todo list item

2007-11-06 Thread Gokulakannan Somasundaram
I have not followed this thread very closely. But just wanted to give my inputs. From the results obtained, the average of all the hash probes is 141.8ms, the average for btree is 168.5, a difference of about 27.The standard deviations are about 23, so this is a statistically significant

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Heikki Linnakangas
Gokulakannan Somasundaram wrote: Just one more thought on the same. This implementation also assumes that there won't be any update chains across pages, which is the current stage. No, it doesn't assume that. Heikki, Is it planned as a optional feature? (I support the optional

Re: [HACKERS] Hash index todo list item

2007-11-06 Thread Jens-Wolfhard Schicke
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Shreya Bhargava wrote: Note that the bottom line for the problems with hash indexes is that the current implementation doesn't offer any advantages over btree indexes. Hash indexes need to be not only as good of a choice as btree indexes but

Re: [HACKERS] Hash index todo list item

2007-11-06 Thread Heikki Linnakangas
Shreya Bhargava wrote: 1. Populate the table with 80 million tuples. 2. Create HASH index on the table. 3. clear both linux cache psql buffers. (exiting psql and restarting it cleared the psql buffers; to clear linux cache, we used drop_cache command) 4. start psql 5. select on an

Re: [HACKERS] should I worry?

2007-11-06 Thread ohp
On Mon, 5 Nov 2007, Tom Lane wrote: Date: Mon, 05 Nov 2007 15:53:10 -0500 From: Tom Lane [EMAIL PROTECTED] To: Heikki Linnakangas [EMAIL PROTECTED] Cc: Rick Gigger [EMAIL PROTECTED], [EMAIL PROTECTED], pgsql-hackers@postgresql.org Subject: Re: [HACKERS] should I worry? Heikki

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Simon Riggs
On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote: Reducing VACUUM time is important, but the real big promise is the ability to do index-only-scans. Have you thought about how index-only scans work work? Seems like we need a rough plan for that before we go and build the

Re: [HACKERS] should I worry?

2007-11-06 Thread Heikki Linnakangas
[EMAIL PROTECTED] wrote: I'd love to find a query against pg_triggers giving the table name for each RI_ConstraintTrigger_xxx. SELECT t.tgname, c.relname, tgconstrname FROM pg_trigger t, pg_class c WHERE t.tgrelid = c.oid AND tgname like 'RI_ConstraintTrigger_%'; -- Heikki Linnakangas

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Andrew Dunstan
Mark Mielke wrote: I'm not sure what RI lookup is. Sorry. :-) RI = Referential Integrity. i.e. Foreign Keys. cheers andrew ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Segmentation fault using digest from pg_crypto

2007-11-06 Thread Marko Kreen
On 11/6/07, Bruce Momjian [EMAIL PROTECTED] wrote: Alvaro Herrera wrote: Tom, how about putting a note about that into next 8.2 minor release notes? (8.3 too?) Something like You need to refresh pgcrypto functions, because since rel 8.2 the code depends on functions being tagged

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Marko Kreen
On 11/6/07, Mark Mielke [EMAIL PROTECTED] wrote: Simon Riggs wrote: On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote: I'm racking my brain trying to think of a query that will benefit from index-only scans without specifically creating covered indexes. Apart from count(*)

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Mark Mielke
Simon Riggs wrote: On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote: I'm racking my brain trying to think of a query that will benefit from index-only scans without specifically creating covered indexes. Apart from count(*) queries and RI lookups. I can't see RI lookups being much

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Marko Kreen
On 11/6/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Jeff Davis wrote: On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote: It's not useful for VACUUM FREEZE, unless we're willing to freeze much more aggressively, and change the meaning of a set bit to all tuples on heap page

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Heikki Linnakangas
Mark Mielke wrote: Simon Riggs wrote: On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote: I'm racking my brain trying to think of a query that will benefit from index-only scans without specifically creating covered indexes. Apart from count(*) queries and RI lookups. I can't see RI

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Marko Kreen
On 11/6/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: Marko Kreen wrote: On 11/6/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: (Gosh, we really need a name for the sort of vacuum. I was about to say we'd still need regular regular VACUUMs :-)) As the new VACUUM variant will be

Re: [HACKERS] Test lab

2007-11-06 Thread Simon Riggs
On Mon, 2007-11-05 at 14:33 -0800, Mark Wong wrote: On 11/4/07, Simon Riggs [EMAIL PROTECTED] wrote: Why don't you post a TODO list for TPC-E somewhere, so people can bite small pieces off of the list. I'm sure there's lots of people can help if we do it that way. This should be a good

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Heikki Linnakangas
Marko Kreen wrote: On 11/6/07, Heikki Linnakangas [EMAIL PROTECTED] wrote: (Gosh, we really need a name for the sort of vacuum. I was about to say we'd still need regular regular VACUUMs :-)) As the new VACUUM variant will be somewhat unsafe, it should not replace regular VACUUM but get

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Heikki Linnakangas
Gregory Stark wrote: Heikki Linnakangas [EMAIL PROTECTED] writes: One problem is that you have to atomically update the visibility map when you update the heap. That means that you have to lock the visibility map page and the heap page at the same time. If the visibility map is in the heap,

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Heikki Linnakangas
Simon Riggs wrote: I'm thinking that looking in the visibility map will have a cost also, so how will we know whether to bother looking? I'm assuming that we won't want to do that lookup in all cases, since it could easily just add pathlength and contention in the normal OLTP case. Yeah, I

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Simon Riggs
On Tue, 2007-11-06 at 13:29 +, Heikki Linnakangas wrote: My list would be: - EXISTS / NOT EXISTS - COUNT(*) Yeah, those are good candidates. Yeah. Many-to-many relationships is one example: OK, thats a very good one. And of course people will start adding columns to indexes, to

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Mark Mielke
Simon Riggs wrote: On Tue, 2007-11-06 at 13:29 +, Heikki Linnakangas wrote: And of course people will start adding columns to indexes, to make use of index-only-scans, once we have the capability. Not too keen on that. Very difficult to judge whether its worth the benefit for

Re: [HACKERS] Hash index todo list item

2007-11-06 Thread Tom Lane
Shreya Bhargava [EMAIL PROTECTED] writes: We performed some probe tests on our patch on hash index and the original btree index to compare the performance between the two. Interesting, but ... From the results obtained, the average of all the hash probes is 141.8ms, the average for btree

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Gokulakannan Somasundaram
On Nov 6, 2007 4:33 PM, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: Just one more thought on the same. This implementation also assumes that there won't be any update chains across pages, which is the current stage. No, it doesn't assume that. Say, if

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Heikki Linnakangas
Gokulakannan Somasundaram wrote: On Nov 6, 2007 4:33 PM, Heikki Linnakangas [EMAIL PROTECTED] wrote: Gokulakannan Somasundaram wrote: Just one more thought on the same. This implementation also assumes that there won't be any update chains across pages, which is the current stage. No, it

Re: [HACKERS] should I worry?

2007-11-06 Thread Tom Lane
[EMAIL PROTECTED] writes: IIUC, I have drop every trigger like this : SELECT t.tgname, c.relname, tgconstrname FROM pg_trigger t, pg_class c WHERE t.tgrelid = c.oid AND tgname like 'RI_ConstraintTrigger_%'AND tgconstrname = 'unnamed'; and I delete all those ancient foreign key WITHOUT

Re: [HACKERS] should I worry?

2007-11-06 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: [EMAIL PROTECTED] wrote: I'd love to find a query against pg_triggers giving the table name for each RI_ConstraintTrigger_xxx. SELECT t.tgname, c.relname, tgconstrname FROM pg_trigger t, pg_class c WHERE t.tgrelid = c.oid AND tgname like

Re: [HACKERS] should I worry?

2007-11-06 Thread ohp
On Tue, 6 Nov 2007, Heikki Linnakangas wrote: Date: Tue, 06 Nov 2007 13:07:23 + From: Heikki Linnakangas [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: Tom Lane [EMAIL PROTECTED], Rick Gigger [EMAIL PROTECTED], pgsql-hackers@postgresql.org Subject: Re: [HACKERS] should I worry?

[HACKERS] Weird type selection choice

2007-11-06 Thread Peter Eisentraut
I noticed this problem in 8.2 and 8.3: pei=# select mod( trunc( 1 ), 2 ); ERROR: 42883: function mod(double precision, integer) does not exist LINE 1: select mod( trunc( 1 ), 2 ); ^ It apparently casts the 1 to double precision to pick the variant trunc(dp)=dp instead of

Re: [HACKERS] Weird type selection choice

2007-11-06 Thread Peter Eisentraut
Am Dienstag, 6. November 2007 schrieb Peter Eisentraut: I noticed this problem in 8.2 and 8.3: pei=# select mod( trunc( 1 ), 2 ); ERROR: 42883: function mod(double precision, integer) does not exist LINE 1: select mod( trunc( 1 ), 2 ); ^ It apparently casts the 1 to double

Re: [HACKERS] Weird type selection choice

2007-11-06 Thread Bernd Helmle
--On Dienstag, November 06, 2007 16:31:05 +0100 Peter Eisentraut [EMAIL PROTECTED] wrote: I noticed this problem in 8.2 and 8.3: pei=# select mod( trunc( 1 ), 2 ); ERROR: 42883: function mod(double precision, integer) does not exist LINE 1: select mod( trunc( 1 ), 2 ); ^ It

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Gokulakannan Somasundaram
By the way, please have a look at how i have converted a index scan into a index only scan in the thick index patch. Currently it doesn't convert those queries which doesn't have where clause. I hope you would be able refine it further. For example, if there is a query like select count(1) from

[HACKERS] regtype sorting broken in 8.3

2007-11-06 Thread Peter Eisentraut
This used to work in 8.2: pei=# select castsource::regtype, casttarget::regtype, castcontext from pg_cast order by 1,2; But int 8.3 it says: ERROR: XX000: could not find pathkey item to sort LOCATION: make_sort_from_pathkeys, createplan.c:2812 -- Peter Eisentraut

Re: [HACKERS] Weird type selection choice

2007-11-06 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: It apparently casts the 1 to double precision to pick the variant trunc(dp)=dp instead of trunc(numeric)=numeric. I was under the impression that we didn't want to cast integers to float types implicitly because this loses information. Clearly, the

[HACKERS] Opinion / advice needed for TODO: function params ref by name

2007-11-06 Thread Gevik Babakhani
Hello, This is about: http://archives.postgresql.org/pgsql-patches/2007-11/msg00028.php In order to resolve function name.parameter name the check in transformColumnRef goes into case 2 which is A.B The ideal way to resolve the func.param in this case is to perform the callback (hook) after

Re: [HACKERS] Opinion / advice needed for TODO: function params ref by name

2007-11-06 Thread Tom Lane
Gevik Babakhani [EMAIL PROTECTED] writes: Unfortunately for func.param is this problematic because qualifiedNameToVar and transformWholeRowRef/ParseFuncOrColumn internally call RTE functions which justifiably throw an ambiguous column error. What's the problem? If it's ambiguous within the

Re: [HACKERS] regtype sorting broken in 8.3

2007-11-06 Thread Tom Lane
Peter Eisentraut [EMAIL PROTECTED] writes: pei=# select castsource::regtype, casttarget::regtype, castcontext from pg_cast order by 1,2; ERROR: XX000: could not find pathkey item to sort LOCATION: make_sort_from_pathkeys, createplan.c:2812 Hmph. When we hacked that function last week, I

Re: [HACKERS] Weird type selection choice

2007-11-06 Thread Peter Eisentraut
Tom Lane wrote: Peter Eisentraut [EMAIL PROTECTED] writes: It apparently casts the 1 to double precision to pick the variant trunc(dp)=dp instead of trunc(numeric)=numeric. I was under the impression that we didn't want to cast integers to float types implicitly because this loses

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Jeff Davis
On Mon, 2007-11-05 at 22:45 +, Heikki Linnakangas wrote: 1) Do as you say above. What are some of the cost trade-offs here? It seems that frequent VACUUM FREEZE runs would keep the visibility map mostly full, but will also cause more writing. I suppose the worst case is that every

Re: [HACKERS] Test lab

2007-11-06 Thread Mark Wong
On Tue, 06 Nov 2007 13:15:02 + Simon Riggs [EMAIL PROTECTED] wrote: On Mon, 2007-11-05 at 14:33 -0800, Mark Wong wrote: On 11/4/07, Simon Riggs [EMAIL PROTECTED] wrote: Why don't you post a TODO list for TPC-E somewhere, so people can bite small pieces off of the list. I'm sure

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Jeff Davis
On Tue, 2007-11-06 at 08:01 -0500, Mark Mielke wrote: Simon Riggs wrote: On Mon, 2007-11-05 at 09:52 +, Heikki Linnakangas wrote: I'm racking my brain trying to think of a query that will benefit from index-only scans without specifically creating covered indexes. Apart from count(*)

[HACKERS] tribble.postgresql.org - planned maintenance downtime

2007-11-06 Thread Stefan Kaltenbrunner
Hi all! There will be planned downtime on tribble.postgresql.org Nov 7(tomorrow) from 11:30-12:00 GMT(estimated) affecting the following services: cvs.postgresql.org wwwmaster.postgresql.org www.pgadmin.org doxygen.postgresql.org Downtime is necessary to implement several changes to the

Re: [HACKERS] should I worry?

2007-11-06 Thread ohp
Dear Tom, On Tue, 6 Nov 2007, Tom Lane wrote: Date: Tue, 06 Nov 2007 10:05:58 -0500 From: Tom Lane [EMAIL PROTECTED] To: [EMAIL PROTECTED] Cc: Heikki Linnakangas [EMAIL PROTECTED], Rick Gigger [EMAIL PROTECTED], pgsql-hackers list pgsql-hackers@postgresql.org Subject: Re:

[HACKERS] EquivalenceClasses vs volatile functions

2007-11-06 Thread Tom Lane
Awhile back there was some discussion about how the system assumes that syntactically equal sort expressions can be considered interchangeable, which falls down if volatile functions are involved: http://archives.postgresql.org/pgsql-general/2006-11/msg01523.php I seem to recall the issue coming

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Gregory Stark
Gokulakannan Somasundaram [EMAIL PROTECTED] writes: For example, if there is a query like select count(1) from table. Then we can scan through all the index pages and the visibility map to get the count. Currently it goes for Full table scan. there should be something like full index scan,

Re: [HACKERS] Visibility map thoughts

2007-11-06 Thread Gregory Stark
Heikki Linnakangas [EMAIL PROTECTED] writes: I don't buy that. I believe at least on some architectures you'd get a word-long load+modify+store, and scribble the neighboring bytes. Hm, I mis-remembered this bit of advice from the glibc info doc. I remembered thinking it was strange when I read

Re: [HACKERS] Weird type selection choice

2007-11-06 Thread Gregory Stark
Peter Eisentraut [EMAIL PROTECTED] writes: The clarity stems from the fact that this is the variant that doesn't lose data whereas the other one does. I think double has a wider range. So you get a choice between losing precision or not being able to store all values. The expression I

Re: [HACKERS] Test lab

2007-11-06 Thread Robert Hodges
Hi everyone, Here are a couple of additions to the performance test lab discussion. I hope you will find these useful. 1.) Test tools. The Bristlecone testing package I presented at the PG Fall 2007 Conference is now available at http:// bristlecone.continuent.org. There are two main

[HACKERS] Beta2 crash / create type + tsearch2

2007-11-06 Thread Radoslaw Zielinski
Hello, Beta2 crashes on restoring a dump of tsearch2-enabled database, as produced by 8.2.x (x==5 IIRC) pg_dump; the shortest way I know to get the SIG11: SET search_path = public, pg_catalog; CREATE TYPE tsvector; CREATE FUNCTION tsvector_in(cstring) RETURNS tsvector AS

[HACKERS] fulltext parser strange behave

2007-11-06 Thread Pavel Stehule
Hello I am writing tsearch2 wrapper and I testing functionality. I found some little bit strange on default parser. It can't parse tags with numbers: test=# select * from parse('h1zluty kun se napil bzlute/b vody/h2'); tokid | token ---+--- 12 | 3 | h1 12 | 1 | zluty

Re: [HACKERS] Test lab

2007-11-06 Thread Guillaume Smet
Hi Robert (small world, I contributed to Sequoia a while ago...), all, On 11/6/07, Robert Hodges [EMAIL PROTECTED] wrote: 2.) Test hardware. We have a number of hosts in Grenoble, France that are available to help set up a European lab.We gave away 4 to the postgresql.fr folks but if

[HACKERS] Beta2 crash / create type + tsearch2

2007-11-06 Thread Radoslaw Zielinski
Hello, Beta2 crashes on restoring a dump of tsearch2-enabled database, as produced by 8.2.x (x==5 IIRC) pg_dump; the shortest way I know to get the SIG11: SET search_path = public, pg_catalog; CREATE TYPE tsvector; CREATE FUNCTION tsvector_in(cstring) RETURNS tsvector AS

Re: [HACKERS] Beta2 crash / create type + tsearch2

2007-11-06 Thread Jeff Davis
On Tue, 2007-11-06 at 23:00 +, Radoslaw Zielinski wrote: Hello, Beta2 crashes on restoring a dump of tsearch2-enabled database, as produced by 8.2.x (x==5 IIRC) pg_dump; the shortest way I know to get the SIG11: Here is a guide to help migrate tsearch2 data from 8.2 to 8.3:

Re: [HACKERS] EquivalenceClasses vs volatile functions

2007-11-06 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: select random() as a, random() as b from foo order by b, a; And it definitely seems the wrong thing for the last one. Ouch. What I'm thinking of doing is adding a field to EquivalenceClass that carries the ressortgroupref of the originating ORDER

[HACKERS] Bug: --with-libxml does not take a location

2007-11-06 Thread Josh Berkus
Peter, Nickolay, ./configure --with-libxml does not accept a location argument. This makes it impossible to configure 8.3 with LibXML on the Mac, because I can't upgrade the main libxml without breaking something, and ./configure doesn't let me specify an alternate location. --with-libxml

Re: [HACKERS] Bug: --with-libxml does not take a location

2007-11-06 Thread Neil Conway
On Wed, 2007-11-07 at 11:38 +0900, Josh Berkus wrote: ./configure --with-libxml does not accept a location argument. This makes it impossible to configure 8.3 with LibXML on the Mac, because I can't upgrade the main libxml without breaking something, and ./configure doesn't let me specify

Re: [HACKERS] Bug: --with-libxml does not take a location

2007-11-06 Thread Josh Berkus
Neil Conway wrote: On Wed, 2007-11-07 at 11:38 +0900, Josh Berkus wrote: ./configure --with-libxml does not accept a location argument. This makes it impossible to configure 8.3 with LibXML on the Mac, because I can't upgrade the main libxml without breaking something, and ./configure

Re: [HACKERS] Bug: --with-libxml does not take a location

2007-11-06 Thread Andrew Dunstan
Josh Berkus wrote: Neil Conway wrote: On Wed, 2007-11-07 at 11:38 +0900, Josh Berkus wrote: ./configure --with-libxml does not accept a location argument. This makes it impossible to configure 8.3 with LibXML on the Mac, because I can't upgrade the main libxml without breaking something,

Re: [HACKERS] Beta2 crash / create type + tsearch2

2007-11-06 Thread Tom Lane
Radoslaw Zielinski [EMAIL PROTECTED] writes: Is contrib/tsearch2 obsolete? Yes. regards, tom lane ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings

Re: [HACKERS] Beta2 crash / create type + tsearch2

2007-11-06 Thread Tom Lane
Jeff Davis [EMAIL PROTECTED] writes: I agree that a dump/restore from 8.2 with tsearch2 to 8.3 with built-in tsearch should not SIGSEGV. That's not what he did, though. Force-feeding contrib/tsearch2 into 8.3 will not work. regards, tom lane

Re: [HACKERS] EquivalenceClasses vs volatile functions

2007-11-06 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: What if the equivalence class is in more than one place in the ORDER BY? That's not a problem. ORDER BY 1,1 is not different from ORDER BY 1, even if the referenced expression is volatile, because the secondary sort key is useless anyway. And what