Re: [HACKERS] GIN index creation extremely slow ?

2006-06-28 Thread Teodor Sigaev
I won't have access to the original testcase and server for a few days but I just redid some testing on a slower personal box of mine with a smaller(but similiar) testset and on that box I could not reproduce that issue. So the problem is either caused by the size of the table or somehow by the

Re: [HACKERS] GIN index creation extremely slow ?

2006-06-28 Thread Martijn van Oosterhout
On Wed, Jun 28, 2006 at 01:09:23PM +0400, Teodor Sigaev wrote: We tested gin with 5 millions records (but not a wikipedia's text, but with blog records which is usually mush shorter) and index creation time was near 8 hours on Dual Xeon 3.2, but we didn't try GiST. In other hand, creating

Re: [HACKERS] GIN index creation extremely slow ?

2006-06-28 Thread Oleg Bartunov
On Wed, 28 Jun 2006, Martijn van Oosterhout wrote: On Wed, Jun 28, 2006 at 01:09:23PM +0400, Teodor Sigaev wrote: We tested gin with 5 millions records (but not a wikipedia's text, but with blog records which is usually mush shorter) and index creation time was near 8 hours on Dual Xeon 3.2,

Re: [HACKERS] GIN index creation extremely slow ?

2006-06-28 Thread Teodor Sigaev
Tom did commit a patch a while ago which made a huge difference in index creation time for tsearch by changing one routine. I don't know if it got backpatched, so it might be worth checking people are working on the same version. I saw that patch, but I still think that 7 minutes is too small

[HACKERS] Help with casting and comparing.

2006-06-28 Thread Tzahi Fadida
Hi, I need help finding out how to determine if two types are equality compatible and compare them. I am using the following call to check for equality between two values: DatumGetBool(\ FunctionCall2((fctx-tupleSetAttEQFunctions[attID]-eq_opr_finfo)\ , lvalue, rvalue)) The

Re: [HACKERS] SO_SNDBUF size is small on win32?

2006-06-28 Thread Yoshiyuki Asaba
Hi, From: Tom Lane [EMAIL PROTECTED] Subject: Re: [HACKERS] SO_SNDBUF size is small on win32? Date: Tue, 27 Jun 2006 14:43:57 -0400 What I would think might help is a patch on the libpq side (because it *does* use a nonblocking socket) to avoid sending more than 8K per WSASend call.

Re: [HACKERS] SO_SNDBUF size is small on win32?

2006-06-28 Thread Tom Lane
Yoshiyuki Asaba [EMAIL PROTECTED] writes: I think libpq interface does not use non-blocking socket. Not unless the Windows port has also disabled pg_set_noblock ... regards, tom lane ---(end of broadcast)--- TIP 3: Have

Re: [HACKERS] Help with casting and comparing.

2006-06-28 Thread Martijn van Oosterhout
On Wed, Jun 28, 2006 at 03:25:57PM +0300, Tzahi Fadida wrote: Hi, I need help finding out how to determine if two types are equality compatible and compare them. snip Currently i only allow two values only of the same type but i wish to allow to compare values like 20.2=?20.2 or 20=?20.0

[HACKERS] optimizing constant quals within outer joins

2006-06-28 Thread Phil Frost
I have an optimization I'd like to see which I think should be pretty easy for someone familiar with the planner code to implement. My situation is this: I have an application using veil[1]. Essentially, I have a schema private and another public. Private contains regular tables, where private

Re: [HACKERS] optimizing constant quals within outer joins

2006-06-28 Thread Martijn van Oosterhout
On Wed, Jun 28, 2006 at 10:35:37AM -0400, Phil Frost wrote: I have an optimization I'd like to see which I think should be pretty easy for someone familiar with the planner code to implement. My situation is this: I have an application using veil[1]. Essentially, I have a schema private and

Re: [HACKERS] Help with casting and comparing.

2006-06-28 Thread Tom Lane
Martijn van Oosterhout kleptog@svana.org writes: On Wed, Jun 28, 2006 at 03:25:57PM +0300, Tzahi Fadida wrote: I need help finding out how to determine if two types are equality compatible and compare them. Fortunatly the backend contains functions that do all this already. Check out

Re: [HACKERS] optimizing constant quals within outer joins

2006-06-28 Thread Phil Frost
On Wed, Jun 28, 2006 at 05:11:59PM +0200, Martijn van Oosterhout wrote: On Wed, Jun 28, 2006 at 10:35:37AM -0400, Phil Frost wrote: I have an optimization I'd like to see which I think should be pretty easy for someone familiar with the planner code to implement. My situation is this: I

Re: [HACKERS] optimizing constant quals within outer joins

2006-06-28 Thread Tom Lane
Phil Frost [EMAIL PROTECTED] writes: The planner in fact can move the function around without changing the output. Not when it's within the nullable side of an outer join --- moving a WHERE clause up out of that would make the difference between no row out, and a null-extended row out, which

Re: [HACKERS] SO_SNDBUF size is small on win32?

2006-06-28 Thread Yoshiyuki Asaba
From: Tom Lane [EMAIL PROTECTED] Subject: Re: [HACKERS] SO_SNDBUF size is small on win32? Date: Wed, 28 Jun 2006 09:54:21 -0400 Yoshiyuki Asaba [EMAIL PROTECTED] writes: I think libpq interface does not use non-blocking socket. Not unless the Windows port has also disabled pg_set_noblock

[HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Tom Lane
Buildfarm member platypus is showing a regression failure that I'm surprised we have not seen before: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=platypusdt=2006-06-28%2014:05:01 Basically what this is showing is that when there is more than one referencing table, the order in which things

Re: [HACKERS] optimizing constant quals within outer joins

2006-06-28 Thread Phil Frost
On Wed, Jun 28, 2006 at 11:40:52AM -0400, Tom Lane wrote: Phil Frost [EMAIL PROTECTED] writes: The planner in fact can move the function around without changing the output. Not when it's within the nullable side of an outer join --- moving a WHERE clause up out of that would make the

[HACKERS] Index corruption

2006-06-28 Thread Marc Munro
We have now experienced index corruption on two separate but identical slony clusters. In each case the slony subscriber failed after attempting to insert a duplicate record. In each case reindexing the sl_log_1 table on the provider fixed the problem. The latest occurrence was on our

[HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as primary keys

2006-06-28 Thread Thomas Hallgren
Martijn van Oosterhout wrote: On Wed, Jun 28, 2006 at 01:56:47PM +0200, Thomas Hallgren wrote: A user that is trusted with installing a C-function in the backend is free to scan the process memory anyway so in what way did that increase the security? IMHO, the only relevant security in that

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Alvaro Herrera
Tom Lane wrote: 1. Find a way to make the processing order consistent (eg by driving it off OID ordering). Doesn't seem easy, but maybe I'm missing an idea. Hmm, what about 1. get the complete list of tables to truncate, AccessShareLock'ed, get their names 2. release locks 3. sort the list

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as primary keys

2006-06-28 Thread Jim C. Nasby
On Wed, Jun 28, 2006 at 06:39:16PM +0200, Thomas Hallgren wrote: Instead, I would like to humbly request the inclusion of a UUID datatype (or an opaque 128 bit datatype) in the core package. It's increasingly ISTM that we get enough requests for this that it's probably worth doing. -- Jim C.

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: 1. Find a way to make the processing order consistent (eg by driving it off OID ordering). Doesn't seem easy, but maybe I'm missing an idea. Hmm, what about 1. get the complete list of tables to truncate, AccessShareLock'ed, get

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as primary keys

2006-06-28 Thread mark
On Wed, Jun 28, 2006 at 06:39:16PM +0200, Thomas Hallgren wrote: Instead, I would like to humbly request the inclusion of a UUID datatype (or an opaque 128 bit datatype) in the core package. It's increasingly common and some databases (MS SQLServer) already have built in support for it. We

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Jim C. Nasby
On Wed, Jun 28, 2006 at 01:13:42PM -0400, Tom Lane wrote: One thing I was toying with was to add an index to pg_constraint on, say, (confrelid, conrelid), and to replace the existing seqscans for FK constraints with scans using this index. The second-column ordering would guarantee everybody

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as

2006-06-28 Thread A.M.
On Wed, June 28, 2006 1:14 pm, [EMAIL PROTECTED] wrote: On Wed, Jun 28, 2006 at 06:39:16PM +0200, Thomas Hallgren wrote: Instead, I would like to humbly request the inclusion of a UUID datatype (or an opaque 128 bit datatype) in the core package. It's increasingly common and some databases

Re: [HACKERS] Index corruption

2006-06-28 Thread Tom Lane
Marc Munro [EMAIL PROTECTED] writes: We have now experienced index corruption on two separate but identical slony clusters. In each case the slony subscriber failed after attempting to insert a duplicate record. In each case reindexing the sl_log_1 table on the provider fixed the problem.

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as

2006-06-28 Thread Jim C. Nasby
On Wed, Jun 28, 2006 at 01:18:39PM -0400, A.M. wrote: On Wed, June 28, 2006 1:14 pm, [EMAIL PROTECTED] wrote: On Wed, Jun 28, 2006 at 06:39:16PM +0200, Thomas Hallgren wrote: Instead, I would like to humbly request the inclusion of a UUID datatype (or an opaque 128 bit datatype) in the

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Andrew Dunstan
Tom Lane wrote: Buildfarm member platypus is showing a regression failure that I'm surprised we have not seen before: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=platypusdt=2006-06-28%2014:05:01 Basically what this is showing is that when there is more than one referencing table, the

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as

2006-06-28 Thread mark
On Wed, Jun 28, 2006 at 12:38:50PM -0500, Jim C. Nasby wrote: On Wed, Jun 28, 2006 at 01:18:39PM -0400, A.M. wrote: On Wed, June 28, 2006 1:14 pm, [EMAIL PROTECTED] wrote: On Wed, Jun 28, 2006 at 06:39:16PM +0200, Thomas Hallgren wrote: Instead, I would like to humbly request the

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as

2006-06-28 Thread Andrew Dunstan
Jim C. Nasby wrote: On Wed, Jun 28, 2006 at 01:18:39PM -0400, A.M. wrote: On Wed, June 28, 2006 1:14 pm, [EMAIL PROTECTED] wrote: On Wed, Jun 28, 2006 at 06:39:16PM +0200, Thomas Hallgren wrote: Instead, I would like to humbly request the inclusion of a UUID datatype (or an

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Alvaro Herrera
Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: 1. Find a way to make the processing order consistent (eg by driving it off OID ordering). Doesn't seem easy, but maybe I'm missing an idea. Hmm, what about 1. get the complete list of tables to truncate,

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as

2006-06-28 Thread Alvaro Herrera
[EMAIL PROTECTED] wrote: I've downloaded the version off pgfoundry.org. It is broken. It leaks memory, and if memory is correct it can cause the client to core dump. Also it couldn't possibly be included in core, since it's based on a GPL'ed UUID library. If you have a more appropiately

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: If this were a significant risk wouldn't we have seen many such failures before now? Hard to tell. It's possibly architecture-dependent, for one thing (MAXALIGN will affect space availability). Since this happened in a parallel regression run, it

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Alvaro Herrera
Alvaro Herrera wrote: Tom Lane wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: 1. Find a way to make the processing order consistent (eg by driving it off OID ordering). Doesn't seem easy, but maybe I'm missing an idea. Hmm, what about 1. get the complete

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: One thing I was toying with was to add an index to pg_constraint on, say, (confrelid, conrelid), and to replace the existing seqscans for FK constraints with scans using this index. I think there is more than one place that would

[HACKERS] Single Index Tuple Chain (SITC) method

2006-06-28 Thread Bruce Momjian
bruce wrote: Greg Stark wrote: Bruce Momjian [EMAIL PROTECTED] writes: PFC wrote: My idea is that if an UPDATE places the new tuple on the same page as the old tuple, it will not create new index entries for any indexes where the key doesn't change.

Re: [HACKERS] Fixed length datatypes. WAS [GENERAL] UUID's as

2006-06-28 Thread Jim C. Nasby
On Wed, Jun 28, 2006 at 01:49:55PM -0400, Andrew Dunstan wrote: Personally I don't buy the misuse objection - we already have plenty of things that can be misused. As long as there is a reasonable valid use and we can make it portable enough, I think there is a good case for including it.

Re: [HACKERS] Single Index Tuple Chain (SITC) method

2006-06-28 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Here is an overview of the SITC method: http://momjian.us/cgi-bin/pgsitc A pretty fundamental problem is that the method assumes it's OK to change the CTID of a live tuple (by swapping its item pointer with some expired version). It is not --- this

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes: Tom Lane wrote: Buildfarm member platypus is showing a regression failure that I'm surprised we have not seen before: http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=platypusdt=2006-06-28%2014:05:01 If this were a significant risk wouldn't we have

Re: [HACKERS] Instability in TRUNCATE regression test

2006-06-28 Thread Tom Lane
I wrote: Alvaro Herrera [EMAIL PROTECTED] writes: Tom Lane wrote: One thing I was toying with was to add an index to pg_constraint on, say, (confrelid, conrelid), and to replace the existing seqscans for FK constraints with scans using this index. I think there is more than one place that

Re: [HACKERS] [GENERAL] UUID's as primary keys

2006-06-28 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Martijn van Oosterhout kleptog@svana.org writes: The input functions get it, the output functions (bpcharout, bpcharsend, etc) don't. Which makes it kind of hard to print a raw value if you don't know how long it's going to be. They used to, but that

Re: [HACKERS] optimizing constant quals within outer joins

2006-06-28 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Phil Frost [EMAIL PROTECTED] writes: The planner in fact can move the function around without changing the output. Not when it's within the nullable side of an outer join --- moving a WHERE clause up out of that would make the difference between no row

Re: [HACKERS] [GENERAL] UUID's as primary keys

2006-06-28 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Hm, so it could be stored on disk without the length header as long as the length header is added to the in-memory representation? I don't think the type system has hooks for reading and storing data to disk though. No, it doesn't, and we'd pay a nonzero