Re: [HACKERS] In Japan with Josh Berkus

2006-02-16 Thread Luke Lonergan
Transcript: Josh: "Can people in the back hear me? Thank you for hosting me in Tokyo, it's a lot of fun for me to come over here. It is also an extremely exciting time to be a PostgreSQL developer. It's just amazing how something that was a hobby, a sideline, um a ... thing that I and pe

Re: [HACKERS] In Japan with Josh Berkus

2006-02-16 Thread Luke Lonergan
Arigato gozaimas! - Luke From: [EMAIL PROTECTED] on behalf of Satoshi Nagayasu Sent: Thu 2/16/2006 10:17 PM To: Luke Lonergan Cc: Bruce Momjian; PostgreSQL-development Subject: Re: [HACKERS] In Japan with Josh Berkus Hi all, Josh's talk is now available at:

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Ron
At 01:47 PM 2/16/2006, Ron wrote: At 12:19 PM 2/16/2006, Scott Lamb wrote: On Feb 16, 2006, at 8:32 AM, Ron wrote: Let's pretend that we have the typical DB table where rows are ~2-4KB apiece. 1TB of storage will let us have 256M-512M rows in such a table. A 32b hash code can be assigned to e

Re: [HACKERS] In Japan with Josh Berkus

2006-02-16 Thread Satoshi Nagayasu
Hi all, Josh's talk is now available at: http://snaga.org/01_Josh_Berkus.mp3 This file is very long, and an interpreter's voice to interpret into Japanese is also recorded. If you want to learn Japanese, please try it! :) Thanks. Luke Lonergan wrote: Drink Sake and eat some Yakitori for us f

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread David Lang
On Thu, 16 Feb 2006, Mark Lewis wrote: On Thu, 2006-02-16 at 17:51 -0500, Greg Stark wrote: Data types which could probably provide a useful function for f would be int2, int4, oid, and possibly int8 and text (at least for SQL_ASCII). How exactly do you imagine doing this for text? I could s

Re: [HACKERS] In Japan with Josh Berkus

2006-02-16 Thread Luke Lonergan
Title: Re: [HACKERS] In Japan with Josh Berkus Drink Sake and eat some Yakitori for us folks in the west.  Maybe shake a robot hand or two while you’re at it :-) - Luke On 2/16/06 2:14 PM, "Bruce Momjian" wrote: FYI, Josh Berkus and I are in Japan to give some presentations.  We return to th

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Mark Lewis
On Thu, 2006-02-16 at 17:51 -0500, Greg Stark wrote: > > > Data types which could probably provide a useful function for f would be > > > int2, int4, oid, and possibly int8 and text (at least for SQL_ASCII). > > How exactly do you imagine doing this for text? > > I could see doing it for char(n)/

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Greg Stark
Markus Schaber <[EMAIL PROTECTED]> writes: > Hmm, to remove redundancy, I'd change the <= to a < and define: > > if a==b then f(a)==f(b) > if a > > Data types which could probably provide a useful function for f would be > > int2, int4, oid, and possibly int8 and text (at least for SQL_ASCII).

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 02:17:36PM -0800, Mark Lewis wrote: > It seems that instead of maintaining a different sorting code path for > each data type, you could get away with one generic path and one > (hopefully faster) path if you allowed data types to optionally support > a 'sortKey' interface b

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Markus Schaber
Hi, Mark, Mark Lewis schrieb: > It seems that instead of maintaining a different sorting code path for > each data type, you could get away with one generic path and one > (hopefully faster) path if you allowed data types to optionally support > a 'sortKey' interface by providing a function f whi

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Mark Lewis
On Thu, 2006-02-16 at 12:15 -0500, Tom Lane wrote: > Once or twice we've kicked around the idea of having some > datatype-specific sorting code paths alongside the general-purpose one, > but I can't honestly see this as being workable from a code maintenance > standpoint. > >

[HACKERS] In Japan with Josh Berkus

2006-02-16 Thread Bruce Momjian
FYI, Josh Berkus and I are in Japan to give some presentations. We return to the USA on February 23. -- Bruce Momjian| http://candle.pha.pa.us pgman@candle.pha.pa.us | (610) 359-1001 + If your life is a hard drive, | 13 Roberts Road + Christ

Re: [HACKERS] how solve diff of API counstruct_md_array between

2006-02-16 Thread Joe Conway
Martijn van Oosterhout wrote: On Thu, Feb 16, 2006 at 08:36:34PM +0100, Pavel Stehule wrote: I use counstruct_md_array function in my Orafunc module. CVS version has diff def now. I am findig way for simple solution of maintaince source code for both version. I have PG_VERSION variable, but it'

[HACKERS] time waiting on patch queue

2006-02-16 Thread Andrew Dunstan
[redirecting to -hackers] Tom Lane wrote: At the moment it's not unusual for nontrivial patches to sit around for a month or two, unless they happen to attract special interest of one of the committers. Yeah. That's just horrible. It makes people lose interest and can hurt because of bi

Re: [HACKERS] how solve diff of API counstruct_md_array between 8.1 and 8.2?

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 08:36:34PM +0100, Pavel Stehule wrote: > Hello > > I use counstruct_md_array function in my Orafunc module. CVS version has > diff def now. I am findig way for simple solution of maintaince source code > for both version. I have PG_VERSION variable, but it's unusable. Is

[HACKERS] how solve diff of API counstruct_md_array between 8.1 and 8.2?

2006-02-16 Thread Pavel Stehule
Hello I use counstruct_md_array function in my Orafunc module. CVS version has diff def now. I am findig way for simple solution of maintaince source code for both version. I have PG_VERSION variable, but it's unusable. Is there way for contrib's autors differentiate PostgreSQL versions? I don

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Neil Conway
On Thu, 2006-02-16 at 12:35 +0100, Steinar H. Gunderson wrote: > glibc-2.3.5/stdlib/qsort.c: > > /* Order size using quicksort. This implementation incorporates > four optimizations discussed in Sedgewick: > > I can't see any references to merge sort in there at all. stdlib/qsort.c defin

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Ron
At 12:19 PM 2/16/2006, Scott Lamb wrote: On Feb 16, 2006, at 8:32 AM, Ron wrote: Let's pretend that we have the typical DB table where rows are ~2-4KB apiece. 1TB of storage will let us have 256M-512M rows in such a table. A 32b hash code can be assigned to each row value such that only exactl

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Dann Corbit
> -Original Message- > From: [EMAIL PROTECTED] [mailto:pgsql-hackers- > [EMAIL PROTECTED] On Behalf Of Markus Schaber > Sent: Thursday, February 16, 2006 5:45 AM > To: pgsql-performance@postgresql.org; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] qsort again (was Re: [PERFORM] Stra

Re: [HACKERS] Generating config stuff from single source

2006-02-16 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: >> The m4 idea seems attractive to me because that's already effectively >> required as part of the autoconf infrastructure (and I think bison >> uses it too these days). > That is true, but I'm

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Scott Lamb
On Feb 16, 2006, at 8:32 AM, Ron wrote: Let's pretend that we have the typical DB table where rows are ~2-4KB apiece. 1TB of storage will let us have 256M-512M rows in such a table. A 32b hash code can be assigned to each row value such that only exactly equal rows will have the same hash

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Tom Lane
"Craig A. James" <[EMAIL PROTECTED]> writes: > You can also use this trick when the optimizer is asked for "fastest first > result." Say you have a cursor on a column of numbers with good > distribution. If you do a bucket sort on the first two or three digits only, > you know the first "page"

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 11:32:55AM -0500, Ron wrote: > At 10:52 AM 2/16/2006, Ron wrote: > >In fact we can do better. > >Using hash codes or what-not to map datums to keys and then sorting > >just the keys and the pointers to those datums followed by an > >optional final pass where we do the actu

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Ron
At 10:52 AM 2/16/2006, Ron wrote: At 09:48 AM 2/16/2006, Martijn van Oosterhout wrote: Where this does become interesting is where we can convert a datum to an integer such that if f(A) > f(B) then A > B. Then we can sort on f(X) first with just integer comparisons and then do a full tuple comp

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Craig A. James
Markus Schaber wrote: Ron wrote: ...and of course if you know enough about the data to be sorted so as to constrain it appropriately, one should use a non comparison based O(N) sorting algorithm rather than any of the general comparison based O(NlgN) methods. Sounds interesting, could you give

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Tom Lane
Ron <[EMAIL PROTECTED]> writes: > Your cost comment basically agrees with mine regarding the cost of > random memory accesses. The good news is that the number of datums > to be examined during the pivot choosing process is small enough that > the datums can fit into CPU cache while the pointer

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create

2006-02-16 Thread Ron
At 09:48 AM 2/16/2006, Martijn van Oosterhout wrote: On Thu, Feb 16, 2006 at 08:22:55AM -0500, Ron wrote: > 3= Especially in modern systems where the gap between internal CPU > bandwidth and memory bandwidth is so great, the overhead of memory > accesses for comparisons and moves is the majority

Re: [HACKERS] Generating config stuff from single source

2006-02-16 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: >> That's fine for users, but what new demands are you about to place on >> developers? Does this require tools not already needed in order to >> build from a CVS pull? (There's sure no xsltproc

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 08:22:55AM -0500, Ron wrote: > 3= Especially in modern systems where the gap between internal CPU > bandwidth and memory bandwidth is so great, the overhead of memory > accesses for comparisons and moves is the majority of the overhead > for both the pivot choosing and th

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Jonah H. Harris
Last night I implemented a non-recursive introsort in C... let me test it a bit more and then I'll post it here for everyone else to try out.On 2/16/06, Markus Schaber <[EMAIL PROTECTED]> wrote:Hi, Ron, Ron wrote:> ...and of course if you know enough about the data to be sorted so as to> constrain

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Markus Schaber
Hi, Ron, Ron wrote: > ...and of course if you know enough about the data to be sorted so as to > constrain it appropriately, one should use a non comparison based O(N) > sorting algorithm rather than any of the general comparison based > O(NlgN) methods. Sounds interesting, could you give us som

Re: [HACKERS] Doubt in parser

2006-02-16 Thread Alvaro Herrera
Dhanaraj wrote: > hi > > currently i looking at the postgres src code. I saw the scanner and > parser implemetations at two different places (src/backend/parser/ and > /src/bakend/bootstrp). Can anybody tell me the purpose of having two > phases?? or will this help to parse the queries at dif

Re: [PERFORM] [HACKERS] qsort again

2006-02-16 Thread Ron
At 07:10 AM 2/16/2006, Florian Weimer wrote: * Neil Conway: > On Wed, 2006-02-15 at 18:28 -0500, Tom Lane wrote: >> It seems clear that our qsort.c is doing a pretty awful job of picking >> qsort pivots, while glibc is mostly managing not to make that mistake. >> I haven't looked at the glibc co

Re: [HACKERS] qsort again (was Re: [PERFORM] Strange Create Index

2006-02-16 Thread Ron
At 06:35 AM 2/16/2006, Steinar H. Gunderson wrote: On Wed, Feb 15, 2006 at 11:30:54PM -0500, Ron wrote: > Even better (and more easily scaled as the number of GPR's in the CPU > changes) is to use > the set {L; L+1; L+2; t>>1; R-2; R-1; R} > This means that instead of 7 random memory accesses, we

Re: [PERFORM] [HACKERS] qsort again

2006-02-16 Thread Sven Geisler
Martijn van Oosterhout schrieb: Last time around there were a number of different algorithms tested. Did anyone run those tests while getting it to count the number of actual comparisons (which could easily swamp the time taken to do the actual sort in some cases)? The last time I did such te

Re: [HACKERS] qsort again

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 01:10:48PM +0100, Florian Weimer wrote: > * Neil Conway: > > > On Wed, 2006-02-15 at 18:28 -0500, Tom Lane wrote: > >> It seems clear that our qsort.c is doing a pretty awful job of picking > >> qsort pivots, while glibc is mostly managing not to make that mistake. > >> I h

Re: [HACKERS] Doubt in parser

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 06:07:25PM +0530, Dhanaraj wrote: > hi > > currently i looking at the postgres src code. I saw the scanner and > parser implemetations at two different places (src/backend/parser/ and > /src/bakend/bootstrp). Can anybody tell me the purpose of having two > phases?? or

[HACKERS] Doubt in parser

2006-02-16 Thread Dhanaraj
hi currently i looking at the postgres src code. I saw the scanner and parser implemetations at two different places (src/backend/parser/ and /src/bakend/bootstrp). Can anybody tell me the purpose of having two phases?? or will this help to parse the queries at different levels? Thanks Dha

Re: [HACKERS] qsort again

2006-02-16 Thread Florian Weimer
* Neil Conway: > On Wed, 2006-02-15 at 18:28 -0500, Tom Lane wrote: >> It seems clear that our qsort.c is doing a pretty awful job of picking >> qsort pivots, while glibc is mostly managing not to make that mistake. >> I haven't looked at the glibc code yet to see what they are doing >> differentl

Re: [HACKERS] Generating config stuff from single source

2006-02-16 Thread Martijn van Oosterhout
On Thu, Feb 16, 2006 at 02:36:01AM +0100, Peter Eisentraut wrote: > We are currently maintaining information about configuration parameters > in at least three places: the documentation, guc.c, and > postgresql.conf.sample. I would like to generate these from a single > source. Computationally

Re: [HACKERS] Blog post on EnterpriseDB...maybe off topic

2006-02-16 Thread Lukas Smith
Christopher Kings-Lynne wrote: http://www.flamingspork.com/blog/2006/02/16/enterprisedb-where-is-the-source/ Any comments on this? Is he referring to EnterpriseDB extensions that they don't make public? I think so. Trying to "battle" the perception that EnterpriseDB is an open source data

Re: [HACKERS] Generating config stuff from single source

2006-02-16 Thread Peter Eisentraut
Am Donnerstag, 16. Februar 2006 02:50 schrieb Tom Lane: > That's fine for users, but what new demands are you about to place on > developers? Does this require tools not already needed in order to > build from a CVS pull? (There's sure no xsltproc on this machine...) It is to be expected that so

Re: [HACKERS] Blog post on EnterpriseDB...maybe off topic

2006-02-16 Thread Rick Gigger
Any comments on this? Is he referring to EnterpriseDB extensions that they don't make public? I've noticed a lot of press lately is mentioning their name next to ingres as an alternative to MySQL, so the MySQL folks might be feeling some Postgres heat from their direction. I also wonder w

Re: [HACKERS] Blog post on EnterpriseDB...maybe off topic

2006-02-16 Thread Luke Lonergan
Christoper, On 2/15/06 11:14 PM, "Christopher Kings-Lynne" <[EMAIL PROTECTED]> wrote: > Any comments on this? Is he referring to EnterpriseDB extensions that > they don't make public? I've noticed a lot of press lately is mentioning their name next to ingres as an alternative to MySQL, so the M