Re: [HACKERS] 7.3 schedule

2002-04-17 Thread Bruce Momjian
I have added these emails to TODO.detail/prepare. --- Karel Zak wrote: > On Fri, Apr 12, 2002 at 12:41:34AM -0400, Neil Conway wrote: > > On Fri, 12 Apr 2002 12:58:01 +0900 > > "Hiroshi Inoue" <[EMAIL PROTECTED]> wrote: > >

Re: Scanner performance (was Re: [HACKERS] 7.3 schedule)

2002-04-16 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Top ten calls: > % cumulative self self total > time seconds secondscalls ms/call ms/call name > 36.95 9.87 9.87 74882482 0.00 0.00 pq_getbyte > 22.80 15.96 6.09 11 553.64 1450.9

Re: Scanner performance (was Re: [HACKERS] 7.3 schedule)

2002-04-16 Thread Bruce Momjian
Peter Eisentraut wrote: > The string literals didn't contain any backslashes, so scanstr is > operating in the best-case scenario here. But for arbitary binary data we > need some escape mechanism, so I don't see much room for improvement > there. > > It seems the real bottleneck is the excessiv

Re: Scanner performance (was Re: [HACKERS] 7.3 schedule)

2002-04-16 Thread Peter Eisentraut
Tom Lane writes: > The regression tests contain no very-long literals. The results I was > referring to concerned cases with string (BLOB) literals in the > hundreds-of-K range; it seems that the per-character loop in the flex > lexer starts to look like a bottleneck when you have tokens that mu

Re: [HACKERS] 7.3 schedule

2002-04-14 Thread Curt Sampson
On Sun, 14 Apr 2002, Barry Lind wrote: > But since the syntax for prepare is: PREPARE AS you > can't easily reuse sql prepared by other PreparedStatement objects since > you don't know if the sql you are about to execute has or has not yet > been prepared or what was used in that prepare. T

Re: [HACKERS] 7.3 schedule

2002-04-14 Thread Barry Lind
Curt Sampson wrote: > On Thu, 11 Apr 2002, Barry Lind wrote: > > >>I'm not sure that JDBC would use this feature directly. When a >>PreparableStatement is created in JDBC there is nothing that indicates >>how many times this statement is going to be used. Many (most IMHO) >>will be used only

Re: [HACKERS] 7.3 schedule

2002-04-14 Thread Curt Sampson
On Thu, 11 Apr 2002, Barry Lind wrote: > I'm not sure that JDBC would use this feature directly. When a > PreparableStatement is created in JDBC there is nothing that indicates > how many times this statement is going to be used. Many (most IMHO) > will be used only once. Well, the particular

Re: [HACKERS] 7.3 schedule

2002-04-14 Thread Brian Bruns
On 13 Apr 2002, Hannu Krosing wrote: > On Fri, 2002-04-12 at 03:04, Brian Bruns wrote: > > On 11 Apr 2002, Hannu Krosing wrote: > > > > > IIRC someone started work on modularising the network-related parts with > > > a goal of supporting DRDA (DB2 protocol) and others in future. > > > > That wa

Re: [HACKERS] 7.3 schedule

2002-04-14 Thread Karel Zak
On Fri, Apr 12, 2002 at 12:51:26PM -0400, Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Certainly a shared cache would be good for apps that connect to issue a > > single query frequently. In such cases, there would be no local cache > > to use. > > We have enough other problem

Re: [HACKERS] 7.3 schedule

2002-04-13 Thread Neil Conway
On Sat, 13 Apr 2002 14:21:50 +0800 "Christopher Kings-Lynne" <[EMAIL PROTECTED]> wrote: > Could this cache mechanism be used to make views fast as well? The current PREPARE/EXECUTE code will speed up queries that use rules of any kind, including views: the query plan is cached after it has been r

Re: [HACKERS] 7.3 schedule

2002-04-13 Thread Tom Lane
"Christopher Kings-Lynne" <[EMAIL PROTECTED]> writes: > thought out way of predicting/limiting their size. (2) How the heck do > you get rid of obsoleted cached plans, if the things stick around in > shared memory even after you start a new backend? (3) A shared cache > requires locking; content

Re: [HACKERS] 7.3 schedule

2002-04-13 Thread Hannu Krosing
On Fri, 2002-04-12 at 03:04, Brian Bruns wrote: > On 11 Apr 2002, Hannu Krosing wrote: > > > IIRC someone started work on modularising the network-related parts with > > a goal of supporting DRDA (DB2 protocol) and others in future. > > That was me, although I've been bogged down lately, and hav

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Christopher Kings-Lynne
> > thought out way of predicting/limiting their size. (2) How the heck do > > you get rid of obsoleted cached plans, if the things stick around in > > shared memory even after you start a new backend? (3) A shared cache > > requires locking; contention among multiple backends to access that > >

Re: Scanner performance (was Re: [HACKERS] 7.3 schedule)

2002-04-12 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > My profiles show that the work spent in the scanner is really minuscule > compared to everything else. Under ordinary circumstances I think that's true ... > (The profile data is from a run of all the regression test files in order > in one session.

Scanner performance (was Re: [HACKERS] 7.3 schedule)

2002-04-12 Thread Peter Eisentraut
Tom Lane writes: > We do have some numbers suggesting that the per-character loop in the > lexer is slow enough to be a problem with very long literals. That is > the overhead that might be avoided with a special protocol. Which loop is that? Doesn't the scanner use buffered input anyway? > H

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Dann Corbit
-Original Message- From: Bruce Momjian [mailto:[EMAIL PROTECTED]] Sent: Friday, April 12, 2002 2:38 PM To: Tom Lane Cc: Neil Conway; [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: Re: [HACKERS] 7.3 schedule Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > &

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Oh, are you thinking that one backend would do the PREPARE and another > > one the EXECUTE? I can't see that working at all. > > Uh, why exactly were you advocating a shared cache then? Wouldn't that > be exactly the *point* of a s

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Oh, are you thinking that one backend would do the PREPARE and another > one the EXECUTE? I can't see that working at all. Uh, why exactly were you advocating a shared cache then? Wouldn't that be exactly the *point* of a shared cache?

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Bruce Momjian
Neil Conway wrote: > On Fri, 12 Apr 2002 12:21:04 -0400 (EDT) > "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > > Tom Lane wrote: > > > A per-backend cache kept in local memory avoids all of these problems, > > > and I have seen no numbers to make me think that a shared plan cache > > > would achieve

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Neil Conway
On Fri, 12 Apr 2002 12:21:04 -0400 (EDT) "Bruce Momjian" <[EMAIL PROTECTED]> wrote: > Tom Lane wrote: > > A per-backend cache kept in local memory avoids all of these problems, > > and I have seen no numbers to make me think that a shared plan cache > > would achieve significantly more performance

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Certainly a shared cache would be good for apps that connect to issue a > single query frequently. In such cases, there would be no local cache > to use. We have enough other problems with the single-query-per-connection scenario that I see no reason t

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Bruce Momjian
Barry Lind wrote: > Oracle's implementation is a shared cache for all plans. This was > introduced in Oracle 6 or 7 (I don't remember which anymore). The net > effect was that in general there was a significant performance > improvement with the shared cache. However poorly written apps can

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Barry Lind
Tom Lane wrote: > Yes, that is the part that was my sticking point last time around. > (1) Because shared memory cannot be extended on-the-fly, I think it is > a very bad idea to put data structures in there without some well > thought out way of predicting/limiting their size. (2) How the heck

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Bruce Momjian
Tom Lane wrote: > Karel Zak <[EMAIL PROTECTED]> writes: > > * The patch allows store query-planns to shared memory and is > > possible EXECUTE it at more backends (over same DB) and planns > > are persistent across connetions. For this feature I create special > > memory con

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Barry Lind
Neil Conway wrote: > I would suggest using it any time you're executing the same query > plan a large number of times. In my experience, this is very common. > There are already hooks for this in many client interfaces: e.g. > PrepareableStatement in JDBC and $dbh->prepare() in Perl DBI. I'm not

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Tom Lane
Karel Zak <[EMAIL PROTECTED]> writes: > * The patch allows store query-planns to shared memory and is > possible EXECUTE it at more backends (over same DB) and planns > are persistent across connetions. For this feature I create special > memory context subsystem (like curre

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Hannu Krosing
On Thu, 2002-04-11 at 22:48, Tom Lane wrote: > Barry Lind <[EMAIL PROTECTED]> writes: > > ... > > Since we > > don't currently provide any information to the user on the relative cost > > of the parse, plan and execute phases, the end user is going to be > > guessing IMHO. > > You can in fact

Re: [HACKERS] 7.3 schedule

2002-04-12 Thread Karel Zak
On Fri, Apr 12, 2002 at 12:41:34AM -0400, Neil Conway wrote: > On Fri, 12 Apr 2002 12:58:01 +0900 > "Hiroshi Inoue" <[EMAIL PROTECTED]> wrote: > > > > Just a confirmation. > > Someone is working on PREPARE/EXECUTE ? > > What about Karel's work ? Right question :-) > I am. My work is based on

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Neil Conway
On Fri, 12 Apr 2002 12:58:01 +0900 "Hiroshi Inoue" <[EMAIL PROTECTED]> wrote: > Tom Lane wrote: > > > > I'm not sure I believe Hannu's numbers, but in any case they're fairly > > irrelevant to the argument about whether a special protocol is useful. > > He wasn't testing textually-long queries, b

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Hiroshi Inoue
Tom Lane wrote: > > I'm not sure I believe Hannu's numbers, but in any case they're fairly > irrelevant to the argument about whether a special protocol is useful. > He wasn't testing textually-long queries, but rather the planning > overhead, which is more or less independent of the length of any

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Tom Lane
Ashley Cambrell <[EMAIL PROTECTED]> writes: > I remember an email Hannu sent (I originally thought Tome sent it but I > found the email*) that said postgresql spends a lot of time parsing sql > (compared to oracle), so if the BE/FE and libpq were extended to support > pg_prepare / pg_bind, then it

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Brian Bruns
On 11 Apr 2002, Hannu Krosing wrote: > IIRC someone started work on modularising the network-related parts with > a goal of supporting DRDA (DB2 protocol) and others in future. That was me, although I've been bogged down lately, and haven't been able to get back to it. DRDA, btw, is not just a

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Neil Conway
On Thu, 11 Apr 2002 11:38:33 -0700 "Barry Lind" <[EMAIL PROTECTED]> wrote: > Neil Conway wrote: > > On Thu, 11 Apr 2002 16:25:24 +1000 > > "Ashley Cambrell" <[EMAIL PROTECTED]> wrote: > > > >>What are the chances that the BE/FE will be altered to take advantage of > >>prepare / execute? Or is it

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Ashley Cambrell
Neil Conway wrote: >On Thu, 11 Apr 2002 16:25:24 +1000 >"Ashley Cambrell" <[EMAIL PROTECTED]> wrote: > >>What are the chances that the BE/FE will be altered to take advantage of >>prepare / execute? Or is it something that will "never happen"? >> > >Is there a need for this? The current patch I'

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Barry Lind
Tom Lane wrote: > It would be interesting to see some stats for the large-BLOB scenarios > being debated here. You could get more support for the position that > something should be done if you had numbers to back it up. Below are some stats you did a few months ago when I was asking a rel

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Tom Lane
Barry Lind <[EMAIL PROTECTED]> writes: > ... > Since we > don't currently provide any information to the user on the relative cost > of the parse, plan and execute phases, the end user is going to be > guessing IMHO. You can in fact get that information fairly easily; set show_parser_stats, s

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Barry Lind
Neil Conway wrote: > On Thu, 11 Apr 2002 16:25:24 +1000 > "Ashley Cambrell" <[EMAIL PROTECTED]> wrote: > >>What are the chances that the BE/FE will be altered to take advantage of >>prepare / execute? Or is it something that will "never happen"? > > > Is there a need for this? The current pa

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Hannu Krosing
On Thu, 2002-04-11 at 18:14, Tom Lane wrote: > Neil Conway <[EMAIL PROTECTED]> writes: > > On the other hand, there are already a few reasons to make some > > changes to the FE/BE protocol (NOTIFY messages, transaction state, > > and now possibly PREPARE/EXECUTE -- anything else?). > > Passing EX

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > On the other hand, there are already a few reasons to make some > changes to the FE/BE protocol (NOTIFY messages, transaction state, > and now possibly PREPARE/EXECUTE -- anything else?). Passing EXECUTE parameters without having them go through the parse

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Marc G. Fournier
On Thu, 11 Apr 2002, Bruce Momjian wrote: > Is anyone feeling we have the 7.3 release nearing? I certainly am not. > I can imagine us going for several more months like this, perhaps > through August. seeing as how we just released v7.2, I don't see a v7.3 even going beta until end of Summer ..

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Neil Conway
On Thu, 11 Apr 2002 16:25:24 +1000 "Ashley Cambrell" <[EMAIL PROTECTED]> wrote: > What are the chances that the BE/FE will be altered to take advantage of > prepare / execute? Or is it something that will "never happen"? Is there a need for this? The current patch I'm working on just does everyt

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread Peter Eisentraut
Nicolas Bazin writes: > For the next release and package it would be good to differentiate the > release candidate to the proper release. They do have different names. -- Peter Eisentraut [EMAIL PROTECTED] ---(end of broadcast)--- TIP 3: if p

Re: [HACKERS] 7.3 schedule

2002-04-11 Thread bpalmer
> We'll need a good beta period this time, because of: I know it's a sore subject, but how about "ALTER TABLE DROP COLUMN" this time around? I've been hearing about it for years now. :) - brandon c: 646-456-5455

Re: [HACKERS] 7.3 schedule

2002-04-10 Thread Nicolas Bazin
release. - Original Message - From: "Ashley Cambrell" <[EMAIL PROTECTED]> To: "PostgreSQL-development" <[EMAIL PROTECTED]> Sent: Thursday, April 11, 2002 4:25 PM Subject: Re: [HACKERS] 7.3 schedule > Christopher Kings-Lynne wrote: > > >>Is

Re: [HACKERS] 7.3 schedule

2002-04-10 Thread Ashley Cambrell
Christopher Kings-Lynne wrote: >>Is anyone feeling we have the 7.3 release nearing? >> > >No way! > >>I certainly am not. >>I can imagine us going for several more months like this, perhaps >>through August. >> > >Easily. I think that the critical path is Tom's schema support. > >We'll need a g

Re: [HACKERS] 7.3 schedule

2002-04-10 Thread Bruce Momjian
Christopher Kings-Lynne wrote: > > Is anyone feeling we have the 7.3 release nearing? > > No way! Good. > > I certainly am not. > > I can imagine us going for several more months like this, perhaps > > through August. > > Easily. I think that the critical path is Tom's schema support. > > W

Re: [HACKERS] 7.3 schedule

2002-04-10 Thread Christopher Kings-Lynne
> Is anyone feeling we have the 7.3 release nearing? No way! > I certainly am not. > I can imagine us going for several more months like this, perhaps > through August. Easily. I think that the critical path is Tom's schema support. We'll need a good beta period this time, because of: * Sch

[HACKERS] 7.3 schedule

2002-04-10 Thread Bruce Momjian
Is anyone feeling we have the 7.3 release nearing? I certainly am not. I can imagine us going for several more months like this, perhaps through August. -- Bruce Momjian| http://candle.pha.pa.us [EMAIL PROTECTED] | (610) 853-3000 + If your life is