Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Wed, Apr 12, 2006 at 05:00:17PM -0400, Tom Lane wrote: Issuer (name and certificate), validity dates, basic constraints, key usage, posslby fingerprint. I think that way madness lies --- do we really want to commit to re-inventing an SSL API that will cover anything someone might want

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Wed, Apr 12, 2006 at 05:25:47PM +0100, Dave Page wrote: The driver implements all versions of the wire protocol itself, but if libpq is available at runtime (it will dynamically load it on platforms that support it) it can use it for connection setup so features like SSL can be provided

Re: [HACKERS] Get explain output of postgresql in Tables

2006-04-13 Thread Dave Page
-Original Message- From: Alvaro Herrera [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 00:28 To: Dave Page Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Get explain output of postgresql in

Re: [HACKERS] Get explain output of postgresql in Tables

2006-04-13 Thread Dave Page
-Original Message- From: Jim C. Nasby [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 01:07 To: Dave Page; [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Get explain output of postgresql in Tables On Wed, Apr 12,

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Martijn van Oosterhout [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 07:58 To: Dave Page Cc: pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries On Wed, Apr 12, 2006 at

[HACKERS] Possible race in UnlockBuffers() and UnpinBuffer()

2006-04-13 Thread Qingqing Zhou
We have a wait-pin-to-1 mechanism in LockBufferForCleanup() like this: 1: bufHdr-wait_backend_pid = MyProcPid; 2: bufHdr-flags |= BM_PIN_COUNT_WAITER; 3: PinCountWaitBuf = bufHdr; 4: UnlockBufHdr_NoHoldoff(bufHdr); 5: LockBuffer(buffer, BUFFER_LOCK_UNLOCK); 6: /*

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote: Well, we had a pure custom implementation of the protocol, had a pure libpq based version and after much discussion decided that the best version of all was the hybrid as it allowed us to hijack features like SSL, Kerberos, pgpass et

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Martijn van Oosterhout [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 09:15 To: Dave Page Cc: pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries On Thu, Apr 13, 2006 at

Re: [HACKERS] Control File

2006-04-13 Thread Bruce Momjian
Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Bruno Almeida do Lago wrote: After that night, I started to ask myself if PostgreSQL should not have a control file to check if expected datafiles are where they should be and JUST warn about missing ones? I don't think this

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Martijn van Oosterhout (kleptog@svana.org) wrote: On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote: Well, we had a pure custom implementation of the protocol, had a pure libpq based version and after much discussion decided that the best version of all was the hybrid as it

Re: [HACKERS] Support Parallel Query Execution in Executor

2006-04-13 Thread Bruce Momjian
Enhanded TODO: * Experiment with multi-threaded backend better resource utilization This would allow a single query to make use of multiple CPU's or multiple I/O channels simultaneously. One idea is to create a background reader that can

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 06:44:12AM -0400, Stephen Frost wrote: Instead of having it hijack the libpq connection and implement the wireline protocol itself, why don't we work on fixing the problems (such as the double-copying that libpq requires) in libpq to allow the driver (and others!) to

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Stephen Frost [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 11:44 To: Martijn van Oosterhout Cc: Dave Page; pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries Instead of having

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Martijn van Oosterhout [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 11:54 To: Dave Page; pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries On Thu, Apr 13, 2006 at 06:44:12AM

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 12:12:25PM +0100, Dave Page wrote: Ok. I'm not sure what this double copying you're referring to is, The libpq driver copies results out of the PGresult struct into the internal QueryResult classes. With libpq out of the loop, data can go straight from the wire

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Martijn van Oosterhout [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 12:34 To: Dave Page Cc: pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries However, w.r.t. the copying, the

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Dave Page (dpage@vale-housing.co.uk) wrote: This has been the subject of discussion for many months and the concencus was that the most effective approach was the hybrid one which has now been moved into CVS tip. Those involved are fully aware of the maintenance issues of implementing the

Re: [HACKERS] please actualize FAQ, broken urls

2006-04-13 Thread Bruce Momjian
Pavel Stehule wrote: Hello 1.11) How can I learn SQL? ... There is also a nice tutorial at http://www.intermedia.net/support/sql/sqltut.shtm, at http://ourworld.compuserve.com/homepages/graeme_birchall/HTM_COOK.HTM, and at http://sqlcourse.com. first link is broken, second moved

Re: [HACKERS] please actualize FAQ, broken urls

2006-04-13 Thread Pavel Stehule
David Fetter dir patch two days ago Pavel From: Bruce Momjian pgman@candle.pha.pa.us To: Pavel Stehule [EMAIL PROTECTED] CC: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] please actualize FAQ, broken urls Date: Thu, 13 Apr 2006 08:09:40 -0400 (EDT) Pavel Stehule wrote: Hello 1.11)

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Stephen Frost [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 12:56 To: Dave Page Cc: Martijn van Oosterhout; pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries There was

Re: [HACKERS] please actualize FAQ, broken urls

2006-04-13 Thread Bruce Momjian
I updated the tutorial list to be a bulleted list, and added the new URL you mentioned below. --- Robert Treat wrote: On Friday 07 April 2006 16:10, Pavel Stehule wrote: Hello 1.11) How can I learn SQL? ... There

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Martijn van Oosterhout (kleptog@svana.org) wrote: On Thu, Apr 13, 2006 at 12:12:25PM +0100, Dave Page wrote: Ok. I'm not sure what this double copying you're referring to is, The libpq driver copies results out of the PGresult struct into the internal QueryResult classes. With

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 12:48:06PM +0100, Dave Page wrote: Anyhoo, as I've said, that approach has now been abandoned anyway in favour of Hiroshi's, so it's him you'd need to convince to change. The rest of us have only just started re-learning the code. Well, I quickly scanned the code in CVS

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Dave Page (dpage@vale-housing.co.uk) wrote: What does the wireline protocol implementation in the ODBC driver do that it can't get through libpq? I can certainly understand the double-copying issue (I complained about that myself when first starting to use libpq) but I think that

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Martijn van Oosterhout (kleptog@svana.org) wrote: Well, I quickly scanned the code in CVS to see what I could find out. Wow, that was quick. :) So in fact what you really want is libpq as a protocol decoder but want to manage your resultset yourself. And you want to be able to let users

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Dave Page
-Original Message- From: Stephen Frost [mailto:[EMAIL PROTECTED] Sent: 13 April 2006 14:03 To: Martijn van Oosterhout Cc: Dave Page; pgsql-hackers@postgresql.org; Hiroshi Inoue Subject: Re: [HACKERS] Practical impediment to supporting multiple SSL libraries * Martijn van

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 08:32:34AM -0400, Stephen Frost wrote: * Martijn van Oosterhout (kleptog@svana.org) wrote: Hmm, the simplest improvement I can think of is one where you register a callback that libpq calls whenever it has received a new tuple. You wouldn't want it on every tuple

[HACKERS] two-argument aggregates and SQL 2003

2006-04-13 Thread Sergey E. Koposov
Hello All, I just thought about implementing some two-argument aggregate functions from SQL 2003 (like CORR(x,y), REGR_SLOPE(x,y) etc...) ( http://www.wiscorp.com/SQL2003Features.pdf , page 10) 1) I looked into the architecture of how the aggregate functions are created and used, and it seemed

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Martijn van Oosterhout (kleptog@svana.org) wrote: On Thu, Apr 13, 2006 at 08:32:34AM -0400, Stephen Frost wrote: You wouldn't want it on every tuple as that'd get expensive through function calls. Why not? Internally we call pqAddTuple for every tuple, calling a user function instead is

Re: [HACKERS] plpgsql by default

2006-04-13 Thread Bruce Momjian
Marc G. Fournier wrote: On Tue, 11 Apr 2006, Joshua D. Drake wrote: Can you guarantee unequivocally that there are absolutely not security issues in plpgsql? Can you guarantee unequivocally that there are absolutely not security issues in PostgreSQL? No, but does that mean

Re: [HACKERS] GPUSort project

2006-04-13 Thread Benny Amorsen
MvO == Martijn van Oosterhout kleptog@svana.org writes: MvO Is this of practical use for run-of-the-mill video cards? -- The article suggests that using the GPU is a win even on a $100 64MB card. The built-in card in most servers is probably not worth bothering with, but many servers offer PCI

Re: [HACKERS] GPUSort project

2006-04-13 Thread mark
On Wed, Apr 12, 2006 at 09:31:52AM +0200, Benny Amorsen wrote: MvO == Martijn van Oosterhout kleptog@svana.org writes: MvO Is this of practical use for run-of-the-mill video cards? -- The article suggests that using the GPU is a win even on a $100 64MB card. The built-in card in most servers

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Tom Lane
Martijn van Oosterhout kleptog@svana.org writes: Right. Would you see value in a more formal libpq hijack-me interface that would support making the initial connection and then handing off the rest to something else? I think this would just be busywork... the way ODBC is doing it seems fine to

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Greg Stark
Martijn van Oosterhout kleptog@svana.org writes: On Thu, Apr 13, 2006 at 12:12:25PM +0100, Dave Page wrote: Ok. I'm not sure what this double copying you're referring to is, The libpq driver copies results out of the PGresult struct into the internal QueryResult classes. With

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Martijn van Oosterhout (kleptog@svana.org) wrote: Why not? Internally we call pqAddTuple for every tuple, calling a user function instead is hardly going to be more expensive. Also, I was thinking of the situation where the user function could set a flag so the eventual caller of (perhaps)

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 09:34:10AM -0400, Stephen Frost wrote: * Martijn van Oosterhout (kleptog@svana.org) wrote: Except in the case of psqlODBC, it wants to be able to malloc/free() each field, which your method doesn't solve. Also, it doesn't solve the duplicate memory use, nor the

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Greg Stark ([EMAIL PROTECTED]) wrote: Martijn van Oosterhout kleptog@svana.org writes: Hmm, the simplest improvement I can think of is one where you register a callback that libpq calls whenever it has received a new tuple. That could be useful for applications but I think a driver

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 11:14:57AM -0400, Greg Stark wrote: That could be useful for applications but I think a driver really wants to retain control of the flow of control. To make use of a callback it would have to have an awkward dance of calling whatever function gives libpq license to

Re: [HACKERS] two-argument aggregates and SQL 2003

2006-04-13 Thread Tom Lane
Sergey E. Koposov [EMAIL PROTECTED] writes: ... Nothing else and nothing internal need not to be changed to insert new two-arg. aggregate functions into the core. Am I right in this ? IIRC the main issues are the syntax of CREATE AGGREGATE and the actual implementation in nodeAgg.c. See

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Martijn van Oosterhout (kleptog@svana.org) wrote: Right, I didn't understand that you meant to be doing this synchronously, as the data came in. I thought it was just another way of retreiving the data already received. But given that a stated reason that psqlODBC didn't use the libpq

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Tom Lane
Stephen Frost [EMAIL PROTECTED] writes: I can see how having a callback would be useful though I think for a good number of cases it's just going to be populating a memory region with it and we could cover that common case by providing an API for exactly that. We already have that: it's

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: Stephen Frost [EMAIL PROTECTED] writes: I can see how having a callback would be useful though I think for a good number of cases it's just going to be populating a memory region with it and we could cover that common case by providing an API for

Re: [HACKERS] Possible race in UnlockBuffers() and UnpinBuffer()

2006-04-13 Thread Tom Lane
Qingqing Zhou [EMAIL PROTECTED] writes: ... However, a possible execution sequence involving another process doing UnpinBuffer() may look like this: unpinner: lockHdr(); read and reset flag; unlockHdr(); waiter: lockHdr(); reset flag; unlockHdr(); ProcCancelWaitForSignal(); unpinner:

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 11:54:33AM -0400, Stephen Frost wrote: snip Sure you could but you're forced to do more copying around of the data (copy into the PGresAttValue, copy out of it into your structure array). If you want something more complex then a callback makes more sense but I'm of

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 12:02:56PM -0400, Greg Stark wrote: There's nothing wrong with a callback interface for applications. They can generally have the callback function update the display or output to a file or whatever they're planning to do with the data. However drivers don't generally

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: I think there's some confusion about what problem this is aiming to solve. I thought the primary problem ODBC and other drivers have is just that they want to be able to fetch whatever records are available instead of waiting for the entire query results to

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Tom Lane ([EMAIL PROTECTED]) wrote: Stephen Frost [EMAIL PROTECTED] writes: It's only the functional equivalent when you think all the world is a Postgres app, which is just not the case. If we are dumping data into a simple memory block in a format dictated by libpq, then we haven't

Re: OS cached buffers (was: [HACKERS] Support Parallel Query Execution

2006-04-13 Thread Bruce Momjian
Jim C. Nasby wrote: On Mon, Apr 10, 2006 at 12:02:56PM -0700, Luke Lonergan wrote: Hannu, On 4/10/06 2:23 AM, Hannu Krosing [EMAIL PROTECTED] wrote: The cost of fetching a page from the OS is not really much of an overhead, Have you tested this ? I have - the overhead

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Greg Stark ([EMAIL PROTECTED]) wrote: Tom Lane [EMAIL PROTECTED] writes: So instead, you'd prefer to *always* store partial tuples in a temporary area, thereby making sure the independent-field-conversions case has performance just as bad as the dependent-conversions case. I can't

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Greg Stark
Tom Lane [EMAIL PROTECTED] writes: Greg Stark [EMAIL PROTECTED] writes: I think there's some confusion about what problem this is aiming to solve. I thought the primary problem ODBC and other drivers have is just that they want to be able to fetch whatever records are available instead

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Stephen Frost
* Greg Stark ([EMAIL PROTECTED]) wrote: Tom Lane [EMAIL PROTECTED] writes: Greg Stark [EMAIL PROTECTED] writes: I think there's some confusion about what problem this is aiming to solve. I thought the primary problem ODBC and other drivers have is just that they want to be able

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Martijn van Oosterhout
On Thu, Apr 13, 2006 at 03:42:44PM -0400, Stephen Frost wrote: You seem to be talking about a much broader set of problems to solve. I'd like to improve the API in general to cover a set of use-cases that I've run into quite a few times (and apparently some others have too as other DBs

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Bruce Momjian
Martijn van Oosterhout wrote: -- Start of PGP signed section. On Thu, Apr 13, 2006 at 03:42:44PM -0400, Stephen Frost wrote: You seem to be talking about a much broader set of problems to solve. I'd like to improve the API in general to cover a set of use-cases that I've run into quite

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Tom Lane
Greg Stark [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: No, that's not what I'm thinking about at all, and I don't think Martijn is either. The point here is that ODBC wants to store the resultset in a considerably different format from what libpq natively provides, and we'd

Re: [HACKERS] Practical impediment to supporting multiple SSL libraries

2006-04-13 Thread Hiroshi Inoue
Stephen Frost wrote: * Martijn van Oosterhout (kleptog@svana.org) wrote: On Thu, Apr 13, 2006 at 08:48:54AM +0100, Dave Page wrote: Well, we had a pure custom implementation of the protocol, had a pure libpq based version and after much discussion decided that the best version of all

Re: [HACKERS] Control File

2006-04-13 Thread Jim C. Nasby
On Thu, Apr 13, 2006 at 04:39:59AM -0400, Bruce Momjian wrote: Tom Lane wrote: Bruce Momjian pgman@candle.pha.pa.us writes: Bruno Almeida do Lago wrote: After that night, I started to ask myself if PostgreSQL should not have a control file to check if expected datafiles are where they

Re: OS cached buffers (was: [HACKERS] Support Parallel Query Execution

2006-04-13 Thread Jim C. Nasby
On Thu, Apr 13, 2006 at 03:38:04PM -0400, Bruce Momjian wrote: Is there any pratcical way to tell the difference between a page comming from the OS cache and one comming from disk? Or maybe for a set of pages an estimate on how many came from cache vs disk? There's some areas where having