Re: [HACKERS] Archiver behavior at shutdown

2008-01-04 Thread Fujii Masao
Simon Riggs wrote: > My original one line change described on bug 3843 seems like the best > solution for 8.3. > +1 Is this change in time for RC1? -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center TEL (03)5860-5115 FAX (03)5463-5490 ---

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Richard Huxton
Simon Riggs wrote: We would keep a dynamic visibility map at *segment* level, showing which segments have all rows as 100% visible. No freespace map data would be held at this level. Small dumb-user question. I take it you've considered some more flexible consecutive-run-of-blocks unit of fl

[HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Glyn Astill
Hi people, I've tried posting on the general list about this, but I never get a reply, so I'm trying here. I have a function that is run each time an INSERT, DELETE or UPDATE happens on a row and log into an audit table. It is based on the info here: http://www.alberton.info/postgresql_table_au

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 10:22 +, Richard Huxton wrote: > Simon Riggs wrote: > > We would keep a dynamic visibility map at *segment* level, showing which > > segments have all rows as 100% visible. No freespace map data would be > > held at this level. > > Small dumb-user question. > > I take i

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 13:06 +0530, Gokulakannan Somasundaram wrote: > a) This proposal would work for the kind of table organizations which > are currently partitioned and maintained based on some kind of > timestamp. Consider one of the use-case. A large Retail firm has a lot > of stores. DBA mai

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Richard Huxton
Simon Riggs wrote: On Fri, 2008-01-04 at 10:22 +, Richard Huxton wrote: Simon Riggs wrote: We would keep a dynamic visibility map at *segment* level, showing which segments have all rows as 100% visible. No freespace map data would be held at this level. Small dumb-user question. I take

[HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Glyn Astill
Hi people, I've tried posting on the general list about this, but I never get a reply, so I'm trying here. I have a function that is run each time an INSERT, DELETE or UPDATE happens on a row and log into an audit table. It is based on the info here: http://www.alberton.info/postgres

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 13:29 +0100, Markus Schiltknecht wrote: > Given that we are operating on segments here, to which the DBA has very > limited information and access, I prefer the term "Segment Exclusion". I > think of that as an optimization of sequential scans on tables with the > above me

Re: [HACKERS] timestamp typedefs

2008-01-04 Thread Alvaro Herrera
Warren Turkal escribió: > On Jan 3, 2008 8:54 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > > I wrote: > > > Do we really need "fhour_t" and "fminute_t" on top of "fsec_t"? > > > This seems like a bad factorization ... > > > > After some more thought: I think that what's bugging me is that "fsec_t" > >

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hello Simon, Simon Riggs wrote: I've come up with an alternative concept to allow us to discuss the particular merits of each. ISTM that this new proposal has considerable potential. Hm.. interesting idea. If we were able to keep track of which sections of a table are now read-only then we w

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hi, Simon Riggs wrote: - any Fact table where measurements/observations/events are accumulated e.g. Web Hits (any Internet events) Call Detail Records Sales Security Events Scientific Measurements Process Control - any Major Entity where new entities are created from a sequence e.g. Orders, Ord

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hi, Simon Riggs wrote: The smaller the partition size the greater the overhead of managing it. Also I've been looking at read-only tables and compression, as you may know. My idea was that in the future we could mark segments as either - read-only - compressed - able to be shipped off to hiera

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Martijn van Oosterhout
On Fri, Jan 04, 2008 at 05:13:21PM +0100, Peter Eisentraut wrote: > Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, > this > just works. I didn't find a way to sniff a Unix-domain socket, however. Looks clear enough. You should be able to test if it works by using stra

[HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Peter Eisentraut
Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however. How should we proceed with this? -- Peter Eisentraut http://developer.postgresql.org/~petere/ diff -ur ../cvs-pgsql/src/backend/postmaster/post

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Peter Eisentraut wrote: > Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, > this > just works. I didn't find a way to sniff a Unix-domain socket, however. > > How should we proceed with this? I am confused by the shortness of this patch. Right now pg_hba.conf has:

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Magnus Hagander
Bruce Momjian wrote: > Peter Eisentraut wrote: >> Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: >>> Peter Eisentraut wrote: Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, this just works. I didn't find a way to sniff a Unix-domain socket, however.

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Magnus Hagander wrote: > Bruce Momjian wrote: > > Peter Eisentraut wrote: > >> Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: > >>> Peter Eisentraut wrote: > Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, > this just works. I didn't find a way to sniff a Uni

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Peter Eisentraut
Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: > Peter Eisentraut wrote: > > Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, > > this just works. I didn't find a way to sniff a Unix-domain socket, > > however. > > > > How should we proceed with this? > > I am confused

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Peter Eisentraut wrote: > Am Freitag, 4. Januar 2008 schrieb Bruce Momjian: > > Peter Eisentraut wrote: > > > Using the attached patch, SSL will act over Unix-domain sockets. AFAICT, > > > this just works. I didn't find a way to sniff a Unix-domain socket, > > > however. > > > > > > How should we

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Andrew Sullivan
On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: > > Agreed. Just a minor note: I find "marked read-only" too strong, as it > implies an impossibility to write. I propose speaking about mostly-read > segments, or optimized for reading or similar. I do want some segments to b

Re: [HACKERS] Index performance

2008-01-04 Thread Andrew Sullivan
On Thu, Jan 03, 2008 at 07:11:07AM +0200, Brian Modra wrote: > Thanks, I think you have me on the right track. I'm testing a vacuum > analyse now to see how long it takes, and then I'll set it up to > automatically run every night (so that it has a chance to complete > before about 6am.) Note that

[HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
There was a serious performance regression in OUTER JOIN planning going from 8.2.4 to 8.2.5. I know Tom came up with some patches to mitigate the issues in 8.2.5, but my testing shows that problems remain in 8.3beta4. The query: SELECT "CH"."caseNo", "CH"."countyNo", "CH"."chargeNo

Re: [HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Richard Huxton
Glyn Astill wrote: Hi people, I've tried posting on the general list about this, but I never get a reply, so I'm trying here. I think you'll probably have more luck with a TCL list than the PG hackers list. However, I've attached some pltcl functions I put together ages ago to do this s

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 13:06 -0500, Andrew Sullivan wrote: > On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: > > > > Agreed. Just a minor note: I find "marked read-only" too strong, as it > > implies an impossibility to write. I propose speaking about mostly-read > > segments

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Yes, it would plug the hole without fully implementing SSL control on > > local sockets. However, the hole is already plugged by using directory > > permissions so I question the need for a partial solution at this point > > in 8.3. >

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Aidan Van Dyk
* Bruce Momjian <[EMAIL PROTECTED]> [080104 13:00]: > > Actually, if you just commit that patch *without* pg_hba modifications, > > it still solves the problem stated, no? Because the client can be > > configured to require ssl and to require server certificate validation, > > and that's the hole

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Yes, it would plug the hole without fully implementing SSL control on > local sockets. However, the hole is already plugged by using directory > permissions so I question the need for a partial solution at this point > in 8.3. As already noted, "fix /tm

Re: [HACKERS] SSL over Unix-domain sockets

2008-01-04 Thread Andrew Sullivan
On Fri, Jan 04, 2008 at 02:37:03PM -0500, Bruce Momjian wrote: > The problem with adding SSL to local sockets is this slippery slope > where we only do part of the job, but it isn't clear where to draw the > line. I don't think "part of the job" for a patch is a slippery slope. It's what you do w

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Andrew Sullivan
On Fri, Jan 04, 2008 at 10:26:54PM +0100, Markus Schiltknecht wrote: > > I'm still puzzled about how a DBA is expected to figure out which > segments to mark. I think that part might be hand-wavy still. But once this facility is there, what's to prevent the current active segment (and the rest

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Markus Schiltknecht
Hi, Simon Riggs wrote: On Fri, 2008-01-04 at 13:06 -0500, Andrew Sullivan wrote: On Fri, Jan 04, 2008 at 01:29:55PM +0100, Markus Schiltknecht wrote: Agreed. Just a minor note: I find "marked read-only" too strong, as it implies an impossibility to write. I propose speaking about mostly-read

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 12:16 PM, in message <[EMAIL PROTECTED]>, "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > problems remain in 8.3beta4. As I poked around at this, it started to seem familiar. I had previously posted about this query's performance under 8.2.4. http://archives.postgre

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Simon Riggs
On Fri, 2008-01-04 at 22:26 +0100, Markus Schiltknecht wrote: > I'm still puzzled about how a DBA is expected to figure out which > segments to mark. Simon, are you assuming we are going to pass on > segment numbers to the DBA one day? No Way! That would stop Richard's idea to make the segment

Re: [HACKERS] Problem with PgTcl auditing function on trigger

2008-01-04 Thread Brett Schwarz
> > > - Original Message > From: Glyn Astill <[EMAIL PROTECTED]> > To: pgsql-hackers@postgresql.org > Sent: Friday, January 4, 2008 5:23:18 AM > Subject: [HACKERS] Problem with PgTcl auditing function on trigger > > Hi people, > > I've tried posting on the general list about this, but

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > As I poked around at this, it started to seem familiar. I had > previously posted about this query's performance under 8.2.4. > http://archives.postgresql.org/pgsql-performance/2007-10/msg00087.php Well, that thread gave some of the missing details,

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 4:46 PM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > I don't see anything in that thread that shows the view definition It was in the first post on the other thread, but for convenience: bigbird=# \d "CaseTypeHistEvent" View "public

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 4:40 PM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > what the heck is "CountyNoT"? bigbird=# \dD "CountyNoT" List of domains Schema | Name| Type | Modifier | Check +---+--+--+-

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 4:29 PM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > Can't do much with this without seeing the table and view definitions > involved. Understood. It was while I was putting that together that it struck me as familiar. They are the same as

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > There was a serious performance regression in OUTER JOIN planning > going from 8.2.4 to 8.2.5. I know Tom came up with some patches to > mitigate the issues in 8.2.5, but my testing shows that problems > remain in 8.3beta4. Can't do much with this wi

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 4:46 PM, in message <[EMAIL PROTECTED]>, Tom Lane <[EMAIL PROTECTED]> wrote: > nor mentions the data types involved. Schema | Name | Type| Modifier | Check +---+-+--+--- p

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
"Kevin Grittner" <[EMAIL PROTECTED]> writes: > On Fri, Jan 4, 2008 at 4:29 PM, in message <[EMAIL PROTECTED]>, > Tom Lane <[EMAIL PROTECTED]> wrote: >> Can't do much with this without seeing the table and view definitions >> involved. > Understood. It was while I was putting that together that

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 4:51 PM, in message <[EMAIL PROTECTED]>, "Kevin Grittner" <[EMAIL PROTECTED]> wrote: > keyEventSeqNo | integer | > COALESCE( > CASE > WHEN d."eventType" IS NOT NULL THEN d."keyEventSeqNo"::smallint > ELSE b."keyEventSeqN

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
Hmm ... I think I've managed to invent a test case, and unfortunately for you, what it shows is that 8.2 is optimizing the query incorrectly. create table t1 (f1 int primary key); create table t2 (f2 int primary key); create table t3 (f3 int primary key); insert into t1 values(53); insert into t2

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Kevin Grittner
>>> On Fri, Jan 4, 2008 at 5:45 PM, in message <[EMAIL PROTECTED]>, "Kevin Grittner" <[EMAIL PROTECTED]> wrote: On Fri, Jan 4, 2008 at 4:51 PM, in message > <[EMAIL PROTECTED]>, "Kevin Grittner" > <[EMAIL PROTECTED]> wrote: > >> keyEventSeqNo | integer | > >> COALESCE( >>

Re: [HACKERS] OUTER JOIN performance regression remains in 8.3beta4

2008-01-04 Thread Tom Lane
I wrote: > [ Pokes at older branches... ] Oh, that's interesting, 8.1 seems to do > the right thing already! Seems that 8.1 does the right thing for the wrong reason :-(. Just like 8.2, it falsely concludes that the f3 = f1 clause can be deleted, but it fails to get rid of every copy of it. The

Re: [HACKERS] Dynamic Partitioning using Segment Visibility Maps

2008-01-04 Thread Robert Treat
On Friday 04 January 2008 17:01, Simon Riggs wrote: > On Fri, 2008-01-04 at 22:26 +0100, Markus Schiltknecht wrote: > > I'm still puzzled about how a DBA is expected to figure out which > > segments to mark. Simon, are you assuming we are going to pass on > > segment numbers to the DBA one day? > >

Re: [HACKERS] timestamp typedefs

2008-01-04 Thread Warren Turkal
On Jan 4, 2008 4:20 AM, Alvaro Herrera <[EMAIL PROTECTED]> wrote: > Perhaps what you want here is to define a type for calculation results > (double/int64). Whether it is used in the code for minutes or hours is > irrelevant to the typedef. Okay...that sounds good. Do you have a good name for it?