Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Simon Riggs
On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: For (2) there's a very simple way of transferring the data between sessions: a) we connect on session 1 as a serializable transaction b) we ask session 1 for its snapshot c) we then connect on

Re: [HACKERS] Some ideas about Vacuum

2008-01-12 Thread Markus Schiltknecht
Hi, Gokulakannan Somasundaram wrote: I'm also not sure it really buys us anything over having a second dead-space-map data structure. The WAL is much larger and serves other purposes which would limit what we can do with it. Ok. One obvious advantage is that it saves the contention

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread NikhilS
Hi, I've proposed an alternative approach, which we've called declarative partitioning which is grammar based. This grammar was developed by Jeff Cohen at Greenplum with some assistance from myself. It is to be completely open source. .. FWIW, I had done some very initial work on

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread NikhilS
Hi, We did look at allowing general functions for partitioning and this was one concern. The other is that we want to enforce that a row only gets inserted into a single partition, so we wanted a declarative syntax where it was relatively easy to check that range and list specifications

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread NikhilS
Hi, On Jan 12, 2008 6:29 AM, Gavin Sherry [EMAIL PROTECTED] wrote: The syntax is half the problem, performance is the other. I will bring the performance issues up in another thread. Yes, we are confident that we can address the performance issues that rule out the existing partitioning

[HACKERS] Postgresql Materialized views

2008-01-12 Thread Jean-Michel Pouré
Dear Friends, In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs. You are well aware of Jonathan Gardner preliminary work: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html When do you plan to add MATERIALIZED VIEWS to

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: [ blanches... ] Can you say security hole? Static on the line, sorry. I'm hearing useful superuser-only capability. ;-) It would be far *more* useful if it didn't have to be superuser-only. And since

Re: [HACKERS] Some ideas about Vacuum

2008-01-12 Thread Tom Lane
Markus Schiltknecht [EMAIL PROTECTED] writes: Since Vacuum process is going to have much more information on what has happened in the database, Why should that be? IMO, collecting the information at transaction time can give you exactly the same information, if not more or better

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread Peter Eisentraut
Jeff Cohen wrote: In order to make such error checking feasible, we would have to restrict the set of predicates you can use in the WHERE clause, so it wouldn't be completely general anyway. Well, with an extensible system such as PostgreSQL you will need to have a partitioning scheme that

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 01:59 +0100, Gavin Sherry wrote: The syntax is half the problem, performance is the other. The syntax looks great to me, but I think it is about 5% of the problem, maybe less. I don't really have any questions about the syntax, but I may have thoughts when the

Re: [HACKERS] Some ideas about Vacuum

2008-01-12 Thread Markus Schiltknecht
Hi, Tom Lane wrote: Well, one of the principal arguments for having VACUUM at all is that it off-loads required maintenance effort from foreground transaction code paths. Off-loading doesn't mean we don't have to do the work, so it's obviously is a compromise. AFAICT, having to write some

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Dave Page
On 12/01/2008, Mark Mielke [EMAIL PROTECTED] wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: [ blanches... ] Can you say security hole? Static on the line, sorry. I'm hearing useful superuser-only capability. ;-) It would be far *more* useful if it didn't

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread Gavin Sherry
On Sat, Jan 12, 2008 at 05:47:30PM +, Simon Riggs wrote: On Sat, 2008-01-12 at 01:59 +0100, Gavin Sherry wrote: The syntax is half the problem, performance is the other. The syntax looks great to me, but I think it is about 5% of the problem, maybe less. I don't really have any

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 12 Jan 2008 10:26:49 -0500 Mark Mielke [EMAIL PROTECTED] wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay*

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Mark Mielke wrote: Counts, because as we all know, PostgreSQL count(*) is slow, and in any case, my count(*) is not on the whole table, but on a subset. Doing this in a general way seems complex to me as it would need to be able to evaluate whether a given INSERT or UPDATE or one of the

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Dave Page wrote: On 12/01/2008, Mark Mielke [EMAIL PROTECTED] wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Marko Kreen
On 1/12/08, Gregory Stark [EMAIL PROTECTED] wrote: Tom Lane [EMAIL PROTECTED] writes: Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: [ blanches... ] Can you say security hole? Static on the line, sorry. I'm hearing useful superuser-only

[HACKERS] getting out boolean value from PQgetValue function

2008-01-12 Thread Arunachalam Parthasarathy
Hello all, I am using pg advisory locks in my application using libpq. When lock is obtained, the result set contains boolean value for the row locked in result set. When i iterate over the result set, i am unable to get the return boolean value in my code. The ptr returned by PQGetValue is

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 11:57 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: [ blanches... ] Can you say security hole? Static on the line, sorry. I'm hearing useful superuser-only capability. ;-) It would be far *more*

Re: [HACKERS] getting out boolean value from PQgetValue function

2008-01-12 Thread Tom Lane
Arunachalam Parthasarathy [EMAIL PROTECTED] writes: When i iterate over the result set, i am unable to get the return boolean value in my code. The ptr returned by PQGetValue is valid, but contains nothing. why is it so? Did you check PQgetisnull()? IIRC PQgetvalue() will return a zero-length

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: On Sat, 2008-01-12 at 11:57 -0500, Tom Lane wrote: It would be far *more* useful if it didn't have to be superuser-only. And since the actual details of the snapshot content are really of zero interest to the user, I think making it pass through his hands

Re: [HACKERS] A note about SIGTERM illusion and reality

2008-01-12 Thread Tom Lane
I wrote: After digging in the man page for init(8) on a couple of machines, I realized that the SIGTERM-then-SIGKILL behavior only applies to processes that are launched directly by init. Actually, after further experimentation, it seems this is very platform-dependent. Current Linux (tested

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 15:42 -0500, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: On Sat, 2008-01-12 at 11:57 -0500, Tom Lane wrote: It would be far *more* useful if it didn't have to be superuser-only. And since the actual details of the snapshot content are really of zero

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 13:27 +0100, Jean-Michel Pouré wrote: Please pick-up this important issue for developpers. There is no need to concentrate on complex issues, when handling materialized views could boost somme web apps. by a factor of 10 or more. It's more complex than you think, but the

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: ... So if we did this via an installable module approach then we would be able to use it much sooner for upgrading to 8.3, rather than waiting for 8.4 I think it borders on ludicrous to imagine making this work with just an installable module and no

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Joshua D. Drake wrote: Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try yourself? If it was so easy, and such a clear win, I think one of the very competent people using PostgreSQL today would

[HACKERS] NOTICE messages during COPY OUT

2008-01-12 Thread Tom Lane
It was pointed out to me off-list that pg_dump fails if a user-defined datatype's output routine is silly enough to emit NOTICE messages: the rest of the table data simply disappears without any warning. What is happening is that libpq's pqGetCopyData3() drops out of COPY_OUT state upon seeing a

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Jonah H. Harris
On Jan 12, 2008 5:31 PM, Mark Mielke [EMAIL PROTECTED] wrote: Joshua D. Drake wrote: I think his email was very well written and a simple request of discussion of alternatives as well as future plans. Agreed, JD. Offensive is relative. I find it offensive when people demand things on one of

[HACKERS] Unexpected log chatter during transaction abort in CVS HEAD

2008-01-12 Thread Tom Lane
About two weeks ago we had a discussion that concluded that notice messages put out by GUC assign hooks should be logged when there's a problem with a postgresql.conf setting, leading to this patch: http://archives.postgresql.org/pgsql-committers/2007-12/msg00298.php I noticed today that running

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Greg Smith
On Sat, 12 Jan 2008, Jean-Michel Pour? wrote: You are well aware of Jonathan Gardner preliminary work: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html There's also PostgreSQL::Snapshots ; intro at http://cunha17.cristianoduarte.pro.br/postgresql/snapshots.en_us.php and main

Re: [HACKERS] timestamp refactor effort

2008-01-12 Thread Tom Lane
Warren Turkal [EMAIL PROTECTED] writes: So...in the vein of my last mail, I have tried to create another patch for refactoring out some of the HAVE_INT64_TIMESTAMP ifdefs in the code in timestamp.c. I have attached the patch. Please let me know if this patch is acceptable and what I can do to

[HACKERS] Make pg_dump suppress COMMENT ON SCHEMA public ?

2008-01-12 Thread Tom Lane
There was some discussion today about restoring pg_dump output as a non-superuser: http://archives.postgresql.org/pgsql-admin/2008-01/msg00128.php In 8.3 we have eliminated one of the major roadblocks to doing that, which is that we now allow non-superuser database owners to create trusted

[HACKERS] ERROR during WAL replay

2008-01-12 Thread Gurjeet Singh
Hi All, We were trying to move a big database from one machine to the other using PITR mechanism. We hit the following LOG message in during the recovery (WAL replay) process LOG: incorrect resource manager data checksum in record at 111/A7738C8 I had used this procedure to do such