[HACKERS] ERROR during WAL replay

2008-01-12 Thread Gurjeet Singh
Hi All, We were trying to move a big database from one machine to the other using PITR mechanism. We hit the following LOG message in during the recovery (WAL replay) process" LOG: incorrect resource manager data checksum in record at 111/A7738C8 I had used this procedure to do such mig

[HACKERS] Make pg_dump suppress COMMENT ON SCHEMA public ?

2008-01-12 Thread Tom Lane
There was some discussion today about restoring pg_dump output as a non-superuser: http://archives.postgresql.org/pgsql-admin/2008-01/msg00128.php In 8.3 we have eliminated one of the major roadblocks to doing that, which is that we now allow non-superuser database owners to create trusted procedu

Re: [HACKERS] timestamp refactor effort

2008-01-12 Thread Tom Lane
"Warren Turkal" <[EMAIL PROTECTED]> writes: > So...in the vein of my last mail, I have tried to create another patch > for refactoring out some of the HAVE_INT64_TIMESTAMP ifdefs in the > code in timestamp.c. I have attached the patch. Please let me know if > this patch is acceptable and what I can

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Greg Smith
On Sat, 12 Jan 2008, Jean-Michel Pour? wrote: You are well aware of Jonathan Gardner preliminary work: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html There's also PostgreSQL::Snapshots ; intro at http://cunha17.cristianoduarte.pro.br/postgresql/snapshots.en_us.php and main

[HACKERS] Unexpected log chatter during transaction abort in CVS HEAD

2008-01-12 Thread Tom Lane
About two weeks ago we had a discussion that concluded that notice messages put out by GUC assign hooks should be logged when there's a problem with a postgresql.conf setting, leading to this patch: http://archives.postgresql.org/pgsql-committers/2007-12/msg00298.php I noticed today that running p

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Jonah H. Harris
On Jan 12, 2008 5:31 PM, Mark Mielke <[EMAIL PROTECTED]> wrote: > Joshua D. Drake wrote: >> I think his email was very well written and a simple request of >> discussion of alternatives as well as future plans. Agreed, JD. > Offensive is relative. I find it offensive when people demand things on

[HACKERS] NOTICE messages during COPY OUT

2008-01-12 Thread Tom Lane
It was pointed out to me off-list that pg_dump fails if a user-defined datatype's output routine is silly enough to emit NOTICE messages: the rest of the table data simply disappears without any warning. What is happening is that libpq's pqGetCopyData3() drops out of COPY_OUT state upon seeing a N

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Joshua D. Drake wrote: Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try yourself? If it was so easy, and such a clear win, I think one of the very competent people using PostgreSQL today would h

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > ... So if we did this via an installable module approach then we > would be able to use it much sooner for upgrading to 8.3, rather than > waiting for 8.4 I think it borders on ludicrous to imagine making this work with just an installable module and no co

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 13:27 +0100, Jean-Michel Pouré wrote: > Please pick-up this important issue for developpers. There is no need to > concentrate on complex issues, when handling materialized views could > boost somme web apps. by a factor of 10 or more. It's more complex than you think, but t

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 15:42 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Sat, 2008-01-12 at 11:57 -0500, Tom Lane wrote: > >> It would be far *more* useful if it didn't have to be superuser-only. > >> And since the actual details of the snapshot content are really of > >>

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Sat, 2008-01-12 at 11:57 -0500, Tom Lane wrote: >> It would be far *more* useful if it didn't have to be superuser-only. >> And since the actual details of the snapshot content are really of >> zero interest to the user, I think making it pass through hi

Re: [HACKERS] getting out boolean value from PQgetValue function

2008-01-12 Thread Tom Lane
"Arunachalam Parthasarathy" <[EMAIL PROTECTED]> writes: > When i iterate over the result set, i am unable to get the return boolean > value in my code. The ptr returned by PQGetValue is valid, but contains > nothing. why is it so? Did you check PQgetisnull()? IIRC PQgetvalue() will return a zero-

Re: [HACKERS] A note about SIGTERM illusion and reality

2008-01-12 Thread Tom Lane
I wrote: > After digging in the man page for init(8) on a couple of machines, > I realized that the SIGTERM-then-SIGKILL behavior only applies to > processes that are launched directly by init. Actually, after further experimentation, it seems this is very platform-dependent. Current Linux (teste

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 11:57 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: > >> [ blanches... ] Can you say "security hole"? > > > Static on the line, sorry. > > I'm hearing "useful superuser-only capability". ;-) > > It w

[HACKERS] getting out boolean value from PQgetValue function

2008-01-12 Thread Arunachalam Parthasarathy
Hello all, I am using pg advisory locks in my application using libpq. When lock is obtained, the result set contains boolean value for the row locked in result set. When i iterate over the result set, i am unable to get the return boolean value in my code. The ptr returned by PQGetValue is valid

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Marko Kreen
On 1/12/08, Gregory Stark <[EMAIL PROTECTED]> wrote: > "Tom Lane" <[EMAIL PROTECTED]> writes: > > Simon Riggs <[EMAIL PROTECTED]> writes: > >> On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: > >>> [ blanches... ] Can you say "security hole"? > > > >> Static on the line, sorry. > >> I'm hearing

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Mark Mielke wrote: Counts, because as we all know, PostgreSQL count(*) is slow, and in any case, my count(*) is not on the whole table, but on a subset. Doing this in a general way seems complex to me as it would need to be able to evaluate whether a given INSERT or UPDATE or one of the depend

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 12 Jan 2008 10:26:49 -0500 Mark Mielke <[EMAIL PROTECTED]> wrote: > Jean-Michel Pouré wrote: > > In my past development projects, I always used the concept of > > Materialized VIEW to speed-up SELECTs over INSERTs > > Unless you are going to

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Dave Page wrote: On 12/01/2008, Mark Mielke <[EMAIL PROTECTED]> wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best wa

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Gregory Stark
"Tom Lane" <[EMAIL PROTECTED]> writes: > Simon Riggs <[EMAIL PROTECTED]> writes: >> On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: >>> [ blanches... ] Can you say "security hole"? > >> Static on the line, sorry. >> I'm hearing "useful superuser-only capability". ;-) > > It would be far *more

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread Gavin Sherry
On Sat, Jan 12, 2008 at 05:47:30PM +, Simon Riggs wrote: > On Sat, 2008-01-12 at 01:59 +0100, Gavin Sherry wrote: > > The syntax is half the problem, performance is the other. > > The syntax looks great to me, but I think it is about 5% of the problem, > maybe less. I don't really have any que

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Dave Page
On 12/01/2008, Mark Mielke <[EMAIL PROTECTED]> wrote: > Jean-Michel Pouré wrote: > > In my past development projects, I always used the concept of > > Materialized VIEW to speed-up SELECTs over INSERTs > > Unless you are going to *pay* for it - you do realize that the best way > to get it implement

Re: [HACKERS] Some ideas about Vacuum

2008-01-12 Thread Markus Schiltknecht
Hi, Tom Lane wrote: Well, one of the principal arguments for having VACUUM at all is that it off-loads required maintenance effort from foreground transaction code paths. Off-loading doesn't mean we don't have to do the work, so it's obviously is a compromise. AFAICT, having to write some D

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 01:59 +0100, Gavin Sherry wrote: > The syntax is half the problem, performance is the other. The syntax looks great to me, but I think it is about 5% of the problem, maybe less. I don't really have any questions about the syntax, but I may have thoughts when the implementatio

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread Peter Eisentraut
Jeff Cohen wrote: > In order to make such error checking > feasible, we would have to restrict the set of predicates you can use > in the WHERE clause, so it wouldn't be completely general anyway. Well, with an extensible system such as PostgreSQL you will need to have a partitioning scheme that

Re: [HACKERS] Some ideas about Vacuum

2008-01-12 Thread Tom Lane
Markus Schiltknecht <[EMAIL PROTECTED]> writes: >> Since Vacuum process is going to >> have much more information on what has happened in the database, > Why should that be? IMO, collecting the information at transaction time > can give you exactly the same information, if not more or better >

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: >> [ blanches... ] Can you say "security hole"? > Static on the line, sorry. > I'm hearing "useful superuser-only capability". ;-) It would be far *more* useful if it didn't have to be superuser-only. A

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try yours

[HACKERS] Postgresql Materialized views

2008-01-12 Thread Jean-Michel Pouré
Dear Friends, In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs. You are well aware of Jonathan Gardner preliminary work: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html When do you plan to add MATERIALIZED VIEWS to

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread NikhilS
Hi, On Jan 12, 2008 6:29 AM, Gavin Sherry <[EMAIL PROTECTED]> wrote: > > > The syntax is half the problem, performance is the other. I will bring > the performance issues up in another thread. Yes, we are confident that > we can address the performance issues that rule out the existing > partit

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread NikhilS
Hi, > We did look at allowing general functions for partitioning and this > was one concern. The other is that we want to enforce that a row > only gets inserted into a single partition, so we wanted a > declarative syntax where it was relatively easy to check that range > and list specifications

Re: [HACKERS] Declarative partitioning grammar

2008-01-12 Thread NikhilS
Hi, > I've proposed an alternative approach, which we've called declarative > partitioning which is grammar based. This grammar was developed by Jeff > Cohen at Greenplum with some assistance from myself. It is to be > completely open source. > <..> FWIW, I had done some very initial work on decl

Re: [HACKERS] Some ideas about Vacuum

2008-01-12 Thread Markus Schiltknecht
Hi, Gokulakannan Somasundaram wrote: I'm also not sure it really buys us anything over having a second dead-space-map data structure. The WAL is much larger and serves other purposes which would limit what we can do with it. Ok. One obvious advantage is that it saves the contention

Re: [HACKERS] Transaction Snapshot Cloning

2008-01-12 Thread Simon Riggs
On Fri, 2008-01-11 at 19:23 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > For (2) there's a very simple way of transferring the data between > > sessions: > > a) we connect on session 1 as a serializable transaction > > b) we ask session 1 for its snapshot > > c) we then con