Re: [HACKERS] Page Checksums

2011-12-18 Thread Heikki Linnakangas
On 17.12.2011 23:33, David Fetter wrote: What: Please find attached a patch for 9.2-to-be which implements page checksums. It changes the page format, so it's an initdb-forcing change. How: In order to ensure that the checksum actually matches the hint bits, this

[HACKERS] Autonomous subtransactions

2011-12-18 Thread Gianni Ciolli
Dear Hackers, I have written some notes about autonomous subtransactions, which have already been touched (at least) in two separate threads; please find them at http://wiki.postgresql.org/wiki/Autonomous_subtransactions It is meant to be an ongoing project, requesting comments and

Re: [HACKERS] Page Checksums

2011-12-18 Thread David Fetter
On Sun, Dec 18, 2011 at 10:14:38AM +0200, Heikki Linnakangas wrote: On 17.12.2011 23:33, David Fetter wrote: What: Please find attached a patch for 9.2-to-be which implements page checksums. It changes the page format, so it's an initdb-forcing change. How: In order

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Jan Urbański
On 18/12/11 04:21, Robert Haas wrote: On Sat, Dec 17, 2011 at 7:50 PM, David E. Wheeler da...@kineticode.com wrote: Love having the start here. I forwarded this message to Claes Jakobsson, creator of the jansson-using pg-json extension. He’s a bit less supportive. He gave me permission to

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Pavel Stehule
2011/12/18 Jan Urbański wulc...@wulczer.org: On 18/12/11 04:21, Robert Haas wrote: On Sat, Dec 17, 2011 at 7:50 PM, David E. Wheeler da...@kineticode.com wrote: Love having the start here. I forwarded this message to Claes Jakobsson, creator of the jansson-using pg-json extension. He’s a

Re: [HACKERS] Page Checksums

2011-12-18 Thread Heikki Linnakangas
On 18.12.2011 10:54, David Fetter wrote: On Sun, Dec 18, 2011 at 10:14:38AM +0200, Heikki Linnakangas wrote: On 17.12.2011 23:33, David Fetter wrote: If this introduces new failure modes, please detail, and preferably demonstrate, just what those new modes are. Hint bits, torn pages -

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes: On Sat, Dec 17, 2011 at 5:02 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I'd like to add some confusion on the implementation choice, because it looks damn too easy now… Guile 2.0 offers an implementation of the ECMAscript language and plscheme

Re: [HACKERS] Command Triggers

2011-12-18 Thread Dimitri Fontaine
Bruce Momjian br...@momjian.us writes: We have a big user community and what _you_ want for this feature is only a small part of our decision on what is needed. Robert's concern that this might not be useful enough for the general use-cases people want is a legitimate, if difficult to hear,

Re: [HACKERS] Patch to allow users to kill their own queries

2011-12-18 Thread Magnus Hagander
On Fri, Dec 16, 2011 at 13:31, Greg Smith g...@2ndquadrant.com wrote: On 12/14/2011 05:24 AM, Magnus Hagander wrote: How about passing a parameter to pg_signal_backend? Making pg_signal_backend(int pid, int sig, bool allow_samerole)? That works, got rid of the parts I didn't like and

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Magnus Hagander
On Sun, Dec 18, 2011 at 10:49, Jan Urbański wulc...@wulczer.org wrote: On 18/12/11 04:21, Robert Haas wrote: On Sat, Dec 17, 2011 at 7:50 PM, David E. Wheeler da...@kineticode.com wrote: Love having the start here. I forwarded this message to Claes Jakobsson, creator of the jansson-using

Re: [HACKERS] Patch to allow users to kill their own queries

2011-12-18 Thread Robert Haas
On Sat, Dec 17, 2011 at 11:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think this argument is bogus: if this is a real issue, then no use of kill() anytime, by anyone, is safe.  In practice I believe that Unix systems avoid recycling PIDs right away so as to offer some protection. I'm not sure

Re: [HACKERS] Command Triggers

2011-12-18 Thread Bruce Momjian
Dimitri Fontaine wrote: Bruce Momjian br...@momjian.us writes: We have a big user community and what _you_ want for this feature is only a small part of our decision on what is needed. Robert's concern that this might not be useful enough for the general use-cases people want is a

Re: [HACKERS] Autonomous subtransactions

2011-12-18 Thread Jim Nasby
On Dec 18, 2011, at 2:28 AM, Gianni Ciolli wrote: I have written some notes about autonomous subtransactions, which have already been touched (at least) in two separate threads; please find them at http://wiki.postgresql.org/wiki/Autonomous_subtransactions The document seems to mix the

[HACKERS] Cursor behavior

2011-12-18 Thread amit sehas
HI, I had a question about the cursor internals implementation. When you Fetch next 'n' results without moving the cursors, is this kind of functionality implemented by firstly executing the whole query and then moving the cursor over the results, or are the movements done on active database

[HACKERS] array behavior

2011-12-18 Thread amit sehas
If a field(attribute) in a type is declared to be an array. Then is it always the case that the array will be packed into a single tuple that resides in the database. There is the TOAST mechanism for oversized tuples but that is still considered to be a single tuple. Is there any circumstance

Re: [HACKERS] Measuring relation free space

2011-12-18 Thread Noah Misch
On Fri, Dec 16, 2011 at 02:02:03AM -0500, Greg Smith wrote: On 12/15/2011 04:11 PM, Noah Misch wrote: Is one of those index figures simply wrong, or do they measure two senses of free space, both of which are interesting to DBAs? I think the bigger one--the one I was aiming to measure--also

Re: [HACKERS] unite recovery.conf and postgresql.conf

2011-12-18 Thread Josh Berkus
Greg, You keep asking the hard questions. I practice. ;-) Right now, I kind of like that it's possible to copy a postgresql.conf file from master to standby and just use it. That should still be possible with the realignment into GUCs: ... long discussion omitted here. I agree that GUC

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Merlin Moncure
On Sat, Dec 17, 2011 at 4:02 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: to add some confusion on the implementation choice, because it looks damn too easy now… Guile 2.0 offers an implementation of the ECMAscript language and plscheme already exists as a PostgreSQL PL extension for

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Andrew Dunstan
On 12/18/2011 12:17 PM, Merlin Moncure wrote: One small note about the json type being an extension -- this probably means the json type oid won't be fixed -- not a huge deal but it could affect some corner cases with binary format consumers. Why would that matter more for JSON than for

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Merlin Moncure
On Sun, Dec 18, 2011 at 11:21 AM, Andrew Dunstan and...@dunslane.net wrote: On 12/18/2011 12:17 PM, Merlin Moncure wrote: One small note about the json type being an extension -- this probably means the json type oid won't be fixed -- not a huge deal but it could affect some corner cases

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Dimitri Fontaine
Merlin Moncure mmonc...@gmail.com writes: Why would that matter more for JSON than for any other non-core type? well, it's a minor headache for all the oid-isn't-in-pgtypes.h types, and only then for high traffic types (which presumably json will be). Extensions are going to be more and more

Re: [HACKERS] Page Checksums

2011-12-18 Thread David Fetter
On Sun, Dec 18, 2011 at 12:19:32PM +0200, Heikki Linnakangas wrote: On 18.12.2011 10:54, David Fetter wrote: On Sun, Dec 18, 2011 at 10:14:38AM +0200, Heikki Linnakangas wrote: On 17.12.2011 23:33, David Fetter wrote: If this introduces new failure modes, please detail, and preferably

Re: [HACKERS] Command Triggers

2011-12-18 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: The main part of my answer, though, is that all the more complex use cases involving command triggers that Robert is offering are in fact possible to implement with what my patch is providing, as soon as you're ok with understanding the content

Re: [HACKERS] Page Checksums

2011-12-18 Thread Heikki Linnakangas
On 18.12.2011 20:44, David Fetter wrote: On Sun, Dec 18, 2011 at 12:19:32PM +0200, Heikki Linnakangas wrote: On 18.12.2011 10:54, David Fetter wrote: On Sun, Dec 18, 2011 at 10:14:38AM +0200, Heikki Linnakangas wrote: On 17.12.2011 23:33, David Fetter wrote: If this introduces new failure

Re: [HACKERS] Page Checksums

2011-12-18 Thread Peter Eisentraut
On sön, 2011-12-18 at 21:34 +0200, Heikki Linnakangas wrote: On 18.12.2011 20:44, David Fetter wrote: Any way to simulate them, even if it's by injecting faults into the source code? Hmm, it's hard to persuade the OS to suffer a torn page on purpose. What you could do is split the

Re: [HACKERS] Page Checksums

2011-12-18 Thread Jesper Krogh
On 2011-12-18 11:19, Heikki Linnakangas wrote: The patch requires that full page writes be on in order to obviate this problem by never reading a torn page. Doesn't help. Hint bit updates are not WAL-logged. I dont know if it would be seen as a half baked feature.. or similar, and I dont

Re: [HACKERS] splitting plpython into smaller parts

2011-12-18 Thread Peter Eisentraut
On tis, 2011-12-06 at 00:58 +0100, Jan Urbański wrote: Rebased against master after the SPI cursor patch has been committed. The first patch removes SPI boilerplate from the cursor functions as well and the second patch creates a plpython_cursor.c file. A side effect of creating a separate

Re: [HACKERS] Command Triggers

2011-12-18 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Hmm ... I don't think that I *am* ok with that. ISTM that we'd then find ourselves with any changes in utility statement parse trees amounting to a user-visible API break, and that's not an acceptable situation. Oh, you mean like exposing the parser for

Re: [HACKERS] splitting plpython into smaller parts

2011-12-18 Thread Jan Urbański
On 18/12/11 20:53, Peter Eisentraut wrote: On tis, 2011-12-06 at 00:58 +0100, Jan Urbański wrote: Rebased against master after the SPI cursor patch has been committed. The first patch removes SPI boilerplate from the cursor functions as well and the second patch creates a plpython_cursor.c

Re: [HACKERS] Command Triggers

2011-12-18 Thread Tom Lane
Dimitri Fontaine dimi...@2ndquadrant.fr writes: Tom Lane t...@sss.pgh.pa.us writes: We really need some higher-level API than the raw parse tree, and I have to admit that I have no idea what that would look like. But exposing parse trees to user-written triggers is a decision that we will

Re: [HACKERS] Command Triggers

2011-12-18 Thread Dimitri Fontaine
Tom Lane t...@sss.pgh.pa.us writes: Well, we don't have any such examples, because frankly the nodeToString representation is pretty damn unfriendly. The only code we have that I tend to agree here, but I know that Jan is convincing enough when he's saying that it is in fact very friendly.

Re: [HACKERS] Page Checksums

2011-12-18 Thread Greg Stark
On Sun, Dec 18, 2011 at 7:51 PM, Jesper Krogh jes...@krogh.cc wrote: I dont know if it would be seen as a half baked feature.. or similar, and I dont know if the hint bit problem is solvable at all, but I could easily imagine checksumming just skipping the hit bit entirely. That was one

Re: [HACKERS] Command Triggers

2011-12-18 Thread Robert Haas
On Sun, Dec 18, 2011 at 5:11 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: I guess XML would be ok but we don't embed powerful enough tools, and JSON might be perfect but we would need to have a full blown datatype and functions to work with that from plpgsql.  What other tree-ish data

Re: [HACKERS] Page Checksums

2011-12-18 Thread Josh Berkus
On 12/18/11 5:55 PM, Greg Stark wrote: There is another way to look at this problem. Perhaps it's worth having a checksum *even if* there are ways for the checksum to be spuriously wrong. Obviously having an invalid checksum can't be a fatal error then but it might still be useful information.

Re: [HACKERS] JSON for PG 9.2

2011-12-18 Thread Merlin Moncure
On Sun, Dec 18, 2011 at 12:26 PM, Dimitri Fontaine dimi...@2ndquadrant.fr wrote: Merlin Moncure mmonc...@gmail.com writes: Why would that matter more for JSON than for any other non-core type? well, it's a minor headache for all the oid-isn't-in-pgtypes.h types, and only then for high traffic

Re: [HACKERS] Page Checksums

2011-12-18 Thread Aidan Van Dyk
On Sun, Dec 18, 2011 at 11:21 PM, Josh Berkus j...@agliodbs.com wrote: On 12/18/11 5:55 PM, Greg Stark wrote: There is another way to look at this problem. Perhaps it's worth having a checksum *even if* there are ways for the checksum to be spuriously wrong. Obviously having an invalid