Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-11 Thread Noah Misch
On Sun, Nov 11, 2012 at 02:18:11AM -0300, Alvaro Herrera wrote: Amit kapila wrote: On Saturday, November 10, 2012 10:19 PM Noah Misch wrote: This patch is now marked Returned with Feedback in the CF, but I see no on-list feedback. Did some review happen? No review happened for this

Re: [HACKERS] Failing SSL connection due to weird interaction with openssl

2012-11-11 Thread Lars Kanis
Am 06.11.2012 21:40, schrieb Robert Haas: On Tue, Oct 23, 2012 at 4:09 AM, Lars Kanis l...@greiz-reinsdorf.de wrote: While investigating a ruby-pg issue [1], we noticed that a libpq SSL connection can fail, if the running application uses OpenSSL for other work, too. Root cause is the thread

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch n...@leadboat.com wrote: On Sun, Nov 04, 2012 at 02:30:38PM -0500, Tom Lane wrote: Matthew Gerber gerber.matt...@gmail.com writes: Here is the command that was executing when the 0xC409 exception was raised: INSERT INTO places

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Matthew Gerber gerber.matt...@gmail.com writes: On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch n...@leadboat.com wrote: It signifies scribbling past the end of the frame's local variables: http://msdn.microsoft.com/en-us/library/8dbf701c.aspx A slight update on this: previously, my insert code

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: Matthew Gerber gerber.matt...@gmail.com writes: On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch n...@leadboat.com wrote: It signifies scribbling past the end of the frame's local variables:

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Matthew Gerber gerber.matt...@gmail.com writes: On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: How long is long? I was seeing queries with around 5000-7000 UNION ALL statements. Hm. I experimented with test queries created like so: perl -e 'print SELECT 1 a, 2 b, 3

Re: [HACKERS] too much pgbench init output

2012-11-11 Thread Tomas Vondra
On 23.10.2012 18:21, Robert Haas wrote: On Tue, Oct 23, 2012 at 12:02 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Tomas Vondra wrote: I've been thinking about this a bit more, and do propose to use an option that determines logging step i.e. number of items (either directly or as a

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 12:22 PM, Tom Lane t...@sss.pgh.pa.us wrote: Matthew Gerber gerber.matt...@gmail.com writes: On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane t...@sss.pgh.pa.us wrote: How long is long? I was seeing queries with around 5000-7000 UNION ALL statements. Hm. I

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Matthew Gerber gerber.matt...@gmail.com writes: Interesting. I really have no idea why mine seemed to fail so much sooner. I recalled my 5k-7k figure from memory, so I might be off on that, but probably not by an order of magnitude. In any case, it sounds like you know how to fix the problem.

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
On Sat, 2012-11-10 at 14:46 +0100, Florian Pflug wrote: The bit indicating that a checksum is present may be lost due to corruption. Though that concern mostly goes away if instead of a separate bit we use a special checksum value, say 0xDEAD, to indicate that the page isn't checksummed,

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Noah Misch
On Sun, Nov 11, 2012 at 12:22:24PM -0500, Tom Lane wrote: perl -e 'print SELECT 1 a, 2 b, 3 c\n; print UNION ALL SELECT 1 a, 2 b, 3 c\n foreach (1..8200);' | psql On the machine I tried this on, it works up to about 8200 and then fails in the way I'd expect: ERROR: stack depth limit

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-11-11 Thread Tomas Vondra
Hi, I've prepared a slightly updated patch, based on the previous review. See it attached. On 18.10.2012 04:28, 花田 茂 wrote: Hi Tomas, On 2012/10/17, at 20:45, Tomas Vondra t...@fuzzy.cz wrote: Dne 17.10.2012 12:34, Shigeru HANADA napsal: Performance test I tested 1000

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Noah Misch
On Sun, Nov 11, 2012 at 10:10:31AM -0500, Matthew Gerber wrote: Matthew Gerber gerber.matt...@gmail.com writes: Here is the command that was executing when the 0xC409 exception was raised: INSERT INTO places (bounding_box,country,full_name,id,name,type,url) VALUES

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
On Fri, 2012-11-09 at 09:57 -0800, Josh Berkus wrote: Huh? Why would a GUC not make sense? How else would you make sure that checksums where on when you started the system? If we stored the information in pg_control, you could check with pg_controldata. We could have a separate utility,

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 2:43 PM, Noah Misch n...@leadboat.com wrote: On Sun, Nov 11, 2012 at 10:10:31AM -0500, Matthew Gerber wrote: Matthew Gerber gerber.matt...@gmail.com writes: Here is the command that was executing when the 0xC409 exception was raised: INSERT INTO

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Pavel Stehule
Hello Does it make sense to store this information in pg_control? That doesn't require adding any new file, and it has the benefit that it's already checksummed. It's available during recovery and can be made available pretty easily in the places where we write data. And the next

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
On Sun, 2012-11-11 at 21:20 +0100, Pavel Stehule wrote: I don't think so GUC are good for this purpouse, but I don't like single purpouse statements too. what do you think about enhancing ALTER DATABASE statement some like ALTER DATABASE name ENABLE CHECKSUMS and ALTER DATABASE name

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-11 Thread Tom Lane
Attached is a complete draft patch against HEAD for this issue. I found a depressingly large number of pre-existing bugs: Practically all WAL record types that touch multiple pages have some bug of this type. In addition to btree_xlog_split, I found that heap_xlog_update, ginRedoDeletePage,

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-11 Thread Tom Lane
[ this time *with* the patch ] Attached is a complete draft patch against HEAD for this issue. I found a depressingly large number of pre-existing bugs: Practically all WAL record types that touch multiple pages have some bug of this type. In addition to btree_xlog_split, I found that

Re: [HACKERS] review: pgbench - aggregation of info written into log

2012-11-11 Thread Tomas Vondra
Hi, attached is a v4 of the patch. There are not many changes, mostly some simple tidying up, except for handling the Windows. After a bit more investigation, it seems to me that we can't really get the same behavior as in other systems - basically the timestamp is unavailable so we can't log

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Andrew Dunstan
On 11/11/2012 05:52 PM, Jeff Davis wrote: On Sun, 2012-11-11 at 21:20 +0100, Pavel Stehule wrote: I don't think so GUC are good for this purpouse, but I don't like single purpouse statements too. what do you think about enhancing ALTER DATABASE statement some like ALTER DATABASE name ENABLE

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-11 Thread Andres Freund
On 2012-11-10 16:24:22 -0500, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: One thing that seems a bit annoying is the use of zero-based backup block indexes in RestoreBackupBlock, when most (not all) of the callers are referencing macros with one-based names (XLR_BKP_BLOCK_1

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Noah Misch n...@leadboat.com writes: So, I can reproduce the lower threshold, but the exception type does not agree with the one Matthew observed. I finally got around to looking at the link you provided about error 0xC409, and realized that I'd been completely confusing it with stack

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 8:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: Noah Misch n...@leadboat.com writes: So, I can reproduce the lower threshold, but the exception type does not agree with the one Matthew observed. I finally got around to looking at the link you provided about error

Re: [HACKERS] Doc patch only relevant - relevant only

2012-11-11 Thread Peter Eisentraut
On Tue, 2012-10-16 at 22:24 -0500, Karl O. Pinc wrote: In a number of places the docs read only relevant, this patch reverses this to read relevant only. committed -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Greg Smith
On 11/11/12 2:56 PM, Jeff Davis wrote: We could have a separate utility, pg_checksums, that can alter the state and/or do an offline verification. And initdb would take an option that would start everything out fully protected with checksums. Adding an initdb option to start out with

Re: [HACKERS] WIP checksums patch

2012-11-11 Thread Greg Smith
On 11/9/12 6:14 PM, Jeff Davis wrote: On Mon, 2012-11-05 at 12:19 -0500, Robert Haas wrote: Yeah. I definitely think that we could shed an enormous amount of complexity by deciding that this is, for now, an option that can only be selected at initdb time. That would remove approximately 85%

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jesper Krogh
On 12/11/12 05:55, Greg Smith wrote: The only guarantee I see that we can give for online upgrades is that after a VACUUM CHECKSUM sweep is done, and every page is known to both have a valid checksum on it and have its checksum bits set, *then* any page that doesn't have both set bits and a

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Greg Smith
On 11/12/12 12:55 AM, Jesper Krogh wrote: I'd just like some rough guard against hardware/OS related data corruption. and that is more likely to hit data-blocks constantly flying in and out of the system. I get that. I think that some of the design ideas floating around since this feature

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-11 Thread Greg Smith
On 11/9/12 11:59 PM, Amit kapila wrote: Please let me know if there are any objections or problems in above method of implementation, else I can go ahead to prepare the patch for the coming CF. It may be the case that the locking scheme Robert described is the best approach here. It seems