Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-11 Thread Greg Smith
On 11/9/12 11:59 PM, Amit kapila wrote: Please let me know if there are any objections or problems in above method of implementation, else I can go ahead to prepare the patch for the coming CF. It may be the case that the locking scheme Robert described is the best approach here. It seems k

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Greg Smith
On 11/12/12 12:55 AM, Jesper Krogh wrote: I'd just like some rough guard against hardware/OS related data corruption. and that is more likely to hit data-blocks constantly flying in and out of the system. I get that. I think that some of the design ideas floating around since this feature was

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jesper Krogh
On 12/11/12 05:55, Greg Smith wrote: The only guarantee I see that we can give for online upgrades is that after a VACUUM CHECKSUM sweep is done, and every page is known to both have a valid checksum on it and have its checksum bits set, *then* any page that doesn't have both set bits and a mat

Re: [HACKERS] WIP checksums patch

2012-11-11 Thread Greg Smith
On 11/9/12 6:14 PM, Jeff Davis wrote: On Mon, 2012-11-05 at 12:19 -0500, Robert Haas wrote: Yeah. I definitely think that we could shed an enormous amount of complexity by deciding that this is, for now, an option that can only be selected at initdb time. That would remove approximately 85% of

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Greg Smith
On 11/11/12 2:56 PM, Jeff Davis wrote: We could have a separate utility, pg_checksums, that can alter the state and/or do an offline verification. And initdb would take an option that would start everything out fully protected with checksums. Adding an initdb option to start out with everything

Re: [HACKERS] Doc patch "only relevant" -> "relevant only"

2012-11-11 Thread Peter Eisentraut
On Tue, 2012-10-16 at 22:24 -0500, Karl O. Pinc wrote: > In a number of places the docs read "only relevant", > this patch reverses this to read "relevant only". committed -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.po

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 8:27 PM, Tom Lane wrote: > Noah Misch writes: > > So, I can reproduce the lower threshold, but the exception type does not > agree > > with the one Matthew observed. > > I finally got around to looking at the link you provided about error > 0xC409, and realized that I

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Noah Misch writes: > So, I can reproduce the lower threshold, but the exception type does not agree > with the one Matthew observed. I finally got around to looking at the link you provided about error 0xC409, and realized that I'd been completely confusing it with stack overflow --- but actu

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-11 Thread Andres Freund
On 2012-11-10 16:24:22 -0500, Tom Lane wrote: > Simon Riggs writes: > >> One thing that seems a bit annoying is the use of zero-based backup > >> block indexes in RestoreBackupBlock, when most (not all) of the callers > >> are referencing macros with one-based names (XLR_BKP_BLOCK_1 etc). > >> Tha

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Andrew Dunstan
On 11/11/2012 05:52 PM, Jeff Davis wrote: On Sun, 2012-11-11 at 21:20 +0100, Pavel Stehule wrote: I don't think so GUC are good for this purpouse, but I don't like single purpouse statements too. what do you think about enhancing ALTER DATABASE statement some like ALTER DATABASE name ENABLE

Re: [HACKERS] review: pgbench - aggregation of info written into log

2012-11-11 Thread Tomas Vondra
Hi, attached is a v4 of the patch. There are not many changes, mostly some simple tidying up, except for handling the Windows. After a bit more investigation, it seems to me that we can't really get the same behavior as in other systems - basically the timestamp is unavailable so we can't log the

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-11 Thread Tom Lane
[ this time *with* the patch ] Attached is a complete draft patch against HEAD for this issue. I found a depressingly large number of pre-existing bugs: Practically all WAL record types that touch multiple pages have some bug of this type. In addition to btree_xlog_split, I found that heap_xlog

Re: [HACKERS] Inadequate thought about buffer locking during hot standby replay

2012-11-11 Thread Tom Lane
Attached is a complete draft patch against HEAD for this issue. I found a depressingly large number of pre-existing bugs: Practically all WAL record types that touch multiple pages have some bug of this type. In addition to btree_xlog_split, I found that heap_xlog_update, ginRedoDeletePage, spgR

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
On Sun, 2012-11-11 at 21:20 +0100, Pavel Stehule wrote: > I don't think so GUC are good for this purpouse, but I don't like > single purpouse statements too. > > what do you think about enhancing ALTER DATABASE statement > > some like > > ALTER DATABASE name ENABLE CHECKSUMS and ALTER DATABASE n

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Pavel Stehule
Hello > >> > Does it make sense to store this information in pg_control? That doesn't >> > require adding any new file, and it has the benefit that it's already >> > checksummed. It's available during recovery and can be made available >> > pretty easily in the places where we write data. >> > >>

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 2:43 PM, Noah Misch wrote: > On Sun, Nov 11, 2012 at 10:10:31AM -0500, Matthew Gerber wrote: > > > > Matthew Gerber writes: > > > > >> Here is the command that was executing when the 0xC409 > exception > > > was > > > > >> raised: > > > > >> INSERT INTO places > (boun

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
On Fri, 2012-11-09 at 09:57 -0800, Josh Berkus wrote: > Huh? Why would a GUC not make sense? How else would you make sure that > checksums where on when you started the system? If we stored the information in pg_control, you could check with pg_controldata. We could have a separate utility, pg_c

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Noah Misch
On Sun, Nov 11, 2012 at 10:10:31AM -0500, Matthew Gerber wrote: > > > Matthew Gerber writes: > > > >> Here is the command that was executing when the 0xC409 exception > > was > > > >> raised: > > > >> INSERT INTO places (bounding_box,country,full_name,id,name,type,url) > > > >> VALUES > > > >>

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2012-11-11 Thread Tomas Vondra
Hi, I've prepared a slightly updated patch, based on the previous review. See it attached. On 18.10.2012 04:28, 花田 茂 wrote:> Hi Tomas, > > On 2012/10/17, at 20:45, Tomas Vondra wrote: >> >> Dne 17.10.2012 12:34, Shigeru HANADA napsal: >>> Performance test >>> >>> I tested 1000 t

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Noah Misch
On Sun, Nov 11, 2012 at 12:22:24PM -0500, Tom Lane wrote: > perl -e 'print "SELECT 1 a, 2 b, 3 c\n"; print "UNION ALL SELECT 1 a, 2 b, 3 > c\n" foreach (1..8200);' | psql > > On the machine I tried this on, it works up to about 8200 and then fails > in the way I'd expect: > > ERROR: stack depth

Re: [HACKERS] Enabling Checksums

2012-11-11 Thread Jeff Davis
On Sat, 2012-11-10 at 14:46 +0100, Florian Pflug wrote: > > The bit indicating that a checksum is present may be lost due to > > corruption. > > Though that concern mostly goes away if instead of a separate bit we use a > special checksum value, say 0xDEAD, to indicate that the page isn't > checks

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Matthew Gerber writes: > Interesting. I really have no idea why mine seemed to fail so much sooner. > I recalled my 5k-7k figure from memory, so I might be off on that, but > probably not by an order of magnitude. In any case, it sounds like you know > how to fix the problem. Should I file this as

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 12:22 PM, Tom Lane wrote: > Matthew Gerber writes: > > On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane wrote: > >> How long is "long"? > > > I was seeing queries with around 5000-7000 "UNION ALL" statements. > > Hm. I experimented with test queries created like so: > > perl

Re: [HACKERS] too much pgbench init output

2012-11-11 Thread Tomas Vondra
On 23.10.2012 18:21, Robert Haas wrote: > On Tue, Oct 23, 2012 at 12:02 PM, Alvaro Herrera > wrote: >> Tomas Vondra wrote: >> >>> I've been thinking about this a bit more, and do propose to use an >>> option that determines "logging step" i.e. number of items (either >>> directly or as a percentag

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Matthew Gerber writes: > On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane wrote: >> How long is "long"? > I was seeing queries with around 5000-7000 "UNION ALL" statements. Hm. I experimented with test queries created like so: perl -e 'print "SELECT 1 a, 2 b, 3 c\n"; print "UNION ALL SELECT 1 a, 2

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 11:19 AM, Tom Lane wrote: > Matthew Gerber writes: > > On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch wrote: > >> It signifies scribbling past the end of the frame's local variables: > >> http://msdn.microsoft.com/en-us/library/8dbf701c.aspx > > > A slight update on this:

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Tom Lane
Matthew Gerber writes: > On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch wrote: >> It signifies scribbling past the end of the frame's local variables: >> http://msdn.microsoft.com/en-us/library/8dbf701c.aspx > A slight update on this: previously, my insert code involved a long > "SELECT ... UNION

Re: [HACKERS] Unresolved error 0xC0000409 on Windows Server

2012-11-11 Thread Matthew Gerber
On Sun, Nov 11, 2012 at 12:23 AM, Noah Misch wrote: > On Sun, Nov 04, 2012 at 02:30:38PM -0500, Tom Lane wrote: > > Matthew Gerber writes: > > >> Here is the command that was executing when the 0xC409 exception > was > > >> raised: > > >> INSERT INTO places (bounding_box,country,full_name,id

Re: [HACKERS] Failing SSL connection due to weird interaction with openssl

2012-11-11 Thread Lars Kanis
Am 06.11.2012 21:40, schrieb Robert Haas: > On Tue, Oct 23, 2012 at 4:09 AM, Lars Kanis wrote: >> While investigating a ruby-pg issue [1], we noticed that a libpq SSL >> connection can fail, if the running application uses OpenSSL for other work, >> too. Root cause is the thread local error queue

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-11 Thread Noah Misch
On Sun, Nov 11, 2012 at 02:18:11AM -0300, Alvaro Herrera wrote: > Amit kapila wrote: > > On Saturday, November 10, 2012 10:19 PM Noah Misch wrote: > > > This patch is now marked Returned with Feedback in the CF, but I see no > > > on-list feedback. Did some review happen? > > > > No review happen