Re: [HACKERS] RangeType internal use

2015-02-05 Thread Kyotaro HORIGUCHI
Hi, from nearby:) I wonder why I cannot find a way to get a range type for a given (sub-) type. I would like to build a RangeType from Datum's of lower and upper bounds. Much like how construct_array() builds an ArrayType from a Datum array of elements given elements' type info. Is there

Re: [HACKERS] Early Setup of instrumentation information in pg_stat_statements

2015-02-05 Thread Amit Kapila
On Thu, Feb 5, 2015 at 8:58 PM, Amit Kapila amit.kapil...@gmail.com wrote: Currently in pg_stat_statements, the setup to track instrumentation/totaltime information is done after ExecutorStart(). Can we do this before ExecutorStart()? On further analyzing, I found that currently it is done

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2015-02-05 Thread Michael Paquier
On Thu, Feb 5, 2015 at 11:58 PM, Michael Paquier wrote: An updated patch is attached. I just noticed that the patch I sent was incorrect: - Parameter name was still wal_availability_check_interval and not wal_retrieve_retry_interval - Documentation was incorrect. Please use the patch attached

Re: [HACKERS] pg_basebackup may fail to send feedbacks.

2015-02-05 Thread Kyotaro HORIGUCHI
Sorry, I misunderstood that. At Wed, 4 Feb 2015 19:22:39 +0900, Fujii Masao masao.fu...@gmail.com wrote in cahgqgwgudgcmnhzinkd37i+jijdkruecrea1ncrs1mmte3r...@mail.gmail.com On Wed, Feb 4, 2015 at 4:58 PM, Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp wrote: I'm very sorry for

Re: [HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2015-02-05 Thread Michael Paquier
On Fri, Jan 30, 2015 at 10:26 PM, Andreas Karlsson wrote: On 01/30/2015 07:48 AM, Michael Paquier wrote: Looking at the latest patch, it seems that in AlterTableGetLockLevel@tablecmds.c we ought to put AT_ReAddConstraint, AT_AddConstraintRecurse and AT_ProcessedConstraint under the same

Re: [HACKERS] RangeType internal use

2015-02-05 Thread Amit Langote
Horiguchi-san, On 06-02-2015 PM 04:34, Kyotaro HORIGUCHI wrote: Hi, from nearby:) Thank you! I wonder why I cannot find a way to get a range type for a given (sub-) type. I would like to build a RangeType from Datum's of lower and upper bounds. Much like how construct_array() builds an

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Syed, Rahila
Hello, /* +* We recheck the actual size even if pglz_compress() report success, +* because it might be satisfied with having saved as little as one byte +* in the compressed data. +*/ + *len = (uint16) compressed_len; + if (*len = orig_len - 1) + return false; +

Re: [HACKERS] Unlikely-to-happen crash in ecpg driver caused by NULL-pointer check not done

2015-02-05 Thread Michael Meskes
On Tue, Feb 03, 2015 at 10:44:17PM +0900, Michael Paquier wrote: All those things are addressed in the patch attached. Fixed a typo and commited. Thanks Michael for fixing and Heikki for reviewing. Michael -- Michael Meskes Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Feb 3, 2015 at 6:26 PM, Noah Yetter nyet...@gmail.com wrote: The obvious objection is, well you should just use foreign tables instead of dblink(). I'll cut a long story short by saying that doesn't work for us. We are using

[HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Jan Wieck
Hi, I have encountered a small instability in the behavior of pgcrypto's pgp_sym_decrypt() function. Attached is a script that can reproduce the problem. It may have to be run repeatedly because the symptom occurs rather seldom. What the script does is to encode a small string with

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/05/2015 01:28 PM, Robert Haas wrote: On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus j...@agliodbs.com wrote: Except that, when setting up servers for customers, one thing I pretty much always do for them is temporarily increase checkpoint_segments for the initial data load. So having

Re: [HACKERS] SSL renegotiation and other related woes

2015-02-05 Thread Heikki Linnakangas
On 01/26/2015 12:14 PM, Andres Freund wrote: Hi, When working on getting rid of ImmediateInterruptOK I wanted to verify that ssl still works correctly. Turned out it didn't. But neither did it in master. Turns out there's two major things we do wrong: 1) We ignore the rule that once called

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus j...@agliodbs.com wrote: Except that, when setting up servers for customers, one thing I pretty much always do for them is temporarily increase checkpoint_segments for the initial data load. So having Postgres do this automatically would be a

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Heikki Linnakangas
On 02/05/2015 11:28 PM, Robert Haas wrote: On Thu, Feb 5, 2015 at 2:11 PM, Josh Berkus j...@agliodbs.com wrote: Default of 4 for min_wal_size? I assume you mean 4 segments; why not 3 as currently? As long as the system has the latitude to ratchet it up when needed, there seems to be little

Re: [HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Jan Wieck
On 02/05/2015 10:58 AM, Tom Lane wrote: Jan Wieck j...@wi3ck.info writes: I have encountered a small instability in the behavior of pgcrypto's pgp_sym_decrypt() function. Attached is a script that can reproduce the problem. It may have to be run repeatedly because the symptom occurs rather

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/04/2015 04:16 PM, David Steele wrote: On 2/4/15 3:06 PM, Robert Haas wrote: Hmmm, I see your point. I spend a lot of time on AWS and in container-world, where disk space is a lot more constrained. However, it probably makes more sense to recommend non-default settings for that

Re: [HACKERS] Possible problem with pgcrypto

2015-02-05 Thread Jan Wieck
On 02/05/2015 01:18 PM, Marko Tiikkaja wrote: On 2/5/15 4:48 PM, Jan Wieck wrote: What the script does is to encode a small string with pgp_sym_encrypt() and then repeatedly try to decrypt it with different wrong passwords. The expected error message for that is of course Wrong key or

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Michael Paquier
On Thu, Feb 5, 2015 at 11:06 PM, Syed, Rahila rahila.s...@nttdata.com wrote: /* +* We recheck the actual size even if pglz_compress() report success, +* because it might be satisfied with having saved as little as one byte +* in the compressed data. +*/ + *len = (uint16)

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Fujii Masao
On Tue, Jan 6, 2015 at 11:09 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Jan 5, 2015 at 10:29 PM, Fujii Masao masao.fu...@gmail.com wrote: On Sun, Dec 28, 2014 at 10:57 PM, Michael Paquier wrote: The patch 1 cannot be applied to the master successfully because of recent

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Andres Freund
On 2015-02-05 09:42:37 -0500, Robert Haas wrote: I previously proposed 100 segments, or 1.6GB. If that seems too large, how about 64 segments, or 1.024GB? I think there will be few people who can't tolerate a gigabyte of xlog under peak load, and an awful lot who will benefit from it. It'd

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: All that having been said, it wouldn't be crazy to try to invent a system to lock this down, but it *would* be complicated. An individual FDW can call its authentication-related options anything it likes; they do not need to be called 'password'. So

[HACKERS] Simplify sleeping while reading/writing from client

2015-02-05 Thread Heikki Linnakangas
Looking again at the code after Andres' interrupt-handling patch series, I got confused by the fact that there are several wait-retry loops in different layers, and reading and writing works slightly differently. I propose the attached, which pulls all the wait-retry logic up to secure_read()

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Robert Haas
On Tue, Feb 3, 2015 at 6:26 PM, Noah Yetter nyet...@gmail.com wrote: The obvious objection is, well you should just use foreign tables instead of dblink(). I'll cut a long story short by saying that doesn't work for us. We are using postgres_fdw to allow our analysts to run queries against

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Heikki Linnakangas
On 02/05/2015 04:47 PM, Andres Freund wrote: On 2015-02-05 09:42:37 -0500, Robert Haas wrote: I previously proposed 100 segments, or 1.6GB. If that seems too large, how about 64 segments, or 1.024GB? I think there will be few people who can't tolerate a gigabyte of xlog under peak load, and

[HACKERS] Early Setup of instrumentation information in pg_stat_statements

2015-02-05 Thread Amit Kapila
Currently in pg_stat_statements, the setup to track instrumentation/totaltime information is done after ExecutorStart(). Can we do this before ExecutorStart()? In particular, I am referring to below code: static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags) { ..

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
On Wed, Feb 4, 2015 at 4:41 PM, Josh Berkus j...@agliodbs.com wrote: That's certainly better, but I think we should go further. Again, you're not committed to using this space all the time, and if you are using it you must have a lot of write activity, which means you are not running on a tin

Re: [HACKERS] [GENERAL] 4B row limit for CLOB tables

2015-02-05 Thread Matthew Kelly
That's assuming that toasting is evenly spread between tables. In my experience, that's not a great bet... Time to create a test: SELECT chunk_id::bigint/10 as id_range, count(*), count(*)/(10::float) density FROM (SELECT chunk_id FROM pg_toast.pg_toast_39000165 WHERE chunk_id

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
Default of 4 for min_wal_size? I assume you mean 4 segments; why not 3 as currently? As long as the system has the latitude to ratchet it up when needed, there seems to be little advantage to raising the minimum. Of course I guess there must be some advantage or Heikki wouldn't have made

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Robert Haas
On Thu, Feb 5, 2015 at 4:42 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Actually, perhaps we should have a boolean setting that just implies min=max, instead of having a configurable minimum?. That would cover all of those reasons pretty well. So we would have a max_wal_size setting,

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread David Steele
On 2/5/15 4:53 PM, Josh Berkus wrote: Actually, perhaps we should have a boolean setting that just implies min=max, instead of having a configurable minimum?. That would cover all of those reasons pretty well. So we would have a max_wal_size setting, and a boolean preallocate_all_wal = on |

[HACKERS] HEADS UP: PGCon 2015 major schedule changes

2015-02-05 Thread Dan Langille
Hello, By request, the format of PGCon 2015 will differ significantly from previous year. Our goal is to give you more of what you want while still keeping the stuff you've always liked. In June 2015, PGCon will be structured as follows: Unconference: 16-17 June 2015 (Tue afternoon all day

Re: [HACKERS] Redesigning checkpoint_segments

2015-02-05 Thread Josh Berkus
On 02/05/2015 01:42 PM, Heikki Linnakangas wrote: There are a few reasons for making the minimum configurable: Any thoughts on what the default minimum should be, if the default max is 1.1GB/64? 1. Creating new segments when you need them is not free, so if you have a workload with occasional

[HACKERS] RangeType internal use

2015-02-05 Thread Amit Langote
Hi, I wonder why I cannot find a way to get a range type for a given (sub-) type. I would like to build a RangeType from Datum's of lower and upper bounds. Much like how construct_array() builds an ArrayType from a Datum array of elements given elements' type info. Is there some way I do not

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-02-05 Thread Naoya Anzai
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed Hi, I'm Naoya Anzai. I've been working as a PostgreSQL

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread David G Johnston
Tom Lane-2 wrote Stephen Frost lt; sfrost@ gt; writes: * Robert Haas ( robertmhaas@ ) wrote: On Thu, Feb 5, 2015 at 10:48 AM, Stephen Frost lt; sfrost@ gt; wrote: And I thought this was about FDW options and not about dblink, really.. The OP is pretty clearly asking about

Re: [HACKERS] GRANT USAGE on FOREIGN SERVER exposes passwords

2015-02-05 Thread Tom Lane
Stephen Frost sfr...@snowman.net writes: * Robert Haas (robertmh...@gmail.com) wrote: On Thu, Feb 5, 2015 at 10:48 AM, Stephen Frost sfr...@snowman.net wrote: And I thought this was about FDW options and not about dblink, really.. The OP is pretty clearly asking about dblink. I was just

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-02-05 Thread Naoya Anzai
The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:tested, failed I'm sorry, I just sent it by mistake. All of them have

[HACKERS] No-rewrite timestamp-timestamptz conversions

2015-02-05 Thread Noah Misch
On Tue, Nov 05, 2013 at 05:02:58PM -0800, Josh Berkus wrote: I'd also love some way of doing a no-rewrite conversion between timestamp and timestamptz, based on the assumption that the original values are UTC time. That's one I encounter a lot. It was such a conversion that motivated me to

Re: [HACKERS] File based Incremental backup v9

2015-02-05 Thread Francesco Canovai
Hi Marco, On Sunday 01 February 2015 00:47:24 Marco Nenciarini wrote: You can find the updated patch attached to this message. I've been testing the v9 patch with checksums enabled and I end up with a lot of warnings like these ones: WARNING: page verification failed, calculated checksum

Re: [HACKERS] Table-level log_autovacuum_min_duration

2015-02-05 Thread Michael Paquier
On Fri, Feb 6, 2015 at 4:50 AM, Naoya Anzai anzai-na...@mxu.nes.nec.co.jp wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, failed Implements feature: tested, failed Spec compliant: tested, failed Documentation:

[HACKERS] pg_basebackup, tablespace mapping and path canonicalization

2015-02-05 Thread Ian Barwick
Hi I stumbled on what appears to be inconsistent handling of double slashes in tablespace paths when using pg_basebackup with the -T/--tablespace-mapping option: ibarwick:postgresql (master)$ mkdir /tmp//foo-old ibarwick:postgresql (master)$ $PG_HEAD/psql 'dbname=postgres port=9595'

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-05 Thread Michael Paquier
Fujii Masao wrote: I wrote This is an inspiration from lz4 APIs. Wouldn't it be buggy for a compression algorithm to return a size of 0 bytes as compressed or decompressed length btw? We could as well make it return a negative value when a failure occurs if you feel more comfortable with it.

Re: [HACKERS] Proposal : REINDEX xxx VERBOSE

2015-02-05 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: We've got a mix of styles for extensible options right now: That we do. So COPY puts the options at the very end, but EXPLAIN and VACUUM put them right after the command name. I prefer the latter style and would vote to adopt it here. Meh.

Re: [HACKERS] Patch: add recovery_timeout option to control timeout of restore_command nonzero status code

2015-02-05 Thread Michael Paquier
Fujii Masao wrote: +TimestampDifference(start_time, stop_time, secs, microsecs); +pg_usleep(interval_msec * 1000L - (100L * secs + 1L * microsecs)); What if the large interval is set and a signal arrives during the sleep? I'm afraid that a signal cannot terminate the sleep on some

Re: [HACKERS] s_lock.h default definitions are rather confused

2015-02-05 Thread Andres Freund
On 2015-01-15 17:59:40 +0100, Andres Freund wrote: On 2015-01-15 11:56:24 -0500, Tom Lane wrote: Andres Freund and...@anarazel.de writes: On 2015-01-15 10:57:10 -0500, Tom Lane wrote: While I'll not cry too hard when we decide to break C89 compatibility, I don't want it to happen