Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Albe Laurenz
Hans-Jürgen Schönig wrote: in addition to that you have the “problem” of transactions. if you failover in the middle of a transaction, strange things might happen from the application point of view. the good thing, however, is that stupid middleware is sometimes not able to handle

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread Thom Brown
On 18 August 2015 at 11:30, Amit Langote langote_amit...@lab.ntt.co.jp wrote: Hi, I would like propose $SUBJECT for this development cycle. Attached is a WIP patch that implements most if not all of what's described below. Some yet unaddressed parts are mentioned below, too. I'll add this

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread PostgreSQL - Hans-Jürgen Schönig
On 18 Aug 2015, at 11:19, Albe Laurenz laurenz.a...@wien.gv.at wrote: Hans-Jürgen Schönig wrote: in addition to that you have the “problem” of transactions. if you failover in the middle of a transaction, strange things might happen from the application point of view. the good thing,

Re: [HACKERS] Raising our compiler requirements for 9.6

2015-08-18 Thread Noah Misch
On Mon, Aug 17, 2015 at 12:06:42PM +0200, Andres Freund wrote: On 2015-08-16 03:31:48 -0400, Noah Misch wrote: I'd love to make it a #warning intead of an error, but unfortunately that's not standard C :( Okay. Other than that benefit, making headers #error-on-FRONTEND mostly lets us

Re: [HACKERS] jsonb array-style subscripting

2015-08-18 Thread Oleg Bartunov
On Mon, Aug 17, 2015 at 11:26 PM, Peter Geoghegan p...@heroku.com wrote: On Mon, Aug 17, 2015 at 12:26 PM, Merlin Moncure mmonc...@gmail.com wrote: ...is a good idea. postgres operators tend to return immutable copies of the item they are referring to. This patch does not add an operator

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Albe Laurenz
Victor Wagner wrote: Rationale = Since introduction of the WAL-based replication into the PostgreSQL, it is possible to create high-availability and load-balancing clusters. However, there is no support for failover in the client libraries. So, only way to provide transparent for

Re: [HACKERS] [Proposal] Table partition + join pushdown

2015-08-18 Thread Kouhei Kaigai
Hello Kondo-san, I briefly checked your patch. Let me put some comments about its design and implementation, even though I have no arguments towards its concept. :-) * Construction of RelOptInfo In your patch, try_hashjoin_pushdown() called by try_hashjoin_path() constructs RelOptInfo of the

Re: [HACKERS] checkpointer continuous flushing

2015-08-18 Thread Fabien COELHO
Hello Amit, So the option is best kept as off for now, without further data, I'm fine with that. One point to think here is on what basis user can decide make this option on, is it predictable in any way? I think one case could be when the data set fits in shared_buffers. Yep. In general,

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread PostgreSQL - Hans-Jürgen Schönig
On 18 Aug 2015, at 10:32, Albe Laurenz laurenz.a...@wien.gv.at wrote: Victor Wagner wrote: Rationale = Since introduction of the WAL-based replication into the PostgreSQL, it is possible to create high-availability and load-balancing clusters. However, there is no support for

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 7:59 AM, Peter Eisentraut pete...@gmx.net wrote: How would we handle decreases at run time? We can prevent =archive - minimal if archiving is running or there are physical replication slots, and we can prevent logical - something less if there are logical replication

[HACKERS] allowing wal_level change at run time

2015-08-18 Thread Peter Eisentraut
In [1], it was discussed to make wal_level changeable at run time (PGC_SIGHUP), as part of an effort to make replication easier to set up and/or provide better defaults. I was wondering what it would take to do that. I looks like increasing the setting is doable, as long as we WAL-log the change

Re: [HACKERS] Test code is worth the space

2015-08-18 Thread David Fetter
On Tue, Aug 18, 2015 at 02:03:19PM +0100, Greg Stark wrote: On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch n...@leadboat.com wrote: I suspect any effort to significantly improve Postgres test coverage is doomed until there's an alternative to pg_regress. There is the

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread Amit Langote
Hi Thom, On Tue, Aug 18, 2015 at 8:02 PM, Thom Brown t...@linux.com wrote: Wow, didn't expect to see that email this morning. A very quick test: CREATE TABLE purchases (purchase_id serial, purchase_time timestamp, item text) partition by range on ((extract(year from

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Magnus Hagander
On Tue, Aug 18, 2015 at 1:59 PM, Peter Eisentraut pete...@gmx.net wrote: In [1], it was discussed to make wal_level changeable at run time (PGC_SIGHUP), as part of an effort to make replication easier to set up and/or provide better defaults. I was wondering what it would take to do that.

Re: [HACKERS] Test code is worth the space

2015-08-18 Thread Greg Stark
On Tue, Aug 18, 2015 at 6:57 AM, Noah Misch n...@leadboat.com wrote: My own position is based on having maintained a pg_regress suite an order of magnitude larger than that. I don't know why that outcome was so different. Comparing the size of test suites by these numbers is impossible because

Re: [HACKERS] Freeze avoidance of very large table.

2015-08-18 Thread Masahiko Sawada
On Mon, Aug 10, 2015 at 11:05 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Mon, Aug 10, 2015 at 12:39 AM, Robert Haas robertmh...@gmail.com wrote: On Thu, Aug 6, 2015 at 11:33 AM, Jim Nasby jim.na...@bluetreble.com wrote: They also provide a level of control over what is and isn't

Re: [HACKERS] missing documentation for partial WAL files

2015-08-18 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Mon, Aug 17, 2015 at 2:50 PM, Peter Eisentraut pete...@gmx.net wrote: The commit message for de76884 contains some important information about the purpose and use of the new .partial WAL files. But I don't see anything about this in the

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-18 Thread Pavel Stehule
Hi 2015-08-17 23:46 GMT+02:00 Tom Lane t...@sss.pgh.pa.us: Jim Nasby jim.na...@bluetreble.com writes: On 8/17/15 9:48 AM, Tom Lane wrote: I'm inclined to think that if we wanted to make this better, the way to improve it would be to detect the error*at compile time*, and get rid of this

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread Marc Mamin
great :-) 2. Creating a partition of a partitioned table CREATE TABLE table_name PARTITION OF partitioned_table_name FOR VALUES values_spec; Where values_spec is: listvalues: [IN] (val1, ...) Would it make sense to allow one complementary partition to the listvalues? listvalues: [[NOT]

Re: [HACKERS] missing documentation for partial WAL files

2015-08-18 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: * Robert Haas (robertmh...@gmail.com) wrote: On Mon, Aug 17, 2015 at 2:50 PM, Peter Eisentraut pete...@gmx.net wrote: The commit message for de76884 contains some important information about the purpose and use of the new .partial WAL files.

Re: [HACKERS] Autonomous Transaction is back

2015-08-18 Thread Merlin Moncure
On Tue, Aug 18, 2015 at 8:17 AM, Robert Haas robertmh...@gmail.com wrote: On Sat, Aug 15, 2015 at 6:47 PM, Noah Misch n...@leadboat.com wrote: CREATE TABLE t (c) AS SELECT 1; BEGIN; UPDATE t SET c = 2 WHERE c = 1; BEGIN_AUTONOMOUS; UPDATE t SET c = 3 WHERE c = 1; UPDATE t SET c = 4 WHERE c

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 12:53 PM, Jaime Casanova jaime.casan...@2ndquadrant.com wrote: This is not completely true, you can always use something like pgbouncer or other middleware to change the server to which clients connect. you still need to solve the fact that you will have a read-only

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread Josh Berkus
Amit, I would like propose $SUBJECT for this development cycle. Attached is a WIP patch that implements most if not all of what's described below. Some yet unaddressed parts are mentioned below, too. I'll add this to the CF-SEP. First of all, wow! Really happy to see this. Syntax ==

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Andres Freund
On 2015-08-18 13:24:54 -0400, Peter Eisentraut wrote: On 8/18/15 12:35 PM, Robert Haas wrote: If archive_mode=on or max_wal_senders0, then we need at least wal_level=archive. Otherwise wal_level=minimal is enough. Totally forgot about max_wal_senders. However, the thread I linked to

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread Corey Huinker
On Tue, Aug 18, 2015 at 6:30 AM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: Hi, I would like propose $SUBJECT for this development cycle. Attached is a WIP patch that implements most if not all of what's described below. Some yet unaddressed parts are mentioned below, too. I'll add

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: On Tue, Aug 18, 2015 at 2:07 PM, Stephen Frost sfr...@snowman.net wrote: SCRAM itself, as has been discussed, supports multiple password verifiers- that's a specific case all by itself, and it's done specifically to address the issue that one or

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Peter Eisentraut
On 8/18/15 12:35 PM, Robert Haas wrote: If archive_mode=on or max_wal_senders0, then we need at least wal_level=archive. Otherwise wal_level=minimal is enough. Totally forgot about max_wal_senders. However, the thread I linked to earlier aimed for a different master plan (or if not, I'm

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 2:07 PM, Stephen Frost sfr...@snowman.net wrote: I would expect there to be people who would run into pg_upgrade complaining, that's why there would be the check. That's actually a much better situation than what happened around standard_conforming_strings. Further,

Re: [HACKERS] [patch] psql tab completion for grant execute

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 5:07 PM, Daniel Verite dan...@manitou-mail.org wrote: When tab-completing after GRANT EXECUTE, currently psql injects PROCEDURE, rather than the expected ON. The code for completing with ON is there, but it's not reached due to falling earlier into another branch, one

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: On Tue, Aug 18, 2015 at 10:06 AM, Stephen Frost sfr...@snowman.net wrote: That was the imputus for my earlier suggestion that in a release or two, we actively make pg_upgrade error (or perhaps warn first, then error, across two releases)

[HACKERS] a few doubts around check_for_isn_and_int8_passing_mismatch

2015-08-18 Thread Robert Haas
pg_upgrade's check.c contains a function called check_for_isn_and_int8_passing_mismatch. If the float8 pass-by-value values differ between the old and new clusters, which is likely to happen on Windows 64 because of cf376a4ad, and if the old cluster contains any databases which contain functions

[HACKERS] Extension upgrade and GUCs

2015-08-18 Thread Paul Ramsey
Hi hackers, I've been wrestling with this one for a while and gone down a couple blind alleys, so time to ask the experts. PostGIS has a couple things different from the extensions that live in contrib, - it has some GUCs - it has a versioned loadable library (postgis-2.1.so, postgis-2.2.so,

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Greg Stark
On Tue, Aug 18, 2015 at 7:19 PM, Robert Haas robertmh...@gmail.com wrote: Sorry, that's a completely bogus argument. We do not need to prevent people from upgrading if they haven't moved off of MD5 authentication; that's just an arbitrary - and IMHO extremely user-hostile - policy decision.

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread Kevin Grittner
Kouhei Kaigai kai...@ak.jp.nec.com wrote: longlbuckets; lbuckets = 1 my_log2(hash_table_bytes / bucket_size); Assert(nbuckets 0); In my case, the hash_table_bytes was 101017630802, and bucket_size was 48. So, my_log2(hash_table_bytes / bucket_size) = 31, then

Re: [HACKERS] Add support for RADIUS passwords longer than 16 octets

2015-08-18 Thread Marko Tiikkaja
On 2015-08-15 17:55, I wrote: The attached patch adds support for RADIUS passwords longer than 16 octets. Improved the coding and comments a bit, new version attached. .m *** a/src/backend/libpq/auth.c --- b/src/backend/libpq/auth.c *** *** 2168,2173 CheckCertAuth(Port

Re: [HACKERS] Test code is worth the space

2015-08-18 Thread David Fetter
On Tue, Aug 18, 2015 at 04:54:07PM +0100, Greg Stark wrote: On Tue, Aug 18, 2015 at 2:16 PM, David Fetter da...@fetter.org wrote: I'm given to understand that this tight coupling is necessary for performance. Are you saying that it could be unwound, or that testing strategies mostly need

Re: [HACKERS] More WITH

2015-08-18 Thread Tom Lane
Craig Ringer cr...@2ndquadrant.com writes: On 18 August 2015 at 01:18, David Fetter da...@fetter.org wrote: FETCH [in WITH] I'd be a huge fan of this one. I'd love to see FETCH in subqueries, too. Currently doing anything like this requires an ugly PL/PgSQL wrapper. The cursor would have

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Tatsuo Ishii
I wonder how extended protocol is handled by this proposal. Suppose load balacing mode is enabled. PQprepare is executed on standby1. Then PQexecPrepared gets called. This may be executed on standby2, which will fail because there's no prepared statement created by the former PQprepare call. Even

Re: [HACKERS] [DESIGN] ParallelAppend

2015-08-18 Thread Amit Kapila
On Thu, Aug 13, 2015 at 5:26 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: On Fri, Aug 7, 2015 at 2:15 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Sure, that is what we should do, however the tricky part would be when the path for doing local scan is extremely cheaper than path for

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread Kouhei Kaigai
I reported a similar issue before. * Re: DBT-3 with SF=20 got failed http://www.postgresql.org/message-id/557a19d1.9050...@2ndquadrant.com I didn't get a server crash at that time, however, palloc() complained about request size = 1GB. So, we may need a couple of overhaul around HashJoin to

Re: [HACKERS] checkpointer continuous flushing

2015-08-18 Thread Amit Kapila
On Tue, Aug 18, 2015 at 12:38 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: Hello Amit, So the option is best kept as off for now, without further data, I'm fine with that. One point to think here is on what basis user can decide make this option on, is it predictable in any way? I think

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Amit Kapila
On Tue, Aug 18, 2015 at 9:48 AM, Victor Wagner vi...@wagner.pp.ru wrote: Behavoir If PQconnectdb encounters connect string with multiple hosts specified, it attempts to establish connection with all these hosts simultaneously, and begins to work with server which responds first,

Re: [HACKERS] More WITH

2015-08-18 Thread Craig Ringer
On 18 August 2015 at 01:18, David Fetter da...@fetter.org wrote: FETCH [in WITH] I'd be a huge fan of this one. I'd love to see FETCH in subqueries, too. Currently doing anything like this requires an ugly PL/PgSQL wrapper. The cursor would have to be known at plan-time so it could be

Re: [HACKERS] How to compile, link and use a C++ extension

2015-08-18 Thread Craig Ringer
On 15 August 2015 at 00:51, Andres Freund and...@anarazel.de wrote: I started my tests by cloning the contrib/worker_spi code, and when transforming the code into C++, I could only note that C++ is not supported in the provided Makefiles. Yes, that doesn't surprise me. Postgres itself

[HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-18 Thread Jeff Janes
When we check a tuple for MVCC, it has to pass checks that the inserting transaction has committed, and that it committed before our snapshot began. And similarly that the deleting transaction hasn't committed, or did so after our snapshot. XidInMVCCSnapshot is (or can be) very much cheaper than

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-18 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: When we check a tuple for MVCC, it has to pass checks that the inserting transaction has committed, and that it committed before our snapshot began. And similarly that the deleting transaction hasn't committed, or did so after our snapshot.

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread Kouhei Kaigai
Kouhei Kaigai kai...@ak.jp.nec.com wrote: longlbuckets; lbuckets = 1 my_log2(hash_table_bytes / bucket_size); Assert(nbuckets 0); In my case, the hash_table_bytes was 101017630802, and bucket_size was 48. So, my_log2(hash_table_bytes / bucket_size)

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread David Rowley
On 19 August 2015 at 08:54, Kevin Grittner kgri...@ymail.com wrote: Kouhei Kaigai kai...@ak.jp.nec.com wrote: longlbuckets; lbuckets = 1 my_log2(hash_table_bytes / bucket_size); Assert(nbuckets 0); In my case, the hash_table_bytes was 101017630802, and

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread David Rowley
On 19 August 2015 at 12:23, Kouhei Kaigai kai...@ak.jp.nec.com wrote: -Original Message- From: David Rowley [mailto:david.row...@2ndquadrant.com] Sent: Wednesday, August 19, 2015 9:00 AM The size of your hash table is 101017630802 bytes, which is: david=# select

Re: [HACKERS] Make HeapTupleSatisfiesMVCC more concurrent

2015-08-18 Thread Tom Lane
I wrote: Just thinking about this ... I wonder why we need to call TransactionIdIsInProgress() at all rather than believing the answer from the snapshot? Under what circumstances could TransactionIdIsInProgress() return true where XidInMVCCSnapshot() had not? I experimented with the attached

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-18 Thread Kouhei Kaigai
On Mon, Aug 17, 2015 at 6:40 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Here is one other thing I could learn from TPC-DS benchmark. The attached query is Q4 of TPC-DS, and its result was towards SF=100. It took long time to compete (about 30min), please see the attached EXPLAIN

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Michael Paquier
On Wed, Aug 19, 2015 at 2:46 AM, Andres Freund and...@anarazel.de wrote: On 2015-08-18 13:24:54 -0400, Peter Eisentraut wrote: But if we tie the effective wal_level to archive_mode or max_wal_senders, both of which are restart-only, then we haven't gained anything. (We would have removed half

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Peter Eisentraut
On 8/18/15 1:46 PM, Andres Freund wrote: I don't think not requiring restarts is sufficient, having to twiddle a bunch of parameters manually still is a lot more effort than people see as necessary. I agree that we want both. But requiring a restart is a hard stop, whereas making

Re: [HACKERS] Error message with plpgsql CONTINUE

2015-08-18 Thread Tom Lane
Pavel Stehule pavel.steh...@gmail.com writes: when I implemented this check in plpgsql_check I found another minor issue in CONTINUE statement - the typename is wrong Hmmm ... a bit of nosing around says that fetch/move and get diagnostics are similarly sloppy. Will fix.

Re: [HACKERS] pgbench - allow backslash-continuations in custom scripts

2015-08-18 Thread Kyotaro HORIGUCHI
Hi, all. I don't think we actually want backslash-continuations. The feature we want is allow SQL statements span multiple lines, and using the psql lexer solves that. We don't need the backslash-continuations when we have that. Sure. The feature *I* initially wanted was to have

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread Kouhei Kaigai
-Original Message- From: David Rowley [mailto:david.row...@2ndquadrant.com] Sent: Wednesday, August 19, 2015 9:00 AM To: Kevin Grittner Cc: Kaigai Kouhei(海外 浩平); pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread Tom Lane
David Rowley david.row...@2ndquadrant.com writes: david=# set work_mem = '94GB'; ERROR: 98566144 is outside the valid range for parameter work_mem (64 .. 2097151) Apparently you're testing on a 32-bit server. 64-bit servers allow work_mem to go up to INT_MAX kilobytes.

Re: [HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread David Rowley
On 19 August 2015 at 12:38, Tom Lane t...@sss.pgh.pa.us wrote: David Rowley david.row...@2ndquadrant.com writes: david=# set work_mem = '94GB'; ERROR: 98566144 is outside the valid range for parameter work_mem (64 .. 2097151) Apparently you're testing on a 32-bit server. 64-bit

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-18 Thread Qingqing Zhou
On Mon, Aug 17, 2015 at 6:40 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Here is one other thing I could learn from TPC-DS benchmark. The attached query is Q4 of TPC-DS, and its result was towards SF=100. It took long time to compete (about 30min), please see the attached EXPLAIN ANALYZE

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Michael Paquier
On Wed, Aug 19, 2015 at 5:30 AM, Greg Stark st...@mit.edu wrote: On Tue, Aug 18, 2015 at 7:19 PM, Robert Haas robertmh...@gmail.com wrote: Sorry, that's a completely bogus argument. We do not need to prevent people from upgrading if they haven't moved off of MD5 authentication; that's just

[HACKERS] proposal: function parse_ident

2015-08-18 Thread Pavel Stehule
Hi I miss a functionality that helps with parsing any identifier to basic three parts - database, schema, objectname. We have this function internally, but it is not available for SQL layer. FUNCTION parse_ident(IN ident text, OUT dbname text, OUT schemaname text, OUT objectname text) Examples:

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 9:40 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: I think SortSupport logic provides a reasonable way to solve this kind of problem. For example, btint4sortsupport() informs a function pointer of the fast version of comparator (btint4fastcmp) which takes two Datum

Re: [HACKERS] Declarative partitioning

2015-08-18 Thread David Fetter
On Tue, Aug 18, 2015 at 07:30:20PM +0900, Amit Langote wrote: Hi, I would like propose $SUBJECT for this development cycle. Attached is a WIP patch that implements most if not all of what's described below. Some yet unaddressed parts are mentioned below, too. I'll add this to the CF-SEP.

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 8/18/15 8:48 AM, Robert Haas wrote: What do you mean by prevent? If the user edits postgresql.conf and reduces the setting, and then reloads the configuration file, they have a right to expect that the changes got applied. We have certain checks

Re: [HACKERS] jsonb array-style subscripting

2015-08-18 Thread Andrew Dunstan
On 08/18/2015 01:11 AM, Kaare Rasmussen wrote: On 2015-08-17 22:33, Josh Berkus wrote: So, both perl and python do not allow deep nesting of assignments. For example: d = { a : { } } d[a][a1][a2] = 42 Traceback (most recent call last): File stdin, line 1, in module KeyError: 'a1' Not

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-18 Thread Robert Haas
On Mon, Aug 10, 2015 at 12:36 AM, Rahila Syed rahilasye...@gmail.com wrote: Hello, Say, 6 bigint counters, 6 float8 counters, and 3 strings up to 80 characters each. So we have a fixed-size chunk of shared memory per backend, and each backend that wants to expose progress information can fill

Re: [HACKERS] Configure with thread sanitizer fails the thread test

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 3:02 PM, Andres Freund and...@anarazel.de wrote: On 2015-08-17 14:31:24 -0300, Alvaro Herrera wrote: The postmaster process in particular runs in a rather unusual arrangement, where most of the interesting stuff does happen in signal handlers. FWIW, I think it might

Re: [HACKERS] Warnings around booleans

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 8:01 AM, Andres Freund and...@anarazel.de wrote: I'd rather see those split into separate commits. Doing polishing and active bugs in one commit imo isn't a good idea if the polishing goes beyond a line or two. +1. -- Robert Haas EnterpriseDB:

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Peter Eisentraut
On 8/18/15 8:48 AM, Robert Haas wrote: On Tue, Aug 18, 2015 at 7:59 AM, Peter Eisentraut pete...@gmx.net wrote: How would we handle decreases at run time? We can prevent =archive - minimal if archiving is running or there are physical replication slots, and we can prevent logical - something

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Stephen Frost
Josh, * Josh Berkus (j...@agliodbs.com) wrote: I don't feel like you've correctly assessed the risk inherent in the md5 auth method, which is that, having captured an md5auth string by whatever means, and attacker can reuse that md5 string on other databases in the network *without* cracking

Re: [HACKERS] jsonb array-style subscripting

2015-08-18 Thread Andrew Dunstan
On 08/18/2015 01:32 AM, Pavel Stehule wrote: Hi 2015-08-17 21:12 GMT+02:00 Jim Nasby jim.na...@bluetreble.com mailto:jim.na...@bluetreble.com: On 8/17/15 12:57 PM, Dmitry Dolgov wrote: * is it interesting for the community? We definitely need better ways to manipulate

Re: [HACKERS] Autonomous Transaction is back

2015-08-18 Thread Robert Haas
On Sat, Aug 15, 2015 at 6:47 PM, Noah Misch n...@leadboat.com wrote: CREATE TABLE t (c) AS SELECT 1; BEGIN; UPDATE t SET c = 2 WHERE c = 1; BEGIN_AUTONOMOUS; UPDATE t SET c = 3 WHERE c = 1; UPDATE t SET c = 4 WHERE c = 2; COMMIT_AUTONOMOUS; ROLLBACK; If you replace the autonomous

Re: [HACKERS] DTrace build dependency rules

2015-08-18 Thread Robert Haas
On Sat, Aug 15, 2015 at 6:45 PM, Mark Johnston ma...@freebsd.org wrote: There seems to be a bug in the make rules when DTrace is enabled. It causes dtrace -G to be invoked twice when building PostgreSQL as a FreeBSD port: once during the build itself, and once during installation. For a long

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 9:50 AM, Tom Lane t...@sss.pgh.pa.us wrote: Peter Eisentraut pete...@gmx.net writes: On 8/18/15 8:48 AM, Robert Haas wrote: What do you mean by prevent? If the user edits postgresql.conf and reduces the setting, and then reloads the configuration file, they have a

[HACKERS] Bug? ExecChooseHashTableSize() got assertion failed with crazy number of rows

2015-08-18 Thread Kouhei Kaigai
Hello, I noticed ExecChooseHashTableSize() in nodeHash.c got failed by Assert(nbuckets 0), when crazy number of rows are expected. BACKTRACE: #0 0x003f79432625 in raise () from /lib64/libc.so.6 #1 0x003f79433e05 in abort () from /lib64/libc.so.6 #2 0x0092600a in

Re: [HACKERS] Freeze avoidance of very large table.

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 7:27 AM, Masahiko Sawada sawada.m...@gmail.com wrote: I have encountered the much cases where pg_stat_statement, pgstattuples are required in production, so I basically agree with moving such extension into core. But IMO, the diagnostic tools for visibility map, heap

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 6:07 AM, PostgreSQL - Hans-Jürgen Schönig postg...@cybertec.at wrote: On 18 Aug 2015, at 11:19, Albe Laurenz laurenz.a...@wien.gv.at wrote: Hans-Jürgen Schönig wrote: in addition to that you have the “problem” of transactions. if you failover in the middle of a

Re: [HACKERS] WIP: SCRAM authentication

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 10:06 AM, Stephen Frost sfr...@snowman.net wrote: That was the imputus for my earlier suggestion that in a release or two, we actively make pg_upgrade error (or perhaps warn first, then error, across two releases) if any of the old verifiers exist. I think there's

Re: [HACKERS] Proposal: Implement failover on libpq connect level.

2015-08-18 Thread Jaime Casanova
On 17 August 2015 at 23:18, Victor Wagner vi...@wagner.pp.ru wrote: Rationale = Since introduction of the WAL-based replication into the PostgreSQL, it is possible to create high-availability and load-balancing clusters. However, there is no support for failover in the client

Re: [HACKERS] Potential GIN vacuum bug

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 5:41 PM, Jeff Janes jeff.ja...@gmail.com wrote: User backends attempt to take the lock conditionally, because otherwise they would cause an autovacuum already holding the lock to cancel itself, which seems quite bad. Not that this a substantial behavior change, in that

Re: [HACKERS] DTrace build dependency rules

2015-08-18 Thread Alvaro Herrera
Robert Haas wrote: On Sat, Aug 15, 2015 at 6:45 PM, Mark Johnston ma...@freebsd.org wrote: The bug is in src/backend/Makefile. probes.o, the dtrace(1)-generated object file, depends on the objfiles.txt for each of the backend subdirs. These files depend in turn on the object files

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Robert Haas
On Tue, Aug 18, 2015 at 12:27 PM, Peter Eisentraut pete...@gmx.net wrote: On 8/18/15 9:50 AM, Tom Lane wrote: IIRC, the reason for having a wal_level parameter separate from those other ones was precisely that only wal_level had to be PGC_POSTMASTER, and you could change the others if you

Re: [HACKERS] allowing wal_level change at run time

2015-08-18 Thread Peter Eisentraut
On 8/18/15 9:50 AM, Tom Lane wrote: IIRC, the reason for having a wal_level parameter separate from those other ones was precisely that only wal_level had to be PGC_POSTMASTER, and you could change the others if you wanted. I think you are thinking of having split archive_mode/archive_command,

Re: [HACKERS] Colon Omitted From pgSQL Assignments

2015-08-18 Thread Pavel Stehule
Hi 2015-08-18 17:41 GMT+02:00 Charles Sheridan cesh...@swbell.net: Hi, I was looking at PL/pgSQL documentation and realized that contrary to spec, I've been omitting the colon ':' from assignments, e.g. writing 'x = 5' rather than the correct 'x := 5' I don't see any error

Re: [HACKERS] missing documentation for partial WAL files

2015-08-18 Thread Robert Haas
On Mon, Aug 17, 2015 at 2:50 PM, Peter Eisentraut pete...@gmx.net wrote: The commit message for de76884 contains some important information about the purpose and use of the new .partial WAL files. But I don't see anything about this in the documentation or another user-visible place. We

[HACKERS] Colon Omitted From pgSQL Assignments

2015-08-18 Thread Charles Sheridan
Hi, I was looking at PL/pgSQL documentation and realized that contrary to spec, I've been omitting the colon ':' from assignments, e.g. writing 'x = 5' rather than the correct 'x := 5' I don't see any error messages about this. I am not aware of any problems due to this. I suppose that

Re: [HACKERS] Test code is worth the space

2015-08-18 Thread Greg Stark
On Tue, Aug 18, 2015 at 2:16 PM, David Fetter da...@fetter.org wrote: I'm given to understand that this tight coupling is necessary for performance. Are you saying that it could be unwound, or that testing strategies mostly need to take it into account, or...? I'm just saying that we