[HACKERS] Making BackgroundWorkerHandle a complete type or offering a worker enumeration API?

2014-12-13 Thread Craig Ringer
Hi all While working on BDR, I've run into a situation that I think highlights a limitation of the dynamic bgworker API that should be fixed. Specifically, when the postmaster crashes and restarts shared memory gets cleared but dynamic bgworkers don't get unregistered, and that's a mess. I have

Re: [HACKERS] Commitfest problems

2014-12-13 Thread Craig Ringer
On 12/12/2014 06:02 AM, Josh Berkus wrote: Speaking as the originator of commitfests, they were *always* intended to be a temporary measure, a step on the way to something else like continuous integration. I'd really like to see the project revisit some of the underlying assumptions that're

Re: [HACKERS] moving from contrib to bin

2014-12-13 Thread Christoph Berg
Re: Alvaro Herrera 2014-12-12 20141212203700.gb1...@alvh.no-ip.org Pardon me for not knowing much about Debian packages, but how would that work exactly? Is it possible to do make install-client, then package the installed files, then rm -rf the install tree, then repeat for

Re: [HACKERS] Commitfest problems

2014-12-13 Thread Craig Ringer
On 12/12/2014 06:01 AM, David G Johnston wrote: The patch list concept should be formalized, and should include a targeted release concept. IMO, the patch list concept should be discarded in favour of a working tree list. At this point, given the challenges the CF process faces, I can't say

[HACKERS] pgbench -f and vacuum

2014-12-13 Thread Tatsuo Ishii
Currently pgbench -f (run custom script) executes vacuum against pgbench_* tables before stating bench marking if -n (or --no-vacuum) is not specified. If those tables do not exist, pgbench fails. To prevent this, -n must be specified. For me this behavior seems insane because -f does not

[HACKERS] moving Orafce from pgFoundry - pgFoundry management

2014-12-13 Thread Pavel Stehule
Hi a Orafce package on pgFoundry is obsolete - we migrated to github few years ago. Please, can somebody modify a description about this migration? Or drop this project there. Regards Pavel

Re: [HACKERS] Reducing lock strength of adding foreign keys

2014-12-13 Thread Andreas Karlsson
On 10/28/2014 01:33 AM, Noah Misch wrote: ALTER TRIGGER is not bad; like you say, change pg_get_triggerdef_worker() the way commit e5550d5 changed pg_get_constraintdef_worker(). DROP TRIGGER is more difficult. pg_constraint.tgqual of a dropped trigger may reference other dropped objects, which

Re: [HACKERS] Compression of full-page-writes

2014-12-13 Thread Amit Kapila
On Tue, Dec 9, 2014 at 10:45 AM, Amit Kapila amit.kapil...@gmail.com wrote: On Mon, Dec 8, 2014 at 3:17 PM, Simon Riggs si...@2ndquadrant.com wrote: On 8 December 2014 at 11:46, Michael Paquier michael.paqu...@gmail.com wrote: I don't really like those new names, but I'd prefer

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-13 Thread Simon Riggs
On 12 December 2014 at 21:40, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 12, 2014 at 1:51 PM, Simon Riggs si...@2ndquadrant.com wrote: What I don't understand is why we aren't working on double buffering, since that cost would be paid in a background process and would be evenly

Re: [HACKERS] Final Patch for GROUPING SETS

2014-12-13 Thread Andrew Gierth
Tom == Tom Lane t...@sss.pgh.pa.us writes: With the high-priority questions out of the way, time to tackle the rest: Tom My single biggest complaint is about the introduction of struct Tom GroupedVar. If we stick with that, we're going to have to teach Tom an extremely large number of

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-13 Thread Michael Paquier
On Fri, Dec 12, 2014 at 11:50 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 10, 2014 at 11:25 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Dec 10, 2014 at 07:40:46PM +0530, Rahila Syed wrote: The tests ran for around 30 mins.Manual checkpoint was run before each

[HACKERS] duplicate #define

2014-12-13 Thread Mark Dilger
In commit 2c03216d831160bedd72d45f712601b6f7d03f1c, the following define occurs twice in src/include/access/xlogrecord.h: #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) It is no big deal, as the definitions don't contradict each other. We could probably live with just

[HACKERS] Custom timestamp format in logs

2014-12-13 Thread Michael Paquier
Hi all, This week, we heard about a user willing to use a custom timestamp format across a set of services to improve the debugability of the whole set, Postgres being one of them. Unfortunately datestyle does not take into account the logs. Would it be worth adding a new GUC able to control the

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

2014-12-13 Thread Julien Rouhaud
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Le 18/11/2014 08:36, Michael Paquier a écrit : On Wed, Oct 22, 2014 at 12:50 AM, Brightwell, Adam adam.brightw...@crunchydatasolutions.com wrote: Though, I would think that the general desire would be to keep the patch relevant ONLY to the

[HACKERS] Status of CF 2014-10 and upcoming 2014-12

2014-12-13 Thread Michael Paquier
Hi all, Looking at the CF app as of today, there is the following status for pending patches: - Needs Review: 18 - Waiting on Author: 1 - Ready for Committer: 8 And the next coming fest that should begin on Monday has this status: - Needs Review: 34 - Waiting on Author: 3 - Ready for Committer:

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

2014-12-13 Thread Michael Paquier
On Sat, Dec 13, 2014 at 11:53 PM, Julien Rouhaud julien.rouh...@dalibo.com wrote: I agree with you about the problems of the v2 patch I originally sent. I think this v3 is the right way of keeping track of .ready files, so it's ok for me. The v3 also still applies well on current head. Simon

Re: [HACKERS] Status of CF 2014-10 and upcoming 2014-12

2014-12-13 Thread Andrew Dunstan
On 12/13/2014 10:00 AM, Michael Paquier wrote: Hi all, Looking at the CF app as of today, there is the following status for pending patches: - Needs Review: 18 - Waiting on Author: 1 - Ready for Committer: 8 And the next coming fest that should begin on Monday has this status: - Needs Review:

Re: [HACKERS] 9.4rc bug in percentile_cont

2014-12-13 Thread Tom Lane
Andrew Gierth and...@tao11.riddles.org.uk writes: Just got a report on IRC of a bug in the array version of percentile_cont; if two of the requested percentiles were between the same pair of input rows, the result could be wrong or an error would be generated. Oooh, good catch. Proposed

Re: [HACKERS] pgbench -f and vacuum

2014-12-13 Thread Tom Lane
Tatsuo Ishii is...@postgresql.org writes: Currently pgbench -f (run custom script) executes vacuum against pgbench_* tables before stating bench marking if -n (or --no-vacuum) is not specified. If those tables do not exist, pgbench fails. To prevent this, -n must be specified. For me this

Re: [HACKERS] split builtins.h to quote.h

2014-12-13 Thread Andrew Dunstan
On 11/08/2014 12:37 AM, Michael Paquier wrote: On Sat, Nov 8, 2014 at 5:55 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Michael Paquier wrote: On Fri, Nov 7, 2014 at 2:31 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I thought the consensus was that the SQL-callable function

Re: [HACKERS] split builtins.h to quote.h

2014-12-13 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/08/2014 12:37 AM, Michael Paquier wrote: Well, yes :) I missed that. Note that I am leaning to Robert's direction as well to do a clear separation... Now if the final consensus is different, then let's use the patch attached that puts the SQL

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-12-13 Thread Andrew Dunstan
On 11/20/2014 02:27 AM, Amit Kapila wrote: On Wed, Nov 19, 2014 at 11:46 PM, Robert Haas robertmh...@gmail.com mailto:robertmh...@gmail.com wrote: On Tue, Nov 18, 2014 at 9:19 AM, Alvaro Herrera alvhe...@2ndquadrant.com mailto:alvhe...@2ndquadrant.com wrote: Right, but they provide same

Re: [HACKERS] add modulo (%) operator to pgbench

2014-12-13 Thread Andrew Dunstan
On 11/24/2014 07:26 AM, Heikki Linnakangas wrote: On 09/25/2014 05:10 AM, Robert Haas wrote: On Wed, Sep 24, 2014 at 2:34 PM, Fabien COELHO coe...@cri.ensmp.fr wrote: Sigh. How to transform a trivial 10 lines patch into a probably 500+ lines project involving flex bison some non trivial

Re: [HACKERS] duplicate #define

2014-12-13 Thread Heikki Linnakangas
On 12/13/2014 04:45 PM, Mark Dilger wrote: In commit 2c03216d831160bedd72d45f712601b6f7d03f1c, the following define occurs twice in src/include/access/xlogrecord.h: #define SizeOfXLogRecordDataHeaderLong (sizeof(uint8) + sizeof(uint32)) It is no big deal, as the definitions don't contradict

Re: [HACKERS] On partitioning

2014-12-13 Thread José Luis Tallón
On 12/12/2014 05:43 AM, Amit Langote wrote: [snip] In case of what we would have called a 'LIST' partition, this could look like ... FOR VALUES (val1, val2, val3, ...) Assuming we only support partition key to contain only one column in such a case. Hmmm…. [...] PARTITION BY LIST(col1 [,

Re: [HACKERS] On partitioning

2014-12-13 Thread José Luis Tallón
On 12/13/2014 03:09 AM, Alvaro Herrera wrote: [snip] Arbitrary SQL expressions (including functions) are not the thing to use for partitioning -- at least that's how I understand this whole discussion. I don't think you want to do proofs as such -- they are expensive. Yup. Plus, it looks like

Re: [HACKERS] moving Orafce from pgFoundry - pgFoundry management

2014-12-13 Thread David Fetter
On Sat, Dec 13, 2014 at 11:19:08AM +0100, Pavel Stehule wrote: Hi a Orafce package on pgFoundry is obsolete - we migrated to github few years ago. Please, can somebody modify a description about this migration? Or drop this project there. Pavel, I've removed pgFoundry references from the

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-12-13 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: On 11/20/2014 02:27 AM, Amit Kapila wrote: Now the part where I would like to receive feedback before revising the patch is on the coding style. It seems to me from Tom's comments that he is not happy with the code, now I am not sure which part of

Re: [HACKERS] pg_rewind in contrib

2014-12-13 Thread David Fetter
On Fri, Dec 12, 2014 at 10:06:32AM -0500, Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: I'd like to include pg_rewind in contrib. I don't object to adding the tool as such, but let's wait to see what happens with Peter's proposal to move contrib command-line tools into

Re: [HACKERS] On partitioning

2014-12-13 Thread Jim Nasby
On 12/12/14, 3:48 PM, Robert Haas wrote: On Fri, Dec 12, 2014 at 4:28 PM, Jim Nasby jim.na...@bluetreble.com wrote: Sure. Mind you, I'm not proposing that the syntax I just mooted is actually for the best. What I'm saying is that we need to talk about it. Frankly, if we're going to require

Re: [HACKERS] On partitioning

2014-12-13 Thread David Fetter
On Fri, Dec 12, 2014 at 09:03:12AM -0500, Robert Haas wrote: Yeah, range and list partition definitions are very similar, but hash partition definitions are a different kettle of fish. I don't think we really need hash partitioning for anything right away - it's pretty useless unless you've

Re: [HACKERS] add modulo (%) operator to pgbench

2014-12-13 Thread Fabien COELHO
As it doesn't have documentation, I'm inclined to say we should mark this as Waiting on Author or Returned with Feedback. I'm planing to have a detailed look at Robert's patch before the end of the year. I could update pgbench documentation while doing that. -- Fabien. -- Sent via

Re: [HACKERS] add modulo (%) operator to pgbench

2014-12-13 Thread Andrew Dunstan
On 12/13/2014 01:19 PM, Fabien COELHO wrote: As it doesn't have documentation, I'm inclined to say we should mark this as Waiting on Author or Returned with Feedback. I'm planing to have a detailed look at Robert's patch before the end of the year. I could update pgbench documentation

Re: [HACKERS] On partitioning

2014-12-13 Thread José Luis Tallón
On 12/13/2014 05:57 PM, José Luis Tallón wrote: On 12/13/2014 03:09 AM, Alvaro Herrera wrote: [snip] Arbitrary SQL expressions (including functions) are not the thing to use for partitioning -- at least that's how I understand this whole discussion. I don't think you want to do proofs as such

Re: [HACKERS] CINE in CREATE TABLE AS ... and CREATE MATERIALIZED VIEW ...

2014-12-13 Thread Andrew Dunstan
On 11/03/2014 07:35 AM, Fabrízio de Royes Mello wrote: On Mon, Nov 3, 2014 at 3:12 AM, Rushabh Lathia rushabh.lat...@gmail.com mailto:rushabh.lat...@gmail.com wrote: Patch looks good, assigning to committer. Thanks for your review! Committed. cheers andrew -- Sent via pgsql-hackers

Re: [HACKERS] Commitfest problems

2014-12-13 Thread Álvaro Hernández Tortosa
On 12/12/14 20:43, Josh Berkus wrote: On 12/12/2014 11:35 AM, Alvaro Herrera wrote: Uh, really? Last I looked at the numbers from SPI treasurer reports, they are not impressive enough to hire a full-time engineer, let alone a senior one. The Linux Foundation has managed to pay for Linus

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-13 Thread Simon Riggs
On 13 December 2014 at 14:36, Michael Paquier michael.paqu...@gmail.com wrote: Something to be aware of btw is that this patch introduces an additional 8 bytes per block image in WAL as it contains additional information to control the compression. In this case this is the uint16 compress_len

[HACKERS] PATCH: Reducing lock strength of trigger and foreign key DDL

2014-12-13 Thread Andreas Karlsson
Hi, I have attached a patch with the current status of my work on reducing the lock level of trigger and foreign key related DDL. This commit reduces the lock level of the following commands from ACCESS EXCLUSIVE to SHARE ROW EXCLUSIVE, plus that it does the same for the referred table of

Re: [HACKERS] pgbench -f and vacuum

2014-12-13 Thread David Rowley
On 14 December 2014 at 04:39, Tom Lane t...@sss.pgh.pa.us wrote: Tatsuo Ishii is...@postgresql.org writes: Currently pgbench -f (run custom script) executes vacuum against pgbench_* tables before stating bench marking if -n (or --no-vacuum) is not specified. If those tables do not exist,

Re: [HACKERS] CINE in CREATE TABLE AS ... and CREATE MATERIALIZED VIEW ...

2014-12-13 Thread Fabrízio de Royes Mello
Em sábado, 13 de dezembro de 2014, Andrew Dunstan and...@dunslane.net escreveu: On 11/03/2014 07:35 AM, Fabrízio de Royes Mello wrote: On Mon, Nov 3, 2014 at 3:12 AM, Rushabh Lathia rushabh.lat...@gmail.com mailto:rushabh.lat...@gmail.com wrote: Patch looks good, assigning to committer.

Re: [HACKERS] operator does not exist: character varying[] character[]

2014-12-13 Thread Jim Nasby
On 12/12/14, 7:16 PM, Tom Lane wrote: Jim Nasby jim.na...@bluetreble.com writes: I'd say that array_eq (and probably _cmp) just needs to be taught to fall back to what oper() does, but this part of the commit message gives me pause: Change the operator search algorithms to look for

Re: [HACKERS] pgbench -f and vacuum

2014-12-13 Thread Tatsuo Ishii
On 14 December 2014 at 04:39, Tom Lane t...@sss.pgh.pa.us wrote: Tatsuo Ishii is...@postgresql.org writes: Currently pgbench -f (run custom script) executes vacuum against pgbench_* tables before stating bench marking if -n (or --no-vacuum) is not specified. If those tables do not exist,

Re: [HACKERS] On partitioning

2014-12-13 Thread Amit Langote
On Sun, Dec 14, 2014 at 1:57 AM, José Luis Tallón jltal...@adv-solutions.net wrote: On 12/13/2014 03:09 AM, Alvaro Herrera wrote: [snip] Arbitrary SQL expressions (including functions) are not the thing to use for partitioning -- at least that's how I understand this whole discussion. I

Re: [HACKERS] On partitioning

2014-12-13 Thread Amit Langote
On Sun, Dec 14, 2014 at 1:40 AM, José Luis Tallón jltal...@adv-solutions.net wrote: On 12/12/2014 05:43 AM, Amit Langote wrote: Amit: mind if I add the DB2 syntax for partitioning to the wiki, too? This might as well help with deciding the final form of partitioning (and define the first

Re: [HACKERS] split builtins.h to quote.h

2014-12-13 Thread Michael Paquier
On Sun, Dec 14, 2014 at 1:00 AM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 11/08/2014 12:37 AM, Michael Paquier wrote: Well, yes :) I missed that. Note that I am leaning to Robert's direction as well to do a clear separation... Now if the final consensus

Re: [HACKERS] pgbench -f and vacuum

2014-12-13 Thread Jim Nasby
On 12/13/14, 6:17 PM, Tatsuo Ishii wrote: Problem with -f implies -n approach is, it breaks backward compatibility. There are use cases using custom script*and* pgbench_* tables. For example the particular user wants to use the standard pgbench tables and is not satisfied with the built in

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-13 Thread Michael Paquier
On Sun, Dec 14, 2014 at 5:45 AM, Simon Riggs si...@2ndquadrant.com wrote: On 13 December 2014 at 14:36, Michael Paquier michael.paqu...@gmail.com wrote: Something to be aware of btw is that this patch introduces an additional 8 bytes per block image in WAL as it contains additional

Re: [HACKERS] pg_basebackup vs. Windows and tablespaces

2014-12-13 Thread Amit Kapila
On Sat, Dec 13, 2014 at 10:48 PM, Tom Lane t...@sss.pgh.pa.us wrote: Andrew Dunstan and...@dunslane.net writes: On 11/20/2014 02:27 AM, Amit Kapila wrote: Now the part where I would like to receive feedback before revising the patch is on the coding style. It seems to me from Tom's