Re: [HACKERS] [Bug] Inconsistent result for inheritance and FOR UPDATE.

2014-12-12 Thread Etsuro Fujita
(2014/12/12 11:33), Etsuro Fujita wrote: (2014/12/12 11:19), Tom Lane wrote: Etsuro Fujita fujita.ets...@lab.ntt.co.jp writes: (2014/12/12 10:37), Tom Lane wrote: Yeah, this is clearly a thinko: really, nothing in the planner should be using get_parse_rowmark(). I looked around for other

Re: [HACKERS] logical column ordering

2014-12-12 Thread Andres Freund
On 2014-12-10 19:06:28 -0800, Josh Berkus wrote: On 12/10/2014 05:14 PM, Stephen Frost wrote: * Andres Freund (and...@2ndquadrant.com) wrote: But the scheduling of commits with regard to the 9.5 schedule actually opens a relevant question: When are we planning to release 9.5? Because

Re: [HACKERS] Review of Refactoring code for sync node detection

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 04:29 AM, Michael Paquier wrote: On Thu, Dec 11, 2014 at 10:07 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I propose the attached (I admit I haven't tested it). Actually if you do it this way I think that it would be worth adding the small optimization Fujii-san

Re: [HACKERS] Review of Refactoring code for sync node detection

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 9:38 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: On 12/12/2014 04:29 AM, Michael Paquier wrote: On Thu, Dec 11, 2014 at 10:07 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I propose the attached (I admit I haven't tested it). Actually if you do it

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/9/14 1:18 PM, Josh Berkus wrote: The one exception I might make above is pg_standby. What do we need this for today, exactly? This was discussed recently and people wanted to keep it. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the server side in there? I think that'd be pg_xlogdump, pg_archivecleanup, pg_upgrade, pg_test_timing,

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/9/14 4:32 PM, Bruce Momjian wrote: On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: (For pg_upgrade you also need to do something about pg_upgrade_support, which is good because that is one very ugly crock.) FYI, pg_upgrade_support was segregated from pg_upgrade only

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the server side in there? I think that'd be pg_xlogdump,

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 15:11:01 +0200, Heikki Linnakangas wrote: On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 03:11 PM, Heikki Linnakangas wrote: On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the server

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 5:46 PM, Tomas Vondra t...@fuzzy.cz wrote: The idea was that if we could increase the load a bit (e.g. using 2 tuples per bucket instead of 1), we will still use a single batch in some cases (when we miss the work_mem threshold by just a bit). The lookups will be

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 10:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 9, 2014 at 4:09 AM, Robert Haas robertmh...@gmail.com wrote: On Sun, Dec 7, 2014 at 9:30 PM, Simon Riggs si...@2ndquadrant.com wrote: * parameter should be SUSET - it doesn't *need* to be set only at

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian br...@momjian.us wrote: compression = 'on' : 1838 secs = 'off' : 1701 secs Different is around 140 secs. OK, so the compression took 2x the cpu and was 8% slower. The only benefit is WAL files are 35% smaller? Compression didn't

Re: [HACKERS] pg_regress writes into source tree

2014-12-12 Thread Alvaro Herrera
Peter Eisentraut wrote: When using a vpath build pg_regress writes the processed input/*.source files into the *source* tree, which isn't supposed to happen. This appears to be a thinko introduced in this patch: e3fc4a97bc8ee82a78605b5ffe79bd4cf3c6213b Oh, I noticed this while doing the

[HACKERS] Streaming replication and WAL archive interactions

2014-12-12 Thread Heikki Linnakangas
There have been a few threads on the behavior of WAL archiving, after a standby server is promoted [1] [2]. In short, it doesn't work as you might expect. The standby will start archiving after it's promoted, but it will not archive files that were replicated from the old master via streaming

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 5:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent on reviewing patches than on managing the process. That's not really

Re: [HACKERS] BUG: *FF WALs under 9.2 (WAS: .ready files appearing on slaves)

2014-12-12 Thread Heikki Linnakangas
On 12/10/2014 04:32 PM, Dennis Kögel wrote: Hi, Am 04.09.2014 um 17:50 schrieb Jehan-Guillaume de Rorthais j...@dalibo.com: Since few months, we occasionally see .ready files appearing on some slave instances from various context. The two I have in mind are under 9.2.x. […] So it seems for

Re: [HACKERS] On partitioning

2014-12-12 Thread Robert Haas
On Thu, Dec 11, 2014 at 11:43 PM, Amit Langote langote_amit...@lab.ntt.co.jp wrote: In case of what we would have called a 'LIST' partition, this could look like ... FOR VALUES (val1, val2, val3, ...) Assuming we only support partition key to contain only one column in such a case. In

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:13:44PM +0200, Heikki Linnakangas wrote: On 12/12/2014 03:11 PM, Heikki Linnakangas wrote: On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 08:27:59 -0500, Robert Haas wrote: On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian br...@momjian.us wrote: compression = 'on' : 1838 secs = 'off' : 1701 secs Different is around 140 secs. OK, so the compression took 2x the cpu and was 8% slower. The only

[HACKERS] pg_rewind in contrib

2014-12-12 Thread Heikki Linnakangas
Hi, I'd like to include pg_rewind in contrib. I originally wrote it as an external project so that I could quickly get it working with the existing versions, and because I didn't feel it was quite ready for production use yet. Now, with the WAL format changes in master, it is a lot more

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 08:11:31AM -0500, Peter Eisentraut wrote: On 12/9/14 4:32 PM, Bruce Momjian wrote: On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: (For pg_upgrade you also need to do something about pg_upgrade_support, which is good because that is one very ugly

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 10:23 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Dec 11, 2014 at 10:33 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Tue, Dec 9, 2014 at 4:09 AM, Robert Haas robertmh...@gmail.com wrote: On Sun, Dec 7, 2014 at 9:30 PM, Simon Riggs

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Alvaro Herrera
Robert Haas wrote: (I note that the proposal to have the CFM review everything is merely one way of meeting the need to have senior people spend more time reviewing. But I assure all of you that I spend as much time reviewing as I can find time for. If someone wants to pay me the same

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote: On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian br...@momjian.us wrote: compression = 'on' : 1838 secs = 'off' : 1701 secs Different is around 140 secs. OK, so the compression took 2x the cpu and was 8% slower.

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/8/14 10:50 PM, Tom Lane wrote: oid2name and vacuumlo, besides being of very dubious general utility, are fails from a namespacing standpoint. If we were to promote them into standard install components I think a minimum requirement should be to rename them to pg_something. (oid2name is

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Andres Freund
Hi, On 2014-12-12 16:13:13 +0200, Heikki Linnakangas wrote: I'd like to include pg_rewind in contrib. I originally wrote it as an external project so that I could quickly get it working with the existing versions, and because I didn't feel it was quite ready for production use yet. Now, with

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 09:18:01 -0500, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote: On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian br...@momjian.us wrote: compression = 'on' : 1838 secs = 'off' : 1701 secs Different is around 140 secs.

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: On 2014-12-12 09:18:01 -0500, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote: On Thu, Dec 11, 2014 at 11:34 AM, Bruce Momjian br...@momjian.us wrote: compression = 'on' : 1838 secs

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Peter Eisentraut
On 12/12/14 8:13 AM, Andres Freund wrote: Wouldn't a make install-server/client targets or something similar actually achieve the same thing? Seems simpler to maintain to me. Adding non-standard makefile targets comes with its own set of maintenance issues. Restructuring the source tree and

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Bruce Momjian wrote: On Fri, Dec 12, 2014 at 03:13:44PM +0200, Heikki Linnakangas wrote: On 12/12/2014 03:11 PM, Heikki Linnakangas wrote: Sounds good. We already separate server and client programs in the docs, and packagers put them in different packages too. This should make

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 09:24:27 -0500, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: Well, the larger question is why wouldn't we just have the user compress the entire WAL file before archiving --- why have each backend do it? Is it the write volume we are

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:20:47PM +0100, Andres Freund wrote: Hi, On 2014-12-12 16:13:13 +0200, Heikki Linnakangas wrote: I'd like to include pg_rewind in contrib. I originally wrote it as an external project so that I could quickly get it working with the existing versions, and because

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 11:32 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Dec 12, 2014 at 9:15 AM, Michael Paquier michael.paqu...@gmail.com wrote: I just meant that the scratch buffers used to store temporarily the compressed and uncompressed data should be palloc'd all the time,

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 9:15 AM, Michael Paquier michael.paqu...@gmail.com wrote: I just meant that the scratch buffers used to store temporarily the compressed and uncompressed data should be palloc'd all the time, even if the switch is off. If they're fixed size, you can just put them on the

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 9:15 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: (I note that the proposal to have the CFM review everything is merely one way of meeting the need to have senior people spend more time reviewing. But I assure all of you that I spend as much

Re: [HACKERS] Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 9:34 AM, Michael Paquier michael.paqu...@gmail.com wrote: I don't think that's a cost worth caring about. OK, I thought it was. Space on the heap that never gets used is basically free. The OS won't actually allocate physical memory unless the pages are actually

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Michael Paquier
On Fri, Dec 12, 2014 at 11:13 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I'd like to include pg_rewind in contrib. I originally wrote it as an external project so that I could quickly get it working with the existing versions, and because I didn't feel it was quite ready for

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Rahila Syed
Hello, Well, the larger question is why wouldn't we just have the user compress the entire WAL file before archiving --- why have each backend do it? Is it the write volume we are saving? IIUC, the idea here is to not only save the on disk size of WAL but to reduce the overhead of flushing WAL

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 03:27:33PM +0100, Andres Freund wrote: On 2014-12-12 09:24:27 -0500, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: Well, the larger question is why wouldn't we just have the user compress the entire WAL file before archiving

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Michael Paquier
On Wed, Dec 10, 2014 at 11:25 PM, Bruce Momjian br...@momjian.us wrote: On Wed, Dec 10, 2014 at 07:40:46PM +0530, Rahila Syed wrote: The tests ran for around 30 mins.Manual checkpoint was run before each test. Compression WAL generated%compressionLatency-avg CPU usage

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 09:46:13 -0500, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 03:27:33PM +0100, Andres Freund wrote: On 2014-12-12 09:24:27 -0500, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 03:22:24PM +0100, Andres Freund wrote: Well, the larger question is why wouldn't we just have the

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Heikki Linnakangas
On 12/12/2014 04:20 PM, Andres Freund wrote: Not sure if the copyright notices in the current form are actually ok? Hmm. We do have such copyright notices in the source tree, but I know that we're trying to avoid it in new code. They had to be there when the code lived as a separate project,

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 23:50:43 +0900, Michael Paquier wrote: I got curious to see how the compression of an entire record would perform and how it compares for small WAL records, and here are some numbers based on the patch attached, this patch compresses the whole record including the block headers,

Re: [HACKERS] pg_rewind in contrib

2014-12-12 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: I'd like to include pg_rewind in contrib. I don't object to adding the tool as such, but let's wait to see what happens with Peter's proposal to move contrib command-line tools into src/bin/. If it should be there it'd be less code churn if it

Re: [HACKERS] Commitfest problems

2014-12-12 Thread David Fetter
On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent on reviewing patches than on managing the process. With utmost respect, Tom,

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 11:27:01 -0300, Alvaro Herrera wrote: We already have src/bin/; the mixture of src/ and bin/ predates us. Of course, the stuff we keep in there is not binaries but source code that produces binaries. As for src/sbin/, we wouldn't install anything to the system's /usr/sbin/ of

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Heikki Linnakangas hlinnakan...@vmware.com writes: On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that involves the server

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 12/9/14 4:32 PM, Bruce Momjian wrote: On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: (For pg_upgrade you also need to do something about pg_upgrade_support, which is good because that is one very ugly crock.) FYI,

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes: On 12/12/14 8:13 AM, Andres Freund wrote: Wouldn't a make install-server/client targets or something similar actually achieve the same thing? Seems simpler to maintain to me. Adding non-standard makefile targets comes with its own set of maintenance

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Andres Freund
On 2014-12-12 07:10:40 -0800, David Fetter wrote: On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent on reviewing patches

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 10:20:58 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On 12/12/14 8:13 AM, Andres Freund wrote: Wouldn't a make install-server/client targets or something similar actually achieve the same thing? Seems simpler to maintain to me. Adding non-standard

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-12 Thread Alex Shulgin
Alex Shulgin a...@commandprompt.com writes: Alex Shulgin a...@commandprompt.com writes: Here's an attempt to revive this patch. Here's the patch rebased against current HEAD, that is including the recently committed action_at_recovery_target option. The default for the new GUC is 'pause',

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Dec 12, 2014 at 9:15 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Robert Haas wrote: (I note that the proposal to have the CFM review everything is merely one way of meeting the need to have senior people spend more time reviewing. But

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Tom Lane wrote: Heikki Linnakangas hlinnakan...@vmware.com writes: On 12/12/2014 03:07 PM, Peter Eisentraut wrote: On 12/9/14 4:10 PM, Alvaro Herrera wrote: Maybe it makes sense to have a distinction between client programs and server programs. Can we have src/sbin/ and move stuff that

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 10:04 AM, Andres Freund and...@anarazel.de wrote: Note that autovacuum and fsync are off. =# select phase, user_diff, system_diff, pg_size_pretty(pre_update - pre_insert), pg_size_pretty(post_update - pre_update) from results; phase| user_diff |

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 11:08:52 -0500, Robert Haas wrote: Unless I'm missing something, this test is showing that FPW compression saves 298MB of WAL for 17.3 seconds of CPU time, as against master. And compressing the whole record saves a further 1MB of WAL for a further 13.39 seconds of CPU time.

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:00 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I'm pretty much -1 on relocating anything that's under src/bin already. I agree. I can't see packagers putting it anywhere except for $SOMETHING/bin in the final install, so what do we get out of dividing it up in

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:12 AM, Andres Freund and...@anarazel.de wrote: On 2014-12-12 11:08:52 -0500, Robert Haas wrote: Unless I'm missing something, this test is showing that FPW compression saves 298MB of WAL for 17.3 seconds of CPU time, as against master. And compressing the whole

Re: [HACKERS] [Bug] Inconsistent result for inheritance and FOR UPDATE.

2014-12-12 Thread Tom Lane
Etsuro Fujita fujita.ets...@lab.ntt.co.jp writes: (2014/12/12 10:37), Tom Lane wrote: Yeah, this is clearly a thinko: really, nothing in the planner should be using get_parse_rowmark(). I looked around for other errors of the same type and found that postgresGetForeignPlan() is also using

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Andres Freund
On 2014-12-12 11:15:46 -0500, Robert Haas wrote: On Fri, Dec 12, 2014 at 11:12 AM, Andres Freund and...@anarazel.de wrote: On 2014-12-12 11:08:52 -0500, Robert Haas wrote: Unless I'm missing something, this test is showing that FPW compression saves 298MB of WAL for 17.3 seconds of CPU

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Joshua D. Drake
On 12/08/2014 07:50 PM, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: Last time this was attempted, the discussion got lost in exactly which extensions are worthy enough to be considered official or something like that. I want to dodge that here by starting at the opposite end: 1.

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I'm not really convinced this is a very good idea. What do we get out of moving everything, or even anything, from contrib? It will make back-patching harder, but more importantly, it will possibly create the false impression that everything we

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 11:14:56 -0500, Robert Haas wrote: I'm not really convinced this is a very good idea. What do we get out of moving everything, or even anything, from contrib? The benefit of moving relevant stuff is that it'll actually be installed by default when installing postgres on many

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:26 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm not really convinced this is a very good idea. What do we get out of moving everything, or even anything, from contrib? It will make back-patching harder, but more importantly,

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:40 AM, Andres Freund and...@2ndquadrant.com wrote: The benefit of moving relevant stuff is that it'll actually be installed by default when installing postgres on many platforms. That's currently often not the case. The contrib umbrella, as used by many other

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Andres Freund
On 2014-12-12 11:42:57 -0500, Robert Haas wrote: I think the amount of effort a simple renamed directory which wholly contains a binary creates is acceptable. Just use patch -p4 instead of patch -p1... That is fine if you are manually applying a patch that touches only that directory,

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2014-12-12 Thread Tomas Vondra
On 12.12.2014 14:19, Robert Haas wrote: On Thu, Dec 11, 2014 at 5:46 PM, Tomas Vondra t...@fuzzy.cz wrote: Regarding the sufficiently small - considering today's hardware, we're probably talking about gigabytes. On machines with significant memory pressure (forcing the temporary files to

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Andres Freund and...@2ndquadrant.com writes: On 2014-12-12 11:42:57 -0500, Robert Haas wrote: And I don't know how well git cherry-pick will follow the moves. Not well if the patch is done in master first. FWIW, I always patch master first, and have zero intention of changing that workflow.

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 10:16:05AM -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On 12/9/14 4:32 PM, Bruce Momjian wrote: On Tue, Dec 9, 2014 at 06:10:02PM -0300, Alvaro Herrera wrote: (For pg_upgrade you also need to do something about pg_upgrade_support, which is good

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Robert Haas wrote: On Fri, Dec 12, 2014 at 11:00 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: So let's put the whole bunch under src/bin/ and be done with it. I'm not really convinced this is a very good idea. What do we get out of moving everything, or even anything, from contrib?

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 05:19:42PM +0100, Andres Freund wrote: On 2014-12-12 11:15:46 -0500, Robert Haas wrote: On Fri, Dec 12, 2014 at 11:12 AM, Andres Freund and...@anarazel.de wrote: On 2014-12-12 11:08:52 -0500, Robert Haas wrote: Unless I'm missing something, this test is showing

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Bruce Momjian
On Fri, Dec 12, 2014 at 10:50:56AM -0500, Tom Lane wrote: Also, one part of the point of the review mechanism is that it's supposed to provide an opportunity for less-senior reviewers to look at parts of the code that they maybe don't know so well, and thereby help grow them into senior

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Andrew Dunstan
On 12/08/2014 01:00 PM, Andrew Dunstan wrote: On 12/08/2014 04:21 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: OK, here is a new patch version that * uses find_coercion_path() to find the cast function if any, as discussed elsewhere * removes calls to getTypeOutputInfo() except

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Andrew Dunstan
On 12/12/2014 01:10 PM, Andrew Dunstan wrote: On 12/08/2014 01:00 PM, Andrew Dunstan wrote: On 12/08/2014 04:21 AM, Alvaro Herrera wrote: Andrew Dunstan wrote: OK, here is a new patch version that * uses find_coercion_path() to find the cast function if any, as discussed elsewhere

Re: [HACKERS] Commitfest problems

2014-12-12 Thread David Fetter
On Fri, Dec 12, 2014 at 04:21:43PM +0100, Andres Freund wrote: On 2014-12-12 07:10:40 -0800, David Fetter wrote: On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 06:30 AM, Robert Haas wrote: Yeah, that would be great, and even better if we could get 2 or 3 positions funded so that the success or failure isn't too much tied to a single individual. But even getting 1 position funded in a stable-enough fashion that someone would be willing

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Simon Riggs
On 12 December 2014 at 18:04, Bruce Momjian br...@momjian.us wrote: Well, it seems we need to see some actual cases where compression does help before moving forward. I thought Amit had some amazing numbers for WAL compression --- has that changed? For background processes, like VACUUM, then

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes: Also here is a patch factored out which applies the find_coercion_pathway change to json.c. I'm inclined to say we should backpatch this to 9.4 (and with a small change 9.3). Thoughts? Meh. Maybe I'm just feeling gunshy because I broke something

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Tomas Vondra
On 12.12.2014 19:07, Bruce Momjian wrote: On Fri, Dec 12, 2014 at 10:50:56AM -0500, Tom Lane wrote: Also, one part of the point of the review mechanism is that it's supposed to provide an opportunity for less-senior reviewers to look at parts of the code that they maybe don't know so well, and

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Simon Riggs
On 12 December 2014 at 15:10, David Fetter da...@fetter.org wrote: On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent on

Re: [HACKERS] jsonb generator functions

2014-12-12 Thread Andrew Dunstan
On 12/12/2014 01:55 PM, Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Also here is a patch factored out which applies the find_coercion_pathway change to json.c. I'm inclined to say we should backpatch this to 9.4 (and with a small change 9.3). Thoughts? Meh. Maybe I'm just

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/11/2014 02:55 PM, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of time I can put into a commitfest is better spent on reviewing patches than on managing the process. Agreed but That means committers/hackers

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/12/2014 06:30 AM, Robert Haas wrote: Yeah, that would be great, and even better if we could get 2 or 3 positions funded so that the success or failure isn't too much tied to a single individual. But even getting 1 position funded in a stable-enough fashion that someone would be willing

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/12/2014 10:59 AM, Simon Riggs wrote: On 12 December 2014 at 15:10, David Fetter da...@fetter.org wrote: On Thu, Dec 11, 2014 at 05:55:56PM -0500, Tom Lane wrote: Josh Berkus j...@agliodbs.com writes: How about *you* run the next one, Tom? I think the limited amount of time I can put

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Alvaro Herrera
Joshua D. Drake wrote: On 12/12/2014 06:30 AM, Robert Haas wrote: Yeah, that would be great, and even better if we could get 2 or 3 positions funded so that the success or failure isn't too much tied to a single individual. But even getting 1 position funded in a stable-enough fashion

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Tomas Vondra
On 11.12.2014 16:06, Bruce Momjian wrote: On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: I will add: 4. commitfest managers have burned out and refuse to do it again Agreed. The fun, if it was ever there, has left the commitfest process. I've never been a CFM, but from my

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 11:35 AM, Alvaro Herrera wrote: Uh, really? Last I looked at the numbers from SPI treasurer reports, they are not impressive enough to hire a full-time engineer, let alone a senior one. The Linux Foundation has managed to pay for Linus Torvalds somehow, so it does sound

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Joshua D. Drake
On 12/12/2014 11:35 AM, Alvaro Herrera wrote: We (not CMD, the community) with proper incentive could fund this. It really wouldn't be that hard. That said, there would have to be a clear understanding of expectations, results, and authority. Uh, really? Yeah I think so. Money can be easy

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Magnus Hagander
On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra t...@fuzzy.cz wrote: On 11.12.2014 16:06, Bruce Momjian wrote: On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: 3) manual processing that could be automated I think the CF site was a huge step forward, but maybe we could improve

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
On 12/12/2014 11:52 AM, Magnus Hagander wrote: On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra t...@fuzzy.cz wrote: On 11.12.2014 16:06, Bruce Momjian wrote: On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: 3) manual processing that could be automated I think the CF site was a

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Christoph Berg
Re: Andres Freund 2014-12-12 20141212152723.go31...@awork2.anarazel.de On 2014-12-12 10:20:58 -0500, Tom Lane wrote: Peter Eisentraut pete...@gmx.net writes: On 12/12/14 8:13 AM, Andres Freund wrote: Wouldn't a make install-server/client targets or something similar actually achieve

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Tom Lane
Christoph Berg c...@df7cb.de writes: However, for PostgreSQL this means lengthy debian/*.install files (the equivalent of %files in rpm spec speak): Right ... If there were separate install-client, install-server, and install-contrib targets, that would probably shorten those files quite a

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Magnus Hagander
On Fri, Dec 12, 2014 at 9:05 PM, Josh Berkus j...@agliodbs.com wrote: On 12/12/2014 11:52 AM, Magnus Hagander wrote: On Fri, Dec 12, 2014 at 8:43 PM, Tomas Vondra t...@fuzzy.cz wrote: On 11.12.2014 16:06, Bruce Momjian wrote: On Wed, Dec 10, 2014 at 11:00:21PM -0800, Josh Berkus wrote: 3)

Re: [HACKERS] moving from contrib to bin

2014-12-12 Thread Alvaro Herrera
Tom Lane wrote: Christoph Berg c...@df7cb.de writes: However, for PostgreSQL this means lengthy debian/*.install files (the equivalent of %files in rpm spec speak): Right ... If there were separate install-client, install-server, and install-contrib targets, that would probably

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Peter Geoghegan
On Fri, Dec 12, 2014 at 12:30 PM, Magnus Hagander mag...@hagander.net wrote: It also automates a bunch of the emailing no? Yes. Please let me know the details (privately or otherwise). I'd like to try it out again. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Josh Berkus
Just as a note abot this one part along (I'll read the rest later). I do have the new version of the CF app more or less ready to deploy, but I got bogged down by thinking I'll do it between two commitfests to not be disruptive. But there has been no between two commitfests. Hopefully I can

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Jim Nasby
On 12/12/14, 2:38 PM, Josh Berkus wrote: Just as a note abot this one part along (I'll read the rest later). I do have the new version of the CF app more or less ready to deploy, but I got bogged down by thinking I'll do it between two commitfests to not be disruptive. But there has been no

Re: [HACKERS] PATCH: hashjoin - gracefully increasing NTUP_PER_BUCKET instead of batching

2014-12-12 Thread Robert Haas
On Fri, Dec 12, 2014 at 11:50 AM, Tomas Vondra t...@fuzzy.cz wrote: On 12.12.2014 14:19, Robert Haas wrote: On Thu, Dec 11, 2014 at 5:46 PM, Tomas Vondra t...@fuzzy.cz wrote: Regarding the sufficiently small - considering today's hardware, we're probably talking about gigabytes. On machines

Re: [HACKERS] Commitfest problems

2014-12-12 Thread Jim Nasby
On 12/12/14, 1:44 PM, Joshua D. Drake wrote: 1. We don't need a full-time engineer to manage a commitfest. We need a manager or PM. I don't think that's actually true. The major points on this thread are that 1) we don't have enough capacity for doing reviews and 2) the CFM has no authority

  1   2   >