Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-12-02 Thread Michael Paquier
On Wed, Dec 3, 2014 at 3:23 PM, Kouhei Kaigai wrote: >> On Fri, Nov 14, 2014 at 02:51:32PM +1300, David Rowley wrote: >> > Likely for most aggregates, like count, sum, max, min, bit_and and >> > bit_or the merge function would be the same as the transition >> > function, as the state type is just

Re: [HACKERS] Wait free LW_SHARED acquisition - v0.2

2014-12-02 Thread Michael Paquier
On Tue, Nov 18, 2014 at 12:33 AM, Robert Haas wrote: > On Mon, Nov 17, 2014 at 10:31 AM, Andres Freund > wrote: >> On 2014-11-17 10:21:04 -0500, Robert Haas wrote: >>> Andres, where are we with this patch? >>> >>> 1. You're going to commit it, but haven't gotten around to it yet. >>> >>> 2. You'

Re: [HACKERS] printing table in asciidoc with psql

2014-12-02 Thread Michael Paquier
On Wed, Dec 3, 2014 at 3:52 PM, Michael Paquier wrote: > This patch has been marked as "Waiting on Author" for a couple of > weeks, and the problems mentioned before have not been completely > addressed, hence marking this patch as returned with feedback. It > would be nice to see progress for the

Re: [HACKERS] printing table in asciidoc with psql

2014-12-02 Thread Michael Paquier
On Mon, Nov 17, 2014 at 7:48 AM, Pavel Stehule wrote: > 2014-11-07 22:37 GMT+01:00 Alvaro Herrera : >> >> >> I did \o /tmp/tst, then >> \dS >> create table "eh | oh" (); >> \dS >> >> and then filtered the output file to HTML. The CREATE TABLE tag ended >> up in the same line as the title of the f

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-12-02 Thread Kouhei Kaigai
> On Fri, Nov 14, 2014 at 02:51:32PM +1300, David Rowley wrote: > > Likely for most aggregates, like count, sum, max, min, bit_and and > > bit_or the merge function would be the same as the transition > > function, as the state type is just the same as the input type. It > > would only be aggregate

Re: [HACKERS] using Core Foundation locale functions

2014-12-02 Thread Peter Geoghegan
On Tue, Dec 2, 2014 at 10:07 PM, Craig Ringer wrote: > On 12/02/2014 12:52 AM, David E. Wheeler wrote: >> Gotta say, I’m thrilled to see movement on this front, and especially >> pleased to see how consensus seems to be building around an abstracted >> interface to keep options open. This platfo

Custom/Foreign-Join-APIs (Re: [HACKERS] [v9.5] Custom Plan API)

2014-12-02 Thread Kouhei Kaigai
> On Tue, Nov 25, 2014 at 3:44 AM, Kouhei Kaigai wrote: > > Today, I had a talk with Hanada-san to clarify which can be a common > > portion of them and how to implement it. Then, we concluded both of > > features can be shared most of the infrastructure. > > Let me put an introduction of join rep

Re: [HACKERS] using Core Foundation locale functions

2014-12-02 Thread Craig Ringer
On 12/02/2014 12:52 AM, David E. Wheeler wrote: > Gotta say, I’m thrilled to see movement on this front, and especially pleased > to see how consensus seems to be building around an abstracted interface to > keep options open. This platform-specific example really highlights the need > for it (I

Re: [HACKERS] using Core Foundation locale functions

2014-12-02 Thread Noah Misch
On Fri, Nov 28, 2014 at 11:43:28AM -0500, Peter Eisentraut wrote: > In light of the recent discussions about using ICU on OS X, I looked > into the Core Foundation locale functions (Core Foundation = traditional > Mac API in OS X, as opposed to the Unix/POSIX APIs). > > Attached is a proof of conc

Re: [HACKERS] On partitioning

2014-12-02 Thread Amit Langote
Hi Robert, From: Robert Haas [mailto:robertmh...@gmail.com] > > * Catalog schema: > > > > CREATE TABLE pg_catalog.pg_partitioned_rel > > ( > >partrelidoidNOT NULL, > >partkindoidNOT NULL, > >partissub bool NOT NULL, > >partkey

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-02 Thread Michael Paquier
On Wed, Dec 3, 2014 at 12:35 PM, Robert Haas wrote: > On Tue, Dec 2, 2014 at 7:16 PM, Michael Paquier > wrote: >> In the latest versions of the patch, control of compression is done >> within full_page_writes by assigning a new value 'compress'. Something >> that I am scared of is that if we enfo

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 7:16 PM, Michael Paquier wrote: > In the latest versions of the patch, control of compression is done > within full_page_writes by assigning a new value 'compress'. Something > that I am scared of is that if we enforce compression when > full_page_writes is off for forcibly-

Re: [HACKERS] About xmllint checking for the validity of postgres.xml in 9.5

2014-12-02 Thread Michael Paquier
On Wed, Dec 3, 2014 at 12:09 PM, Michael Paquier wrote: > Since commit 5d93ce2d, the output of xmllint is checked by passing > --valid to it. Isn't that a regression with what we were doing for > pre-9.4 versions? For example, with 9.4 and older versions it is > possible to compile man pages even

[HACKERS] About xmllint checking for the validity of postgres.xml in 9.5

2014-12-02 Thread Michael Paquier
Hi all, Since commit 5d93ce2d, the output of xmllint is checked by passing --valid to it. Isn't that a regression with what we were doing for pre-9.4 versions? For example, with 9.4 and older versions it is possible to compile man pages even if the xml spec is not entirely valid when using docbook

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Peter Geoghegan
On Tue, Dec 2, 2014 at 5:28 PM, Peter Geoghegan wrote: > Attached, revised patchset makes these updates. Whoops. Missed some obsolete comments. Here is a third commit that makes a further small modification to one comment. -- Peter Geoghegan From 8d1aba80f95e05742047cba5bd83d8f17aa5ef37 Mon Sep

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Peter Geoghegan
On Tue, Dec 2, 2014 at 2:16 PM, Peter Geoghegan wrote: > On Tue, Dec 2, 2014 at 2:07 PM, Robert Haas wrote: >> Well, maybe you should make the updates we've agreed on and I can take >> another look at it. > > Agreed. Attached, revised patchset makes these updates. I continue to use the sortsuppo

Re: [HACKERS] Many processes blocked at ProcArrayLock

2014-12-02 Thread Michael Paquier
On Tue, Dec 2, 2014 at 5:07 PM, Xiaoyulei wrote: > Test configuration: > Hardware: > 4P intel server, 60 core 120 hard thread. > Memory:512G > SSD:2.4T > > PG: > max_connections = 160 # (change requires restart) > shared_buffers = 32GB > work_mem = 128MB >

Re: [HACKERS] [v9.5] Custom Plan API

2014-12-02 Thread Kouhei Kaigai
> -Original Message- > From: Simon Riggs [mailto:si...@2ndquadrant.com] > Sent: Thursday, November 27, 2014 8:48 PM > To: Kaigai Kouhei(海外 浩平) > Cc: Robert Haas; Thom Brown; Kohei KaiGai; Tom Lane; Alvaro Herrera; Shigeru > Hanada; Stephen Frost; Andres Freund; PgHacker; Jim Mlodgenski; Pet

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jim Nasby
On 12/2/14, 2:22 PM, Jeff Janes wrote: Or maybe I overestimate how hard it would be to make vacuum restartable. You would have to save a massive amount of state (upto maintenance_work_mem tid list, the block you left off on both the table and all of the indexes in that table), and you would s

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-02 Thread Michael Paquier
On Wed, Dec 3, 2014 at 2:17 AM, Robert Haas wrote: > On Wed, Nov 26, 2014 at 11:00 PM, Michael Paquier > wrote: >> On Wed, Nov 26, 2014 at 8:27 PM, Syed, Rahila >> wrote: >>> Don't we need to initialize doPageCompression similar to doPageWrites in >>> InitXLOGAccess? >> Yep, you're right. I m

Re: [HACKERS] Nitpicky doc corrections for BRIN functions of pageinspect

2014-12-02 Thread Michael Paquier
On Wed, Dec 3, 2014 at 12:23 AM, Alvaro Herrera wrote: > Peter Geoghegan wrote: >> On Wed, Nov 19, 2014 at 8:02 PM, Michael Paquier >> wrote: >> > Just a small thing I noticed while looking at pageinspect.sgml, the >> > set of SQL examples related to BRIN indexes uses lower-case characters >> > f

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Tom Lane
Peter Geoghegan writes: > On Tue, Dec 2, 2014 at 2:21 PM, Robert Haas wrote: >> Right, and what I'm saying is that maybe the "applicability" flag >> shouldn't be stored in the SortSupport object, but passed down as an >> argument. > But then how does that information get to any given sortsupport

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Peter Geoghegan
On Tue, Dec 2, 2014 at 2:21 PM, Robert Haas wrote: > Right, and what I'm saying is that maybe the "applicability" flag > shouldn't be stored in the SortSupport object, but passed down as an > argument. But then how does that information get to any given sortsupport routine? That's the place that

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 5:16 PM, Peter Geoghegan wrote: > On Tue, Dec 2, 2014 at 2:07 PM, Robert Haas wrote: >> Well, maybe you should make the updates we've agreed on and I can take >> another look at it. > > Agreed. > >> But I didn't think that I was proposing to change >> anything about the lev

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Peter Geoghegan
On Tue, Dec 2, 2014 at 2:07 PM, Robert Haas wrote: > Well, maybe you should make the updates we've agreed on and I can take > another look at it. Agreed. > But I didn't think that I was proposing to change > anything about the level at which the decision about whether to > abbreviate or not was

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Alex Shulgin
Andres Freund writes: > On 2014-12-02 17:25:14 +0300, Alex Shulgin wrote: >> I'd be in favor of a solution that works the same way as before the >> patch, without the need for extra trigger files, etc., but that doesn't >> seem to be nearly possible. Whatever tricks we might employ will likely

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 4:21 PM, Peter Geoghegan wrote: >>> I'm not sure about that. I'd prefer to have tuplesort (and one or two >>> other sites) set the "abbreviation is possible in principle" flag. >>> Otherwise, sortsupport needs to assume that the leading attribute is >>> going to be the abbre

Re: [HACKERS] 9.2 recovery/startup problems

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 11:54 AM, Jeff Janes wrote: > During abort processing after getting a SIGTERM, the back end truncates > 59288 to zero size, and unlinks all the other files (including 59288_init). > The actual removal of 59288 is left until the checkpoint. So if you SIGTERM > the backend, t

Re: [HACKERS] using custom scan nodes to prototype parallel sequential scan

2014-12-02 Thread Bruce Momjian
On Fri, Nov 14, 2014 at 02:51:32PM +1300, David Rowley wrote: > Likely for most aggregates, like count, sum, max, min, bit_and and bit_or the > merge function would be the same as the transition function, as the state type > is just the same as the input type. It would only be aggregates like avg()

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Peter Geoghegan
On Tue, Dec 2, 2014 at 1:00 PM, Robert Haas wrote: > I'd prefer not to have a #define in pg_config_manual.h. Only stuff > that we expect a reasonably decent number of users to need to change > should be in that file, and this is too marginal for that. If anybody > other than the developers of th

Re: [HACKERS] Selectivity estimation for inet operators

2014-12-02 Thread Emre Hasegeli
> Actually, there's a second large problem with this patch: blindly > iterating through all combinations of MCV and histogram entries makes the > runtime O(N^2) in the statistics target. I made up some test data (by > scanning my mail logs) and observed the following planning times, as > reported

Re: [HACKERS] superuser() shortcuts

2014-12-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Dec 2, 2014 at 2:35 PM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> On Tue, Dec 2, 2014 at 11:29 AM, Stephen Frost wrote: > >> > It includes the actual error message, unlike what we have today which is > >> > gu

Re: [HACKERS] Doing better at HINTing an appropriate column within errorMissingColumn()

2014-12-02 Thread Robert Haas
On Tue, Nov 25, 2014 at 7:13 PM, Peter Geoghegan wrote: > Alright, so let me summarize what I think are the next steps in > working towards getting this patch committed. I should produce a new > revision which: > > * Uses default costings. > > * Applies a generic final quality check that enforces

Re: [HACKERS] Selectivity estimation for inet operators

2014-12-02 Thread Emre Hasegeli
> I spent a fair chunk of the weekend hacking on this patch to make > it more understandable and fix up a lot of what seemed to me pretty > clear arithmetic errors in the "upper layers" of the patch. However, > I couldn't quite convince myself to commit it, because the business > around estimation

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

2014-12-02 Thread Robert Haas
On Tue, Nov 25, 2014 at 1:38 PM, Peter Geoghegan wrote: > On Tue, Nov 25, 2014 at 4:01 AM, Robert Haas wrote: >> - This appears to needlessly reindent the comments for PG_CACHE_LINE_SIZE. > > Actually, the word "only" is removed (because PG_CACHE_LINE_SIZE has a > new client now). So it isn't qui

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Andres Freund
On 2014-12-02 17:25:14 +0300, Alex Shulgin wrote: > I'd be in favor of a solution that works the same way as before the > patch, without the need for extra trigger files, etc., but that doesn't > seem to be nearly possible. Whatever tricks we might employ will likely > be defeated by the fact that

Re: [HACKERS] superuser() shortcuts

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 2:35 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Tue, Dec 2, 2014 at 11:29 AM, Stephen Frost wrote: >> > It includes the actual error message, unlike what we have today which is >> > guidence as to what's required to get past the permission

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Andres Freund
On 2014-12-02 12:22:42 -0800, Jeff Janes wrote: > Or maybe I overestimate how hard it would be to make vacuum > restartable. That's a massive project. Which is why I'm explicitly *not* suggesting that. What I instead suggest is a separate threshhold after which vacuum isn't going to abort automati

Re: [HACKERS] Review of GetUserId() Usage

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 2:50 PM, Stephen Frost wrote: >> 1. It makes more of the crappy error message change that Andres and I >> already objected to on the other thread. Whether you disagree with >> those objections or not, don't make an end-run around them by putting >> more of the same stuff in

Re: [HACKERS] Review of GetUserId() Usage

2014-12-02 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: > > 3. It messes around with pg_signal_backend(). There are currently no > > cases in which pg_signal_backend() throws an error, which is good, > > because it lets you write queries against pg_stat_activity() that > > don't fail halfway through, even if

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 11:41 AM, Andres Freund wrote: > On 2014-12-02 11:23:31 -0800, Jeff Janes wrote: > > I think it would be more promising to work on downgrading lock strengths > so > > that fewer things conflict, and it would be not much more work than what > > you propose. > > I think you *

Re: [HACKERS] Removing INNER JOINs

2014-12-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Nov 30, 2014 at 12:51 PM, Tom Lane wrote: > > Bottom line, given all the restrictions on whether the optimization can > > happen, I have very little enthusiasm for the whole idea. I do not think > > the benefit will be big enough to justify t

Re: [HACKERS] Review of GetUserId() Usage

2014-12-02 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Sat, Nov 29, 2014 at 3:00 PM, Stephen Frost wrote: > > * Stephen Frost (sfr...@snowman.net) wrote: > > Updated patch attached which also changes the error messages (which > > hadn't been updated in the prior versions and really should be).

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Andres Freund
On 2014-12-02 11:23:31 -0800, Jeff Janes wrote: > I think it would be more promising to work on downgrading lock strengths so > that fewer things conflict, and it would be not much more work than what > you propose. I think you *massively* underestimate the effort required to to lower lock levels.

Re: [HACKERS] superuser() shortcuts

2014-12-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Dec 2, 2014 at 11:29 AM, Stephen Frost wrote: > > It includes the actual error message, unlike what we have today which is > > guidence as to what's required to get past the permission denied error. > > > > In other words, I disagree that the

Re: [HACKERS] Getting references for OID

2014-12-02 Thread Borodin Vladimir
> Hello, > > How can I get all depend objects for oid of object? For example, I have oid > of db db_id and I want to get all oids of namespaces, which contains this db. > Thank you! Hello, Dmitry. Actually, this list is for developers (as described here [0]). You should ask this question on

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 11:12 AM, Josh Berkus wrote: > On 12/02/2014 11:08 AM, Andres Freund wrote: > > On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: > >> On 12/02/2014 10:35 AM, Alvaro Herrera wrote: > >>> If the table is large, the time window for this to happen is large > also; > >>> there m

Re: [HACKERS] Sequence Access Method WIP

2014-12-02 Thread Andres Freund
On 2014-11-24 13:16:24 +0200, Heikki Linnakangas wrote: > To be clear: I don't think this API is very good for its stated purpose, for > implementing global sequences for use in a cluster. For the reasons I've > mentioned before. I'd like to see two changes to this proposal: > > 1. Make the AM im

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Andres Freund
On 2014-12-02 11:12:40 -0800, Josh Berkus wrote: > On 12/02/2014 11:08 AM, Andres Freund wrote: > > On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: > >> On 12/02/2014 10:35 AM, Alvaro Herrera wrote: > >>> If the table is large, the time window for this to happen is large also; > >>> there might ne

Re: [HACKERS] Removing INNER JOINs

2014-12-02 Thread Robert Haas
On Sun, Nov 30, 2014 at 12:51 PM, Tom Lane wrote: > Bottom line, given all the restrictions on whether the optimization can > happen, I have very little enthusiasm for the whole idea. I do not think > the benefit will be big enough to justify the amount of mess this will > introduce. This optimi

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Josh Berkus
On 12/02/2014 11:08 AM, Andres Freund wrote: > On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: >> On 12/02/2014 10:35 AM, Alvaro Herrera wrote: >>> If the table is large, the time window for this to happen is large also; >>> there might never be a time window large enough between two lock >>> acqu

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Andres Freund
On 2014-12-02 11:02:07 -0800, Josh Berkus wrote: > On 12/02/2014 10:35 AM, Alvaro Herrera wrote: > > If the table is large, the time window for this to happen is large also; > > there might never be a time window large enough between two lock > > acquisitions for one autovacuum run to complete in a

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Josh Berkus
On 12/02/2014 10:35 AM, Alvaro Herrera wrote: > If the table is large, the time window for this to happen is large also; > there might never be a time window large enough between two lock > acquisitions for one autovacuum run to complete in a table. This > starves the table from vacuuming complete

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Josh Berkus
On 12/02/2014 10:31 AM, Alvaro Herrera wrote: > Josh Berkus wrote: >> On 12/02/2014 06:25 AM, Alex Shulgin wrote: > >>> Whatever tricks we might employ will likely >>> be defeated by the fact that the oldschool user will fail to *include* >>> recovery.conf in the main conf file. >> >> Well, can we

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Alvaro Herrera
Robert Haas wrote: > On Sat, Nov 29, 2014 at 11:46 PM, Jim Nasby wrote: > > What do you mean by "never succeed"? Is it skipping a large number of pages? > > Might re-trying the locks within the same vacuum help, or are the user locks > > too persistent? > > You are confused. He's talking about t

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Alvaro Herrera
Josh Berkus wrote: > On 12/02/2014 06:25 AM, Alex Shulgin wrote: > > Whatever tricks we might employ will likely > > be defeated by the fact that the oldschool user will fail to *include* > > recovery.conf in the main conf file. > > Well, can we merge this patch and then fight out what to do for

[HACKERS] Getting references for OID

2014-12-02 Thread Воронин Дмитрий
Hello, How can I get all depend objects for oid of object? For example, I have oid of db db_id and I want to get all oids of namespaces, which contains this db. Thank you! -- С уважением, Дмитрий Воронин 

Re: [HACKERS] How about a option to disable autovacuum cancellation on lock conflict?

2014-12-02 Thread Robert Haas
On Sat, Nov 29, 2014 at 11:46 PM, Jim Nasby wrote: > What do you mean by "never succeed"? Is it skipping a large number of pages? > Might re-trying the locks within the same vacuum help, or are the user locks > too persistent? You are confused. He's talking about the relation-level lock that vac

Re: [HACKERS] Review of GetUserId() Usage

2014-12-02 Thread Robert Haas
On Sat, Nov 29, 2014 at 3:00 PM, Stephen Frost wrote: > * Stephen Frost (sfr...@snowman.net) wrote: >> * Stephen Frost (sfr...@snowman.net) wrote: >> > Attached is a patch to address the pg_cancel/terminate_backend and the >> > statistics info as discussed previously. It sounds like we're coming

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Josh Berkus
On 12/02/2014 06:25 AM, Alex Shulgin wrote: >> I am not in favor of this part. It may be better to let the users know >> > that their old configuration is not valid anymore with an error. This >> > patch cuts in the flesh with a huge axe, let's be sure that users do >> > not ignore the side pain ef

Re: [HACKERS] Testing DDL deparsing support

2014-12-02 Thread Alvaro Herrera
Robert Haas wrote: > On Thu, Nov 27, 2014 at 11:43 PM, Ian Barwick wrote: > > A simple schedule to demonstrate this is available; execute from the > > src/test/regress/ directory like this: > > > > ./pg_regress \ > > --temp-install=./tmp_check \ > > --top-builddir=../../.. \ > >

Re: [HACKERS] Add shutdown_at_recovery_target option to recovery.conf

2014-12-02 Thread Robert Haas
On Fri, Nov 28, 2014 at 11:59 AM, Petr Jelinek wrote: >>> I'm a bit late to the party, but wouldn't >>> >>> recovery_target_action = ... >>> >>> have been a better name for this? It'd be in line with the other >>> recovery_target_* parameters, and also a bit shorter than the imho >>> somewhat ugly

Re: [HACKERS] patch : Allow toast tables to be moved to a different tablespace

2014-12-02 Thread Robert Haas
On Fri, Nov 28, 2014 at 11:38 AM, Alex Shulgin wrote: > Should I add this patch to the next CommitFest? If you don't want it to get forgotten about, yes. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] Fillfactor for GIN indexes

2014-12-02 Thread Robert Haas
On Fri, Nov 28, 2014 at 4:27 AM, Alexander Korotkov wrote: > On Fri, Nov 21, 2014 at 8:12 AM, Michael Paquier > wrote: >> Please find attached a simple patch adding fillfactor as storage parameter >> for GIN indexes. The default value is the same as the one currently aka 100 >> to have the pages

Re: [HACKERS] Using pg_rewind for differential backup

2014-12-02 Thread Robert Haas
On Fri, Nov 28, 2014 at 2:49 AM, Heikki Linnakangas wrote: > It also would be quite straightforward to write a separate tool to do just > that. Would be better than conflating pg_rewind with this. You could use > pg_rewind as the basis for it - it's under the same license as PostgreSQL. If we had

Re: [HACKERS] Testing DDL deparsing support

2014-12-02 Thread Robert Haas
On Thu, Nov 27, 2014 at 11:43 PM, Ian Barwick wrote: > DDL deparsing is a feature that allows collection of DDL commands as they > are > executed in a server, in some flexible, complete, and fully-contained format > that allows manipulation, storage, and transmission. This feature has > several >

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-02 Thread Robert Haas
On Wed, Nov 26, 2014 at 11:00 PM, Michael Paquier wrote: > On Wed, Nov 26, 2014 at 8:27 PM, Syed, Rahila wrote: >> Don't we need to initialize doPageCompression similar to doPageWrites in >> InitXLOGAccess? > Yep, you're right. I missed this code path. > >> Also , in the earlier patches compres

[HACKERS] Report search_path value back to the client.

2014-12-02 Thread Alexander Kukushkin
Hi, As of now postgres is reporting only "really" important variables, but among them there are also not so important, like 'application_name'. The only reason to report it, was: "help pgbouncer, and perhaps other connection poolers". Change was introduced by commit: 59ed94ad0c9f74a3f057f359316c84

Re: [HACKERS] why is PG_AUTOCONF_FILENAME is pg_config_manual.h?

2014-12-02 Thread Robert Haas
On Thu, Nov 27, 2014 at 12:56 PM, Fujii Masao wrote: > On Thu, Nov 27, 2014 at 11:58 PM, Peter Eisentraut wrote: >> Surely that's not a value that we expect users to be able to edit. Is >> pg_config_manual.h just abused as a place that's included everywhere? >> >> (I suggest utils/guc.h as a bet

Re: [HACKERS] 9.2 recovery/startup problems

2014-12-02 Thread Jeff Janes
On Tue, Dec 2, 2014 at 7:41 AM, Robert Haas wrote: > On Wed, Nov 26, 2014 at 7:13 PM, Jeff Janes wrote: > > If I do a pg_ctl stop -mf, then both files go away. If I do a pg_ctl > stop > > -mi, then neither goes away. It is only with the /sbin/reboot that I get > > the fatal combination of _ini

Re: [HACKERS] [PATCH] HINT: pg_hba.conf changed since last config reload

2014-12-02 Thread Robert Haas
On Thu, Nov 27, 2014 at 8:49 AM, Bruce Momjian wrote: > On Thu, Nov 6, 2014 at 05:46:42PM -0500, Peter Eisentraut wrote: >> Finally, the fact that a configuration change is in progress is >> privileged information. Unprivileged users can deduct from the presence >> of this message that administr

Re: [HACKERS] Add CREATE support to event triggers

2014-12-02 Thread Robert Haas
> Normally you would replicate between an older master and a newer > replica, so this shouldn't be an issue. I find it unlikely that we > would de-support some syntax that works in an older version: it would > break pg_dump, for one thing. The most common way in which we break forward-compatibili

Re: [HACKERS] superuser() shortcuts

2014-12-02 Thread Robert Haas
On Tue, Dec 2, 2014 at 11:29 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Wed, Nov 26, 2014 at 10:12 AM, Stephen Frost wrote: >> > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> >> In the context at hand, I think most of the messages in question are >> >> currently phr

Re: [HACKERS] copy.c handling for RLS is insecure

2014-12-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Nov 27, 2014 at 2:03 AM, Stephen Frost wrote: > > Alright, I've done the change to use the RangeVar from CopyStmt, but > > also added a check wherein we verify that the relation's OID returned > > from the planned query is the same as the rela

Re: [HACKERS] superuser() shortcuts

2014-12-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Nov 26, 2014 at 10:12 AM, Stephen Frost wrote: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> In the context at hand, I think most of the messages in question are > >> currently phrased like "must be superuser to do X". I'd be fine with > >>

Re: [HACKERS] initdb: Improve error recovery.

2014-12-02 Thread Robert Haas
On Thu, Nov 27, 2014 at 7:28 AM, Mats Erik Andersson wrote: > I would like to improve error recovery of initdb when the > password file is empty. The present code declares "Error 0" > as the cause of failure. It suffices to use ferrer() since > fgets() returns NULL also at a premature EOF. > > In

Re: [HACKERS] test_shm_mq failing on mips*

2014-12-02 Thread Christoph Berg
Re: Robert Haas 2014-12-02 > I can't tell from this exactly what's going wrong. Questions: > > 1. Are there any background worker processes running at the same time? > If so, how many? We'd expect to see 3. > 2. Can we printout of the following variables in stack frame 3 > (test_shm_mq_pipeli

Re: [HACKERS] copy.c handling for RLS is insecure

2014-12-02 Thread Robert Haas
On Thu, Nov 27, 2014 at 2:03 AM, Stephen Frost wrote: > Alright, I've done the change to use the RangeVar from CopyStmt, but > also added a check wherein we verify that the relation's OID returned > from the planned query is the same as the relation's OID that we did the > RLS check on- if they're

Re: [HACKERS] dblink_get_connections() result for no connections

2014-12-02 Thread Robert Haas
On Tue, Nov 25, 2014 at 3:49 PM, Tom Lane wrote: > While fooling around with the array_agg(anyarray) patch, I noted > that dblink_get_connections() returns NULL if there are no active > connections. It seems like an empty array might be a saner > definition --- thoughts? I don't think it matters

Re: [HACKERS] memory explosion on planning complex query

2014-12-02 Thread Robert Haas
On Wed, Nov 26, 2014 at 7:24 PM, Andrew Dunstan wrote: > On 11/26/2014 05:00 PM, Andrew Dunstan wrote: >> Attached is some anonymized DDL for a fairly complex schema from a >> PostgreSQL Experts client. Also attached is an explain query that runs >> against the schema. The client's problem is that

Re: [HACKERS] 9.2 recovery/startup problems

2014-12-02 Thread Robert Haas
On Wed, Nov 26, 2014 at 7:13 PM, Jeff Janes wrote: > If I do a pg_ctl stop -mf, then both files go away. If I do a pg_ctl stop > -mi, then neither goes away. It is only with the /sbin/reboot that I get > the fatal combination of _init being gone but the other still present. Eh? That sounds won

Re: [HACKERS] superuser() shortcuts

2014-12-02 Thread Robert Haas
On Wed, Nov 26, 2014 at 10:12 AM, Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> In the context at hand, I think most of the messages in question are >> currently phrased like "must be superuser to do X". I'd be fine with >> changing that to "permission denied to do X", but not

Re: [HACKERS] On partitioning

2014-12-02 Thread Robert Haas
On Tue, Nov 25, 2014 at 8:20 PM, Amit Langote wrote: >> Before going too much further with this I'd mock up schemas for your >> proposed catalogs and a list of DDL operations to be supported, with >> the corresponding syntax, and float that here for comment. More people should really comment on t

Re: [HACKERS] Nitpicky doc corrections for BRIN functions of pageinspect

2014-12-02 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Wed, Nov 19, 2014 at 8:02 PM, Michael Paquier > wrote: > > Just a small thing I noticed while looking at pageinspect.sgml, the > > set of SQL examples related to BRIN indexes uses lower-case characters > > for reserved keywords. This has been introduced by 7516f52. > >

Re: [HACKERS] test_shm_mq failing on mips*

2014-12-02 Thread Robert Haas
On Tue, Nov 25, 2014 at 10:42 AM, Christoph Berg wrote: > Re: To Robert Haas 2014-11-24 <20141124200824.ga22...@msg.df7cb.de> >> > Does it fail every time when run on a machine where it fails sometimes? >> >> So far there's a consistent host -> fail-or-not mapping, but most >> mips/mipsel build ho

Re: [HACKERS] pg_stat_statement normalization fails due to temporary tables

2014-12-02 Thread Andres Freund
On 2014-12-02 09:59:00 -0500, Tom Lane wrote: > Andres Freund writes: > > pg_stat_statement's query normalization fails when temporary tables are > > used in a query. That's because JumbleRangeTable() uses the relid in the > > RTE to build the query fingerprint. I think in this case pgss from 9.1

Re: [HACKERS] pg_stat_statement normalization fails due to temporary tables

2014-12-02 Thread Tom Lane
Andres Freund writes: > pg_stat_statement's query normalization fails when temporary tables are > used in a query. That's because JumbleRangeTable() uses the relid in the > RTE to build the query fingerprint. I think in this case pgss from 9.1 > actually would do better than 9.2+ as the hash looku

Re: [HACKERS] [v9.5] Custom Plan API

2014-12-02 Thread Robert Haas
On Tue, Nov 25, 2014 at 3:44 AM, Kouhei Kaigai wrote: > Today, I had a talk with Hanada-san to clarify which can be a common portion > of them and how to implement it. Then, we concluded both of features can be > shared most of the infrastructure. > Let me put an introduction of join replacement b

Re: [HACKERS] Turning recovery.conf into GUCs

2014-12-02 Thread Alex Shulgin
Michael Paquier writes: > On Tue, Dec 2, 2014 at 1:58 AM, Alex Shulgin wrote: >> Here's the patch rebased against current HEAD, that is including the >> recently committed action_at_recovery_target option. > If this patch gets in, it gives a good argument to jump to 10.0 IMO. > That's not a bad

[HACKERS] pg_stat_statement normalization fails due to temporary tables

2014-12-02 Thread Andres Freund
Hi, pg_stat_statement's query normalization fails when temporary tables are used in a query. That's because JumbleRangeTable() uses the relid in the RTE to build the query fingerprint. I think in this case pgss from 9.1 actually would do better than 9.2+ as the hash lookup previously didn't use th

Re: [HACKERS] excessive amounts of consumed memory (RSS), triggering OOM killer

2014-12-02 Thread Tomas Vondra
Dne 2 Prosinec 2014, 10:59, Tomas Vondra napsal(a): > Dne 2014-12-02 02:52, Tom Lane napsal: >> Tomas Vondra writes: >> >>> Also, this explains the TopMemoryContext size, but not the RSS size >>> (or am I missing something)? >> >> Very possibly you're left with "islands" that prevent reclaiming ve

Re: [HACKERS] Role Attribute Bitmask Catalog Representation

2014-12-02 Thread Stephen Frost
Adam, * Adam Brightwell (adam.brightw...@crunchydatasolutions.com) wrote: > Ok. Though, this would affect how CATUPDATE is handled. Peter Eisentraut > previously raised a question about whether superuser checks should be > included with catupdate which led me to create the following post. > > h

[HACKERS] Serialization exception : Who else was involved?

2014-12-02 Thread Olivier MATROT
Hello, I'm using PostgreSQL .9.2.8 on Windows from a .NET application using Npgsql. I'm working in the Radiology Information System field. We have thousands of users against a big accounting database. We're using the SERIALIZABLE isolation level to ensure data consistency. Because of

Re: [HACKERS] excessive amounts of consumed memory (RSS), triggering OOM killer

2014-12-02 Thread Tomas Vondra
Dne 2014-12-02 02:52, Tom Lane napsal: Tomas Vondra writes: On 2.12.2014 01:33, Tom Lane wrote: What I suspect you're looking at here is the detritus of creation of a huge number of memory contexts. mcxt.c keeps its own state about existing contents in TopMemoryContext. So, if we posit that th

Re: [HACKERS] using Core Foundation locale functions

2014-12-02 Thread Peter Geoghegan
On Fri, Nov 28, 2014 at 8:43 AM, Peter Eisentraut wrote: > At the moment, this is probably just an experiment that shows where > refactoring and better abstractions might be suitable if we want to > support multiple locale libraries. If we want to pursue ICU, I think > this could be a useful thir

[HACKERS] Many processes blocked at ProcArrayLock

2014-12-02 Thread Xiaoyulei
Test configuration: Hardware: 4P intel server, 60 core 120 hard thread. Memory:512G SSD:2.4T PG: max_connections = 160 # (change requires restart) shared_buffers = 32GB work_mem = 128MB maintenance_work_me