Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-17 Thread Craig Ringer
On 9 February 2016 at 03:00, Joshua D. Drake wrote: > > I think this further points to the need for more reviewers and less > feature pushes. There are fundamental features that we could use, this is > one of them. It is certainly more important than say pgLogical or BDR (not > that those aren't

Re: [HACKERS] WAL logging problem in 9.4.3?

2016-02-17 Thread Michael Paquier
On Thu, Feb 4, 2016 at 3:24 PM, Heikki Linnakangas wrote: > I dropped the ball on this one back in July, so here's an attempt to revive > this thread. > > I spent some time fixing the remaining issues with the prototype patch I > posted earlier, and rebased that on top of current git master. See a

Re: [HACKERS] Declarative partitioning

2016-02-17 Thread Amit Langote
On 2016/02/16 21:57, Robert Haas wrote: > On Fri, Jan 15, 2016 at 5:48 AM, Amit Langote > wrote: >> If we have a CREATE statement for each partition, how do we generalize >> that to partitions at different levels? For example, if we use something >> like the following to create a partition of pare

Re: [HACKERS] ALTER ROLE SET/RESET for multiple options

2016-02-17 Thread Michael Paquier
On Wed, Feb 17, 2016 at 7:23 PM, Masahiko Sawada wrote: > On Wed, Feb 17, 2016 at 7:14 PM, Robert Haas wrote: >> On Wed, Feb 17, 2016 at 3:22 PM, Masahiko Sawada >> wrote: >>> On Sat, Feb 13, 2016 at 2:45 PM, Robert Haas wrote: On Fri, Feb 12, 2016 at 1:35 PM, Payal Singh wrote: > Th

Re: [HACKERS] proposal: function parse_ident

2016-02-17 Thread Pavel Stehule
Hi 2016-02-17 14:02 GMT+01:00 Teodor Sigaev : > I missed some of my edits. Updated patch with those in place attached. >> > > I'm sorry, but special chararacter still isn't escaped correctly in error > messages: > > % select > parse_ident(E'X\rX

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-02-17 Thread Pavel Stehule
2016-02-17 16:54 GMT+01:00 Catalin Iacob : > On Wed, Feb 17, 2016 at 3:32 PM, Pavel Stehule > wrote: > >> Python 3 has keyword only arguments. It occurs to me they're exactly > >> for "optional extra stuff" like detail, hint etc. > >> Python 2 doesn't have that notion but you can kind of fake it

Re: [HACKERS] WIP: Access method extendability

2016-02-17 Thread Michael Paquier
On Wed, Feb 17, 2016 at 11:56 PM, Teodor Sigaev wrote: > 11 I'd really like to see regression tests (TAP-tests?) for replication with > generic xlog. The recovery test suite can help to accomplish that. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make c

Re: [HACKERS] Fix handling of invalid sockets returned by PQsocket()

2016-02-17 Thread Michael Paquier
On Thu, Feb 18, 2016 at 1:58 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> Hi all, >> >> After looking at Alvaro's message mentioning the handling of >> PQsocket() for invalid sockets, I just had a look by curiosity at >> other calls of this routine, and found a couple of issues: >> 1) In

Re: [HACKERS] PostgreSQL Audit Extension

2016-02-17 Thread Bruce Momjian
On Wed, Feb 17, 2016 at 01:59:09PM +0530, Robert Haas wrote: > On Wed, Feb 17, 2016 at 5:20 AM, Bruce Momjian wrote: > > On Fri, Feb 5, 2016 at 01:16:25PM -0500, Stephen Frost wrote: > >> Looking at pgaudit and the other approaches to auditing which have been > >> developed (eg: applications whic

[HACKERS] pg_ctl promote wait

2016-02-17 Thread Peter Eisentraut
It would be nice if pg_ctl promote supported the -w (wait) option. How could pg_ctl determine when the promotion has finished? We could query pg_is_in_recovery(), but that would require database access, which we got rid of in pg_ctl. We could check when recovery.conf is gone or recovery.done app

Re: [HACKERS] Figures in docs

2016-02-17 Thread Tatsuo Ishii
> What we need is tooling around a new file format that is similar or > analogous to our existing tooling, so that it is easy to edit, easy to > review, easy to build, easy to test, and so on. Otherwise, the image > files will not get maintained properly. > > As an example, if an image contains t

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Peter Eisentraut
On 2/17/16 9:08 PM, Michael Paquier wrote: > On Thu, Feb 18, 2016 at 11:02 AM, Peter Eisentraut wrote: >> On 2/17/16 5:20 PM, Josh berkus wrote: >>> I have a use-case for this feature, at part of it containerized >>> PostgreSQL. Right now, there is certain diagnostic information (like >>> timeline

Re: [HACKERS] Figures in docs

2016-02-17 Thread Peter Eisentraut
On 2/17/16 8:23 PM, Tatsuo Ishii wrote: > Do we really need git history for each figure? It seems we are waiting > for a solution which will never realize. What we need is tooling around a new file format that is similar or analogous to our existing tooling, so that it is easy to edit, easy to rev

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Michael Paquier
On Thu, Feb 18, 2016 at 11:02 AM, Peter Eisentraut wrote: > On 2/17/16 5:20 PM, Josh berkus wrote: >> I have a use-case for this feature, at part of it containerized >> PostgreSQL. Right now, there is certain diagnostic information (like >> timeline) which is exposed ONLY in pg_controldata. > > I'

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Peter Eisentraut
On 2/17/16 5:20 PM, Josh berkus wrote: > I have a use-case for this feature, at part of it containerized > PostgreSQL. Right now, there is certain diagnostic information (like > timeline) which is exposed ONLY in pg_controldata. I'm talking about the pg_config() function, not pg_controldata. --

Re: [HACKERS] custom function for converting human readable sizes to bytes

2016-02-17 Thread Vitaly Burovoy
On 2/17/16, Dean Rasheed wrote: > On 17 February 2016 at 00:39, Vitaly Burovoy > wrote: >> Now parse_memory_unit returns -1024 for bytes as divider, constant >> "bytes" has moved there. >> Add new memory_units_bytes_hint which differs from an original >> memory_units_int by "bytes" size unit. >>

Re: [HACKERS] Figures in docs

2016-02-17 Thread Tatsuo Ishii
>> What's wrong with LibreOffice? > > Is there any reason to think it doesn't have the same disease mentioned > in the previously-cited thread, namely that any change trashes pretty > much the whole file? > > That might be okay for things that we only change once every ten years > or so, but othe

Re: [HACKERS] Figures in docs

2016-02-17 Thread Tom Lane
Tatsuo Ishii writes: >> Because no one has been able to propose a good format for storing and >> editing pictures. > What's wrong with LibreOffice? Is there any reason to think it doesn't have the same disease mentioned in the previously-cited thread, namely that any change trashes pretty much t

Re: [HACKERS] Figures in docs

2016-02-17 Thread Tatsuo Ishii
> On 2/16/16 8:17 PM, Tatsuo Ishii wrote: >> It seems there's no figures/diagrams in our docs. I vaguely recall that >> we used to have a few diagrams in our docs. If so, was there any >> technical reason to remove them? > > Because no one has been able to propose a good format for storing and > e

Re: [HACKERS] Figures in docs

2016-02-17 Thread Tatsuo Ishii
> Gavin Flower wrote: >> On 18/02/16 10:38, Peter Eisentraut wrote: >> >On 2/16/16 8:17 PM, Tatsuo Ishii wrote: >> >>It seems there's no figures/diagrams in our docs. I vaguely recall that >> >>we used to have a few diagrams in our docs. If so, was there any >> >>technical reason to remove them? >>

Re: [HACKERS] Figures in docs

2016-02-17 Thread Tatsuo Ishii
> Gavin Flower wrote: >> On 18/02/16 10:38, Peter Eisentraut wrote: >> >On 2/16/16 8:17 PM, Tatsuo Ishii wrote: >> >>It seems there's no figures/diagrams in our docs. I vaguely recall that >> >>we used to have a few diagrams in our docs. If so, was there any >> >>technical reason to remove them? >>

Re: [HACKERS] extend pgbench expressions with functions

2016-02-17 Thread Michael Paquier
On Wed, Feb 17, 2016 at 5:22 PM, Fabien COELHO wrote: >> \set aid 1 + 1 >> pgbench -f addition.sql -t 5000 >> >> I have the following: >> HEAD: 3.5~3.7M TPS >> list method: 3.6~3.7M TPS >> array method: 3.4~3.5M TPS >> So all approaches have a comparable performance. > > Yep, the execution tra

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Joe Conway
On 02/17/2016 03:34 PM, Josh berkus wrote: > On 02/17/2016 03:02 PM, Tom Lane wrote: >> Joe Conway writes: >>> On 02/17/2016 02:14 PM, Tom Lane wrote: I thought we'd agreed on requiring superuser access for this function. I concur that letting just anyone see the config data is inappropr

[HACKERS] Restructuring Paths to allow per-Path targetlist info

2016-02-17 Thread Tom Lane
Although it may not be obvious to the naked eye, I have been working away on that long-overdue upper-planner pathification project. One thing that's necessary to make that happen is to keep track of the targetlists of upper plan levels within the Path data structures, since once you get past the s

Re: [HACKERS] pglogical - how to use pglogical.conflict_resolution

2016-02-17 Thread xujian
sorry, please ignore it.It is a setting, not function, I can set it withset pglogical.conflict_resolution = 'error' thanks JamesFrom: jame...@outlook.com To: pgsql-hackers@postgresql.org Subject: pglogical - how to use pglogical.conflict_resolution Date: Wed, 17 Feb 2016 18:30:35 -0500 Hello,

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Josh berkus
On 02/17/2016 03:02 PM, Tom Lane wrote: Joe Conway writes: On 02/17/2016 02:14 PM, Tom Lane wrote: I thought we'd agreed on requiring superuser access for this function. I concur that letting just anyone see the config data is inappropriate. It does not let anyone see config data out of the

[HACKERS] pglogical - how to use pglogical.conflict_resolution

2016-02-17 Thread xujian
Hello, I setup the logical replication with pglogical on pg95, it works fine, however, I want to set the conflict resolution according to the document below=The configuration of the conflicts resolver is done via the`pglogical.conflict_resolution` setting. The support

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Tom Lane
Joe Conway writes: > On 02/17/2016 02:14 PM, Tom Lane wrote: >> I thought we'd agreed on requiring superuser access for this function. >> I concur that letting just anyone see the config data is inappropriate. > It does not let anyone see config data out of the box: > + CREATE VIEW pg_config AS

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Andrew Dunstan
On 02/17/2016 05:14 PM, Tom Lane wrote: Peter Eisentraut writes: On 2/17/16 12:15 PM, Joe Conway wrote: Ok, removed the documentation on the function pg_config() and pushed. I still have my serious doubts about this, especially not even requiring superuser access for this information. Coul

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Josh berkus
On 02/17/2016 01:31 PM, Peter Eisentraut wrote: On 1/31/16 7:34 AM, Michael Paquier wrote: I am marking this patch as returned with feedback for now, not all the issues have been fixed yet, and there are still no docs (the conclusion being that people would like to have this stuff, right?). Feel

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-17 Thread Jim Nasby
On 2/8/16 4:39 PM, Peter Geoghegan wrote: On Mon, Feb 8, 2016 at 2:35 PM, Andres Freund wrote: I think having a public git tree, that contains the current state, is greatly helpful for that. Just announce that you're going to screw wildly with history, and that you're not going to be terribly c

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Joe Conway
On 02/17/2016 02:14 PM, Tom Lane wrote: > Peter Eisentraut writes: >> On 2/17/16 12:15 PM, Joe Conway wrote: >>> Ok, removed the documentation on the function pg_config() and pushed. > >> I still have my serious doubts about this, especially not even requiring >> superuser access for this informa

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Tom Lane
Peter Eisentraut writes: > On 2/17/16 12:15 PM, Joe Conway wrote: >> Ok, removed the documentation on the function pg_config() and pushed. > I still have my serious doubts about this, especially not even requiring > superuser access for this information. Could someone explain why we > need this?

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-02-17 Thread Jim Nasby
On 2/17/16 9:03 AM, Dean Rasheed wrote: I'm not totally opposed to specifying a column sort order in psql, and perhaps there's a way to support both 'cols' and 'col_order' options in psql, since there are different situations where one or the other might be more useful. Yeah. If there was some

Re: [HACKERS] Figures in docs

2016-02-17 Thread Alvaro Herrera
Gavin Flower wrote: > On 18/02/16 10:38, Peter Eisentraut wrote: > >On 2/16/16 8:17 PM, Tatsuo Ishii wrote: > >>It seems there's no figures/diagrams in our docs. I vaguely recall that > >>we used to have a few diagrams in our docs. If so, was there any > >>technical reason to remove them? > >Becaus

Re: [HACKERS] Figures in docs

2016-02-17 Thread Gavin Flower
On 18/02/16 10:38, Peter Eisentraut wrote: On 2/16/16 8:17 PM, Tatsuo Ishii wrote: It seems there's no figures/diagrams in our docs. I vaguely recall that we used to have a few diagrams in our docs. If so, was there any technical reason to remove them? Because no one has been able to propose a

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Peter Eisentraut
On 2/17/16 12:15 PM, Joe Conway wrote: > On 02/17/2016 02:32 AM, Michael Paquier wrote: >> Actually, having second-thoughts on the matter, why is is that >> necessary to document the function pg_config()? The functions wrapping >> a system view just have the view documented, take for example >> pg_

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-02-17 Thread Constantin S. Pan
I was testing with number of workers starting at 0. The 0 case is (in theory) equivalent to master branch. But I should certainly compare to the master branch, you are right. Will do that shortly. On Wed, 17 Feb 2016 12:26:05 -0800 Peter Geoghegan wrote: > On Wed, Feb 17, 2016 at 7:55 AM, Consta

Re: [HACKERS] Figures in docs

2016-02-17 Thread Peter Eisentraut
On 2/16/16 8:17 PM, Tatsuo Ishii wrote: > It seems there's no figures/diagrams in our docs. I vaguely recall that > we used to have a few diagrams in our docs. If so, was there any > technical reason to remove them? Because no one has been able to propose a good format for storing and editing pict

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Peter Eisentraut
On 1/31/16 7:34 AM, Michael Paquier wrote: > I am marking this patch as returned with feedback for now, not all the > issues have been fixed yet, and there are still no docs (the > conclusion being that people would like to have this stuff, right?). > Feel free to move it to the next CF should a ne

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-02-17 Thread Peter Eisentraut
On 2/9/16 11:21 AM, Daniel Verite wrote: > Note that NULL values in the column that pivots are discarded > by \crosstabview, because NULL as the name of a column does not > make sense. Why not? All you're doing is printing it out, and psql is quite capable of printing a null value. -- Sent vi

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Alvaro Herrera
Craig Ringer wrote: > Improving this probably needs DDL deparse to be smarter. Rather than just > emitting something that can be reconstructed into the SQL text of the DDL > it needs to emit one or more steps that are semantically the same but allow > us to skip the rewrite. Along the lines of: >

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-02-17 Thread Peter Geoghegan
On Wed, Feb 17, 2016 at 7:55 AM, Constantin S. Pan wrote: > 4. Hit the 8x speedup limit. Made some analysis of the reasons (see the > attached plot or the data file). Did you actually compare this to the master branch? I wouldn't like to assume that the one worker case was equivalent. Obviously t

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-02-17 Thread Oleg Bartunov
On Wed, Feb 17, 2016 at 6:55 PM, Constantin S. Pan wrote: > On Sat, 16 Jan 2016 01:38:39 +0300 > "Constantin S. Pan" wrote: > > > The task of building GIN can require lots of time and eats 100 % CPU, > > but we could easily make it use more than a 100 %, especially since we > > now have parallel

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-02-17 Thread Daniel Verite
Jim Nasby wrote: > > ORDER BY name > > \crosstabview cols = (select to_char(d, 'Mon') from > > generate_series('2000-01-01'::date, '2000-12-01', '1 month') d) > > My concern with that is that often you don't know what the columns will > be, because you don't know what exact data the qu

Re: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates

2016-02-17 Thread Peter Geoghegan
On Wed, Feb 17, 2016 at 2:12 AM, Robert Haas wrote: > Committed, except I left out one comment hunk that I wasn't convinced about. Thank you. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Joe Conway
On 02/17/2016 02:32 AM, Michael Paquier wrote: > Actually, having second-thoughts on the matter, why is is that > necessary to document the function pg_config()? The functions wrapping > a system view just have the view documented, take for example > pg_show_all_file_settings, pg_show_all_settings,

Re: [HACKERS] Fix handling of invalid sockets returned by PQsocket()

2016-02-17 Thread Alvaro Herrera
Michael Paquier wrote: > Hi all, > > After looking at Alvaro's message mentioning the handling of > PQsocket() for invalid sockets, I just had a look by curiosity at > other calls of this routine, and found a couple of issues: > 1) In vacuumdb.c, init_slot() does not check for the return value of

Re: [HACKERS] [PATCH] Code refactoring related to -fsanitize=use-after-scope

2016-02-17 Thread Martin Liška
On 02/15/2016 08:20 PM, Tom Lane wrote: > I bet a nickel that this is triggered by the goto leading into those > variables' scope ("goto process_inner_tuple" at line 2038 in HEAD). > That probably bypasses the "unpoison" step. > > However, doesn't this represent a bug in the sanitizer rather than

Re: [HACKERS] Figures in docs

2016-02-17 Thread Alexander Lakhin
Hello, In fact we use "make postgres.xml" to get a single XML, which we translate. We convert it to .po using xml2po (with some modifications), and then convert it back to xml (and then to sgml files with our custom script). So we have not changed sgml to xml files, and if it's more appropriate

Re: [HACKERS] Figures in docs

2016-02-17 Thread Alexander Lakhin
17.02.2016 09:17, Tatsuo Ishii wrote: >> Hi. >> >> In DocBook 4.2 sgml dtd, figure tag is supported already. >> that was implemented for multi output format. > Ok, there's no technical problems with figures then. MySQL docs has > some nice figures. I am jealous. The "figure" tag is just a placeho

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

2016-02-17 Thread Tom Lane
I just had a rather disturbing thought to the effect that this entire design --- ie, parallel workers taking out locks for themselves --- is fundamentally flawed. As far as I can tell from README.parallel, parallel workers are supposed to exit (and, presumably, release their locks) before the lead

Re: [HACKERS] Tsvector editing functions

2016-02-17 Thread Stas Kelvich
> > On 02 Feb 2016, at 20:10, Teodor Sigaev wrote: > > Some notices: > > 1 tsin in documentation doesn't look like a good name. Changed to vector > similar to other places. > > 2 I did some editorization about freeing memory/forgotten names etc Thanks. > > 3 It seems to me that tsvector_un

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-02-17 Thread Constantin S. Pan
On Sat, 16 Jan 2016 01:38:39 +0300 "Constantin S. Pan" wrote: > The task of building GIN can require lots of time and eats 100 % CPU, > but we could easily make it use more than a 100 %, especially since we > now have parallel workers in postgres. > > The process of building GIN looks like this:

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-02-17 Thread Catalin Iacob
On Wed, Feb 17, 2016 at 3:32 PM, Pavel Stehule wrote: >> Python 3 has keyword only arguments. It occurs to me they're exactly >> for "optional extra stuff" like detail, hint etc. >> Python 2 doesn't have that notion but you can kind of fake it since >> you get an args tuple and a kwargs dictionary

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

2016-02-17 Thread Tom Lane
Robert Haas writes: > On Tue, Feb 16, 2016 at 2:59 AM, Tom Lane wrote: >> Not to be neglected also is that (I believe) this gives the right answer, >> whereas isolationtester's existing query is currently completely broken by >> parallel queries, and it doesn't understand non-conflicting lock mod

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-02-17 Thread Dean Rasheed
On 17 February 2016 at 02:32, Jim Nasby wrote: > On 2/11/16 4:21 AM, Dean Rasheed wrote: >> >> Thinking about this some more though, perhaps*sorting* the columns is >> the wrong way to be thinking about it. Perhaps a better approach would >> be to allow the columns to be*listed* (optionally, usi

Re: [HACKERS] [patch] Proposal for \crosstabview in psql

2016-02-17 Thread Dean Rasheed
On 15 February 2016 at 14:08, Daniel Verite wrote: > Dean Rasheed wrote: > >> My biggest problem is with the sorting, for all the reasons discussed >> above. There is absolutely no reason for \crosstabview to be >> re-sorting rows -- they should just be left in the original query >> result

Re: [HACKERS] WIP: Access method extendability

2016-02-17 Thread Teodor Sigaev
Next version of patch is attached: * Documentation for CREATE ACCESS METHOD command is added. * Refactoring and comments for bloom contrib. Cool work, nice to see. Some comments 1 create-am.7.patch: function lookup_index_am_handler_func() why doesn't it emit error in case of emtpy input? I

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

2016-02-17 Thread Robert Haas
On Tue, Feb 16, 2016 at 2:59 AM, Tom Lane wrote: > Andres Freund writes: >> I wonder if we shouldn't just expose a 'which pid is process X waiting >> for' API, implemented serverside. That's generally really useful, and >> looks like it's actually going to be less complicated than that >> query..

Re: [HACKERS] Fuzzy substring searching with the pg_trgm extension

2016-02-17 Thread Artur Zakirov
On 12.02.2016 20:56, Teodor Sigaev wrote: On Thu, Feb 11, 2016 at 9:56 AM, Teodor Sigaev wrote: 1 - sml_limit to similarity_limit. sml_threshold is difficult to write I think, similarity_limit is more simple. It seems to me that threshold is right word by meaning. sml_threshold is my choice.

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-02-17 Thread Pavel Stehule
Hi 2016-02-17 7:34 GMT+01:00 Catalin Iacob : > After I unzip the patch it doesn't apply. > patch says: > patch: Only garbage was found in the patch input. > > It's a combined diff, the git-diff manual says this about it: > Chunk header format is modified to prevent people from accidentally >

Re: [HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-17 Thread Tom Lane
Robert Haas writes: > I guess it's worth pointing out that storing the groupLeader in the > PROCLOCK, as we do currently, avoids this issue entirely. Right, that's why I framed it as "can we salvage this simplification" (referring to removal of that field). It may be best to just fix the existin

Re: [HACKERS] [PROPOSAL] Improvements of Hunspell dictionaries support

2016-02-17 Thread Artur Zakirov
On 16.02.2016 18:14, Artur Zakirov wrote: I attached a new version of the patch. Sorry for noise. I attached new version of the patch. I saw mistakes in DecodeFlag(). This patch fix them. -- Artur Zakirov Postgres Professional: http://www.postgrespro.com Russian Postgres Company *** a/doc/s

[HACKERS] Prepared Statement support for Parallel query

2016-02-17 Thread Amit Kapila
Commit d1b7c1ffe72e86932b5395f29e006c3f503bc53d has added the support for passing bind parameters to parallel workers, however prepared statement which uses bind parameters wasn't enabled for parallel query as the 'Execute' message in FE-BE protocol can pass the row_count which can make parallel pl

Re: [HACKERS] proposal: function parse_ident

2016-02-17 Thread Teodor Sigaev
I missed some of my edits. Updated patch with those in place attached. I'm sorry, but special chararacter still isn't escaped correctly in error messages: % select parse_ident(E'X\rXX'); XX""X T

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Craig Ringer
On 17 February 2016 at 18:39, Konstantin Knizhnik wrote: > Ok, what about the following plan: > > 1. Support custom WAL records (as far as I know 2ndQuadrant has such > patch). > 2. Add one more function to logical decoding allowing to deal with custom > records. > > So the idea is that we someho

Re: [HACKERS] [REVIEW] In-core regression tests for replication, cascading, archiving, PITR, etc.

2016-02-17 Thread Michael Paquier
On Fri, Feb 5, 2016 at 4:17 AM, Michael Paquier wrote: > Thanks for your enthusiasm. Now, to do an auto-critic of my patch: > + if ($params{allows_streaming}) > + { > + print $conf "wal_level = hot_standby\n"; > + print $conf "max_wal_senders = 5\n"; > +

Re: [HACKERS] commitfest application doesn't see new patch

2016-02-17 Thread Pavel Stehule
2016-02-17 9:24 GMT+01:00 Magnus Hagander : > On Wed, Feb 17, 2016 at 6:08 AM, Pavel Stehule > wrote: > >> Hi >> >> In a entry https://commitfest.postgresql.org/9/525/ the commitfest >> application show last patch >> >> Latest attachment (incompat.py) at 2016-02-04 09:13:55 from Catalin Iacob >>

[HACKERS] Pushing down sorted joins

2016-02-17 Thread Ashutosh Bapat
Hi All, Now that we have join pushdown support in postgres_fdw, we can leverage the sort pushdown mechanism for base relations to work for pushed down joins as well. PFA patch for the same. The code to find useful set of pathkeys and then generate paths for each list of pathkeys is moved into a fu

Re: [HACKERS] tsearch_extras extension

2016-02-17 Thread Stas Kelvich
> On 17 Feb 2016, at 08:14, Oleg Bartunov wrote: > > > > On Wed, Feb 17, 2016 at 6:57 AM, Tim Abbott wrote: > Just following up here since I haven't gotten a reply -- I'd love to work > with someone from the Postgres community on a plan to make the tsearch_extras > functionality available a

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Konstantin Knizhnik
Ok, what about the following plan: 1. Support custom WAL records (as far as I know 2ndQuadrant has such patch). 2. Add one more function to logical decoding allowing to deal with custom records. So the idea is that we somehow record DDL in WAL (for example using executor hook), then them are

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2016-02-17 Thread Michael Paquier
On Wed, Feb 17, 2016 at 10:13 AM, Michael Paquier wrote: > On Tue, Feb 16, 2016 at 11:36 PM, Joe Conway wrote: >> Thanks! > > OK. I think I'm good now. Thanks for the quick update. Actually, having second-thoughts on the matter, why is is that necessary to document the function pg_config()? The

Re: [HACKERS] ALTER ROLE SET/RESET for multiple options

2016-02-17 Thread Masahiko Sawada
On Wed, Feb 17, 2016 at 7:14 PM, Robert Haas wrote: > On Wed, Feb 17, 2016 at 3:22 PM, Masahiko Sawada > wrote: >> On Sat, Feb 13, 2016 at 2:45 PM, Robert Haas wrote: >>> On Fri, Feb 12, 2016 at 1:35 PM, Payal Singh wrote: The feature seems to work as described, but is it necessary to enc

Re: [HACKERS] Figures in docs

2016-02-17 Thread Ioseph Kim
On 수, 2016-02-17 at 12:14 +0300, Alexander Lakhin wrote: > 17.02.2016 09:17, Tatsuo Ishii wrote: > >> Hi. > >> > >> In DocBook 4.2 sgml dtd, figure tag is supported already. > >> that was implemented for multi output format. > > Ok, there's no technical problems with figures then. MySQL docs has

Re: [HACKERS] ALTER ROLE SET/RESET for multiple options

2016-02-17 Thread Robert Haas
On Wed, Feb 17, 2016 at 3:22 PM, Masahiko Sawada wrote: > On Sat, Feb 13, 2016 at 2:45 PM, Robert Haas wrote: >> On Fri, Feb 12, 2016 at 1:35 PM, Payal Singh wrote: >>> The feature seems to work as described, but is it necessary to enclose >>> multiple GUC settings in a parenthesis? This seems

Re: [HACKERS] Re: Reusing abbreviated keys during second pass of ordered [set] aggregates

2016-02-17 Thread Robert Haas
On Wed, Dec 23, 2015 at 12:19 PM, Peter Geoghegan wrote: > On Tue, Dec 22, 2015 at 2:59 PM, Robert Haas wrote: >> OK, so I've gone ahead and committed and back-patched that. Can you >> please rebase and repost the remainder as a 9.6 proposal? > > I attach a rebased patch for 9.6 only. Committed

Re: [HACKERS] custom function for converting human readable sizes to bytes

2016-02-17 Thread Dean Rasheed
On 17 February 2016 at 00:39, Vitaly Burovoy wrote: > On 2/16/16, Vitaly Burovoy wrote: >> On 2/16/16, Dean Rasheed wrote: >>> Fixing that in parse_memory_unit() would be messy because it assumes a >>> base unit of kB, so it would require a negative multiplier, and >>> pg_size_bytes() would have

[HACKERS] Re: Should PostgresFDW ImportForeignSchema should import the remote table default expression?

2016-02-17 Thread Rushabh Lathia
Oh I just found out that IMPORT FOREIGN do have import_default and import_not_null options. Got the answer, sorry for noise. On Wed, Feb 17, 2016 at 1:31 PM, Rushabh Lathia wrote: > Here is the test: > > -- create database > postgres=# create database foo; > CREATE DATABASE > postgres=# \c foo

Re: [HACKERS] ALTER ROLE SET/RESET for multiple options

2016-02-17 Thread Masahiko Sawada
On Sat, Feb 13, 2016 at 2:45 PM, Robert Haas wrote: > On Fri, Feb 12, 2016 at 1:35 PM, Payal Singh wrote: >> The feature seems to work as described, but is it necessary to enclose >> multiple GUC settings in a parenthesis? This seems a deviation from the >> usual syntax of altering multiple set

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Craig Ringer
On 17 February 2016 at 16:27, Andres Freund wrote: > On 2016-02-17 09:33:56 +0800, Craig Ringer wrote: > > Some DDL operations don't translate well to a series of replicatable > > actions. The case I hit the most is > > > >ALTER TABLE mytable ADD COLUMN somecolumn sometype NOT NULL DEFAULT >

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Craig Ringer
On 17 February 2016 at 16:24, Konstantin Knizhnik wrote: > Thanks for your explanation. I have to agree with your arguments that in > general case replication of DDL statement using logical decoding seems to > be problematic. But we are mostly considering logical decoding in quite > limited cont

Re: [HACKERS] Identifying a message in emit_log_hook.

2016-02-17 Thread Simon Riggs
On 17 February 2016 at 08:34, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > > > I'm guessing this would require making the pre-translated error text > > available to plugins as well as translated form. > > If I understand you correctly, edata->messgage_id in my patch is > just what

Re: [HACKERS] Identifying a message in emit_log_hook.

2016-02-17 Thread Kyotaro HORIGUCHI
Hello, thank you for the opinions, At Tue, 16 Feb 2016 10:57:33 +0100, Pavel Stehule wrote in > Hi > > 2016-02-16 10:47 GMT+01:00 Kyotaro HORIGUCHI < > horiguchi.kyot...@lab.ntt.co.jp>: > > > Hello. > > > > I'm planning to change the error level of a specific error > > message. We can use em

Re: [HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-17 Thread Andres Freund
On 2016-02-16 23:33:47 -0500, Tom Lane wrote: > Yes, exactly. I'm not certain if there are any real platforms where > a pointer-sized write wouldn't be atomic (it sure sounds inefficient > for that to be true), but we have not assumed that to date and I'd > just as soon not start here. FWIW, ther

Re: [HACKERS] PostgreSQL Audit Extension

2016-02-17 Thread Robert Haas
On Wed, Feb 17, 2016 at 5:20 AM, Bruce Momjian wrote: > On Fri, Feb 5, 2016 at 01:16:25PM -0500, Stephen Frost wrote: >> Looking at pgaudit and the other approaches to auditing which have been >> developed (eg: applications which sit in front of PG and essentially >> have to reimplement large bit

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Andres Freund
On 2016-02-17 09:33:56 +0800, Craig Ringer wrote: > Some DDL operations don't translate well to a series of replicatable > actions. The case I hit the most is > >ALTER TABLE mytable ADD COLUMN somecolumn sometype NOT NULL DEFAULT > some_function(); > > This is executed (simplified) by taking

Re: [HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-17 Thread Robert Haas
On Wed, Feb 17, 2016 at 10:03 AM, Tom Lane wrote: > Robert Haas writes: >> Hmm, that's true. I don't think it actually matters all that much, >> because proclock->tag.myProc->lockGroupLeader == NULL has pretty much >> the same effect as if proclock->tag.myProc->lockGroupLeader == >> proclock->ta

Re: [HACKERS] pglogical - logical replication contrib module

2016-02-17 Thread Konstantin Knizhnik
Hi Craig, Thanks for your explanation. I have to agree with your arguments that in general case replication of DDL statement using logical decoding seems to be problematic. But we are mostly considering logical decoding in quite limited context: replication between two identical Postgres data

Re: [HACKERS] commitfest application doesn't see new patch

2016-02-17 Thread Magnus Hagander
On Wed, Feb 17, 2016 at 6:08 AM, Pavel Stehule wrote: > Hi > > In a entry https://commitfest.postgresql.org/9/525/ the commitfest > application show last patch > > Latest attachment (incompat.py) at 2016-02-04 09:13:55 from Catalin Iacob > > > although last patch is from yesterday > > Attachment

Re: [HACKERS] extend pgbench expressions with functions

2016-02-17 Thread Fabien COELHO
Hello Michaël, \set aid 1 + 1 pgbench -f addition.sql -t 5000 I have the following: HEAD: 3.5~3.7M TPS list method: 3.6~3.7M TPS array method: 3.4~3.5M TPS So all approaches have a comparable performance. Yep, the execution trace is pretty similar in all cases, maybe with a little more

[HACKERS] Should PostgresFDW ImportForeignSchema should import the remote table default expression?

2016-02-17 Thread Rushabh Lathia
Here is the test: -- create database postgres=# create database foo; CREATE DATABASE postgres=# \c foo You are now connected to database "foo" as user "rushabh". -- Create remote table with default expression foo=# create table test ( a int , b int default 200 ); CREATE TABLE foo=# \c postgres You