Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

2016-02-13 Thread Simon Riggs
On 10 February 2016 at 16:36, Tom Lane wrote: > Robert Haas writes: > > On Wed, Feb 10, 2016 at 11:08 AM, Heikki Linnakangas > wrote: > >> (Sorry if this was discussed already, I haven't been paying attention) > >> > >> LWLockAssign()

Re: [HACKERS] Way to check whether a particular block is on the shared_buffer?

2016-02-13 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Saturday, February 13, 2016 1:46 PM > To: Kaigai Kouhei(海外 浩平) > Cc: Jim Nasby; pgsql-hackers@postgresql.org; Amit Langote > Subject: Re:

Re: [HACKERS] WIP: SCRAM authentication

2016-02-13 Thread Michael Paquier
On Sat, Feb 13, 2016 at 3:05 AM, David Steele wrote: > On 11/16/15 8:53 AM, Michael Paquier wrote: >> On Sat, Sep 5, 2015 at 9:31 AM, Bruce Momjian wrote: >>> On Fri, Sep 4, 2015 at 04:51:33PM -0400, Stephen Frost wrote: > Coming in late, but can you explain how multiple

Re: [HACKERS] Small PATCH: check of 2 Perl modules

2016-02-13 Thread Michael Paquier
On Sat, Feb 13, 2016 at 1:47 PM, Robert Haas wrote: > On Fri, Feb 12, 2016 at 8:20 AM, Eugene Kazakov > wrote: >> TAP-tests need two Perl modules: Test::More and IPC::Run. >> >> The patch adds checking of modules in configure.in and configure. > >

[HACKERS] Defaults for replication/backup

2016-02-13 Thread Magnus Hagander
I know we've had many discussions about the defaults, so hey, let's bring out the paint-cans and do the bikeshed all over again. I would suggest a couple of changes to the default values in parameters related to backup and replication. The reason for this is to make it "easier to do the right

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Michael Paquier
On Sat, Feb 13, 2016 at 4:37 PM, Fabien COELHO wrote: > >> For example, I just realized that this patch allows values to be >> either a double or an integer and extends the operators to handle >> double values. But variables can still only be integers. > > Indeed. That's

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Michael Paquier
On Sat, Feb 13, 2016 at 10:15 PM, Magnus Hagander wrote: > So, I suggest the following changes to the defaults: > wal_level=hot_standby > max_wal_senders=10 > max_replication_slots=10 10 seems a bit high. I would think that max_wal_senders and max_replication_slots set at 3 are sufficient enough,

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Andres Freund
On 2016-02-13 22:37:33 +0900, Michael Paquier wrote: > On Sat, Feb 13, 2016 at 10:15 PM, Magnus Hagander wrote: > > So, I suggest the following changes to the defaults: > > wal_level=hot_standby > > max_wal_senders=10 > > max_replication_slots=10 +1. I'm inclined to set slots a bit higher than

[HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

2016-02-13 Thread Greg Stark
The tests worked fine on faster build animals, right? And the clobber animals are much much slower So it seems perfectly sensible that their deadlock timeout would just have to be much much higher to have the same behaviour. I see nothing wrong in just setting deadlock timeout to a minute or

Re: [HACKERS] GinPageIs* don't actually return a boolean

2016-02-13 Thread Michael Paquier
On Sat, Feb 13, 2016 at 1:48 AM, Tom Lane wrote: > Andres Freund writes: >> On February 12, 2016 5:29:44 PM GMT+01:00, Tom Lane >> wrote: >>> We should standardize on the "((var & FLAG) != 0)" >>> pattern, which works reliably in all

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Magnus Hagander
On Sat, Feb 13, 2016 at 2:39 PM, Andres Freund wrote: > On 2016-02-13 22:37:33 +0900, Michael Paquier wrote: > > On Sat, Feb 13, 2016 at 10:15 PM, Magnus Hagander wrote: > > > So, I suggest the following changes to the defaults: > > > wal_level=hot_standby > > >

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Joshua D. Drake
On 02/13/2016 05:37 AM, Michael Paquier wrote: On Sat, Feb 13, 2016 at 10:15 PM, Magnus Hagander wrote: So, I suggest the following changes to the defaults: wal_level=hot_standby max_wal_senders=10 max_replication_slots=10 10 seems a bit high. I would think that max_wal_senders and

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Fabien COELHO
Hello Michaël, - if the function & double stuff are separated ? - for the double part, if variables can be double ? I just double-checked and could not see a clear use case mentioned in this thread for double return types, Alas there is one: non uniform random functions which use a

Re: [HACKERS] [COMMITTERS] pgsql: Code cleanup in the wake of recent LWLock refactoring.

2016-02-13 Thread Tom Lane
Simon Riggs writes: > On 10 February 2016 at 16:36, Tom Lane wrote: >> FWIW, I wasn't paying attention either, but I'm convinced by Robert's >> argument. Avoiding coupling between extensions is worth an API break. > Old APIs - why can't we keep it?

Re: [HACKERS] Crash with old Windows on new CPU

2016-02-13 Thread Tom Lane
Christian Ullrich writes: > * Robert Haas wrote: >> Thanks for the report and patch. Regrettably I haven't the Windows >> knowledge to have any idea whether it's right or wrong, but hopefully >> someone who knows Windows will jump in here. > In commitfest now. FWIW, I'm a

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Joshua D. Drake
Hello, I would like to add the idea of having archiving on by default. Not everyone uses streaming replication, some people use PITR. The one that I see is archive_command and I am not sure how to deal with that. Sincerely, JD -- Command Prompt, Inc.

Re: [HACKERS] Re: [COMMITTERS] pgsql: Add some isolation tests for deadlock detection and resolution.

2016-02-13 Thread Tom Lane
Greg Stark writes: > The tests worked fine on faster build animals, right? And the clobber > animals are much much slower So it seems perfectly sensible that their > deadlock timeout would just have to be much much higher to have the same > behaviour. I see nothing wrong in

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Andres Freund
Hi, On 2016-02-13 07:13:55 -0800, Joshua D. Drake wrote: > I would like to add the idea of having archiving on by default. Not everyone > uses streaming replication, some people use PITR. The one that I see is > archive_command and I am not sure how to deal with that. Since that requires

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Magnus Hagander
On Sat, Feb 13, 2016 at 4:16 PM, Andres Freund wrote: > Hi, > > On 2016-02-13 07:13:55 -0800, Joshua D. Drake wrote: > > I would like to add the idea of having archiving on by default. Not > everyone > > uses streaming replication, some people use PITR. The one that I see is

Re: [HACKERS] pg_basebackup vs WAL fetching

2016-02-13 Thread Craig Ringer
On 12 February 2016 at 22:37, Magnus Hagander wrote: > Right now, pg_basebackup can only use replication slots with WAL > streaming. It's intended, I believe, to be used when you use pg_basebackup > to set up a new replica, to close the gap before starting a replica (ref >

Re: [HACKERS] proposal: PL/Pythonu - function ereport

2016-02-13 Thread Pavel Stehule
Hi I am sending new version. Current version does: 1. the plpy utility functions can use all ErrorData fields, 2. there are no new functions, 3. via GUC plpythonu.legacy_custom_exception we can return previous behave, 4. only exception Error is raised with natural structure - no composite value

Re: [HACKERS] pg_basebackup vs WAL fetching

2016-02-13 Thread Andres Freund
On 2016-02-13 23:25:06 +0800, Craig Ringer wrote: > Internally replication slots have an ephemeral form, where the slot is > dropped automatically on crash recovery. And upon release (including a http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Fabien COELHO
But variables can still only be integers. Version 28 attached has double variables, although typing is based on guessing because it is just a string. Any comment? [...] two separate enhancements in here. One of them is to support a new data type (doubles) and the other is to support

Re: [HACKERS] Crash with old Windows on new CPU

2016-02-13 Thread Christian Ullrich
On February 13, 2016 4:10:34 PM Tom Lane wrote: > Christian Ullrich writes: >> * Robert Haas wrote: >>> Thanks for the report and patch. Regrettably I haven't the Windows >>> knowledge to have any idea whether it's right or wrong, but hopefully >>>

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Tom Lane
Magnus Hagander writes: > Yes, these changes will increase some of the default overhead. My argument > against that is that anybody who actually cares about that overhead is > going to be tuning their database *anyway*, so they can just change things > back to the old

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Magnus Hagander
On Sat, Feb 13, 2016 at 4:52 PM, Tom Lane wrote: > Magnus Hagander writes: > > Yes, these changes will increase some of the default overhead. My > argument > > against that is that anybody who actually cares about that overhead is > > going to be tuning

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Tom Lane
Magnus Hagander writes: > On Sat, Feb 13, 2016 at 4:52 PM, Tom Lane wrote: >> It would be easier to sell this if we had some numbers for the amount of >> overhead it would add for people *not* using the features. I do not think >> I've ever seen, eg,

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Andres Freund
On 2016-02-13 11:10:58 -0500, Tom Lane wrote: > Magnus Hagander writes: > > The big thing is, IIRC, that we turn off the optimizations in > > min_wal_level. *most* people will see no impact of their regular > > application runtime from that, but it might definitely have an

Re: [HACKERS] Crash with old Windows on new CPU

2016-02-13 Thread Yury Zhuravlev
Tom Lane wrote: Lastly, I'd like to see some discussion of what side effects "_set_FMA3_enable(0);" has ... I rather doubt that it's really a magic-elixir-against-crashes-with-no-downsides. That would give us some context to estimate the risks of this code executing when it's not really needed.

Re: [HACKERS] Crash with old Windows on new CPU

2016-02-13 Thread Christian Ullrich
* From: Christian Ullrich > On February 13, 2016 4:10:34 PM Tom Lane wrote: > > > Christian Ullrich writes: > > Lastly, I'd like to see some discussion of what side effects > > "_set_FMA3_enable(0);" has ... I rather doubt that it's really > > a

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Michael Paquier
On Sun, Feb 14, 2016 at 12:37 AM, Fabien COELHO wrote: > The two features are highly intermix, so it can only be dependent patches, > first to add a function infrastructure and probably some support for doubles > altough it would not be used, then to add doubles & their functions. > > A real pain

[HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-13 Thread Robert Haas
On Sat, Feb 13, 2016 at 1:32 PM, Tom Lane wrote: > Robert Haas writes: >> Introduce group locking to prevent parallel processes from deadlocking. > > I'm fairly unhappy about this patch, because it has introduced a large > amount of new complexity into

Re: [HACKERS] Bug in StartupSUBTRANS

2016-02-13 Thread Jeff Janes
On Tue, Feb 9, 2016 at 10:33 AM, Simon Riggs wrote: > On 9 February 2016 at 18:42, Jeff Janes wrote: >> >> While testing the crash resilience of the recent 2-part-commit >> improvements, I've run into a problem where sometimes after a crash >> the

Re: [HACKERS] Defaults for replication/backup

2016-02-13 Thread Robert Haas
On Sat, Feb 13, 2016 at 11:31 AM, Andres Freund wrote: > On 2016-02-13 11:10:58 -0500, Tom Lane wrote: >> Magnus Hagander writes: >> > The big thing is, IIRC, that we turn off the optimizations in >> > min_wal_level. *most* people will see no impact of

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Robert Haas
On Sat, Feb 13, 2016 at 6:19 PM, Michael Paquier wrote: > On Sun, Feb 14, 2016 at 12:37 AM, Fabien COELHO wrote: >> The two features are highly intermix, so it can only be dependent patches, >> first to add a function infrastructure and probably some support for doubles

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Robert Haas
On Sat, Feb 13, 2016 at 10:37 AM, Fabien COELHO wrote: > A real pain is the documentation, because it means writing a documentation > with only integer functions, then overwriting it with doubles. This is dumb > work, really, for the sake of "a cleaner git history", the

Re: [HACKERS] Way to check whether a particular block is on the shared_buffer?

2016-02-13 Thread Robert Haas
On Sat, Feb 13, 2016 at 7:29 AM, Kouhei Kaigai wrote: >> I suppose there's no theoretical reason why the buffer couldn't go >> from all-visible to not-all-visible and back to all-visible again all >> during the time you are copying it. >> > The backend process that is

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Fabien COELHO
Hello Robert, You know, you make comments like this pretty much every time anybody suggests that you should change anything in any patch you write. Well, not everytime... I'm tired of doing and undoing things: I do a limited A because I'm cautious not to spend too much time that would go

Re: [HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-13 Thread Craig Ringer
On 14 February 2016 at 08:05, Robert Haas wrote: > First, the overall concept here is that processes can either be a > member of a lock group or a member of no lock group. The concept of a > lock group is formally separate from the concept of a parallel group > created

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Fabien COELHO
So I would be fine to do a portion of the legwork and extract from this patch something smaller that adds only functions as a first step, with the minimum set of functions I mentioned upthread. Robert, Alvaro, Fabien, does that sound fine to you? Thanks, but this is my c*, I have a few

Re: [HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-13 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 14 February 2016 at 08:05, Robert Haas wrote: > > First, the overall concept here is that processes can either be a > > member of a lock group or a member of no lock group. The concept of a > > lock group is formally

Re: [HACKERS] Crash with old Windows on new CPU

2016-02-13 Thread Craig Ringer
On 13 February 2016 at 23:45, Christian Ullrich wrote: > > > Maybe that's actually > > what's needed, but it sure looks fishy. And what connection does the > > build toolchain version have to the runtime environment anyway? > > The CRT version is tied to the compiler

Re: [HACKERS] proposal: function parse_ident

2016-02-13 Thread Pavel Stehule
Hi Jim 2016-02-11 8:27 GMT+01:00 Pavel Stehule > > > ok > >> >> Also added test for invalid characters. >> >> I think "strict" would be more in line with other uses in code. There are >> currently no other occurrences of 'strictmode' in the code. There are loads >> of

Re: [HACKERS] extend pgbench expressions with functions

2016-02-13 Thread Michael Paquier
On Sun, Feb 14, 2016 at 10:05 AM, Robert Haas wrote: > On Sat, Feb 13, 2016 at 6:19 PM, Michael Paquier > wrote: >> On Sun, Feb 14, 2016 at 12:37 AM, Fabien COELHO wrote: >>> The two features are highly intermix, so it can only be dependent

Re: [HACKERS] Re: [COMMITTERS] pgsql: Introduce group locking to prevent parallel processes from deadl

2016-02-13 Thread Robert Haas
On Sat, Feb 13, 2016 at 9:20 PM, Craig Ringer wrote: > The case that comes to mind for me is in logical decoding, for decoding > prepared xacts. Being able to make the prepared xact a member of a "lock > group" along with the decoding session's xact may provide a solution