Re: [HACKERS] increasing the default WAL segment size

2017-04-07 Thread Beena Emerson
I ran tests and following are the details: Machine details: Architecture: ppc64le Byte Order:Little Endian CPU(s):192 On-line CPU(s) list: 0-191 Thread(s) per core:8 Core(s) per socket:1 Socket(s): 24 NUMA node(s): 4 Model:

[HACKERS] Comment typo in publicationcmd.c

2017-04-07 Thread Masahiko Sawada
Hi all, Attached fixes a typo in publicationcmd.c file. s/om/on/ Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center fix_typo_in_publicationcmds_c.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 08:21 AM, Noah Misch wrote: On Thu, Apr 06, 2017 at 09:46:29PM +0300, Heikki Linnakangas wrote: On 04/06/2017 08:36 AM, Noah Misch wrote: On Tue, Mar 07, 2017 at 02:36:13PM +0200, Heikki Linnakangas wrote: I didn't include the last-minute changes to the way you specify this in pg

Re: [HACKERS] Comment typo in publicationcmd.c

2017-04-07 Thread Magnus Hagander
On Fri, Apr 7, 2017 at 9:00 AM, Masahiko Sawada wrote: > Hi all, > > Attached fixes a typo in publicationcmd.c file. > > s/om/on/ > Applied, thanks. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Magnus Hagander
Jumping late into this one, apologies if these opinions have already been up and discarded. On Fri, Apr 7, 2017 at 9:28 AM, Heikki Linnakangas wrote: > On 04/07/2017 08:21 AM, Noah Misch wrote: > >> On Thu, Apr 06, 2017 at 09:46:29PM +0300, Heikki Linnakangas wrote: >> >>> On 04/06/2017 08:36 AM

Re: [HACKERS] Compiler warning in costsize.c

2017-04-07 Thread Michael Paquier
On Tue, Apr 4, 2017 at 9:42 PM, Michael Paquier wrote: > On Wed, Apr 5, 2017 at 2:54 AM, Tom Lane wrote: >> (I'm personally not that much in love with PG_USED_FOR_ASSERTS_ONLY, >> because it tends to confuse pgindent.) > > I would be incline to just do that, any other solution I can think of > is

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Fabien COELHO
Hello Corey, \if defined varname \if sql boolean expression to send to server \if compare value operator value I'm still thinking:-) Independently of the my aethetical complaint against having a pretty unusual keyword prefix syntax, how would you envision a \set assig

Re: [HACKERS] pgbench - allow to store select results into variables

2017-04-07 Thread Fabien COELHO
Hello Tatsuo, Ok, I will move the patch to the next cf. Done. If I understand correctly, the patch is moved because of the unrelated issue that variables cannot be utf8 in pgbench, and it is a condition to consider this patch that existing pgbench variables (set with \set) can be utf8?

Re: [HACKERS] Compiler warning in costsize.c

2017-04-07 Thread Michael Paquier
On Fri, Apr 7, 2017 at 12:38 AM, Michael Paquier wrote: > On Tue, Apr 4, 2017 at 9:42 PM, Michael Paquier > wrote: >> On Wed, Apr 5, 2017 at 2:54 AM, Tom Lane wrote: >>> (I'm personally not that much in love with PG_USED_FOR_ASSERTS_ONLY, >>> because it tends to confuse pgindent.) >> >> I would

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 10:38 AM, Magnus Hagander wrote: So here's a wild idea. What if we just call it "sha256"? Does the user actually care about it being scram, or is scram just an implementation detail for them? That way when the next one shows up, it'll be sha512 or whatever. It happens to use scram u

Re: [HACKERS] pgbench - allow to store select results into variables

2017-04-07 Thread Tatsuo Ishii
> If I understand correctly, the patch is moved because of the unrelated > issue that variables cannot be utf8 in pgbench, and it is a condition > to consider this patch that existing pgbench variables (set with \set) > can be utf8? I'm not sure if it is "unrelated" because the new feature relies

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-07 Thread Heikki Linnakangas
On 04/06/2017 11:16 PM, Simon Riggs wrote: or it can just ignore the list and send what it wants anyway, probably leading to client disconnect. It would need to follow one of the requested protocols, but mark the request as doomed. Otherwise we'd be revealing information. That's what SCRAM does

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-07 Thread Heikki Linnakangas
On 04/06/2017 11:05 PM, Tom Lane wrote: Perhaps we could turn this around: have the client send (in the connection request packet) a list of auth protocols it thinks it is able to handle. (I'm envisioning this as being more or less fixed for any one version of any one client, since it would basic

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-07 Thread Craig Ringer
On 7 April 2017 at 16:33, Heikki Linnakangas wrote: > That list of supported authentication methods would need to be included in > the startup message. Unfortunately, there is no way to add options to the > startup message, without breaking compatibility with old servers. If there > is an option

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Craig Ringer
On 7 April 2017 at 15:59, Heikki Linnakangas wrote: > On 04/07/2017 10:38 AM, Magnus Hagander wrote: >> Not sure if it would be doable in the code, but we could also have: >> host all all example.com scram method=sha256plus,sha256 >> >> or something like that. Which would fit within the current s

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Magnus Hagander
On Fri, Apr 7, 2017 at 9:59 AM, Heikki Linnakangas wrote: > On 04/07/2017 10:38 AM, Magnus Hagander wrote: > >> So here's a wild idea. What if we just call it "sha256"? Does the user >> actually care about it being scram, or is scram just an implementation >> detail for them? That way when the ne

Re: [HACKERS] Proposal : For Auto-Prewarm.

2017-04-07 Thread Mithun Cy
On Thu, Apr 6, 2017 at 4:12 AM, Andres Freund wrote: > On 2017-03-13 18:45:00 +0530, Mithun Cy wrote: >> I have implemented a similar logic now. The prewarm bgworker will >> launch a sub-worker per database in the dump file. And, each >> sub-worker will load its database block info. The sub-worker

Re: [HACKERS] Letting the client choose the protocol to use during a SASL exchange

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 11:57 AM, Craig Ringer wrote: On 7 April 2017 at 16:33, Heikki Linnakangas wrote: That list of supported authentication methods would need to be included in the startup message. Unfortunately, there is no way to add options to the startup message, without breaking compatibility w

Re: [HACKERS] pgbench - allow to store select results into variables

2017-04-07 Thread Fabien COELHO
If I understand correctly, the patch is moved because of the unrelated issue that variables cannot be utf8 in pgbench, and it is a condition to consider this patch that existing pgbench variables (set with \set) can be utf8? I'm not sure if it is "unrelated" because the new feature relies on e

[HACKERS] ExecPrepareExprList and per-query context

2017-04-07 Thread Amit Langote
As of b8d7f053c5c, ExecPrepareExprList is (must be?) used instead of ExecPrepareExpr when the caller wants to initialize expressions in a list, for example, FormIndexDatum. ExecPrepareExpr doesn't require the caller to have switched to per-query context, because it itself will. Same is not howeve

Re: [HACKERS] Declarative partitioning - another take

2017-04-07 Thread Etsuro Fujita
On 2016/12/14 16:20, Etsuro Fujita wrote: On 2016/12/09 19:46, Maksim Milyutin wrote: I would like to work on two tasks: - insert (and eventually update) tuple routing for foreign partition. - the ability to create an index on the parent and have all of the children inherit it; The first one

[HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Prakash Itnal
Hello, We currently use psotgres 9.3 in our products. Recently we upgraded to postgres 9.6. But with 9.6 we have seen a drastic reduction in throughput. After analyzing carefully I found that "planner time" in 9.6 is very high. Below are the details: Scenario: 1 Create a table with 10 rows. 2

[HACKERS] pg_export_snapshot doc

2017-04-07 Thread Tatsuo Ishii
pg_export_snapshot() cannot be used during recovery (i.e. on standby servers), but it's not documented. IMO this is a bug and should be fixed. Patch attached. Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese:http://www.sraoss.co.jp diff --git

Re: [HACKERS] Declarative partitioning - another take

2017-04-07 Thread Maksim Milyutin
On 07.04.2017 13:05, Etsuro Fujita wrote: On 2016/12/14 16:20, Etsuro Fujita wrote: On 2016/12/09 19:46, Maksim Milyutin wrote: I would like to work on two tasks: - insert (and eventually update) tuple routing for foreign partition. - the ability to create an index on the parent and have all

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-07 Thread Magnus Hagander
On Fri, Apr 7, 2017 at 6:29 AM, Tom Lane wrote: > Andres Freund writes: > > On 2017-04-07 13:07:59 +0900, Michael Paquier wrote: > >> On Fri, Apr 7, 2017 at 1:01 PM, Tom Lane wrote: > >>> Still, it's not very clear why we need to cater for building just libpq > >>> rather than the whole distrib

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Álvaro Hernández Tortosa
On 07/04/17 11:05, Magnus Hagander wrote: On Fri, Apr 7, 2017 at 9:59 AM, Heikki Linnakangas > wrote: On 04/07/2017 10:38 AM, Magnus Hagander wrote: So here's a wild idea. What if we just call it "sha256"? Does the user actually care about i

Re: [HACKERS] Supporting huge pages on Windows

2017-04-07 Thread Magnus Hagander
On Wed, Apr 5, 2017 at 9:15 AM, Tsunakawa, Takayuki < tsunakawa.ta...@jp.fujitsu.com> wrote: > From: pgsql-hackers-ow...@postgresql.org > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Andres Freund > > As I asked before, why can't we delete all privs and add the explicitly > > needed

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 05:30 AM, Michael Paquier wrote: On Fri, Apr 7, 2017 at 2:47 AM, Heikki Linnakangas wrote: On 04/06/2017 08:42 PM, Heikki Linnakangas wrote: There is for example this portion in the new tables: +static const Codepoint prohibited_output_chars[] = +{ + 0xD800, 0xF8FF,

[HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Yorick Peterse
The attached patch updates the hot-standby documentation (in the high availability section) so it explicitly mentions that certain settings need to be applied to servers in a particular order. For example, it states that if you increase a certain setting (e.g. max_connections) you need to do so on

Re: [HACKERS] Implementation of SASLprep for SCRAM-SHA-256

2017-04-07 Thread Michael Paquier
On Fri, Apr 7, 2017 at 8:58 PM, Heikki Linnakangas wrote: > On 04/07/2017 05:30 AM, Michael Paquier wrote: >> I am really wondering if this should not reflect the real range >> reported by the RFC. I understand that you have grouped things to save >> a couple of bytes, but that would protect from

Re: [HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Aleksander Alekseev
Hi Yorick, > The attached patch updates the hot-standby documentation (in the high > availability section) so it explicitly mentions that certain settings > need to be applied to servers in a particular order. For example, it > states that if you increase a certain setting (e.g. max_connections) >

Re: [HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Stephen Frost
Aleksander, Yorick, * Aleksander Alekseev (a.aleks...@postgrespro.ru) wrote: > > The attached patch updates the hot-standby documentation (in the high > > availability section) so it explicitly mentions that certain settings > > need to be applied to servers in a particular order. For example, it

Re: [HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Yorick Peterse
Ha! It seems I indeed had it the other way around. I suppose that's what happens when writing a patch late at night. Somewhat ironically I did have the other correct in my Git commit message. Attached is an updated version of the patch that corrects the order in the documentation. Yorick diff --g

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 02:00 PM, Magnus Hagander wrote: On Fri, Apr 7, 2017 at 6:29 AM, Tom Lane wrote: Yeah. For win32.mak, the key question is whether there is still anybody who'd have an insurmountable problem with building the whole distro via src/tools/msvc/ rather than just building libpq with w

Re: [HACKERS] postgres_fdw: support parameterized foreign joins

2017-04-07 Thread Arthur Zakirov
On 05.04.2017 12:20, Etsuro Fujita wrote: Rebased. Attached is an updated version created on top of the latest patch "epqpath-for-foreignjoin" [1]. Other changes: * Added a bit more regression tests with FOR UPDATE clause to see if CreateLocalJoinPath works well for parameterized foreign join

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-07 Thread Tom Lane
Heikki Linnakangas writes: > I just tested it. After adding all the missing files to the makefile, > I'm getting an error: >> .\Release\libpq.dll.manifest : general error c1010070: Failed to load and >> parse >> the manifest. The system cannot find the file specified. > This seems be the same

Re: [HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Aleksander Alekseev
Hi Yorick, > Attached is an updated version of the patch that corrects the order in > the documentation. Looks promising. I would recommend to add this patch to the next commitfest [1]. Otherwise there is a chance that it will be lost. [1] https://commitfest.postgresql.org/14/ -- Best regards,

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-07 Thread Andrew Dunstan
On 04/07/2017 09:58 AM, Tom Lane wrote: >> This seems be the same as the 2nd error that was reported back in 2013: >> https://www.postgresql.org/message-id/CAJ2%3DPVQcW8UGNnSy%3DOw%3DvUK2zpjowTkzUS1B864REa7LOT140Q%40mail.gmail.com. > Well, if it's been broken since (at least) 2013, and we've had

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-04-07 Thread Masahiko Sawada
On Wed, Mar 29, 2017 at 11:14 PM, Masahiko Sawada wrote: > On Wed, Mar 22, 2017 at 2:49 AM, Masahiko Sawada > wrote: >> On Thu, Mar 16, 2017 at 2:37 PM, Vinayak Pokale >> wrote: >>> The following review has been posted through the commitfest application: >>> make installcheck-world: tested, pa

Re: [HACKERS] Parallel Append implementation

2017-04-07 Thread Andres Freund
Hi, On 2017-04-07 11:44:39 +0530, Amit Khandekar wrote: > On 6 April 2017 at 07:33, Andres Freund wrote: > > On 2017-04-05 14:52:38 +0530, Amit Khandekar wrote: > >> This is what the earlier versions of my patch had done : just add up > >> per-subplan parallel_workers (1 for non-partial subplan a

Re: [HACKERS] Supporting huge pages on Windows

2017-04-07 Thread Andres Freund
On 2017-04-07 13:57:07 +0200, Magnus Hagander wrote: > On Wed, Apr 5, 2017 at 9:15 AM, Tsunakawa, Takayuki < > tsunakawa.ta...@jp.fujitsu.com> wrote: > > > From: pgsql-hackers-ow...@postgresql.org > > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Andres Freund > > > As I asked before,

[HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Aleksander Alekseev
Hi. Recently I've discovered that if there are multiple values of the same parameter in postgresql.conf PostgreSQL will silently use the last one. It looks like not the best approach to me. For instance, user can find the first value in the config file and expect that it will be used, etc. I sugg

Re: [HACKERS] src/interfaces/libpq shipping nmake-related Makefiles

2017-04-07 Thread Andres Freund
On 2017-04-07 13:00:39 +0200, Magnus Hagander wrote: > Insurmountable, probably not. The big difference is that you don't need > *any* dependencies to build a libpq using win32.mak, but you need many of > them (to start with, perl...) to build using the built-in one. For people > who want to build

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Andres Freund
On 2017-04-07 18:14:27 +0300, Aleksander Alekseev wrote: > Hi. > > Recently I've discovered that if there are multiple values of the same > parameter in postgresql.conf PostgreSQL will silently use the last one. > It looks like not the best approach to me. For instance, user can find > the first v

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Tatsuo Ishii
>> Recently I've discovered that if there are multiple values of the same >> parameter in postgresql.conf PostgreSQL will silently use the last one. >> It looks like not the best approach to me. For instance, user can find >> the first value in the config file and expect that it will be used, etc.

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Aleksander Alekseev
Andres, Tatsuo, Thank you for sharing your thoughts. > -1 - I frequently just override earlier parameters by adding an > include at the end of the file. Also, with postgresql.auto.conf it's > even more common to override parameters. > -1 from me too by the same reason Andres said. I see no pro

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Andres Freund
On 2017-04-07 18:29:40 +0300, Aleksander Alekseev wrote: > Andres, Tatsuo, > > Thank you for sharing your thoughts. > > > -1 - I frequently just override earlier parameters by adding an > > include at the end of the file. Also, with postgresql.auto.conf it's > > even more common to override para

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread David Steele
On 4/7/17 11:22 AM, Tatsuo Ishii wrote: >>> Recently I've discovered that if there are multiple values of the same >>> parameter in postgresql.conf PostgreSQL will silently use the last one. >>> It looks like not the best approach to me. For instance, user can find >>> the first value in the config

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread Euler Taveira
2017-04-07 12:14 GMT-03:00 Aleksander Alekseev : > Recently I've discovered that if there are multiple values of the same > parameter in postgresql.conf PostgreSQL will silently use the last one. > It looks like not the best approach to me. For instance, user can find > the first value in the conf

Re: [HACKERS] [PATCH] Warn users about duplicate configuration parameters

2017-04-07 Thread David G. Johnston
On Fri, Apr 7, 2017 at 8:29 AM, Aleksander Alekseev < a.aleks...@postgrespro.ru> wrote: > Andres, Tatsuo, > > Thank you for sharing your thoughts. > > > -1 - I frequently just override earlier parameters by adding an > > include at the end of the file. Also, with postgresql.auto.conf it's > > eve

Re: [HACKERS] [PATCH] Document the order of changing certain settings when using hot-standby servers

2017-04-07 Thread Yorick Peterse
Done! It can be found at https://commitfest.postgresql.org/14/1110/ Thanks for reviewing thus far :) Yorick -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_basebackup: Allow use of arbitrary compression program

2017-04-07 Thread Jeff Janes
On Thu, Apr 6, 2017 at 7:04 PM, Michael Harris wrote: > Hello, > > Back in pg 9.2, we hacked a copy of pg_basebackup to add a command > line option which would allow the user to specify an arbitrary > external program (potentially including arguments) to be used to > compress the tar backup. > >

Re: [HACKERS] Duplicate assignment in Unicode/convutils.pm

2017-04-07 Thread Heikki Linnakangas
On 04/07/2017 09:32 AM, Kyotaro HORIGUCHI wrote: Hi, I found that convutils.pl contains a harmless duplicate assignemnt. my $out = {f => $fname, l => $., code => hex($1), ucs => hex($2), comment => $4, direction => BOTH,

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Pavel Stehule
2017-04-07 9:52 GMT+02:00 Fabien COELHO : > > Hello Corey, > > \if defined varname \if sql boolean expression to send to server \if compare value operator value >>> >>> I'm still thinking:-) >>> >>> Independently of the my aethetical complaint against having

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Merlin Moncure
On Fri, Apr 7, 2017 at 5:16 AM, Prakash Itnal wrote: > Hello, > > We currently use psotgres 9.3 in our products. Recently we upgraded to > postgres 9.6. But with 9.6 we have seen a drastic reduction in throughput. > After analyzing carefully I found that "planner time" in 9.6 is very high. > Below

Re: [HACKERS] Compiler warning in costsize.c

2017-04-07 Thread Tom Lane
Michael Paquier writes: > Bah. This actually fixes nothing. Attached is a different patch that > really addresses the problem, by removing the variable because we > don't want planner_rt_fetch() to run for non-Assert builds. I don't really like any of these fixes, because they take the code furth

[HACKERS] valgrind errors around dsa.c

2017-04-07 Thread Andres Freund
Hi, newly added tests exercise parallel bitmap scans. And they trigger valgrind errors: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=skink&dt=2017-04-07%2007%3A10%3A01 ==4567== VALGRINDERROR-BEGIN ==4567== Conditional jump or move depends on uninitialised value(s) ==4567==at 0x5F

Re: [HACKERS] ExecPrepareExprList and per-query context

2017-04-07 Thread Tom Lane
Amit Langote writes: > Should ExecPrepareExprList also switch to estate->es_query_cxt? Good point; I'm surprised we haven't noted any failures from that. We surely want the entire result data structure to be in the same memory context. There are not very many callers right now, and I guess they

[HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andres Freund
Hi, There's two machines that recently report changes in deadlock detector output: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax&dt=2017-04-05%2018%3A58%3A04 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=friarbird&dt=2017-04-07%2004%3A20%3A01 both just failed twice in a

[HACKERS] pgbench --progress-timestamp no longer works correctly

2017-04-07 Thread Jeff Janes
--progress-timestamp is supposed to make -P report a Unix Epoch time stamp, for easy correlation with the entries in other log files (like the postgres server log file using %n). But that broke in this commit: commit 1d63f7d2d180c8708bc12710254eb7b45823440f Author: Tom Lane Date: Mon Jan 2 13:

Re: [HACKERS] SCRAM authentication, take three

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 3:59 AM, Heikki Linnakangas wrote: > I think the "SCRAM" part is more important than "SHA-256", so -1 on that. I agree. The point here isn't that we're using a better hashing method, even if a lot of people *think* that's the point. The point is we're using a modern algor

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andrew Dunstan
On 04/07/2017 12:57 PM, Andres Freund wrote: > Hi, > > There's two machines that recently report changes in deadlock detector > output: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=hyrax&dt=2017-04-05%2018%3A58%3A04 > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=friarbird&

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Tom Lane
Andrew Dunstan writes: > On 04/07/2017 12:57 PM, Andres Freund wrote: >> I don't think any recent changes are supposed to affect deadlock >> detector behaviour? > Both these machines have CLOBBER_CACHE_ALWAYS set. And on both machines > recent changes have made the isolation tests run much much l

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Tomas Vondra
On 04/07/2017 06:31 PM, Merlin Moncure wrote: On Fri, Apr 7, 2017 at 5:16 AM, Prakash Itnal wrote: Hello, We currently use psotgres 9.3 in our products. Recently we upgraded to postgres 9.6. But with 9.6 we have seen a drastic reduction in throughput. After analyzing carefully I found that "pl

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2017-04-07 Thread Robert Haas
On Thu, Mar 9, 2017 at 5:49 PM, Robert Haas wrote: > However, I just realized that in > both this case and in the case of group XID clearing, we weren't > advertising a wait event for the PGSemaphoreLock calls that are part > of the group locking machinery. I think we should fix that, because a >

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 12:28 PM, Tom Lane wrote: > Andrew Dunstan writes: >> On 04/07/2017 12:57 PM, Andres Freund wrote: >>> I don't think any recent changes are supposed to affect deadlock >>> detector behaviour? > >> Both these machines have CLOBBER_CACHE_ALWAYS set. And on both machines >> re

Re: [HACKERS] parallel bitmapscan isn't exercised in regression tests

2017-04-07 Thread Andres Freund
On 2017-04-06 13:43:55 -0700, Andres Freund wrote: > On 2017-04-06 10:00:32 +0530, Dilip Kumar wrote: > > On Tue, Apr 4, 2017 at 5:51 AM, Dilip Kumar wrote: > > > Sure I can do that, In attached patch, I only fixed the problem of not > > > executing the bitmap test. Now, I will add few cases to c

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Andres Freund
On 2017-04-07 12:49:22 -0500, Kevin Grittner wrote: > On Fri, Apr 7, 2017 at 12:28 PM, Tom Lane wrote: > > Andrew Dunstan writes: > >> On 04/07/2017 12:57 PM, Andres Freund wrote: > >>> I don't think any recent changes are supposed to affect deadlock > >>> detector behaviour? > > > >> Both these

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Tom Lane
Tomas Vondra writes: > On 04/07/2017 06:31 PM, Merlin Moncure wrote: >> I think your math is off. Looking at your attachments, planning time >> is 0.056ms, not 0.56ms. This is in no way relevant to performance on >> the order of your measured TPS. How are you measuring TPS? > Not sure where d

Re: [HACKERS] UPDATE of partition key

2017-04-07 Thread Robert Haas
On Wed, Apr 5, 2017 at 5:54 AM, Amit Langote wrote: > Marked as ready for committer. Andres seems to have changed the status of this patch to "Needs review" and then, 30 seconds later, to "Waiting on author", but there's no actual email on the thread explaining what his concerns were. I'm going

Re: [HACKERS] UPDATE of partition key

2017-04-07 Thread Andres Freund
On 2017-04-07 13:55:51 -0400, Robert Haas wrote: > On Wed, Apr 5, 2017 at 5:54 AM, Amit Langote > wrote: > > Marked as ready for committer. > > Andres seems to have changed the status of this patch to "Needs > review" and then, 30 seconds later, to "Waiting on author" > there's no actual email on

[HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Joe Conway
Apparently INSERT and SELECT on the parent partitioned table skip normal acl checks on the partitions. Is that intended behavior? 8<--- test=# create user part_test; CREATE ROLE test=# test=# create table t1 (id int) partition by range ((id % 4)); CREATE TABLE test=# create

Re: [HACKERS] [PATCH] Remove unused argument in btree_xlog_split

2017-04-07 Thread Robert Haas
On Thu, Apr 6, 2017 at 8:21 AM, Aleksander Alekseev wrote: > Hi Robert, > >> Hmm. I don't see anything wrong with that, particularly, but it seems >> we also don't need the distinction between XLOG_BTREE_SPLIT_L and >> XLOG_BTREE_SPLIT_L_ROOT or likewise between XLOG_BTREE_SPLIT_R and >> XLOG_BTR

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Keith Fiske
On Fri, Apr 7, 2017 at 2:05 PM, Joe Conway wrote: > Apparently INSERT and SELECT on the parent partitioned table skip normal > acl checks on the partitions. Is that intended behavior? > > 8<--- > test=# create user part_test; > CREATE ROLE > test=# > test=# create table t1

Re: [HACKERS] postgres_fdw bug in 9.6

2017-04-07 Thread Robert Haas
On Mon, Apr 3, 2017 at 2:12 AM, Etsuro Fujita wrote: > On 2017/04/01 1:32, Jeff Janes wrote: >> On Thu, Mar 23, 2017 at 5:20 AM, Etsuro Fujita >> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >> Done. Attached is a new version of the patch. >> Is the fix for 9.6.3 going to be just a back port o

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: > I'd rather fix the issue, than remove the tests entirely. Seems quite > possible to handle blocking on Safesnapshot in a similar manner as > pg_blocking_pids? I'll see what I can figure out. -- Kevin Grittner -- Sent via pgsql-hacker

Re: [HACKERS] partitioned tables and contrib/sepgsql

2017-04-07 Thread Mike Palmiotto
On Thu, Apr 6, 2017 at 5:52 PM, Joe Conway wrote: > On 04/06/2017 12:35 PM, Tom Lane wrote: >> Joe Conway writes: >>> Any thoughts on whether 0001a and 0001b ought to be backpatched? I'm >>> thinking not given the lack of past complaints but it might make sense >>> to do. >> >> I think 0001a abso

[HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
Hi, When I started writing this, there were the following reamining CF items, minus bugfix ones which aren't bound by the code freeze. I think it makes sense to go through those and see whether it's realistic to commit any of them. Ready for Committer: Add GUCs for predicate lock promotion thre

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Kevin Grittner
On Fri, Apr 7, 2017 at 1:37 PM, Andres Freund wrote: > I think it makes sense to go through those and see whether it's > realistic to commit any of them. > > Ready for Committer: > > Add GUCs for predicate lock promotion thresholds: > - claimed by Kevin, should be easy enough I was planning on p

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Unique Joins > - Tom's discussing things with David, not sure. This one was already included-and-removed from 9.6, Tom had said he'd give it priority during the current cycle as I recall. It seems unfair that it's still waiting for review on the last day of pg10's last com

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Peter Geoghegan
On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: > Write Amplification Reduction Method (WARM) > - fair number of people don't think it's ready for v10. > - can't move to next fest because it's waiting-on-author, which doesn't > allow that. Doesn't strike me as a useful restriction. I agr

Re: [HACKERS] Partitioned tables vs GRANT

2017-04-07 Thread Tom Lane
Joe Conway writes: > Apparently INSERT and SELECT on the parent partitioned table skip normal > acl checks on the partitions. Is that intended behavior? Yes, this matches normal inheritance behavior. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Tom Lane
Andres Freund writes: > I think it makes sense to go through those and see whether it's > realistic to commit any of them. > Unique Joins > - Tom's discussing things with David, not sure. Working on this one today. > Generic type subscripting > - still some review back and forth > - probably sh

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
On 2017-04-07 15:45:33 -0300, Alvaro Herrera wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. > > I'm going over this one now with Pavan, with the intent of getting it in > committable shape. > > I may be biased, but the claimed pe

Re: [HACKERS] Performance issue with postgres9.6

2017-04-07 Thread Merlin Moncure
On Fri, Apr 7, 2017 at 12:54 PM, Tom Lane wrote: > Tomas Vondra writes: >> On 04/07/2017 06:31 PM, Merlin Moncure wrote: >>> I think your math is off. Looking at your attachments, planning time >>> is 0.056ms, not 0.56ms. This is in no way relevant to performance on >>> the order of your measur

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Tom Lane
Alvaro Herrera writes: > Andres Freund wrote: >> Write Amplification Reduction Method (WARM) >> - fair number of people don't think it's ready for v10. > I'm going over this one now with Pavan, with the intent of getting it in > committable shape. I have to agree with Andres that this is not som

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 2:53 PM, Tom Lane wrote: > Alvaro Herrera writes: >> Andres Freund wrote: >>> Write Amplification Reduction Method (WARM) >>> - fair number of people don't think it's ready for v10. > >> I'm going over this one now with Pavan, with the intent of getting it in >> committable

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Fabien COELHO
Hello Pavel, I wish I could have an explanation about why the :?varname (or some other variant) syntax I suggested has a "namespace" issue. The advantage that I see is that although it is obviously ugly, it is ugly in the continuity of the various :["'?]varname syntaxes already offered and it

Re: [HACKERS] postgres_fdw bug in 9.6

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 2:33 PM, Robert Haas wrote: > On Mon, Apr 3, 2017 at 2:12 AM, Etsuro Fujita > wrote: >> On 2017/04/01 1:32, Jeff Janes wrote: >>> On Thu, Mar 23, 2017 at 5:20 AM, Etsuro Fujita >>> mailto:fujita.ets...@lab.ntt.co.jp>> wrote: >>> Done. Attached is a new version of the p

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. Given the number of votes against putting this on pg10, I am going to back off from this patch now, with an ey

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Andres Freund
On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: > Peter Geoghegan wrote: > > > - can't move to next fest because it's waiting-on-author, which doesn't > > > allow that. Doesn't strike me as a useful restriction. > > > > I agree that that CF app restriction makes little sense. > > What the

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Robert Haas
On Fri, Apr 7, 2017 at 3:30 PM, Andres Freund wrote: > On 2017-04-07 16:28:03 -0300, Alvaro Herrera wrote: >> Peter Geoghegan wrote: >> > > - can't move to next fest because it's waiting-on-author, which doesn't >> > > allow that. Doesn't strike me as a useful restriction. >> > >> > I agree tha

Re: [HACKERS] [sqlsmith] Unpinning error in parallel worker

2017-04-07 Thread Robert Haas
On Wed, Apr 5, 2017 at 10:18 AM, Robert Haas wrote: >>> Ugh, OK. I committed this, but I think this whole file needs a visit >>> from the message style police. >> >> Like this? > > I was thinking of maybe not creating two separate (translatable) > messages, and just using "could not attach to dyn

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Peter Geoghegan
On Fri, Apr 7, 2017 at 12:28 PM, Alvaro Herrera wrote: > Peter Geoghegan wrote: >> On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: >> > Write Amplification Reduction Method (WARM) >> > - fair number of people don't think it's ready for v10. > > Given the number of votes against putting this

Re: [HACKERS] Undefined psql variables

2017-04-07 Thread Pavel Stehule
2017-04-07 21:04 GMT+02:00 Fabien COELHO : > > Hello Pavel, > > I wish I could have an explanation about why the :?varname (or some other >>> variant) syntax I suggested has a "namespace" issue. >>> >>> The advantage that I see is that although it is obviously ugly, it is >>> ugly >>> in the conti

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2017-04-07 Thread Andres Freund
Hi, I've *not* read the history of this thread. So I really might be missing some context. > From e37d29c26210a0f23cd2e9fe18a264312fecd383 Mon Sep 17 00:00:00 2001 > From: Claudio Freire > Date: Mon, 12 Sep 2016 23:36:42 -0300 > Subject: [PATCH] Vacuum: allow using more than 1GB work mem > >

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > My offer to work with you on amcheck verification of WARM invariants > remains open. If nothing else, structuring things so that verification > is possible may clarify your design. Formalizing the preconditions, > postconditions, and legal states for on-disk structures mig

Re: [HACKERS] BRIN desummarization writes junk WAL records

2017-04-07 Thread Alvaro Herrera
Tom Lane wrote: > The proximate cause of the exception seems to be that > brinSetHeapBlockItemptr is being passed pagesPerRange = 0, > which is problematic since HEAPBLK_TO_REVMAP_INDEX tries to > divide by that. Looking one level down, the bogus value > seems to be coming out of an xl_brin_desum

Re: [HACKERS] recent deadlock regression test failures

2017-04-07 Thread Thomas Munro
On Sat, Apr 8, 2017 at 6:35 AM, Kevin Grittner wrote: > On Fri, Apr 7, 2017 at 12:52 PM, Andres Freund wrote: > >> I'd rather fix the issue, than remove the tests entirely. Seems quite >> possible to handle blocking on Safesnapshot in a similar manner as >> pg_blocking_pids? > > I'll see what I

[HACKERS] monitoring.sgml missing tag

2017-04-07 Thread Erik Rijkers
monitoring.sgml has one tag missing--- doc/src/sgml/monitoring.sgml.orig 2017-04-07 22:37:55.388708334 +0200 +++ doc/src/sgml/monitoring.sgml 2017-04-07 22:38:16.582047695 +0200 @@ -1275,6 +1275,7 @@ ProcArrayGroupUpdate Waiting for group leader to clear transaction

  1   2   >