Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2019-09-10 Thread Michael Paquier
On Tue, Sep 10, 2019 at 08:29:43AM +0530, Amit Kapila wrote: > Good thought, but I think even if we want to change the name of > tuple_data_split, it might be better done separately. Yes, that's not the problem of this patch. Not sure if it actually makes sense either to change it. The

Re: doc: update PL/pgSQL sample loop function

2019-09-10 Thread Amit Kapila
On Sun, Sep 1, 2019 at 9:09 AM Amit Kapila wrote: > > The current example shows the usage of looping in plpgsql, so as such > there is no correctness issue, but OTOH there is no harm in updating > the example as proposed by Ian Barwick. Does anyone else see any > problem with this idea? If we

Re: SIGQUIT on archiver child processes maybe not such a hot idea?

2019-09-10 Thread Kyotaro Horiguchi
At Wed, 11 Sep 2019 11:01:24 +0900 (Tokyo Standard Time), Kyotaro Horiguchi wrote in <20190911.110124.96874741.horikyota@gmail.com> > At Wed, 11 Sep 2019 01:36:15 +, "Tsunakawa, Takayuki" > wrote in > <0A3221C70F24FB45833433255569204D1FD33579@G01JPEXMBYT05> > > From: Tom Lane

Re: ICU for global collation

2019-09-10 Thread Andrey Borodin
> 21 авг. 2019 г., в 12:23, Peter Eisentraut > написал(а): > > On 2019-08-21 08:56, Andrey Borodin wrote: >> postgres=# create database a template template0 collation_provider icu >> lc_collate 'en_US.utf8'; >> CREATE DATABASE >> postgres=# \c a >> 2019-08-21 11:43:40.379 +05 [41509] FATAL:

Re: [patch]socket_timeout in interfaces/libpq

2019-09-10 Thread Michael Paquier
On Tue, Sep 10, 2019 at 03:38:21PM +0900, Michael Paquier wrote: > I recall having a negative impression on the patch when first looking > at it, and still have the same impression when looking at the last > version. Just with a quick look, assuming that you can bypass all > cleanup operations

Re: [bug fix] Produce a crash dump before main() on Windows

2019-09-10 Thread Michael Paquier
On Wed, Sep 11, 2019 at 04:15:24AM +, Tsunakawa, Takayuki wrote: > From: Michael Paquier [mailto:mich...@paquier.xyz] >> Imagine an application which relies on Postgres, still does *not* start >> it as a service but uses "pg_ctl start" >> automatically. This could be triggered as part of

Re: Feature improvement: can we add queryId for pg_catalog.pg_stat_activity view?

2019-09-10 Thread Michael Paquier
On Wed, Aug 07, 2019 at 09:03:21AM +, Evgeny Efimkin wrote: > The new status of this patch is: Ready for Committer I may be wrong of course, but it looks that this is wanted and the current shape of the patch looks sensible: - Register the query ID using a backend entry. - Only consider the

RE: [bug fix] Produce a crash dump before main() on Windows

2019-09-10 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:mich...@paquier.xyz] > Imagine an application which relies on Postgres, still does *not* start > it as a service but uses "pg_ctl start" > automatically. This could be triggered as part of another service startup > which calls say system(), or as another script.

Re: A problem about partitionwise join

2019-09-10 Thread Richard Guo
Hi Alvaro, Thank you for reviewing this patch. On Wed, Sep 11, 2019 at 4:48 AM Alvaro Herrera from 2ndQuadrant < alvhe...@alvh.no-ip.org> wrote: > So in this patch, the input restrictlist is modified to include the > clauses generated by generate_join_implied_equalities_for_all. That > doesn't

Re: Change atoi to strtol in same place

2019-09-10 Thread Michael Paquier
On Tue, Sep 10, 2019 at 08:03:32AM -0400, Robert Haas wrote: > -1. I think it's very useful to have routines for this sort of thing > that return an error message rather than emitting an error report > directly. That gives the caller a lot more control. Please let me counter-argue here. There

Re: [bug fix] Produce a crash dump before main() on Windows

2019-09-10 Thread Michael Paquier
On Tue, Sep 10, 2019 at 06:44:48AM +, Tsunakawa, Takayuki wrote: > We don't have to call pgwin32_is_service() to determine whether we > call SetErrorMode() in postmaster.c because: > > * The dialog box doesn't appear under Windows service, whether > * PostgreSQL itself starts as a service or

RE: SIGQUIT on archiver child processes maybe not such a hot idea?

2019-09-10 Thread Tsunakawa, Takayuki
From: David Steele [mailto:da...@pgmasters.net] > > Can't we use SIGKILL instead of SIGINT/SIGTERM to stop the grandchildren, > just in case they are slow to respond to or ignore SIGINT/SIGTERM? That > matches the idea of pg_ctl's immediate shutdown. > > -1, at least not immediately. Archivers

Re: MSVC buildfarm critters are not running modules' TAP tests

2019-09-10 Thread Michael Paquier
On Mon, Sep 09, 2019 at 09:43:06AM +0900, Michael Paquier wrote: > On Sun, Sep 08, 2019 at 07:44:16PM -0400, Tom Lane wrote: >> The whole thing should be back-patched into branches that have >> any affected tests. (But, please, not till after beta4 is >> tagged.) > > Sure. Don't worry about

Re: SIGQUIT on archiver child processes maybe not such a hot idea?

2019-09-10 Thread Kyotaro Horiguchi
At Wed, 11 Sep 2019 01:36:15 +, "Tsunakawa, Takayuki" wrote in <0A3221C70F24FB45833433255569204D1FD33579@G01JPEXMBYT05> > From: Tom Lane [mailto:t...@sss.pgh.pa.us] > > SIGTERM, which needs to be adjusted. For another, its > > SIGQUIT handler does exit(1) not _exit(2), which seems rather >

PG12 Beta 4 Press Release

2019-09-10 Thread Jonathan S. Katz
Hi, Please see attached draft of the PG12 Beta 4 press release. I went through the list of open items that were resolved before beta 4[1] for the detailed please. Please let me know if I described any of them incorrectly, or if you believe that any other fixes should be on the list. Thanks!

RE: SIGQUIT on archiver child processes maybe not such a hot idea?

2019-09-10 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > SIGTERM, which needs to be adjusted. For another, its > SIGQUIT handler does exit(1) not _exit(2), which seems rather > dubious ... should we make it more like the rest? I think > the reasoning there might've been that if some DBA decides to > SIGQUIT

Re: Bug in GiST paring heap comparator

2019-09-10 Thread Nikita Glukhov
On 09.09.2019 22:47, Alexander Korotkov wrote: On Mon, Sep 9, 2019 at 8:32 PM Nikita Glukhov wrote: On 08.09.2019 22:32, Alexander Korotkov wrote: On Fri, Sep 6, 2019 at 5:44 PM Alexander Korotkov wrote: I'm going to push both if no objections. So, pushed! Two years ago there was a

Re: Multivariate MCV list vs. statistics target

2019-09-10 Thread Tomas Vondra
On Tue, Sep 03, 2019 at 02:38:56PM -0400, Alvaro Herrera wrote: On 2019-Aug-01, Tomas Vondra wrote: I'll move it to the next CF. Aside from the issues pointed by Kyotaro-san in his review, I still haven't made my mind about whether to base the use statistics targets set for the attributes.

Re: Libpq support to connect to standby server as priority

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
On 2019-Sep-11, Tsunakawa, Takayuki wrote: > From: Alvaro Herrera from 2ndQuadrant [mailto:alvhe...@alvh.no-ip.org] > > Remaining patchset attached (per my count it's v13 of your patchset. > > I'm afraid those weren't attached. Oh, oops. Here they are then. -- Álvaro Herrera

RE: Libpq support to connect to standby server as priority

2019-09-10 Thread Tsunakawa, Takayuki
From: Alvaro Herrera from 2ndQuadrant [mailto:alvhe...@alvh.no-ip.org] > Testing protocol version 2 is difficult! Almost every single test fails > because of error messages being reported differently; and streaming > replication (incl. pg_basebackup) doesn't work at all because it's not >

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-10 Thread Julien Rouhaud
Hello, On Sun, Sep 8, 2019 at 11:45 AM Imai Yoshikazu wrote: > > I also saw the codes and have one comment. Thanks for looking at this patch! > [0002 patch] > In pgss_planner_hook: > > + /* calc differences of buffer counters. */ > + bufusage =

Re: Planning counters in pg_stat_statements (using pgss_store)

2019-09-10 Thread Julien Rouhaud
On Fri, Sep 6, 2019 at 4:19 PM Tomas Vondra wrote: > > On Wed, Sep 04, 2019 at 07:19:47PM +0300, Sergei Kornilov wrote: > > > > ... > > > >Results: > > > > test | mode | average_tps | degradation_perc > >--+--+-+-- > >

Re: [PATCH] Opclass parameters

2019-09-10 Thread Nikita Glukhov
On 11.09.2019 1:03, Tomas Vondra wrote: On Tue, Sep 10, 2019 at 04:30:41AM +0300, Nikita Glukhov wrote: 2. New AM method amattoptions().   amattoptions() is used to specify per-column AM-specific options.   The example is signature length for bloom indexes (patch #3). I'm somewhat

Re: [PATCH] Improve performance of NOTIFY over many databases (v2)

2019-09-10 Thread Tom Lane
Martijn van Oosterhout writes: > The original three patches have been collapsed into one as given the > changes discussed it didn't make sense to keep them separate. There > are now two patches (the third is just to help with testing): > Patch 1: Tracks the listening backends in a list so

Re: [PATCH] Opclass parameters

2019-09-10 Thread Nikita Glukhov
On 11.09.2019 1:14, Tomas Vondra wrote: BTW, is there a place where we actually verify the signature of the new am proc? Because I only see code like this: +    case OPCLASS_OPTIONS_PROC: +    ok = true; +    break; in all "validate" functions. See assignProcTypes() at

Re: [PATCH] Opclass parameters

2019-09-10 Thread Tomas Vondra
On Wed, Sep 11, 2019 at 12:03:58AM +0200, Tomas Vondra wrote: On Tue, Sep 10, 2019 at 04:30:41AM +0300, Nikita Glukhov wrote: On 04.09.2019 1:02, Alvaro Herrera wrote: On 2019-Jun-11, Tomas Vondra wrote: 1) We need a better infrastructure to parse opclass parameters. For example the

Re: [PATCH] Opclass parameters

2019-09-10 Thread Tomas Vondra
On Tue, Sep 10, 2019 at 04:30:41AM +0300, Nikita Glukhov wrote: On 04.09.2019 1:02, Alvaro Herrera wrote: On 2019-Jun-11, Tomas Vondra wrote: 1) We need a better infrastructure to parse opclass parameters. For example the gtsvector_options does this: I think this is part of what Nikolay's

Re: Unaccent extension python script Issue in Windows

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
Thanks! I have pushed this patch. I didn't test on Windows, but I did verify that it works with python2 and 3 on my Linux machine. CLDR has made release 35 already, upon download of which the script generates a few more lines in the unaccent.rules file, as attached. -- Álvaro Herrera

Re: A problem about partitionwise join

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
So in this patch, the input restrictlist is modified to include the clauses generated by generate_join_implied_equalities_for_all. That doesn't seem okay -- doesn't it affect downstream usage of the restrictlist in the caller of set_joinrel_size_estimates? I wonder if it's possible to do this by

Re: Duplicated LSN in ReorderBuffer

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
Hello, On 2019-Sep-06, Andres Freund wrote: > On 2019-08-19 08:51:43 -0700, Andres Freund wrote: > > On August 19, 2019 7:43:28 AM PDT, Alvaro Herrera > > wrote: > > >Never mind. I was able to reproduce it later, and verify that Andres' > > >proposed strategy doesn't seem to fix the problem.

Re: [PATCH] Move user options to the end of the command in pg_upgrade

2019-09-10 Thread Tom Lane
I wrote: > Patrik Novotny writes: >> thanks for you reply. You're right, and I apologise for the confusion. >> Options I was talking about are specified via the `--old-options` >> parameter of the pg_upgrade (ex.: --old-options '-c >> unix_socket_directories=/run') >> Mentioning of the

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
On 2019-Sep-09, Paul Guo wrote: > > > > Thank for rebasing. > > > > I didn't like 0001 very much. > > > > * It seems now would be the time to stop pretending we're using a file > > called recovery.conf; I know we still support older server versions that > > use that file, but it sounds like we

Re: unlogged sequences

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
On 2019-Aug-01, Thomas Munro wrote: > On Wed, Jun 26, 2019 at 6:38 AM Andres Freund wrote: > > On 2019-06-20 09:30:34 +0200, Peter Eisentraut wrote: > > > I'm looking for feedback from those who have worked on tableam and > > > storage manager to see what the right interfaces are or whether some

Re: Re: libpq host/hostaddr/conninfo inconsistencies

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
I propose to settle this issue by applying "Robert's changes two and three", which has been +1'd by two people already and I also accept myself as improvement. I don't think any further changes are required. diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index

Re: another look at macOS SIP

2019-09-10 Thread Tom Lane
Peter Eisentraut writes: > I have figured out another solution to the problem that macOS SIP > defeats the use of DYLD_LIBRARY_PATH for running the temp-install > regression tests. It's not without problems either, but it might show a > path forward. > ... > The precise issue is that /bin/sh

another look at macOS SIP

2019-09-10 Thread Peter Eisentraut
I have figured out another solution to the problem that macOS SIP defeats the use of DYLD_LIBRARY_PATH for running the temp-install regression tests. It's not without problems either, but it might show a path forward. First of all, I think I now know the exact mechanism by which this breakage

Re: Useless code in RelationCacheInitializePhase3

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
Hello, On 2019-Apr-13, Tom Lane wrote: > Andres Freund writes: > > I think it probably would work for catalog tables, as it's coded right > > now. There's no catalog lookups RelationInitTableAccessMethod() for > > tables that return true for IsCatalogTable(). In fact, I think we should > >

Re: Libpq support to connect to standby server as priority

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
On 2019-Sep-09, Alvaro Herrera from 2ndQuadrant wrote: > Question about 0001. In the CONNECTION_SETENV state, you end by falling > through to the CONNECTION_CHECK_TARGET case; but in that switch it seems > a bit unnatural to do that. I think doing "goto keep_trying" is another > way of doing

Re: [PATCH] Move user options to the end of the command in pg_upgrade

2019-09-10 Thread Tom Lane
Patrik Novotny writes: > thanks for you reply. You're right, and I apologise for the confusion. > Options I was talking about are specified via the `--old-options` > parameter of the pg_upgrade (ex.: --old-options '-c > unix_socket_directories=/run') > Mentioning of the environment variable came

Re: [PATCH] Move user options to the end of the command in pg_upgrade

2019-09-10 Thread Patrik Novotny
Hi Tom, thanks for you reply. You're right, and I apologise for the confusion. Options I was talking about are specified via the `--old-options` parameter of the pg_upgrade (ex.: --old-options '-c unix_socket_directories=/run') Mentioning of the environment variable came only from my own

Re: [PATCH] Move user options to the end of the command in pg_upgrade

2019-09-10 Thread Tom Lane
Patrik Novotny writes: > when the environment variable POSTGRESQL_UPGRADE_PGUPGRADE_OPTIONS is > used to specify options for pg_upgrade, options related to > unix_socket_directory/ies are being overridden by hardcoded options, > making it difficult to upgrade in some usecases. > The attached

Re: Pulling up direct-correlated ANY_SUBLINK

2019-09-10 Thread Tom Lane
Richard Guo writes: > Currently we do not try to pull up sub-select of type ANY_SUBLINK if it > refers to any Vars of the parent query, as indicated in the code snippet > below: > if (contain_vars_of_level((Node *) subselect, 1)) > return NULL; > Why do we have this check? Because

Re: accounting for memory used for BufFile during hash joins

2019-09-10 Thread Tomas Vondra
On Thu, Sep 05, 2019 at 09:54:33AM -0700, Melanie Plageman wrote: On Tue, Sep 3, 2019 at 9:36 AM Alvaro Herrera wrote: On 2019-Jul-11, Tomas Vondra wrote: > On Wed, Jul 10, 2019 at 04:51:02PM -0700, Melanie Plageman wrote: > > I think implementing support for parallel hashjoin or explicitly

Re: Specifying attribute slot for storing/reading statistics

2019-09-10 Thread Tomas Vondra
Hi, Please don't top-post. If you're not responding to parts of the e-mail, then don't quote it. On Fri, Sep 06, 2019 at 12:50:33PM +0200, Esteban Zimanyi wrote: Dear Tom Many thanks for your quick reply. Indeed both solutions you proposed can be combined together in order to solve all the

Re: [proposal] de-TOAST'ing using a iterator

2019-09-10 Thread Binguo Bao
Alvaro Herrera 于2019年9月4日周三 上午4:12写道: > > +static void > > +init_toast_buffer(ToastBuffer *buf, int32 size, bool compressed) > > +{ > > + buf->buf = (const char *) palloc0(size); > > This API is weird -- you always palloc the ToastBuffer first, then call > init_toast_bufer on it. Why not

Re: Avoiding hash join batch explosions with extreme skew and weird stats

2019-09-10 Thread Tomas Vondra
On Fri, Sep 06, 2019 at 10:54:13AM -0700, Melanie Plageman wrote: On Thu, Sep 5, 2019 at 10:35 PM Thomas Munro wrote: Seems like a good time for me to try to summarise what I think the main problems are here: 1. The match-bit storage problem already discussed. The tuples that each process

Re: Change atoi to strtol in same place

2019-09-10 Thread Robert Haas
On Tue, Sep 10, 2019 at 1:36 AM Michael Paquier wrote: > The error handling is awkward. I think that you should just call > pg_log_error in pg_strtoint64_range instead of returning an error > string as you do. You could do that by passing down the option name > to the routine, and generate a

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-09-10 Thread Noah Misch
[Casual readers with opinions on GUC naming: consider skipping to the end.] MarkBufferDirtyHint() writes WAL even when rd_firstRelfilenodeSubid or rd_createSubid is set; see attached test case. It needs to skip WAL whenever RelationNeedsWAL() returns false. On Tue, Aug 27, 2019 at 03:49:32PM

[PATCH] Move user options to the end of the command in pg_upgrade

2019-09-10 Thread Patrik Novotny
Hi, when the environment variable POSTGRESQL_UPGRADE_PGUPGRADE_OPTIONS is used to specify options for pg_upgrade, options related to unix_socket_directory/ies are being overridden by hardcoded options, making it difficult to upgrade in some usecases. The attached patch changes the order of those

Re: [bug fix] Produce a crash dump before main() on Windows

2019-09-10 Thread Alvaro Herrera from 2ndQuadrant
On 2019-Sep-10, Tsunakawa, Takayuki wrote: > Agreed. Fixed. I changed the CF status to "need review." I have changed it again to "ready for committer". We could really use help from a windows-enabled committer on this patch, I think. -- Álvaro Herrera

Re: Berserk Autovacuum (let's save next Mandrill)

2019-09-10 Thread Amit Kapila
On Tue, Jul 23, 2019 at 1:53 PM Masahiko Sawada wrote: > > > To invoke autovacuum even on insert-only tables we would need check > the number of inserted tuples since last vacuum. I think we can keep > track of the number of inserted tuples since last vacuum to the stats > collector and add the

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-09-10 Thread Asim R P
Hi Anastasia On Thu, Aug 22, 2019 at 9:43 PM Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > > But during the review, I found a bug in the current implementation. > New behavior must apply to crash-recovery only, now it applies to archiveRecovery too. > That can cause a silent loss

Re: FETCH FIRST clause WITH TIES option

2019-09-10 Thread Surafel Temesgen
On Fri, Sep 6, 2019 at 5:07 PM Alvaro Herrera from 2ndQuadrant < alvhe...@alvh.no-ip.org> wrote: > On 2019-Sep-06, Surafel Temesgen wrote: > > > > ... yet this doesn't appear to have resulted in any change in the code, > > > or I just missed it. Are you going to update the patch per that? > > >

Re: Does PostgreSQL support debian Linux on Arm CPU Platform?

2019-09-10 Thread Dagfinn Ilmari Mannsåker
writes: > Hi,I just want know does PostgreSQL support debian Linux with ARM CPU > Platform,Thank you! The PostgreSQL community provided packages (https://apt.postgresql.org/) are only built for amd64, i386 and ppc64el, but Debian itself ships PostgreSQL on every architecture it supports. Each

Re: Pulling up direct-correlated ANY_SUBLINK

2019-09-10 Thread Antonin Houska
Richard Guo wrote: > Can we try to pull up direct-correlated ANY SubLink with the help of > LATERAL? > By this way, we can convert the query: > > select * from a where a.i = ANY(select i from b where a.j > b.j); > > To: > > select * from a SEMI JOIN lateral(select * from b where a.j > b.j) >

Re: Remove page-read callback from XLogReaderState.

2019-09-10 Thread Kyotaro Horiguchi
Attached is new version: - Rebased. Cleaned up - Rebased to the current master - Fixed a known bug in the first step patch. It caused timeline-following failure on a standby of a promoted primary. - Fixed confused naming and setting of the parameter includes_paeg_header. - Removed useless

Re: WIP: BRIN multi-range indexes

2019-09-10 Thread Alexander Korotkov
Hi, Tomas! I took a look at this patchset. On Tue, Jun 11, 2019 at 8:31 PM Tomas Vondra wrote: > Attached is this patch series, rebased on top of current master and the > opclass parameters patch [1]. I previously planned to keep those two > efforts separate for a while, but I decided to give

Pulling up direct-correlated ANY_SUBLINK

2019-09-10 Thread Richard Guo
Hi, Currently we do not try to pull up sub-select of type ANY_SUBLINK if it refers to any Vars of the parent query, as indicated in the code snippet below: JoinExpr * convert_ANY_sublink_to_join(PlannerInfo *root, SubLink *sublink, Relids available_rels) { ...

RE: [bug fix] Produce a crash dump before main() on Windows

2019-09-10 Thread Tsunakawa, Takayuki
From: Tom Lane [mailto:t...@sss.pgh.pa.us] > Alvaro Herrera from 2ndQuadrant writes: > > Well, IMV this is a backpatchable, localized bug fix. > > I dunno. This thread is approaching two years old, and a quick > review shows few signs that we actually have any consensus on > making behavioral

Re: [patch]socket_timeout in interfaces/libpq

2019-09-10 Thread Michael Paquier
On Wed, Jun 26, 2019 at 11:56:28AM +, nagaura.ryo...@fujitsu.com wrote: > It seems that you did not think so at that time. > # Please refer to [1] > > I don't think all the reviewers are completely negative. I recall having a negative impression on the patch when first looking at it, and

Re: Replication & recovery_min_apply_delay

2019-09-10 Thread Michael Paquier
On Tue, Sep 10, 2019 at 12:46:49AM +0300, Alexander Korotkov wrote: > On Wed, Sep 4, 2019 at 4:37 PM Konstantin Knizhnik > wrote: >> receivedUpto is just static variable in xlog.c, maintained by WAL receiver. >> But as I mentioned above, WAL receiver is not started at the moment when >> we need