Re: pgbench - allow to create partitioned tables

2019-10-02 Thread Fabien COELHO
Thanks, attached is a patch with minor modifications which I am planning to push after one more round of review on Thursday morning IST unless there are more comments by anyone else. Pushed. Thanks! -- Fabien.

Re: PATCH: standby crashed when replay block which truncated in standby but failed to truncate in master node

2019-10-02 Thread Michael Paquier
On Thu, Oct 03, 2019 at 01:49:34PM +0900, Fujii Masao wrote: > But this can cause subsequent recovery to always fail with invalid-pages error > and the server not to start up. This is bad. So, to allviate the situation, > I'm thinking it would be worth adding something like igore_invalid_pages >

Re: PATCH: standby crashed when replay block which truncated in standby but failed to truncate in master node

2019-10-02 Thread Fujii Masao
On Fri, Sep 27, 2019 at 3:14 PM Michael Paquier wrote: > > On Thu, Sep 26, 2019 at 01:13:56AM +0900, Fujii Masao wrote: > > On Tue, Sep 24, 2019 at 10:41 AM Michael Paquier > > wrote: > >> This also points out that there are other things to worry about than > >> interruptions, as for example

Re: Hooks for session start and end, take two

2019-10-02 Thread Michael Paquier
On Wed, Oct 02, 2019 at 09:09:53PM -0700, Andres Freund wrote: > On 2019-10-03 11:03:39 +0900, Michael Paquier wrote: >> The idea here is to have a hook which can be triggered at the start of >> a process which can be externally triggered, which I guess is normal >> even for WAL senders not

Re: Hooks for session start and end, take two

2019-10-02 Thread Andres Freund
Hi, On 2019-10-03 11:03:39 +0900, Michael Paquier wrote: > On Wed, Oct 02, 2019 at 10:23:54AM -0700, Andres Freund wrote: > > - the startup hook isn't actually guaranteed to be able to write > > anything, we might be in hot-standby > > Right. > > > - the startup hook doesn't just run in

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-10-02 Thread Michael Paquier
On Wed, Oct 02, 2019 at 08:28:09PM +0300, Alexey Kondratov wrote: > I've directly followed your guess and tried to elaborate pg_rewind test > cases and... It seems I've caught a few bugs: > > 1) --dry-run actually wasn't completely 'dry'. It did update target > controlfile, which could cause

Re: pgbench - allow to create partitioned tables

2019-10-02 Thread Amit Kapila
On Tue, Oct 1, 2019 at 10:20 AM Amit Kapila wrote: > On Mon, Sep 30, 2019 at 5:17 PM Fabien COELHO wrote: > > > > > I don't want to introduce a new pattern in tests which people can then > > > tomorrow copy at other places even though such code is not required. > > > OTOH, if there is a genuine

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Michael Paquier
On Wed, Oct 02, 2019 at 02:55:39PM -0400, Tom Lane wrote: > Mark Dilger writes: >> (I'm sitting on a few patches until v12 goes out the door from some >> conversations with you several months ago, and perhaps I'll include a >> patch for this cleanup, too, when time comes for v13 patch sets to

Re: Hooks for session start and end, take two

2019-10-02 Thread Michael Paquier
On Wed, Oct 02, 2019 at 01:51:43PM -0400, Tom Lane wrote: > Thus, we can't run any shutdown hooks before this one, if indeed we > consider it to be a shutdown hook at all. > > A possible fix is to do it as the first action in proc_exit, but that will > fall foul of Andres' points about not

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-10-02 Thread Tomas Vondra
On Wed, Oct 02, 2019 at 04:27:30AM +0530, Amit Kapila wrote: On Tue, Oct 1, 2019 at 7:21 PM Tomas Vondra wrote: On Tue, Oct 01, 2019 at 06:55:52PM +0530, Amit Kapila wrote: > >On further testing, I found that the patch seems to have problems with >toast. Consider below scenario: >Session-1

Re: WIP: Generic functions for Node types using generated metadata

2019-10-02 Thread Andres Freund
Hi, On 2019-10-02 14:47:22 -0400, Tom Lane wrote: > Robert Haas writes: > > On Wed, Oct 2, 2019 at 12:03 PM Tom Lane wrote: > >> I'm afraid that's going to be a deal-breaker for lots of people. > >> It's fine for prototyping the idea but we'll need to find another > >> implementation before we

Re: WIP: Generic functions for Node types using generated metadata

2019-10-02 Thread Andres Freund
Hi, On 2019-10-02 14:30:08 -0400, Robert Haas wrote: > On Wed, Oct 2, 2019 at 12:03 PM Tom Lane wrote: > > I'm afraid that's going to be a deal-breaker for lots of people. > > It's fine for prototyping the idea but we'll need to find another > > implementation before we can move to commit. > >

Re: Is querying SPITupleTable with SQL possible?

2019-10-02 Thread Chapman Flack
On 10/2/19 2:53 PM, Tom Mercha wrote: > However, I'm not quite so sure how I can query over the > EphemeralNamedRelation using SQL? Could someone indicate where I can > find an example? You could look in the documentation for CREATE TRIGGER in PG 10 or later, specifically the clauses like

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Tom Lane
Mark Dilger writes: > (I'm sitting on a few patches until v12 goes out the door from some > conversations with you several months ago, and perhaps I'll include a > patch for this cleanup, too, when time comes for v13 patch sets to be > submitted. That would be now. We already ran one CF for

Re: Is querying SPITupleTable with SQL possible?

2019-10-02 Thread Tom Mercha
On 02/10/2019 16:11, Tom Lane wrote: > Tom Mercha writes: >> I am using PostgreSQL's SPI to execute a simple SQL query (SELECT * FROM >> ...) via SPI_exec. As a a result, I get an SPITupleTable with the >> results of my query. >> Now that I have the SPITupleTable, I was wondering if it would be

Re: Collation versioning

2019-10-02 Thread Peter Eisentraut
On 2018-09-05 23:18, Thomas Munro wrote: > On Wed, Sep 5, 2018 at 12:10 PM Christoph Berg wrote: >>> So, it's not ideal but perhaps worth considering on the grounds that >>> it's better than nothing? >> >> Ack. > > Ok, here's a little patch like that. > > postgres=# select collname,

Re: WIP: Generic functions for Node types using generated metadata

2019-10-02 Thread Tom Lane
Robert Haas writes: > On Wed, Oct 2, 2019 at 12:03 PM Tom Lane wrote: >> I'm afraid that's going to be a deal-breaker for lots of people. >> It's fine for prototyping the idea but we'll need to find another >> implementation before we can move to commit. > Why do you think it will be a

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Mark Dilger
On 10/2/19 11:02 AM, Tom Lane wrote: Mark Dilger writes: On 10/2/19 8:46 AM, Tom Lane wrote: Right. I think that in general it's bad practice for an initializer to not specify all fields/elements of the target. There are numerous locations in the code that raise warnings when

Re: WIP: Generic functions for Node types using generated metadata

2019-10-02 Thread Robert Haas
On Wed, Oct 2, 2019 at 12:03 PM Tom Lane wrote: > I'm afraid that's going to be a deal-breaker for lots of people. > It's fine for prototyping the idea but we'll need to find another > implementation before we can move to commit. Why do you think it will be a deal-breaker for lots of people? I

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Tom Lane
Mark Dilger writes: > On 10/2/19 8:46 AM, Tom Lane wrote: >> Right. I think that in general it's bad practice for an initializer >> to not specify all fields/elements of the target. > There are numerous locations in the code that raise warnings when > -Wmissing-field-initializers is handed to

Re: Hooks for session start and end, take two

2019-10-02 Thread Tom Lane
Andres Freund writes: > On 2019-10-02 23:52:31 +0900, Fujii Masao wrote: >> Regarding session end hook, you can do the almost same thing as that hook >> by calling on_shmem_exit(), before_shmem_exit() or on_proc_exit() in >> other hook like session start hook. This approach also has the same

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-10-02 Thread Peter Geoghegan
On Mon, Sep 30, 2019 at 7:39 PM Peter Geoghegan wrote: > I've found that my "regular pgbench, but with a low cardinality index > on pgbench_accounts(abalance)" benchmark works best with the specific > heuristics used in the patch, especially over many hours. I ran pgbench without the

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Mark Dilger
On 10/2/19 8:46 AM, Tom Lane wrote: Joe Nelson writes: Isaac Morland wrote: I hope you'll forgive a noob question. Why does the "After" initialization for the boolean array have {0} rather than {false}? I think using a value other than {0} potentially gives the incorrect impression that

Re: Two pg_rewind patches (auto generate recovery conf and ensure clean shutdown)

2019-10-02 Thread Alexey Kondratov
Hi Alvaro, On 30.09.2019 20:13, Alvaro Herrera wrote: OK, I pushed this patch as well as Alexey's test patch. It all works for me, and the coverage report shows that we're doing the new thing ... though only in the case that rewind *is* required. There is no test to verify the case where

Re: Hooks for session start and end, take two

2019-10-02 Thread Andres Freund
Hi, Moving the discussion of the issues leading to this being reverted back to -hackers: On 2019-10-02 23:52:31 +0900, Fujii Masao wrote: > On Wed, Oct 2, 2019 at 10:08 PM Alvaro Herrera > wrote: > > > > On 2019-Oct-02, Michael Paquier wrote: > > > > > On Wed, Oct 02, 2019 at 01:27:50PM +0900,

format of pg_upgrade loadable_libraries warning

2019-10-02 Thread Justin Pryzby
Regarding the previous thread and commit here: https://www.postgresql.org/message-id/flat/20180713162815.GA3835%40momjian.us https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=60e3bd1d7f92430b24b710ecf0559656eb8ed499 I'm suggesting to reformat the warning, which I found to be

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> * I think you missed s/walk/mutate/ in some of the comments you > Tom> copied into query_tree_mutator. > ... where? The only mention of "walk" near query_tree_mutator is in its > header comment, which I didn't touch. Wup, sorry, I

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> * Please run it through pgindent. Otherwise v13+ are going to be Tom> randomly different from older branches in this area, once we next Tom> pgindent HEAD. gotcha. Tom> * I think you missed s/walk/mutate/ in some of the comments you Tom> copied into

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Tom Lane
Andrew Gierth writes: > OK. So here's the final patch. > (For the benefit of anyone in -hackers not following the original thread > in -general, the problem here is that expressions in window framing > clauses were not being walked or mutated by query_tree_walker / > query_tree_mutator. This has

Re: dropdb --force

2019-10-02 Thread Pavel Stehule
st 2. 10. 2019 v 5:20 odesílatel Amit Kapila napsal: > On Thu, Sep 26, 2019 at 7:18 PM Pavel Stehule > wrote: > >> >> út 24. 9. 2019 v 14:52 odesílatel Amit Kapila >> napsal: >> >>> >>> One other minor comment: >>> + >>> + This will also fail, if the connections do not terminate in 5 >>>

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> I'm going to leave the assertion out for now and put in a comment >> for future reference. Tom> WFM. At this point it's clear it would be a separate piece of work Tom> not something to slide into the bug-fix patch, anyway. OK. So here's the final patch.

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Tom Lane
Andrew Gierth writes: > I'm going to leave the assertion out for now and put in a comment for > future reference. WFM. At this point it's clear it would be a separate piece of work not something to slide into the bug-fix patch, anyway. regards, tom lane

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Joe Nelson
> If so, I don't suppose it's possible to give empty braces: > > bool nulls[Natts_pg_attribute] = {}; GNU does add this capability as a nonstandard language extension, but according to the C99 standard, no.

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Andrew Gierth
> "Tom" == Tom Lane writes: Tom> Hm. transformRuleStmt already does special-case utility statements Tom> to some extent, so my inclination would be to make it do more of Tom> that. However, it looks like that might end up with rather Tom> spaghetti-ish code, as that function is kind of

Re: My buildfarm member now giving permission denied

2019-10-02 Thread Kyle Evans
On Tue, Oct 1, 2019 at 8:32 PM Larry Rosenman wrote: > > On 10/01/2019 8:27 pm, Larry Rosenman wrote: > > FreeBSD SVN rev: > > r352600 - - 1.69G 2019-09-22 13:13 > > r352873 NR / 43.1G 2019-09-29 16:36 > > > > I went from r352600 to r352873 and now I'm getting

Re: WIP: Generic functions for Node types using generated metadata

2019-10-02 Thread Tom Lane
Robert Haas writes: > It seems like the approach Andres is proposing here could work pretty > well. I think the biggest possible problem is that any semi-serious > developer will basically have to have LLVM installed. To build the > software, you wouldn't need LLVM unless you want to build with

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Isaac Morland
On Wed, 2 Oct 2019 at 11:34, Joe Nelson wrote: > Isaac Morland wrote: > > I hope you'll forgive a noob question. Why does the "After" > > initialization for the boolean array have {0} rather than {false}? > > I think using a value other than {0} potentially gives the incorrect > impression that

Re: WIP: Generic functions for Node types using generated metadata

2019-10-02 Thread Robert Haas
On Fri, Aug 30, 2019 at 9:03 AM Fabien COELHO wrote: > I have found this thread: > > https://www.postgresql.org/message-id/flat/E1cq93r-0004ey-Mp%40gemulon.postgresql.org > > It seems that comments from committers discouraged me to go on… :-) For > instance Robert wanted a "checker", which is

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Tom Lane
Joe Nelson writes: > Isaac Morland wrote: >> I hope you'll forgive a noob question. Why does the "After" >> initialization for the boolean array have {0} rather than {false}? > I think using a value other than {0} potentially gives the incorrect > impression that the value is used for *all*

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Joe Nelson
Isaac Morland wrote: > I hope you'll forgive a noob question. Why does the "After" > initialization for the boolean array have {0} rather than {false}? I think using a value other than {0} potentially gives the incorrect impression that the value is used for *all* elements of the array/structure,

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Tom Lane
Andrew Gierth writes: > "Tom" == Tom Lane writes: > Tom> Also, in HEAD I'd be inclined to add assertions about utilityStmt > Tom> being NULL. > Tried this. The assertion is hit: > ... > Any suggestions where best to fix this? transformRuleStmt could be > taught to skip a lot of the per-Query

Re: Possible bug: SQL function parameter in window frame definition

2019-10-02 Thread Andrew Gierth
[moving to -hackers, removing OP and -general] > "Tom" == Tom Lane writes: Tom> Also, in HEAD I'd be inclined to add assertions about utilityStmt Tom> being NULL. Tried this. The assertion is hit: #3 0x00bb9144 in ExceptionalCondition (conditionName=0xd3c7a9

Re: let's make the list of reportable GUCs configurable (was Re: Add %r substitution for psql prompts to show recovery status)

2019-10-02 Thread Robert Haas
On Thu, Jul 11, 2019 at 3:16 PM Dave Cramer wrote: > On Thu, 11 Jul 2019 at 15:07, Robert Haas wrote: >> On Thu, Jul 11, 2019 at 2:29 PM Dave Cramer wrote: >> > So if I understand this correctly if user bob has altered his search path >> > and there is a security-definer function called owned

Re: [HACKERS] Restricting maximum keep segments by repslots

2019-10-02 Thread Jehan-Guillaume de Rorthais
On Tue, 30 Jul 2019 21:30:45 +0900 (Tokyo Standard Time) Kyotaro Horiguchi wrote: > Thanks for reviewing! > > At Thu, 27 Jun 2019 16:22:56 +0200, Jehan-Guillaume de Rorthais > wrote in <20190627162256.4f4872b8@firost> > > Hi all, > > > > Being interested by this feature, I did a patch review.

Re: Consider low startup cost in add_partial_path

2019-10-02 Thread James Coleman
On Sat, Sep 28, 2019 at 7:21 PM James Coleman wrote: > Now the trick is to figure out a way to demonstrate it in test :) > > Basically we need: > Path A: Can short circuit with LIMIT but has high total cost > Path B: Can’t short circuit with LIMIT but has lower total cost > > (Both must be

Re: Is querying SPITupleTable with SQL possible?

2019-10-02 Thread Tom Lane
Tom Mercha writes: > I am using PostgreSQL's SPI to execute a simple SQL query (SELECT * FROM > ...) via SPI_exec. As a a result, I get an SPITupleTable with the > results of my query. > Now that I have the SPITupleTable, I was wondering if it would be > possible to later query over it further

Re: [HACKERS] Block level parallel vacuum

2019-10-02 Thread Masahiko Sawada
On Sat, Sep 21, 2019 at 9:31 PM Amit Kapila wrote: > > On Fri, Jun 7, 2019 at 12:03 PM Masahiko Sawada wrote: > > > > Since the previous version patch conflicts with current HEAD, I've > > attached the updated version patches. > > > Thank you for reviewing this patch! > Review comments: >

Re: Fix parsing of identifiers in jsonpath

2019-10-02 Thread Nikita Glukhov
Attached v2 patch rebased onto current master. On 18.09.2019 18:10, Nikita Glukhov wrote: Unfortunately, jsonpath lexer, in contrast to jsonpath parser, was written by Teodor and me without a proper attention to the stanard. JSON path lexics is is borrowed from the external ECMAScript [1],

Ordering of header file inclusion

2019-10-02 Thread vignesh C
Hi, I noticed that some of the header files inclusion is not ordered as per the usual standard that is followed. The attached patch contains the fix for the order in which the header files are included. Let me know your thoughts on the same. Regards, Vignesh EnterpriseDB:

Re: Recovery performance of DROP DATABASE with many tablespaces

2019-10-02 Thread Fujii Masao
On Tue, Jul 10, 2018 at 3:04 PM Michael Paquier wrote: > > On Thu, Jul 05, 2018 at 01:42:20AM +0900, Fujii Masao wrote: > > TBH, I have no numbers measured by the test. > > One question about your test is; how did you measure the *recovery time* > > of DROP DATABASE? Since it's *recovery*

Re: Recovery performance of DROP DATABASE with many tablespaces

2019-10-02 Thread Fujii Masao
On Thu, Jul 5, 2018 at 5:15 PM Simon Riggs wrote: > > On 4 June 2018 at 17:46, Fujii Masao wrote: > > Hi, > > > > My colleague encountered the problem that WAL replay took a long time > > in the standby with large shared_buffers when he dropped the database > > using many tablespaces. As far as

RE: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-02 Thread Smith, Peter
From: Amit Kapila Sent: Wednesday, 2 October 2019 9:42 AM > I don't have any strong opinion on this, but I would mildly prefer to > initialize boolean array with false just for the sake of readability (we > generally initializing booleans with false). Done. Please see attached updated patch.

Re: Standby accepts recovery_target_timeline setting?

2019-10-02 Thread Stephen Frost
Greetings, * David Steele (da...@pgmasters.net) wrote: > On 9/28/19 1:26 PM, Fujii Masao wrote: > > On Sun, Sep 29, 2019 at 12:51 AM David Steele wrote: > > > > Yeah, more checks would be necessary. IMO easy fix is to forbid not only > > recovery target parameters but also any recovery

Re: Inconsistent usage of BACKEND_* symbols

2019-10-02 Thread Michael Paquier
On Mon, Sep 30, 2019 at 04:39:59PM +0900, Kyotaro Horiguchi wrote: > @@ -2740,8 +2740,8 @@ pmdie(SIGNAL_ARGS) > { > /* autovac workers are told to shut down immediately */ > /* and bgworkers too; does this need tweaking? */ > -

Re: Attempt to consolidate reading of XLOG page

2019-10-02 Thread Antonin Houska
Kyotaro Horiguchi wrote: > At Tue, 01 Oct 2019 08:28:03 +0200, Antonin Houska wrote > in <2188.1569911283@antos> > > Kyotaro Horiguchi wrote: > > > > The problem of walsender.c is that its implementation of XLogRead() > > > > does not > > > > care about the TLI of the previous read. If the

Re: some PostgreSQL 12 release notes comments

2019-10-02 Thread Stephen Frost
Greetings, * Peter Eisentraut (peter.eisentr...@2ndquadrant.com) wrote: > On 2019-09-17 22:22, Tom Lane wrote: > > Peter Eisentraut writes: > >> * Add GSSAPI encryption support (Robbie Harwood, Stephen Frost) > >> This allows TCP/IP connections to be encrypted when using GSSAPI > >>

Re: pg_wal/RECOVERYHISTORY file remains after archive recovery

2019-10-02 Thread Masahiko Sawada
On Wed, Oct 2, 2019 at 3:58 PM Michael Paquier wrote: > > On Mon, Sep 30, 2019 at 05:07:08PM +0900, Masahiko Sawada wrote: > > Thank you for updating! The comment in your patch is much better. > > Thanks, done and back-patched down to 9.5. Thank you! Regards, -- Masahiko Sawada

[Bug fix] ECPG: ECPG preprocessor outputs "unsupported feature will be passed to server" even if the command is supported

2019-10-02 Thread higuchi.dais...@fujitsu.com
Hi, I found wrong message output when ECPG preprocessed some SQL commands . For example, ECPG application has following command, ECPG outputs "unsupported feature will be passed to server". --- EXEC SQL CREATE SCHEMA IF NOT EXISTS hollywood; --- I attached sample code to

Re: pg_wal/RECOVERYHISTORY file remains after archive recovery

2019-10-02 Thread Michael Paquier
On Mon, Sep 30, 2019 at 05:07:08PM +0900, Masahiko Sawada wrote: > Thank you for updating! The comment in your patch is much better. Thanks, done and back-patched down to 9.5. -- Michael signature.asc Description: PGP signature

Re: Keep compiler silence (clang 10, implicit conversion from 'long' to 'double' )

2019-10-02 Thread Yuya Watari
Horiguchi-san, On Tue, Oct 1, 2019 at 3:41 PM Kyotaro Horiguchi wrote: > I found a trick seems workable generically (*1). (2.0 * > (PG_INT64_MAX/2 + 1)) will generate the value next to the > PG_INT64_MAX based on some assumptions > (*1). IS_DOUBLE_SAFE_IN_INT64() below would be able to check if