Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Dilip Kumar
On Tue, Oct 22, 2019 at 12:03 PM Fabien COELHO wrote: > > > Hello, > > While developing pgbench to allow partitioned tabled, I reproduced the > string management style used in the corresponding functions, but was > pretty unhappy with that kind of pattern: > > snprintf(buf + strlen(buf), s

回复:Bug about drop index concurrently

2019-10-22 Thread 李杰(慎追)
Hi all, I am sorry to bother you again. I want to consult again, about the last time I raised a bug about drop index, are you going to deal with it in the future? Is it to ban it or to propose a repair plan, what is your next plan? Sincerely look forward to your reply and thanks. adger. ---

Re: Ordering of header file inclusion

2019-10-22 Thread Amit Kapila
On Mon, Oct 21, 2019 at 11:04 PM vignesh C wrote: > > On Mon, Oct 21, 2019 at 8:47 AM Amit Kapila wrote: > > > > > Thanks for the suggestions. > Updated patch contains the fix based on Tom Lane's Suggestion. > Let me know your thoughts for further revision if required. > Few comments on 0001-Ord

Re: Questions/Observations related to Gist vacuum

2019-10-22 Thread Dilip Kumar
On Tue, Oct 22, 2019 at 10:53 AM Amit Kapila wrote: > > On Tue, Oct 22, 2019 at 10:50 AM Dilip Kumar wrote: > > > > On Tue, Oct 22, 2019 at 9:10 AM Amit Kapila wrote: > > > > > > On Fri, Oct 18, 2019 at 4:51 PM Dilip Kumar wrote: > > > > > > > > I have prepared a first version of the patch. Cu

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Jeevan Ladhe
> > > I haven't read the complete patch. But, I have noticed that many > places you changed the variable declaration from c to c++ style (i.e > moved the declaration in the for loop). IMHO, generally in PG, we > don't follow this convention. Is there any specific reason to do > this? > +1. The

v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Devrim Gündüz
Hi, When I run pg_basebackup in v12 against v11, standby server fails to connecto primary with the following error: 2019-10-22 09:28:23.673 UTC [2375] FATAL: could not connect to the primary server: invalid connection option "gssencmode" When I remove this from recovery.conf, it works fine. Lo

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Fabien COELHO
Hello Dilip, - for (i = 0; i < nbranches * scale; i++) + for (int i = 0; i < nbranches * scale; i++) ... - for (i = 0; i < ntellers * scale; i++) + for (int i = 0; i < ntellers * scale; i++) { I haven't read the complete patch. But, I have noticed that many places you changed the variable d

Re: Ordering of header file inclusion

2019-10-22 Thread Amit Kapila
On Tue, Oct 22, 2019 at 12:56 PM Amit Kapila wrote: > > On Mon, Oct 21, 2019 at 11:04 PM vignesh C wrote: > > > > On Mon, Oct 21, 2019 at 8:47 AM Amit Kapila wrote: > > > > > > > > Thanks for the suggestions. > > Updated patch contains the fix based on Tom Lane's Suggestion. > > Let me know your

Re: v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Michael Paquier
On Tue, Oct 22, 2019 at 12:32:53PM +0300, Devrim Gündüz wrote: > When I run pg_basebackup in v12 against v11, standby server fails to connecto > primary with the following error: > > 2019-10-22 09:28:23.673 UTC [2375] FATAL: could not connect to the primary > server: invalid connection option "gs

Re: v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Devrim Gündüz
Hi, On Tue, 2019-10-22 at 19:16 +0900, Michael Paquier wrote: > You are referring to the connection string generated in > primary_conninfo here, right? Right. > It would be nice to be more compatible here. This can be simply fixed by > having an extra filter in GenerateRecoveryConfig() (differ

Re: Proposal: Make use of C99 designated initialisers for nulls/values arrays

2019-10-22 Thread Amit Kapila
On Tue, Oct 22, 2019 at 12:35 AM Tom Lane wrote: > > Stephen Frost writes: > > * Joe Nelson (j...@begriffs.com) wrote: > >> If it's being put behind a macro then *stylistically* it shouldn't > >> matter whether {} or {0} is chosen, right? In which case {0} would > >> be a better choice because it

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Fabien COELHO
Hello Jeevan, I haven't read the complete patch. But, I have noticed that many places you changed the variable declaration from c to c++ style (i.e moved the declaration in the for loop). IMHO, generally in PG, we don't follow this convention. Is there any specific reason to do this? +1.

Re: dropdb --force

2019-10-22 Thread Pavel Stehule
út 22. 10. 2019 v 5:09 odesílatel Amit Kapila napsal: > On Mon, Oct 21, 2019 at 2:15 PM Pavel Stehule > wrote: > > > > po 21. 10. 2019 v 10:25 odesílatel Amit Kapila > napsal: > >> > >> > >> Sorry, but I am not able to understand the reason. Are you worried > >> about the comments atop CountOt

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Dilip Kumar
On Tue, Oct 22, 2019 at 3:30 PM Fabien COELHO wrote: > > > Hello Dilip, > > > - for (i = 0; i < nbranches * scale; i++) > > + for (int i = 0; i < nbranches * scale; i++) > > ... > > - for (i = 0; i < ntellers * scale; i++) > > + for (int i = 0; i < ntellers * scale; i++) > > { > > > > I haven't r

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Jeevan Ladhe
On Tue, Oct 22, 2019 at 4:36 PM Fabien COELHO wrote: > > Hello Jeevan, > > >> I haven't read the complete patch. But, I have noticed that many > >> places you changed the variable declaration from c to c++ style (i.e > >> moved the declaration in the for loop). IMHO, generally in PG, we > >> do

Re: intermittent test failure on Windows

2019-10-22 Thread Andrew Dunstan
On 10/21/19 2:58 PM, Tom Lane wrote: > Andrew Dunstan writes: >> Bowerbird (Visual Studio 2017 / Windows 10 pro) just had a failure on >> the pg_ctl test : >> >> I surmise that what's happening here

Re:Re: [BUG] standby node can not provide service even it replays all log files

2019-10-22 Thread Thunder
Update the patch. 1. The STANDBY_SNAPSHOT_PENDING state is set when we replay the first XLOG_RUNNING_XACTS and the sub transaction ids are overflow. 2. When we log XLOG_RUNNING_XACTS in master node, can we assume that all xact IDS < oldestRunningXid are considered finished? 3. If we can assume t

Re: v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Stephen Frost
Greetings, * Devrim Gündüz (dev...@gunduz.org) wrote: > On Tue, 2019-10-22 at 19:16 +0900, Michael Paquier wrote: > > You are referring to the connection string generated in > > primary_conninfo here, right? > > Right. I'm awful suspicious that there's other similar cases beyond this particular

Re: v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Michael Paquier
On Tue, Oct 22, 2019 at 09:06:03AM -0400, Stephen Frost wrote: > Here's just a quick rough-up of a patch (it compiles, I haven't tried it > out more than that) that adds in a check to skip gssencmode on older > versions. If it seems like a reasonable approach then I can test it out > and deal with

Re: v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Stephen Frost
Greetings, * Michael Paquier (mich...@paquier.xyz) wrote: > On Tue, Oct 22, 2019 at 09:06:03AM -0400, Stephen Frost wrote: > > Here's just a quick rough-up of a patch (it compiles, I haven't tried it > > out more than that) that adds in a check to skip gssencmode on older > > versions. If it seem

Re: pgbench - refactor init functions with buffers

2019-10-22 Thread Fabien COELHO
The patch does not apply on master, needs rebase. Hmmm. "git apply pgbench-buffer-1.patch" works for me on current master. Also, I got some whitespace errors. It possible, but I cannot see any. Could you be more specific? For me it failing, see below: $ git log -1 commit ad4b7aeb84434c9

Re: Ordering of header file inclusion

2019-10-22 Thread vignesh C
On Tue, Oct 22, 2019 at 12:56 PM Amit Kapila wrote: > Few comments on 0001-Ordering-of-header-files-in-contrib-dir-oct21.patch > --- > 1. > --- a/contrib/isn/isn.c > +++ b/contrib/isn/isn.c > @@

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-10-22 Thread Tomas Vondra
On Tue, Oct 22, 2019 at 10:30:16AM +0530, Dilip Kumar wrote: On Fri, Oct 18, 2019 at 5:32 PM Amit Kapila wrote: On Mon, Oct 14, 2019 at 3:09 PM Dilip Kumar wrote: > > On Thu, Oct 3, 2019 at 4:03 AM Tomas Vondra > wrote: > > > > > > Sure, I wasn't really proposing to adding all stats from tha

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-10-22 Thread Tomas Vondra
On Tue, Oct 22, 2019 at 11:01:48AM +0530, Dilip Kumar wrote: On Tue, Oct 22, 2019 at 10:46 AM Amit Kapila wrote: On Thu, Oct 3, 2019 at 1:18 PM Dilip Kumar wrote: > > I have attempted to test the performance of (Stream + Spill) vs > (Stream + BGW pool) and I can see the similar gain what Alex

Re: Ordering of header file inclusion

2019-10-22 Thread vignesh C
On Tue, Oct 22, 2019 at 3:42 PM Amit Kapila wrote: > Review for 0003-Ordering-of-header-files-remaining-dir-oct21 > - > 1. > --- a/src/bin/pg_basebackup/pg_recvlogical.c > +++ b/src/bin/pg_basebackup/pg_recvlog

Re: 回复:Bug about drop index concurrently

2019-10-22 Thread Tomas Vondra
On Mon, Oct 21, 2019 at 10:36:04AM +0800, 李杰(慎追) wrote: Thanks for the quick reply. And sorry I haven’t got back to you sooner . I have seen this backtrace in the core file, and I also looked at the bug in the standby because there is no lock in the drop index concurrently. I'm a bit confuse

Re: 回复:Bug about drop index concurrently

2019-10-22 Thread Tomas Vondra
On Tue, Oct 22, 2019 at 03:24:41PM +0800, 李杰(慎追) wrote: Hi all, I am sorry to bother you again. I want to consult again, about the last time I raised a bug about drop index, are you going to deal with it in the future? Is it to ban it or to propose a repair plan, what is your next plan? It d

Duplicate Workers entries in some EXPLAIN plans

2019-10-22 Thread Maciek Sakrejda
Hello, I originally reported this in pgsql-bugs [0], but there wasn't much feedback there, so moving the discussion here. When using JSON, YAML, or XML-format EXPLAIN on a plan that uses a parallelized sort, the Sort nodes list two different entries for "Workers", one for the sort-related info, an

Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions

2019-10-22 Thread Alexey Kondratov
On 22.10.2019 20:22, Tomas Vondra wrote: On Tue, Oct 22, 2019 at 11:01:48AM +0530, Dilip Kumar wrote: On Tue, Oct 22, 2019 at 10:46 AM Amit Kapila wrote:   In general, yours and Alexy's test results show that there is merit by having workers applying such transactions.   OTOH, as noted above

Re: Declaring a strict function returns not null / eval speed

2019-10-22 Thread Tom Lane
Andres Freund writes: > On 2019-10-20 10:27:19 -0400, Tom Lane wrote: >> "RETURNS NOT NULL", perhaps? That'd have the advantage of not requiring >> any new keyword. > That could work. Actually, I think we probably don't need any SQL representation of this at all, because if what you're going to

Re: Declaring a strict function returns not null / eval speed

2019-10-22 Thread Christoph Berg
Re: Tom Lane 2019-10-22 <821.1571771...@sss.pgh.pa.us> > Actually, I think we probably don't need any SQL representation of this > at all, because if what you're going to do with it is omit logically > necessary null-value checks, then a wrong setting would trivially crash > the server. Therefore,

Re: Declaring a strict function returns not null / eval speed

2019-10-22 Thread Andres Freund
Hi, On 2019-10-22 15:06:50 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2019-10-20 10:27:19 -0400, Tom Lane wrote: > >> "RETURNS NOT NULL", perhaps? That'd have the advantage of not requiring > >> any new keyword. > > > That could work. > > Actually, I think we probably don't need any

Re: stress test for parallel workers

2019-10-22 Thread Justin Pryzby
I want to give some conclusion to our occurance of this, which now I think was neither an instance nor indicitive of any bug. Summary: postgres was being kill -9 by a deployment script, after it "timed out". Thus no log messages. I initially experienced this while testing migration of a customer

Re: Bug about drop index concurrently

2019-10-22 Thread Tomas Vondra
On Fri, Oct 18, 2019 at 05:00:54PM +0200, Tomas Vondra wrote: Hi, I can trivially reproduce this - it's enough to create a master-standby setup, and then do this on the master CREATE TABLE t (a int, b int); INSERT INTO t SELECT i, i FROM generate_series(1,1) s(i); and run pgbench with this

Re: Missing error_context_stack = NULL in AutoVacWorkerMain()

2019-10-22 Thread Michael Paquier
On Mon, Oct 21, 2019 at 12:53:27AM -0400, Tom Lane wrote: > Oh --- looking closer, proc_exit itself will clear error_context_stack > before doing much. So a problem would only occur if we suffered an error > during EmitErrorReport, which seems somewhat unlikely. Still, it's bad > that this code i

Re: [PATCH] Do not use StdRdOptions in Access Methods

2019-10-22 Thread Amit Langote
Hi Nikolay, Sorry for the late reply. On Fri, Oct 11, 2019 at 7:54 PM Nikolay Shaplov wrote: > В письме от четверг, 10 октября 2019 г. 17:17:30 MSK пользователь Amit Langote > написал: > > I read comments that Tomas left at: > > https://www.postgresql.org/message-id/20190727173841.7ypzo4xuzizvij

Re: [PATCH] use separate PartitionedRelOptions structure to store partitioned table options

2019-10-22 Thread Amit Langote
Hi Nikolay, Sorry for the late reply. On Fri, Oct 11, 2019 at 7:38 PM Nikolay Shaplov wrote: > В Thu, 10 Oct 2019 15:50:05 +0900 > Amit Langote пишет: > > > I think it is bad idea to suggest option adder to ad it to > > > StdRdOption, we already have a big mess there. Better if he add it > > >

Re: Obsolete comment in partbounds.c

2019-10-22 Thread Amit Langote
On Mon, Oct 21, 2019 at 5:44 PM Etsuro Fujita wrote: > On Sat, Oct 19, 2019 at 5:56 PM Etsuro Fujita wrote: > > On Fri, Oct 18, 2019 at 6:56 PM Alvaro Herrera > > wrote: > > > On 2019-Oct-18, Etsuro Fujita wrote: > > > > While reviewing the partitionwise-join patch, I noticed $Subject,ie, > > >

Re: d25ea01275 and partitionwise join

2019-10-22 Thread Amit Langote
Hi Justin, On Mon, Oct 14, 2019 at 5:02 AM Justin Pryzby wrote: > > On Thu, Sep 19, 2019 at 05:15:37PM +0900, Amit Langote wrote: > > Please find attached updated patches. > > Tom pointed me to this thread, since we hit it in 12.0 > https://www.postgresql.org/message-id/flat/16802.1570989962%40ss

Re: Remove obsolete options for createuser

2019-10-22 Thread Michael Paquier
On Mon, Oct 21, 2019 at 01:33:08PM +0900, Michael Paquier wrote: > The commit in question is 8ae0d47 from 2005. So let's remove it. It > is not even documented for ages. > > Perhaps somebody thinks it is not a good idea? Done. A similar move could be done for --encrypted which has been made a

Re: [PATCH] Do not use StdRdOptions in Access Methods

2019-10-22 Thread Michael Paquier
On Wed, Oct 23, 2019 at 11:16:25AM +0900, Amit Langote wrote: > IMO, parts of the patch that only refactors the existing code should > be first in the list as it is easier to review, especially if it adds > no new concepts. In this case, your patch to break StdRdOptions into > more manageable chun

Re: [BUG] standby node can not provide service even it replays all log files

2019-10-22 Thread Kyotaro Horiguchi
Hello. At Tue, 22 Oct 2019 20:42:21 +0800 (CST), Thunder wrote in > Update the patch. > > 1. The STANDBY_SNAPSHOT_PENDING state is set when we replay the first > XLOG_RUNNING_XACTS and the sub transaction ids are overflow. > 2. When we log XLOG_RUNNING_XACTS in master node, can we assume that

Re: [BUG] standby node can not provide service even it replays all log files

2019-10-22 Thread Kyotaro Horiguchi
At Wed, 23 Oct 2019 12:51:19 +0900 (JST), Kyotaro Horiguchi wrote in > Hello. > > At Tue, 22 Oct 2019 20:42:21 +0800 (CST), Thunder wrote > in > > Update the patch. > > > > 1. The STANDBY_SNAPSHOT_PENDING state is set when we replay the first > > XLOG_RUNNING_XACTS and the sub transaction

Re: Ordering of header file inclusion

2019-10-22 Thread vignesh C
On Tue, Oct 22, 2019 at 11:05 PM vignesh C wrote: > > On Tue, Oct 22, 2019 at 3:42 PM Amit Kapila wrote: > > Review for 0003-Ordering-of-header-files-remaining-dir-oct21 > > - > > 1. > > --- a/src/bin/pg_baseb

Re: Ordering of header file inclusion

2019-10-22 Thread Amit Kapila
On Tue, Oct 22, 2019 at 3:41 PM Amit Kapila wrote: > > On Tue, Oct 22, 2019 at 12:56 PM Amit Kapila wrote: > > > > On Mon, Oct 21, 2019 at 11:04 PM vignesh C wrote: > > > > > > On Mon, Oct 21, 2019 at 8:47 AM Amit Kapila > > > wrote: > > This patch series has broadly changed the code to organi

Re: Clean up MinGW def file generation

2019-10-22 Thread Peter Eisentraut
On 2019-10-21 00:07, Peter Eisentraut wrote: > On 2019-10-20 10:26, Peter Eisentraut wrote: >> On 2019-10-18 15:00, Tom Lane wrote: >>> Yeah, the comment that Peter complained about is mine. I believe the >>> desire to avoid depending on "sed" at build time was focused on our >>> old support for b

Creating foreign key on partitioned table is too slow

2019-10-22 Thread kato-...@fujitsu.com
Hello To benchmark with tpcb model, I tried to create a foreign key in the partitioned history table, but backend process killed by OOM. the number of partitions is 8192. I tried in master(commit: ad4b7aeb84). I did the same thing in another server which has 200GB memory, but creating foreign k

Re: configure fails for perl check on CentOS8

2019-10-22 Thread Kyotaro Horiguchi
Hello. At Mon, 21 Oct 2019 08:29:39 -0400, Andrew Dunstan wrote in > > On 10/20/19 7:36 PM, Tom Lane wrote: > > Andrew Dunstan writes: > >> On 10/20/19 1:23 PM, Tom Lane wrote: > >>> The right way to fix it, likely, is to add CFLAGS_SL while performing this > >>> particular autoconf test, as

Re: v12 pg_basebackup fails against older servers (take two)

2019-10-22 Thread Michael Paquier
On Tue, Oct 22, 2019 at 09:53:45AM -0400, Stephen Frost wrote: > Yeah.. Something along those lines definitely seems like it'd be better > as that would force anyone adding new options to explicitly say which > server version the option makes sense for. Would it make sense to have a > minimum and

回复:回复:Bug about drop index concurrently

2019-10-22 Thread 李杰(慎追)
> >I'm a bit confused. You shouldn't see any crashes and/or core files in >this scenario, for two reasons. Firstly, I assume you're running a >regular build without asserts. Secondly, I had to add an extra assert to >trigger the failure. So what core are you talking about? > Sorry, I should explain

回复:Bug about drop index concurrently

2019-10-22 Thread 李杰(慎追)
Ah ha ha , this is great, I am very ashamed of my English expression, did not let you clearly understand my mail. now, I am very glad that you can understand this. I sincerely hope that I can help you. I am also a postgres fan, a freshly graduated student. We have all confirmed that this bug wi