Re: should INSERT SELECT use a BulkInsertState?

2021-11-04 Thread Daniel Gustafsson
> On 27 Sep 2021, at 02:08, Justin Pryzby wrote: > I split off the simple part again. If there's no interest in the 0001 patch > alone, then I guess it should be closed in the CF. Since the thread has stalled, maybe that's the best course of action here. Any objections from anyone on the threa

Re: should INSERT SELECT use a BulkInsertState?

2021-09-26 Thread Justin Pryzby
On Mon, May 11, 2020 at 03:19:34PM +0900, Michael Paquier wrote: > On Fri, May 08, 2020 at 02:25:45AM -0500, Justin Pryzby wrote: > > Seems to me it should, at least conditionally. At least if there's a > > function > > scan or a relation or .. > > > > I mentioned a bit about our use-case here: >

RE: should INSERT SELECT use a BulkInsertState?

2021-03-22 Thread houzj.f...@fujitsu.com
Hi, About the 0002-patch [Check for volatile defaults]. I wonder if we can check the volatile default value by traversing "query->targetList" in planner. IMO, the column default expression was written into the targetList, and the current parallel-safety check travere the "query->targetList" to

Re: should INSERT SELECT use a BulkInsertState?

2021-03-18 Thread Zhihong Yu
Hi, + mtstate->ntuples > bulk_insert_ntuples && + bulk_insert_ntuples >= 0) I wonder why bulk_insert_ntuples == 0 is included in the above. It seems bulk_insert_ntuples having value of 0 should mean not enabling bulk insertions. + } + else + { nit: the else should be on

Re: should INSERT SELECT use a BulkInsertState?

2021-03-18 Thread Ibrar Ahmed
On Mon, Mar 8, 2021 at 2:18 PM houzj.f...@fujitsu.com < houzj.f...@fujitsu.com> wrote: > > > > I am very interested in this patch, and I plan to do some > > > > experiments with the > > > patch. > > > > Can you please rebase the patch because it seems can not applied to > > > > the > > > master no

RE: should INSERT SELECT use a BulkInsertState?

2021-03-08 Thread houzj.f...@fujitsu.com
> > > I am very interested in this patch, and I plan to do some > > > experiments with the > > patch. > > > Can you please rebase the patch because it seems can not applied to > > > the > > master now. > > > > Thanks for your interest. > > > > I was sitting on a rebased version since the bulk FDW p

RE: should INSERT SELECT use a BulkInsertState?

2021-03-08 Thread houzj.f...@fujitsu.com
> > I am very interested in this patch, and I plan to do some experiments with > > the > patch. > > Can you please rebase the patch because it seems can not applied to the > master now. > > Thanks for your interest. > > I was sitting on a rebased version since the bulk FDW patch will cause > co

Re: should INSERT SELECT use a BulkInsertState?

2021-02-21 Thread Justin Pryzby
On Mon, Feb 22, 2021 at 02:25:22AM +, houzj.f...@fujitsu.com wrote: > > > Yes, you can see that I've copied the checks from copy. > > > Like copy, some checks are done once, in ExecInitModifyTable, outside > > > of the ExecModifyTable "loop". > > > > > > This squishes some commits together. > >

Re: should INSERT SELECT use a BulkInsertState?

2021-01-20 Thread Justin Pryzby
On Sat, Dec 05, 2020 at 01:59:41PM -0600, Justin Pryzby wrote: > On Thu, Dec 03, 2020 at 10:59:34AM +0530, Bharath Rupireddy wrote: > > On Wed, Dec 2, 2020 at 10:24 PM Justin Pryzby wrote: > > > > > > One loose end in this patch is how to check for volatile default > > > expressions. > > > > I t

Re: should INSERT SELECT use a BulkInsertState?

2020-12-05 Thread Justin Pryzby
On Thu, Dec 03, 2020 at 10:59:34AM +0530, Bharath Rupireddy wrote: > On Wed, Dec 2, 2020 at 10:24 PM Justin Pryzby wrote: > > > > One loose end in this patch is how to check for volatile default > > expressions. > > I think we should be doing all the necessary checks in the planner and > have a

Re: should INSERT SELECT use a BulkInsertState?

2020-12-02 Thread Bharath Rupireddy
On Wed, Dec 2, 2020 at 10:24 PM Justin Pryzby wrote: > > One loose end in this patch is how to check for volatile default expressions. > > copyfrom.c is a utility statement, so it can look at the parser's column list: > COPY table(c1,c2)... > > However, for INSERT, in nodeModifyTable.c, it looks l

Re: should INSERT SELECT use a BulkInsertState?

2020-12-02 Thread Justin Pryzby
One loose end in this patch is how to check for volatile default expressions. copyfrom.c is a utility statement, so it can look at the parser's column list: COPY table(c1,c2)... However, for INSERT, in nodeModifyTable.c, it looks like parsing, rewriting, and planning are done, at which point I do

Re: should INSERT SELECT use a BulkInsertState?

2020-11-29 Thread Justin Pryzby
On Mon, Nov 23, 2020 at 08:00:20PM -0600, Justin Pryzby wrote: > On Mon, Nov 02, 2020 at 12:45:51PM -0600, Justin Pryzby wrote: > > On Mon, Nov 02, 2020 at 07:53:45AM +0100, Luc Vlaming wrote: > > > On 30.10.20 05:51, Justin Pryzby wrote: > > > > On Thu, Oct 22, 2020 at 01:29:53PM +0100, Simon Rigg

Re: should INSERT SELECT use a BulkInsertState?

2020-11-23 Thread Justin Pryzby
On Mon, Nov 02, 2020 at 12:45:51PM -0600, Justin Pryzby wrote: > On Mon, Nov 02, 2020 at 07:53:45AM +0100, Luc Vlaming wrote: > > On 30.10.20 05:51, Justin Pryzby wrote: > > > On Thu, Oct 22, 2020 at 01:29:53PM +0100, Simon Riggs wrote: > > > > On Fri, 16 Oct 2020 at 22:05, Justin Pryzby > > > >

Re: should INSERT SELECT use a BulkInsertState?

2020-11-02 Thread Justin Pryzby
On Mon, Nov 02, 2020 at 07:53:45AM +0100, Luc Vlaming wrote: > On 30.10.20 05:51, Justin Pryzby wrote: > > On Thu, Oct 22, 2020 at 01:29:53PM +0100, Simon Riggs wrote: > > > On Fri, 16 Oct 2020 at 22:05, Justin Pryzby wrote: > > > > > > > > > I made this conditional on BEGIN BULK/SET bulk, so I'l

Re: should INSERT SELECT use a BulkInsertState?

2020-11-01 Thread Luc Vlaming
On 30.10.20 05:51, Justin Pryzby wrote: On Thu, Oct 22, 2020 at 01:29:53PM +0100, Simon Riggs wrote: On Fri, 16 Oct 2020 at 22:05, Justin Pryzby wrote: I made this conditional on BEGIN BULK/SET bulk, so I'll solicit comments on that. I think it would be better if this was self-tuning. So t

Re: should INSERT SELECT use a BulkInsertState?

2020-10-29 Thread Justin Pryzby
On Thu, Oct 22, 2020 at 01:29:53PM +0100, Simon Riggs wrote: > On Fri, 16 Oct 2020 at 22:05, Justin Pryzby wrote: > > > > > I made this conditional on BEGIN BULK/SET bulk, so I'll solicit > > > > comments on that. > > I think it would be better if this was self-tuning. So that we don't > alloca

Re: should INSERT SELECT use a BulkInsertState?

2020-10-22 Thread Simon Riggs
On Fri, 16 Oct 2020 at 22:05, Justin Pryzby wrote: > > > I made this conditional on BEGIN BULK/SET bulk, so I'll solicit comments > > > on that. I think it would be better if this was self-tuning. So that we don't allocate a bulkinsert state until we've done say 100 (?) rows inserted. If there

Re: should INSERT SELECT use a BulkInsertState?

2020-10-22 Thread Simon Riggs
On Thu, 4 Jun 2020 at 18:31, Andres Freund wrote: > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > > Seems to me it should, at least conditionally. At least if there's a > > function > > scan or a relation or .. > > Well, the problem is that this can cause very very significant > regressi

Re: should INSERT SELECT use a BulkInsertState?

2020-10-16 Thread Justin Pryzby
On Sat, Sep 19, 2020 at 08:32:15AM -0500, Justin Pryzby wrote: > On Sun, Jul 12, 2020 at 08:57:00PM -0500, Justin Pryzby wrote: > > On Thu, Jun 04, 2020 at 10:30:47AM -0700, Andres Freund wrote: > > > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > > > > Seems to me it should, at least conditi

Re: should INSERT SELECT use a BulkInsertState?

2020-09-19 Thread Justin Pryzby
On Sun, Jul 12, 2020 at 08:57:00PM -0500, Justin Pryzby wrote: > On Thu, Jun 04, 2020 at 10:30:47AM -0700, Andres Freund wrote: > > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > > > Seems to me it should, at least conditionally. At least if there's a > > > function > > > scan or a relation

Re: should INSERT SELECT use a BulkInsertState?

2020-07-12 Thread Justin Pryzby
On Thu, Jun 04, 2020 at 10:30:47AM -0700, Andres Freund wrote: > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > > Seems to me it should, at least conditionally. At least if there's a > > function > > scan or a relation or .. > > Well, the problem is that this can cause very very significan

Re: should INSERT SELECT use a BulkInsertState?

2020-07-12 Thread Daniel Gustafsson
> On 4 Jun 2020, at 19:30, Andres Freund wrote: > On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: >> Seems to me it should, at least conditionally. At least if there's a >> function >> scan or a relation or .. > > Well, the problem is that this can cause very very significant > regressions.

Re: should INSERT SELECT use a BulkInsertState?

2020-06-04 Thread Andres Freund
Hi, On 2020-05-08 02:25:45 -0500, Justin Pryzby wrote: > Seems to me it should, at least conditionally. At least if there's a function > scan or a relation or .. Well, the problem is that this can cause very very significant regressions. As in 10x slower or more. The ringbuffer can cause constan

Re: should INSERT SELECT use a BulkInsertState?

2020-05-10 Thread Michael Paquier
On Fri, May 08, 2020 at 02:25:45AM -0500, Justin Pryzby wrote: > Seems to me it should, at least conditionally. At least if there's a function > scan or a relation or .. > > I mentioned a bit about our use-case here: > https://www.postgresql.org/message-id/20200219173742.GA30939%40telsasoft.com >

Re: should INSERT SELECT use a BulkInsertState?

2020-05-10 Thread Justin Pryzby
On Fri, May 08, 2020 at 02:25:45AM -0500, Justin Pryzby wrote: > Seems to me it should, at least conditionally. At least if there's a function > scan or a relation or .. > > I mentioned a bit about our use-case here: > https://www.postgresql.org/message-id/20200219173742.GA30939%40telsasoft.com >