Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Dec 22, 2015 at 2:00 AM, Michael Paquier wrote: > On Tue, Dec 22, 2015 at 3:52 PM, Amit Langote > wrote: >> On 2015/12/22 15:24, Michael Paquier wrote: >>> As this debate continues, I think that moving this patch to the next >>>

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/22 15:24, Michael Paquier wrote: > On Thu, Dec 10, 2015 at 1:32 AM, Robert Haas wrote: >> If we get the feature - join pushdown for postgres_fdw - >> working, then we might get some feedback from users about what they >> like about it or don't, and certainly if

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Thu, Dec 10, 2015 at 1:32 AM, Robert Haas wrote: > On Wed, Dec 9, 2015 at 3:22 AM, Etsuro Fujita > wrote: >> Sorry, my explanation might be not enough, but I'm not saying to hide the >> subplan. I think it would be better to show the

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Dec 22, 2015 at 3:52 PM, Amit Langote wrote: > On 2015/12/22 15:24, Michael Paquier wrote: >> As this debate continues, I think that moving this patch to the next >> CF would make the most sense then.. So done this way. > > Perhaps, this ended (?) with the

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/09 13:26, Kouhei Kaigai wrote: On Tue, Dec 8, 2015 at 10:00 PM, Etsuro Fujita wrote: I think the actual regression test outputs are fine, and that your desire to suppress part of the plan tree from showing up in the EXPLAIN output is misguided. I like

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Wed, Dec 9, 2015 at 3:22 AM, Etsuro Fujita wrote: > Sorry, my explanation might be not enough, but I'm not saying to hide the > subplan. I think it would be better to show the subplan somewhere in the > EXPLAIN outout, but I'm not sure that it's a good idea to

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Dec 8, 2015 at 5:49 AM, Etsuro Fujita wrote: > On 2015/12/08 3:06, Tom Lane wrote: >> Robert Haas writes: >>> I think the core system likely needs visibility into where paths and >>> plans are present in node trees, and putting them

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Dec 8, 2015 at 10:00 PM, Etsuro Fujita wrote: >> I think the actual regression test outputs are fine, and that your >> desire to suppress part of the plan tree from showing up in the >> EXPLAIN output is misguided. I like it just the way it is. To >> prevent

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Tue, Dec 8, 2015 at 10:00 PM, Etsuro Fujita > wrote: > >> I think the actual regression test outputs are fine, and that your > >> desire to suppress part of the plan tree from showing up in the > >> EXPLAIN output is misguided. I like it just the way it is. To

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/09 1:13, Robert Haas wrote: On Tue, Dec 8, 2015 at 5:49 AM, Etsuro Fujita wrote: I'd like to discuss the next thing about his patch. As I mentioned in [1], the following change in the patch will break the EXPLAIN output. @@ -205,6 +218,11 @@

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Mon, Dec 7, 2015 at 12:25 AM, Etsuro Fujita wrote: >> Instead, I think we should go the opposite direction and pass the >> outerplan to GetForeignPlan after all. I was lulled into a full sense >> of security by the realization that every FDW that uses this feature

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Robert Haas writes: > I think the core system likely needs visibility into where paths and > plans are present in node trees, and putting them somewhere inside > fdw_private would be going in the opposite direction. Absolutely. You don't really want FDWs having to take

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/05 5:15, Robert Haas wrote: On Tue, Dec 1, 2015 at 10:20 PM, Etsuro Fujita wrote: One thing I can think of is that we can keep both the structure of a ForeignPath node and the API of create_foreignscan_path as-is. The latter is a good thing for FDW

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Dec 1, 2015 at 10:20 PM, Etsuro Fujita wrote: > One thing I can think of is that we can keep both the structure of a > ForeignPath node and the API of create_foreignscan_path as-is. The latter > is a good thing for FDW authors. And IIUC the patch you posted

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/02 1:54, Robert Haas wrote: On Fri, Nov 27, 2015 at 1:25 AM, Kouhei Kaigai wrote: Sorry, I don't understand this. In my understanding, fdw_recheck_quals can be defined for a foreign join, regardless of the join type, Yes, "can be defined", but will not be

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/02 14:54, Kouhei Kaigai wrote: On 2015/12/02 1:41, Robert Haas wrote: On Thu, Nov 26, 2015 at 7:59 AM, Etsuro Fujita wrote: The attached patch adds: Path *fdw_outerpath field to ForeignPath node. FDW driver can set arbitrary but one path-node here.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/02 1:53, Robert Haas wrote: On Fri, Nov 27, 2015 at 1:33 AM, Etsuro Fujita wrote: Plan *plan = >scan.plan; @@ -3755,7 +3763,7 @@ make_foreignscan(List *qptlist, /* cost will be filled in by create_foreignscan_plan */

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hello, thank you for taking time for this. At Tue, 1 Dec 2015 14:56:54 -0500, Robert Haas wrote in > On Thu, Nov 26, 2015 at 12:04 AM, Kouhei Kaigai wrote: > > This patch is not

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Sorry, I made a mistake. At Wed, 02 Dec 2015 10:29:17 +0900 (Tokyo Standard Time), Kyotaro HORIGUCHI wrote in <20151202.102917.50152198.horiguchi.kyot...@lab.ntt.co.jp> > Hello, thank you for editing. > > At Tue, 1 Dec 2015 14:56:54 -0500, Robert Haas

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/12/02 1:41, Robert Haas wrote: On Thu, Nov 26, 2015 at 7:59 AM, Etsuro Fujita wrote: The attached patch adds: Path *fdw_outerpath field to ForeignPath node. FDW driver can set arbitrary but one path-node here. After that, this path-node shall be transformed

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Fri, Nov 27, 2015 at 1:33 AM, Etsuro Fujita wrote: Plan *plan = >scan.plan; > @@ -3755,7 +3763,7 @@ make_foreignscan(List *qptlist, > /* cost will be filled in by create_foreignscan_plan */ > plan->targetlist = qptlist; >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Fri, Nov 27, 2015 at 1:25 AM, Kouhei Kaigai wrote: >> Sorry, I don't understand this. In my understanding, fdw_recheck_quals >> can be defined for a foreign join, regardless of the join type, >> > Yes, "can be defined", but will not be workable if either side of > joined

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Robert Haas writes: > On Fri, Nov 27, 2015 at 1:33 AM, Etsuro Fujita >> One option to avoid that >> is to set the fdw_outerplan in ExecInitForeignScan as in my patch [1], or >> BeginForeignScan as you proposed. That breaks the equivalence that the Plan >> tree and the

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Thu, Nov 26, 2015 at 12:04 AM, Kouhei Kaigai wrote: > > The attached patch adds: Path *fdw_outerpath field to ForeignPath node. > > FDW driver can set arbitrary but one path-node here. > > After that, this path-node shall be transformed to plan-node by > > createplan.c,

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/12/02 1:41, Robert Haas wrote: > > On Thu, Nov 26, 2015 at 7:59 AM, Etsuro Fujita > > wrote: > >>> The attached patch adds: Path *fdw_outerpath field to ForeignPath node. > >>> FDW driver can set arbitrary but one path-node here. > >>> After that, this

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Thu, Nov 26, 2015 at 12:04 AM, Kouhei Kaigai wrote: > The attached patch adds: Path *fdw_outerpath field to ForeignPath node. > FDW driver can set arbitrary but one path-node here. > After that, this path-node shall be transformed to plan-node by > createplan.c, then

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Thu, Nov 26, 2015 at 7:59 AM, Etsuro Fujita wrote: >> The attached patch adds: Path *fdw_outerpath field to ForeignPath node. >> FDW driver can set arbitrary but one path-node here. >> After that, this path-node shall be transformed to plan-node by >> createplan.c,

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/27 0:14, Kouhei Kaigai wrote: >> The documentation says as following so I think the former has. >> >> # I don't understhad what 'can or must' means, though... 'can and >> # must'? >> >> + Also, this callback can or must recheck scan qualifiers and join >> + conditions which are

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/27 0:14, Kouhei Kaigai wrote: On 2015/11/26 14:04, Kouhei Kaigai wrote: The attached patch adds: Path *fdw_outerpath field to ForeignPath node. FDW driver can set arbitrary but one path-node here. After that, this path-node shall be transformed to plan-node by createplan.c, then

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

.com > Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual > > On 2015/11/27 0:14, Kouhei Kaigai wrote: > > >> The documentation says as following so I think the former has. > >> > >> # I don't understhad what 'can or must' means, though... 'can

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/26 14:04, Kouhei Kaigai wrote: On 2015/11/24 2:41, Robert Haas wrote: On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai wrote: One subplan means FDW driver run an entire join sub-tree with local alternative sub-plan; that is my expectation for the majority case.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/11/26 14:04, Kouhei Kaigai wrote: > >> On 2015/11/24 2:41, Robert Haas wrote: > >>> On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai > >>> wrote: > One subplan means FDW driver run an entire join sub-tree with local > alternative sub-plan; that is my

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> At Thu, 26 Nov 2015 05:04:32 +, Kouhei Kaigai wrote > in <9a28c8860f777e439aa12e8aea7694f801176...@bpxm15gp.gisp.nec.co.jp> > > > On 2015/11/24 2:41, Robert Haas wrote: > > > > On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai > > > > wrote: > > >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Tuesday, November 24, 2015 12:45 PM > To: Robert Haas; Kaigai Kouhei(海外 浩平) > Cc: Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers@postgresql.org; Shigeru Hanada > Subject: Re: [HACKERS] For

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hello, At Thu, 26 Nov 2015 05:04:32 +, Kouhei Kaigai wrote in <9a28c8860f777e439aa12e8aea7694f801176...@bpxm15gp.gisp.nec.co.jp> > > On 2015/11/24 2:41, Robert Haas wrote: > > > On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai > > > wrote: > > >>

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai wrote: > >> On Thu, Nov 19, 2015 at 6:39 AM, Kouhei Kaigai > >> wrote: > >> > So, are you suggesting to make a patch that allows ForeignScan to have > >> > multiple sub-plans right now? Or, one

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/24 2:41, Robert Haas wrote: On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai wrote: One subplan means FDW driver run an entire join sub-tree with local alternative sub-plan; that is my expectation for the majority case. What I'm imagining is that we'd add

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/20 22:45, Kouhei Kaigai wrote: I wrote: * This patch means we can define fdw_recheck_quals even for the case of foreign tables with non-NIL fdw_scan_tlist. However, we discussed in another thread [1] that such foreign tables might break EvalPlanQual tests. Where are we on that

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/09 9:26, Kouhei Kaigai wrote: The attached patch is an adjusted version of the previous one. There seems to be no changes to make_foreignscan. Is that OK? Best regards, Etsuro Fujita -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Fri, Nov 20, 2015 at 12:11 AM, Kouhei Kaigai wrote: >> On Thu, Nov 19, 2015 at 6:39 AM, Kouhei Kaigai wrote: >> > So, are you suggesting to make a patch that allows ForeignScan to have >> > multiple sub-plans right now? Or, one sub-plan? >> >> Two:

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/11/19 12:32, Robert Haas wrote: > > On Tue, Nov 17, 2015 at 8:47 PM, Kouhei Kaigai wrote: > >> The attached patch is the portion cut from the previous EPQ recheck > >> patch. > > > Thanks, committed. > > Thanks, Robert and KaiGai-san. > > Sorry, I'm a bit late

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/19 12:32, Robert Haas wrote: On Tue, Nov 17, 2015 at 8:47 PM, Kouhei Kaigai wrote: The attached patch is the portion cut from the previous EPQ recheck patch. Thanks, committed. Thanks, Robert and KaiGai-san. Sorry, I'm a bit late to the party. Here are

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/20 6:57, Robert Haas wrote: On Wed, Nov 18, 2015 at 10:54 PM, Etsuro Fujita wrote: Noted, but let's do it that way and move on. It would be a shame if we didn't end up with a working FDW join pushdown system in 9.6 because of a disagreement on this

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Wed, Nov 18, 2015 at 10:54 PM, Etsuro Fujita wrote: >> Noted, but let's do it that way and move on. It would be a shame if >> we didn't end up with a working FDW join pushdown system in 9.6 >> because of a disagreement on this point. > > Another idea would be to

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Thu, Nov 19, 2015 at 6:39 AM, Kouhei Kaigai wrote: > So, are you suggesting to make a patch that allows ForeignScan to have > multiple sub-plans right now? Or, one sub-plan? Two:

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Tue, Nov 17, 2015 at 10:22 PM, Kouhei Kaigai wrote: > > Apart from EPQ rechecks, the above aggressive push-down idea allows to send > > contents of multiple relations to the remote side. In this case, ForeignScan > > needs to have multiple sub-plans. > > > > For

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Thu, Nov 19, 2015 at 6:39 AM, Kouhei Kaigai wrote: > > So, are you suggesting to make a patch that allows ForeignScan to have > > multiple sub-plans right now? Or, one sub-plan? > > Two: > > http://www.postgresql.org/message-id/CA+TgmoYZeje+ot1kX4wdoB7R7DPS0CWXAzfqZ-

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/19 12:34, Robert Haas wrote: On Tue, Nov 17, 2015 at 9:30 PM, Etsuro Fujita wrote: I suppose you (and KaiGai-san) are probably right, but I really fail to see it actually doing that. Noted, but let's do it that way and move on. It would be a shame if

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Nov 17, 2015 at 9:30 PM, Etsuro Fujita wrote: > I suppose you (and KaiGai-san) are probably right, but I really fail to see > it actually doing that. Noted, but let's do it that way and move on. It would be a shame if we didn't end up with a working FDW join

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Nov 17, 2015 at 8:47 PM, Kouhei Kaigai wrote: > The attached patch is the portion cut from the previous EPQ recheck > patch. Thanks, committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Nov 17, 2015 at 10:22 PM, Kouhei Kaigai wrote: > Apart from EPQ rechecks, the above aggressive push-down idea allows to send > contents of multiple relations to the remote side. In this case, ForeignScan > needs to have multiple sub-plans. > > For example, please

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Thu, Nov 12, 2015 at 12:54 AM, Etsuro Fujita wrote: > Really? I think there would be not a little burden on an FDW author; when > postgres_fdw delegates to the subplan to the remote server, for example, it > would need to create a remote join query by looking at

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Sun, Nov 8, 2015 at 7:26 PM, Kouhei Kaigai wrote: > The attached patch is an adjusted version of the previous one. > Even though it co-exists a new callback and fdw_recheck_quals, > the callback is kicked first as follows. This seems excessive to me: why would we need an

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Sun, Nov 8, 2015 at 7:26 PM, Kouhei Kaigai wrote: > > The attached patch is an adjusted version of the previous one. > > Even though it co-exists a new callback and fdw_recheck_quals, > > the callback is kicked first as follows. > > This seems excessive to me: why

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> > On Sun, Nov 8, 2015 at 7:26 PM, Kouhei Kaigai wrote: > > > The attached patch is an adjusted version of the previous one. > > > Even though it co-exists a new callback and fdw_recheck_quals, > > > the callback is kicked first as follows. > > > > This seems excessive to

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/18 3:19, Robert Haas wrote: On Thu, Nov 12, 2015 at 12:54 AM, Etsuro Fujita wrote: Really? I think there would be not a little burden on an FDW author; when postgres_fdw delegates to the subplan to the remote server, for example, it would need to create

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Nov 17, 2015 at 6:51 PM, Kouhei Kaigai wrote: > It just intends to keep code symmetry with custom-scan case, so not > a significant reason. > And, I expected ForeignScan will also need multiple sub-plans soon > to support more intelligent push-down like: >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Tue, Nov 17, 2015 at 6:51 PM, Kouhei Kaigai wrote: > > It just intends to keep code symmetry with custom-scan case, so not > > a significant reason. > > And, I expected ForeignScan will also need multiple sub-plans soon > > to support more intelligent push-down like: >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/13 11:31, Kouhei Kaigai wrote: On 2015/11/12 2:53, Robert Haas wrote: From my point of view I'm now thinking this solution has two parts: (1) Let foreign scans have inner and outer subplans. For this purpose, we only need one, but it's no more work to enable both, so we may as

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/13 13:44, Kyotaro HORIGUCHI wrote: I wrote: What I think is, I see zero evidence that there is a good use-case for an FDW to do something other than doing an ExecProcNode in the callback routine, as I said below, so I don't see the need to add such a routine while that would cause

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/11/13 13:44, Kyotaro HORIGUCHI wrote: > > I wrote: > >>> What I think is, I > >>> see zero evidence that there is a good use-case for an FDW to do > >>> something other than doing an ExecProcNode in the callback routine, as I > >>> said below, so I don't see the need to add such a

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Robert and Kaigai-san, Sorry, I sent in an unfinished email. On 2015/11/12 15:30, Kouhei Kaigai wrote: On 2015/11/12 2:53, Robert Haas wrote: On Sun, Nov 8, 2015 at 11:13 PM, Etsuro Fujita wrote: To test this change, I think we should update the postgres_fdw

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Horiguchi-san, On 2015/11/12 16:10, Kyotaro HORIGUCHI wrote: I really don't see why you're fighting on this point. Making this a generic feature will require only a few extra lines of code for FDW authors. If this were going to cause some great inconvenience for FDW authors, then I'd agree it

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hello, I also uncertain about what exactly is the blocker.. At Fri, 13 Nov 2015 02:31:53 +, Kouhei Kaigai wrote in <9a28c8860f777e439aa12e8aea7694f80116f...@bpxm15gp.gisp.nec.co.jp> > > Sorry, my explanation was not enough. The reason for that is that in > > the above

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Thursday, November 12, 2015 6:54 PM > To: Kaigai Kouhei(海外 浩平); Robert Haas > Cc: Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers@postgresql.org; Shigeru Hanada > Subject: Re: [HACKERS] For

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Sun, Nov 8, 2015 at 11:13 PM, Etsuro Fujita wrote: > To test this change, I think we should update the postgres_fdw patch so as > to add the RecheckForeignScan. > > Having said that, as I said previously, I don't see much value in adding the > callback routine, to

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/12 2:53, Robert Haas wrote: On Sun, Nov 8, 2015 at 11:13 PM, Etsuro Fujita wrote: To test this change, I think we should update the postgres_fdw patch so as to add the RecheckForeignScan. Having said that, as I said previously, I don't see much value in

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hello, > > I really don't see why you're fighting on this point. Making this a > > generic feature will require only a few extra lines of code for FDW > > authors. If this were going to cause some great inconvenience for FDW > > authors, then I'd agree it isn't worth it. But I see zero

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Thursday, November 12, 2015 2:54 PM > To: Robert Haas > Cc: Kaigai Kouhei(海外 浩平); Tom Lane; Kyotaro HORIGUCHI; > pgsql-hackers@postgresql.org; Shigeru Hanada > Subject: Re: [

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/09 13:40, Kouhei Kaigai wrote: Having said that, as I said previously, I don't see much value in adding the callback routine, to be honest. I know KaiGai-san considers that that would be useful for custom joins, but I don't think that that would be useful even for foreign joins,

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/11/09 13:40, Kouhei Kaigai wrote: > >> Having said that, as I said previously, I don't see much value in adding > >> the callback routine, to be honest. I know KaiGai-san considers that > >> that would be useful for custom joins, but I don't think that that would > >> be useful even for

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Kaigai Kouhei(海外 浩平) > Sent: Sunday, November 08, 2015 12:38 AM > To: 'Robert Haas' > Cc: Etsuro Fujita; Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers@postgresql.org; > Shigeru Hanada > Subject: Re: [HACKERS] Foreign join pushdown vs EvalP

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> > > > @@ -85,6 +86,18 @@ ForeignRecheck(ForeignScanState *node, TupleTableSlot > *slot) > > > > ResetExprContext(econtext); > > > > + /* > > +* FDW driver has to recheck visibility of EPQ tuple towards > > +* the scan qualifiers once it gets pushed

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/09 9:26, Kouhei Kaigai wrote: I think ForeignRecheck should first call ExecQual to test fdw_recheck_quals. If it returns false, return false. If it returns true, then give the FDW callback a chance, if one is defined. If that returns false, return false. If we haven't yet

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Fri, Nov 6, 2015 at 9:42 AM, Kouhei Kaigai wrote: > > This patch needs to be rebased. > > One thing different from the latest version is fdw_recheck_quals of > > ForeignScan was added. So, ... > > > > (1) Principle is that FDW driver knows what qualifiers were pushed

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Fri, Nov 6, 2015 at 9:42 AM, Kouhei Kaigai wrote: > This patch needs to be rebased. > One thing different from the latest version is fdw_recheck_quals of > ForeignScan was added. So, ... > > (1) Principle is that FDW driver knows what qualifiers were pushed down > and how

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

ers@postgresql.org; > Shigeru Hanada > Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual > > On Tue, Nov 3, 2015 at 8:15 AM, Kouhei Kaigai <kai...@ak.jp.nec.com> wrote: > > A challenge is that junk wholerow references on behalf of ROW_MARK_COPY > > are injecte

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Nov 3, 2015 at 8:15 AM, Kouhei Kaigai wrote: > A challenge is that junk wholerow references on behalf of ROW_MARK_COPY > are injected by preprocess_targetlist(). It is earlier than the main path > consideration by query_planner(), thus, it is not predictable how

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/04 18:50, Etsuro Fujita wrote: On 2015/11/04 17:10, Kouhei Kaigai wrote: On 2015/10/28 6:04, Robert Haas wrote: On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita wrote: Sorry, my explanation was not correct. (Needed to take in caffeine.) What I'm

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hello, At Thu, 5 Nov 2015 01:58:00 +, Kouhei Kaigai wrote in <9a28c8860f777e439aa12e8aea7694f801162...@bpxm15gp.gisp.nec.co.jp> > > So, as the third way, I propose to resurrect the abandoned > > ForeinJoinState seems to be for the unearthed requirements. FDW > >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hello, The attached small patch is what I have in mind now. fdwroutine->ExecForeignScan may be unset if the FDW does nothing special. And all the FDW routine needs is the node. > Subject: [PATCH] Allow substitute ExecScan body for ExecForignScan > > ForeignScan node may return joined tuple.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

Hi, I've caught up again. > OK, so if we all agree that the joined-tuple optimization is just an > option for the case where all the component tables use ROW_MARK_COPY, > I'd propose to leave that for 9.6. I still think that ExecScan is called under EPQ recheck without EQP tuple for the *scan*.

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

l.org; shigeru.han...@gmail.com > Subject: Re: [HACKERS] Foreign join pushdown vs EvalPlanQual > > Hi, I've caught up again. > > > OK, so if we all agree that the joined-tuple optimization is just an > > option for the case where all the component tables use ROW_MARK_COPY, > >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/03 22:15, Kouhei Kaigai wrote: A challenge is that junk wholerow references on behalf of ROW_MARK_COPY are injected by preprocess_targetlist(). It is earlier than the main path consideration by query_planner(), thus, it is not predictable how remote query shall be executed at this

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On 2015/10/28 6:04, Robert Haas wrote: > > On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita > > wrote: > >> Sorry, my explanation was not correct. (Needed to take in caffeine.) What > >> I'm concerned about is the following: > >> > >> SELECT * FROM localtab JOIN

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Wednesday, November 04, 2015 5:11 PM > To: Kaigai Kouhei(海外 浩平); Robert Haas > Cc: Tom Lane; Kyotaro HORIGUCHI; pgsql-hackers@postgresql.org; Shigeru Hanada > Subject: Re: [HAC

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/04 17:10, Kouhei Kaigai wrote: On 2015/10/28 6:04, Robert Haas wrote: On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita wrote: Sorry, my explanation was not correct. (Needed to take in caffeine.) What I'm concerned about is the following: SELECT * FROM

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/11/04 17:28, Kouhei Kaigai wrote: I think we need to consider a general solution that can be applied not only to the case where the component tables in a foreign join all use ROW_MARK_COPY but to the case where those tables use different rowmark types such as ROW_MARK_COPY and

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Thu, Oct 29, 2015 at 6:05 AM, Kouhei Kaigai wrote: > > In this case, the EPQ slot to store the joined tuple is still > > a challenge to be solved. > > > > Is it possible to use one or any of EPQ slots that are setup for > > base relations but represented by

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/10/28 6:04, Robert Haas wrote: On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita wrote: Sorry, my explanation was not correct. (Needed to take in caffeine.) What I'm concerned about is the following: SELECT * FROM localtab JOIN (ft1 LEFT JOIN ft2 ON ft1.x =

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Thu, Oct 29, 2015 at 6:05 AM, Kouhei Kaigai wrote: > In this case, the EPQ slot to store the joined tuple is still > a challenge to be solved. > > Is it possible to use one or any of EPQ slots that are setup for > base relations but represented by ForeignScan/CustomScan?

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita > wrote: > > Sorry, my explanation was not correct. (Needed to take in caffeine.) What > > I'm concerned about is the following: > > > > SELECT * FROM localtab JOIN (ft1 LEFT JOIN ft2 ON ft1.x = ft2.x) ON > >

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Tue, Oct 20, 2015 at 12:39 PM, Etsuro Fujita wrote: > Sorry, my explanation was not correct. (Needed to take in caffeine.) What > I'm concerned about is the following: > > SELECT * FROM localtab JOIN (ft1 LEFT JOIN ft2 ON ft1.x = ft2.x) ON > localtab.id = ft1.id

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/10/20 9:36, Kouhei Kaigai wrote: Even if we fetch whole-row of both side, join pushdown is exactly working because we can receive less number of rows than local join + 2 of foreign- scan. (If planner works well, we can expect join-path that increases number of rows shall be dropped.)

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/10/21 13:34, Kouhei Kaigai wrote: On 2015/10/20 13:11, Etsuro Fujita wrote: On 2015/10/20 5:34, Robert Haas wrote: No. You just need to populate fdw_recheck_quals correctly, same as for the scan case. As I said yesterday, that opinion of me is completely wrong. Sorry for the

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Tuesday, October 20, 2015 1:11 PM > To: Robert Haas > Cc: Tom Lane; Kaigai Kouhei(海外 浩平); Kyotaro HORIGUCHI; > pgsql-hackers@postgresql.org; Shigeru Hanada > Subject: Re: [HACKERS]

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On Mon, Oct 19, 2015 at 3:45 AM, Etsuro Fujita wrote: As Tom mentioned, just recomputing the original join tuple is not good enough. We would need to rejoin the test tuples for the baserels even if ROW_MARK_COPY is in use. Consider: A=# BEGIN; A=# UPDATE t SET a

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/10/20 13:11, Etsuro Fujita wrote: On 2015/10/20 5:34, Robert Haas wrote: On Mon, Oct 19, 2015 at 3:45 AM, Etsuro Fujita wrote: As Tom mentioned, just recomputing the original join tuple is not good enough. We would need to rejoin the test tuples for the

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

> -Original Message- > From: Etsuro Fujita [mailto:fujita.ets...@lab.ntt.co.jp] > Sent: Wednesday, October 21, 2015 12:31 PM > To: Robert Haas > Cc: Tom Lane; Kaigai Kouhei(海外 浩平); Kyotaro HORIGUCHI; > pgsql-hackers@postgresql.org; Shigeru Hanada > Subject: Re: [

Re: [HACKERS] Foreign join pushdown vs EvalPlanQual

On 2015/10/17 9:58, Robert Haas wrote: But with Etsuro Fujita's patch, and I think what you have proposed has been similar, how are you going to do it? The proposal is to call the recheck method and hope for the best, but what is the recheck method going to do? Where is it going to get the

  1   2   3   >