Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Fri, Mar 4, 2016 at 2:37 PM, Andres Freund wrote: > On 2016-02-12 15:56:45 +0100, Andres Freund wrote: >> Hi, >> >> >> On 2016-02-10 23:26:20 -0500, Robert Haas wrote: >> > I think the part about whacking around the FDW API is a little more >> > potentially objectionable to

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On 2016-02-12 15:56:45 +0100, Andres Freund wrote: > Hi, > > > On 2016-02-10 23:26:20 -0500, Robert Haas wrote: > > I think the part about whacking around the FDW API is a little more > > potentially objectionable to others, so I want to hold off doing that > > unless a few more people chime in

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

PM >> To: Kaigai Kouhei(海外 浩平) >> Cc: Andres Freund; Amit Kapila; pgsql-hackers >> Subject: Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan >> support >> on readfuncs.c) >> >> On Wed, Feb 10, 2016 at 1:25 AM, Kouhei Kaigai <kai...@ak.jp.ne

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

Hi, On 2016-02-10 23:26:20 -0500, Robert Haas wrote: > I think the part about whacking around the FDW API is a little more > potentially objectionable to others, so I want to hold off doing that > unless a few more people chime in with +1. Perhaps we could start a > new thread to talk about

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

2016-02-13 0:11 GMT+09:00 Robert Haas : > On Fri, Feb 12, 2016 at 9:56 AM, Andres Freund wrote: >> On 2016-02-10 23:26:20 -0500, Robert Haas wrote: >>> I think the part about whacking around the FDW API is a little more >>> potentially objectionable to

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

bject: Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan > support > on readfuncs.c) > > On Wed, Feb 10, 2016 at 1:25 AM, Kouhei Kaigai <kai...@ak.jp.nec.com> wrote: > > It is pretty good! > > > > The attached patch (primary one) implements the above id

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Feb 10, 2016 at 1:25 AM, Kouhei Kaigai wrote: > It is pretty good! > > The attached patch (primary one) implements the above idea. > > Now ExtensibleNode works as a basis structure of data container, > regardless of CustomScan and ForeignScan. > Also, fdw_private and

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> -Original Message- > From: Robert Haas [mailto:robertmh...@gmail.com] > Sent: Monday, February 08, 2016 11:59 PM > To: Kaigai Kouhei(海外 浩平) > Cc: Andres Freund; Amit Kapila; pgsql-hackers > Subject: ##freemail## Re: CustomScan in a larger structure (RE: [HACKERS] &g

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Sun, Feb 7, 2016 at 7:28 PM, Kouhei Kaigai wrote: > The new callbacks of T_ExtensibleNode will replace the necessity to > form and deform process of private values, like as: > https://github.com/pg-strom/devel/blob/master/src/gpujoin.c#L114 Yeah. > It transforms a

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Wed, Feb 3, 2016 at 11:57 PM, Kouhei Kaigai wrote: > > At this moment, I tried to write up description at nodes/nodes.h. > > The amount of description is about 100lines. It is on a borderline > > whether we split off this chunk into another header file, in my sense. > >

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Feb 3, 2016 at 11:57 PM, Kouhei Kaigai wrote: > At this moment, I tried to write up description at nodes/nodes.h. > The amount of description is about 100lines. It is on a borderline > whether we split off this chunk into another header file, in my sense. > > > On

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Wed, Jan 27, 2016 at 9:36 PM, Robert Haas wrote: > > On Mon, Jan 25, 2016 at 8:06 PM, Kouhei Kaigai wrote: > >> Sorry for my late response. I've been unavailable to have enough > >> time to touch code for the last 1.5 month. > >> > >> The

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Feb 3, 2016 at 8:00 PM, Kouhei Kaigai wrote: >> Well, looking at this a bit more, it seems like the documentation >> you've written here is really misplaced. The patch is introducing a >> new facility that applies to both CustomScan and ForeignScan, but the >>

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Jan 27, 2016 at 9:36 PM, Robert Haas wrote: > On Mon, Jan 25, 2016 at 8:06 PM, Kouhei Kaigai wrote: >> Sorry for my late response. I've been unavailable to have enough >> time to touch code for the last 1.5 month. >> >> The attached patch is a

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

bject: Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan > support > on readfuncs.c) > > On Wed, Feb 3, 2016 at 8:00 PM, Kouhei Kaigai <kai...@ak.jp.nec.com> wrote: > >> Well, looking at this a bit more, it seems like the documentation > >> you've wr

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Mon, Jan 25, 2016 at 8:06 PM, Kouhei Kaigai wrote: > > Sorry for my late response. I've been unavailable to have enough > > time to touch code for the last 1.5 month. > > > > The attached patch is a revised one to handle private data of > > foregn/custom scan node more

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Thu, Jan 28, 2016 at 10:18 PM, Kouhei Kaigai wrote: > Do you think we shall allow to register same extensible node name for > different node types? Like, "GpuJoin" for any of CustomPath, CustomScan > and CustomScanState. Or, do we avoid this using different name for each?

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Thu, Jan 28, 2016 at 10:18 PM, Kouhei Kaigai wrote: > > Do you think we shall allow to register same extensible node name for > > different node types? Like, "GpuJoin" for any of CustomPath, CustomScan > > and CustomScanState. Or, do we avoid this using different name

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Mon, Jan 25, 2016 at 8:06 PM, Kouhei Kaigai wrote: > Sorry for my late response. I've been unavailable to have enough > time to touch code for the last 1.5 month. > > The attached patch is a revised one to handle private data of > foregn/custom scan node more gracefully.

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

e- > From: Kaigai Kouhei(海外 浩平) > Sent: Wednesday, December 02, 2015 5:52 PM > To: 'Robert Haas' > Cc: Andres Freund; Amit Kapila; pgsql-hackers > Subject: Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan > support > on readfuncs.c) > > > On Thu, No

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Wed, Dec 2, 2015 at 4:06 AM, Andres Freund wrote: > > On 2015-12-02 08:52:20 +, Kouhei Kaigai wrote: > >> diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c > >> index 26264cb..c4bb76e 100644 > >> --- a/src/backend/nodes/copyfuncs.c > >> +++

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Dec 2, 2015 at 4:06 AM, Andres Freund wrote: > On 2015-12-02 08:52:20 +, Kouhei Kaigai wrote: >> diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c >> index 26264cb..c4bb76e 100644 >> --- a/src/backend/nodes/copyfuncs.c >> +++

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Thu, Nov 26, 2015 at 5:27 AM, Kouhei Kaigai wrote: > > I'm now implementing. The above design perfectly works on ForeignScan. > > On the other hands, I'd like to have deeper consideration for CustomScan. > > > > My recent patch adds LibraryName and SymbolName on

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On 2015-12-02 08:52:20 +, Kouhei Kaigai wrote: > diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c > index 26264cb..c4bb76e 100644 > --- a/src/backend/nodes/copyfuncs.c > +++ b/src/backend/nodes/copyfuncs.c > @@ -635,8 +635,12 @@ _copyWorkTableScan(const WorkTableScan

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Thu, Nov 26, 2015 at 5:27 AM, Kouhei Kaigai wrote: > I'm now implementing. The above design perfectly works on ForeignScan. > On the other hands, I'd like to have deeper consideration for CustomScan. > > My recent patch adds LibraryName and SymbolName on CustomScanMethods

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

ustomScan in a larger structure (RE: [HACKERS] CustomScan > support > on readfuncs.c) > > > On Thu, Nov 19, 2015 at 10:11 PM, Kouhei Kaigai <kai...@ak.jp.nec.com> > > wrote: > > > The above two points are for the case if and when extension want to use > > >

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Thu, Nov 19, 2015 at 10:11 PM, Kouhei Kaigai wrote: > The above two points are for the case if and when extension want to use > variable length fields for its private fields. > So, nodeAlloc() callback is not a perfect answer for the use case because > length of the

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Thu, Nov 19, 2015 at 10:11 PM, Kouhei Kaigai wrote: > > The above two points are for the case if and when extension want to use > > variable length fields for its private fields. > > So, nodeAlloc() callback is not a perfect answer for the use case because > > length of

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Mon, Nov 16, 2015 at 9:03 PM, Kouhei Kaigai wrote: > > I guess we will put a pointer to static ExtensibleNodeMethods structure > > on ForeignScan, CustomScan, CustomScanState and etc... > > I think that makes it confusing. What I'd prefer to do is have only > the name

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Mon, Nov 16, 2015 at 9:03 PM, Kouhei Kaigai wrote: > I guess we will put a pointer to static ExtensibleNodeMethods structure > on ForeignScan, CustomScan, CustomScanState and etc... I think that makes it confusing. What I'd prefer to do is have only the name stored in

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Nov 11, 2015 at 11:13 PM, Kouhei Kaigai wrote: > I agree with we have no reason why only custom-scan is allowed to have > serialize/deserialize capability. I can implement an equivalent stuff > for foreign-scan also, and it is helpful for extension authors,

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Wed, Nov 11, 2015 at 11:13 PM, Kouhei Kaigai wrote: > > I agree with we have no reason why only custom-scan is allowed to have > > serialize/deserialize capability. I can implement an equivalent stuff > > for foreign-scan also, and it is helpful for extension authors,

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> > On Wed, Nov 11, 2015 at 3:29 PM, Andres Freund wrote: > > > On 2015-11-11 14:59:33 -0500, Robert Haas wrote: > > >> I don't see this as being a particularly good idea. The same issue > > >> exists for FDWs, and we're just living with it in that case. > > > > > > It's

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> On Wed, Nov 11, 2015 at 3:29 PM, Andres Freund wrote: > > On 2015-11-11 14:59:33 -0500, Robert Haas wrote: > >> I don't see this as being a particularly good idea. The same issue > >> exists for FDWs, and we're just living with it in that case. > > > > It's absolutely

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

> Robert Haas writes: > > On Wed, Nov 11, 2015 at 3:29 PM, Andres Freund wrote: > >> Just better serialization doesn't actually help all that much. Being > >> able to conveniently access data directly, i.e. as fields in a struct, > >> makes code rather

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Mon, Nov 9, 2015 at 10:26 PM, Kouhei Kaigai wrote: > It is a relevant topic of readfuncs support for custom-scan. > > Unlike CustomPath and CustomScanState, we don't allow custom-scan > provider to define own and larger structure that embeds CustomScan > at head of the

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On 2015-11-11 14:59:33 -0500, Robert Haas wrote: > I don't see this as being a particularly good idea. The same issue > exists for FDWs, and we're just living with it in that case. It's absolutely horrible there. I don't see why that's a justification for much. To deal with the lack of

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

Robert Haas writes: > On Wed, Nov 11, 2015 at 3:29 PM, Andres Freund wrote: >> Just better serialization doesn't actually help all that much. Being >> able to conveniently access data directly, i.e. as fields in a struct, >> makes code rather more

Re: CustomScan in a larger structure (RE: [HACKERS] CustomScan support on readfuncs.c)

On Wed, Nov 11, 2015 at 3:29 PM, Andres Freund wrote: > On 2015-11-11 14:59:33 -0500, Robert Haas wrote: >> I don't see this as being a particularly good idea. The same issue >> exists for FDWs, and we're just living with it in that case. > > It's absolutely horrible there. I