Re: Extracting only the columns needed for a query

2020-06-30 Thread Soumyadeep Chakraborty
Hello, Melanie and me will be posting a separate thread with the scanCols patch listed here, a patch to capture the set of cols in RETURNING and a group of patches to pass down the list of cols to various table AM functions together as a patch set. This will take some time. Thus, we are

Re: Extracting only the columns needed for a query

2020-06-23 Thread Melanie Plageman
On Fri, Mar 13, 2020 at 12:09 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > I've also noticed that for partitioned tables every partition is > evaluated separately. IIRC they structure cannot differ, does it > makes sense then? Good point. At some point, we had discussed only collecting

Re: Extracting only the columns needed for a query

2020-06-18 Thread Melanie Plageman
On Fri, Mar 13, 2020 at 12:09 PM Dmitry Dolgov <9erthali...@gmail.com> wrote: > In general implemented functionality looks good. I've checked how it > works on the existing tests, almost everywhere required columns were not > missing in scanCols (which is probably the most important part). >

Re: Extracting only the columns needed for a query

2020-03-13 Thread Dmitry Dolgov
> On Tue, Feb 18, 2020 at 03:26:16PM -0800, Melanie Plageman wrote: > > > > I believe it would be beneficial to add this potential API extension > > patch into > > > the thread (as an example of an interface defining how scanCols could be > > used) > > > and review them together. > > > > As for

Re: Extracting only the columns needed for a query

2020-02-18 Thread Melanie Plageman
On Fri, Jan 31, 2020 at 9:52 AM Melanie Plageman wrote: > I'm bumping this to the next commitfest because I haven't had a chance > to address the questions posed by Dmitry. I'm sure I'll get to it in > the next few weeks. > > > I believe it would be beneficial to add this potential API extension

Re: Extracting only the columns needed for a query

2020-02-14 Thread Pengzhou Tang
> > > On Sat, Jun 15, 2019 at 10:02 AM Tom Lane wrote: > > > > > > Another reason for having the planner do this is that presumably, in > > > an AM that's excited about this, the set of fetched columns should > > > play into the cost estimates for the scan. I've not been paying > > > enough

Re: Extracting only the columns needed for a query

2020-01-31 Thread Ashwin Agrawal
On Wed, Jan 15, 2020 at 7:54 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > For UPDATE, we need all of the columns in the table because of the > > table_lock() API's current expectation that the slot has all of the > > columns populated. If we want UPDATE to only need to insert the column >

Re: Extracting only the columns needed for a query

2020-01-31 Thread Melanie Plageman
I'm bumping this to the next commitfest because I haven't had a chance to address the questions posed by Dmitry. I'm sure I'll get to it in the next few weeks. > I believe it would be beneficial to add this potential API extension patch into > the thread (as an example of an interface defining

Re: Extracting only the columns needed for a query

2020-01-15 Thread Dmitry Dolgov
> On Thu, Jan 02, 2020 at 05:21:55PM -0800, Melanie Plageman wrote: > > That makes me think that maybe the function name, > extract_used_columns() is bad, though. Maybe extract_scan_columns()? > I tried this in the attached, updated patch. Thanks, I'll take a look at the new version. Following

Re: Extracting only the columns needed for a query

2020-01-07 Thread Melanie Plageman
I just wanted to address a question we got about making scanCols instead of using RelOptInfo->attr_needed. David Rowley had asked: > For planning, isn't this information already available via either > targetlists or from the RelOptInfo->attr_needed array combined with > min_attr and max_attr?

Re: Extracting only the columns needed for a query

2020-01-02 Thread Melanie Plageman
On Tue, Dec 17, 2019 at 2:57 AM Dmitry Dolgov <9erthali...@gmail.com> wrote: > > Thanks for the patch! If I understand correctly from this thread, > approach B is more preferable, so I've concentrated on the patch 0001 > and have a few commentaries/questions: > Thanks so much for the review! >

Re: Extracting only the columns needed for a query

2019-12-17 Thread Dmitry Dolgov
> On Tue, Jul 16, 2019 at 06:49:10PM -0700, Melanie Plageman wrote: > > We implemented Approach B in the attached patch set (patch 0001) and > then implemented Approach A (patch 0002) to sanity check the pruned > list of columns to scan we were getting at plan-time. > We emit a notice in SeqNext()

Re: Extracting only the columns needed for a query

2019-07-17 Thread Melanie Plageman
On Sat, Jun 15, 2019 at 10:01 AM Tom Lane wrote: > Melanie Plageman writes: > > While hacking on zedstore, we needed to get a list of the columns to > > be projected--basically all of the columns needed to satisfy the > > query. The two use cases we have for this is > > 1) to pass this column

Re: Extracting only the columns needed for a query

2019-07-16 Thread Melanie Plageman
We implemented Approach B in the attached patch set (patch 0001) and then implemented Approach A (patch 0002) to sanity check the pruned list of columns to scan we were getting at plan-time. We emit a notice in SeqNext() if the two column sets differ. Currently, for all of the queries in the

Re: Extracting only the columns needed for a query

2019-06-16 Thread Ashwin Agrawal
On Sat, Jun 15, 2019 at 10:02 AM Tom Lane wrote: > > Approach B: after parsing and/or after planning > > If we wanted to do something about this, making the planner record > the set of used columns seems like the thing to do. We could avoid > the expense of doing it when it's not needed by

Re: Extracting only the columns needed for a query

2019-06-16 Thread Corey Huinker
> > The thing that most approaches to this have fallen down on is triggers --- > that is, a trigger function might access columns mentioned nowhere in the > SQL text. (See 8b6da83d1 for a recent example :-() If you have a plan > for dealing with that, then ... > Well, if we had a trigger

Re: Extracting only the columns needed for a query

2019-06-15 Thread Tom Lane
Melanie Plageman writes: > While hacking on zedstore, we needed to get a list of the columns to > be projected--basically all of the columns needed to satisfy the > query. The two use cases we have for this is > 1) to pass this column list down to the AM layer for the AM to leverage it > 2) for

Re: Extracting only the columns needed for a query

2019-06-14 Thread David Rowley
On Sat, 15 Jun 2019 at 13:46, Melanie Plageman wrote: > > While hacking on zedstore, we needed to get a list of the columns to > be projected--basically all of the columns needed to satisfy the > query. The two use cases we have for this is > 1) to pass this column list down to the AM layer for