Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-17 Thread Haribabu Kommi
On Mon, Mar 17, 2014 at 11:45 AM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Hello, The attached patches are revised ones according to the latest custom-plan interface patch (v11). The cache-scan module was re-implemented on the newer interface, and also I noticed the extension does not

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-17 Thread Kouhei Kaigai
KaiGai Kohei kai...@ak.jp.nec.com -Original Message- From: Haribabu Kommi [mailto:kommi.harib...@gmail.com] Sent: Tuesday, March 18, 2014 11:14 AM To: Kaigai Kouhei(海外 浩平) Cc: Kohei KaiGai; Tom Lane; PgHacker; Robert Haas Subject: Re: contrib/cache_scan (Re: [HACKERS] What's needed

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-12 Thread Kouhei Kaigai
...@postgresql.org] On Behalf Of Haribabu Kommi Sent: Wednesday, March 12, 2014 1:14 PM To: Kohei KaiGai Cc: Kaigai Kouhei(海外 浩平); Tom Lane; PgHacker; Robert Haas Subject: Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?) On Thu, Mar 6, 2014 at 10:15 PM, Kohei

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-12 Thread Haribabu Kommi
On Wed, Mar 12, 2014 at 5:26 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: Thanks for your efforts! Head patched Diff Select - 500K772ms2659ms-200% Insert - 400K 3429ms 1948ms 43%

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-11 Thread Haribabu Kommi
On Thu, Mar 6, 2014 at 10:15 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2014-03-06 18:17 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: I will update you later regarding the performance test results. I ran the performance test on the cache scan patch and below are the readings.

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-06 Thread Haribabu Kommi
On Tue, Mar 4, 2014 at 3:07 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: 4. + cchunk = ccache_vacuum_tuple(ccache, ccache-root_chunk, ctid); + if (pchunk != NULL pchunk != cchunk) + ccache_merge_chunk(ccache, pchunk); + pchunk = cchunk; The merge_chunk is called

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-06 Thread Kohei KaiGai
2014-03-06 18:17 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: On Tue, Mar 4, 2014 at 3:07 PM, Kouhei Kaigai kai...@ak.jp.nec.com wrote: 4. + cchunk = ccache_vacuum_tuple(ccache, ccache-root_chunk, ctid); + if (pchunk != NULL pchunk != cchunk) + ccache_merge_chunk(ccache,

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-26 Thread Kouhei Kaigai
Thanks for the information, I will apply other patches also and start testing. When try to run the pgbench test, by default the cache-scan plan is not chosen because of more cost. So I increased the cpu_index_tuple_cost to a maximum value or by turning off index_scan, so that the

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-25 Thread Haribabu Kommi
On Tue, Feb 25, 2014 at 11:13 AM, Haribabu Kommi kommi.harib...@gmail.comwrote: Thanks for the information, I will apply other patches also and start testing. When try to run the pgbench test, by default the cache-scan plan is not chosen because of more cost. So I increased the

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-24 Thread Haribabu Kommi
On Tue, Feb 25, 2014 at 10:44 AM, Kouhei Kaigai kai...@ak.jp.nec.comwrote: Thanks for your testing, Getting some compilation warnings while compiling the extension and also I am not able to load the extension because of undefined symbol get_restriction_qual_cost. It seems to me you

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-23 Thread Haribabu Kommi
On Fri, Feb 21, 2014 at 2:19 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Hello, The attached patch is a revised one for cache-only scan module on top of custom-scan interface. Please check it. Thanks for the revised patch. Please find some minor comments. 1. memcpy(dest, tuple,

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-12 Thread Kohei KaiGai
2014-02-12 14:59 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: I reviewed all the three patches. The first 1 and 2 core PostgreSQL patches are fine. And I have comments in the third patch related to cache scan. Thanks for your volunteering. 1. +# contrib/dbcache/Makefile Makefile

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-12 Thread Haribabu Kommi
On Thu, Feb 13, 2014 at 2:42 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2014-02-12 14:59 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: 7. In ccache_find_tuple function this Assert(i_min + 1 cchunk-ntups); can go wrong when only one tuple present in the block with the equal item

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-12 Thread Kouhei Kaigai
/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?) On Thu, Feb 13, 2014 at 2:42 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: 2014-02-12 14:59 GMT+09:00 Haribabu Kommi kommi.harib...@gmail.com: 7. In ccache_find_tuple function this Assert(i_min + 1 cchunk

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-02-11 Thread Haribabu Kommi
On Sat, Feb 8, 2014 at 1:09 PM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Hello, Because of time pressure in the commit-fest:Jan, I tried to simplifies the patch for cache-only scan into three portions; (1) add a hook on heap_page_prune for cache invalidation on vacuuming a particular page.

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: The cache-only table scan, being in subject line, is an alternative scan logic towards sequential scan if all the referenced columns are cached. This seems like a pretty dubious idea to me --- you're talking about expending large amounts of memory on a

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Robert Haas
On Tue, Nov 12, 2013 at 9:45 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: It is a brief design proposal of a feature I'd like to implement on top of custom-scan APIs. Because it (probably) requires a few additional base features not only custom-scan, I'd like to see feedback from the hackers.

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Kohei KaiGai
2013/11/12 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: The cache-only table scan, being in subject line, is an alternative scan logic towards sequential scan if all the referenced columns are cached. This seems like a pretty dubious idea to me --- you're talking

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Kohei KaiGai
2013/11/12 Robert Haas robertmh...@gmail.com: On Tue, Nov 12, 2013 at 9:45 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: It is a brief design proposal of a feature I'd like to implement on top of custom-scan APIs. Because it (probably) requires a few additional base features not only

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: 2013/11/12 Tom Lane t...@sss.pgh.pa.us: There's no possible way you'll finish this for 9.4. Yes, I understand it is not possible to submit whole of the patch until CF3 deadline. So, I'd like to find out a way to implement it as an extension using

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Claudio Freire
On Tue, Nov 12, 2013 at 11:45 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Hello, It is a brief design proposal of a feature I'd like to implement on top of custom-scan APIs. Because it (probably) requires a few additional base features not only custom-scan, I'd like to see feedback from the

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Kohei KaiGai
2013/11/12 Claudio Freire klaussfre...@gmail.com: On Tue, Nov 12, 2013 at 11:45 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: Hello, It is a brief design proposal of a feature I'd like to implement on top of custom-scan APIs. Because it (probably) requires a few additional base features not

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Kohei KaiGai
2013/11/12 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: 2013/11/12 Tom Lane t...@sss.pgh.pa.us: There's no possible way you'll finish this for 9.4. Yes, I understand it is not possible to submit whole of the patch until CF3 deadline. So, I'd like to find out a way to

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Tom Lane
Kohei KaiGai kai...@kaigai.gr.jp writes: So, are you thinking it is a feasible approach to focus on custom-scan APIs during the upcoming CF3, then table-caching feature as use-case of this APIs on CF4? Sure. If you work on this extension after CF3, and it reveals that the custom scan stuff

Re: [HACKERS] What's needed for cache-only table scan?

2013-11-12 Thread Kohei KaiGai
2013/11/12 Tom Lane t...@sss.pgh.pa.us: Kohei KaiGai kai...@kaigai.gr.jp writes: So, are you thinking it is a feasible approach to focus on custom-scan APIs during the upcoming CF3, then table-caching feature as use-case of this APIs on CF4? Sure. If you work on this extension after CF3,