Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2014-04-04 Thread Andres Freund
Hi, On 2014-01-14 20:58:20 +0900, KONDO Mitsumasa wrote: > I will test some join sqls performance and TPC-3 benchmark in this or next > week. This patch has been marked as "Waiting For Author" for nearly two months now. Marked as "Returned with Feedback". Greetings, Andres Freund -- Sent vi

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2014-01-14 Thread Claudio Freire
On Tue, Jan 14, 2014 at 8:58 AM, KONDO Mitsumasa wrote: > > In my past patch, it is significant bug which is mistaken caluculation of > offset in posix_fadvise():-( However it works well without problem in > pgbench. > Because pgbench transactions are always random access... Did you notice any d

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2014-01-14 Thread Claudio Freire
On Tue, Jan 14, 2014 at 8:58 AM, KONDO Mitsumasa wrote: > > In my past patch, it is significant bug which is mistaken caluculation of > offset in posix_fadvise():-( However it works well without problem in > pgbench. > Because pgbench transactions are always random access... Did you notice any d

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2014-01-14 Thread KONDO Mitsumasa
Hi, I fix and submit this patch in CF4. In my past patch, it is significant bug which is mistaken caluculation of offset in posix_fadvise():-( However it works well without problem in pgbench. Because pgbench transactions are always random access... And I test my patch in DBT-2 benchmark. Resul

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-17 Thread KONDO Mitsumasa
(2013/12/17 21:29), Simon Riggs wrote: These are interesting results. Good research. Thanks! They also show that the benefit of this is very specific to the exact task being performed. I can't see any future for a setting that applies to everything or nothing. We must be more selective. This

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-17 Thread Simon Riggs
On 17 December 2013 11:50, KONDO Mitsumasa wrote: > Unpatched PG is faster than patched in A and B query. It was about 1.3 times > faster. Result of A query as expected, because patched PG cannot execute > readahead at all. So cache cold situation is bad for patched PG. However, it > might good f

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-17 Thread KONDO Mitsumasa
Hi, I fixed the patch to improve followings. - Can compile in MacOS. - Change GUC name enable_kernel_readahead to readahead_strategy. - Change POSIX_FADV_SEQUNENTIAL to POISX_FADV_NORMAL when we select sequential access strategy, this reason is later... I tested simple two access pate

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-12 Thread Simon Riggs
On 12 December 2013 13:43, Mitsumasa KONDO wrote: >> Your tests seem to relate to pgbench. Do we have tests on more BI related >> tasks? > > Yes, off-course! We will need another benchmark test before conclusion of > this patch. > What kind of benchmark do you have? I suggest isolating SeqScan

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-12 Thread Mitsumasa KONDO
2013/12/12 Simon Riggs > On 14 November 2013 12:09, KONDO Mitsumasa > wrote: > > > For your information of effect of this patch, I got results of pgbench > which are > > in-memory-size database and out-memory-size database, and postgresql.conf > > settings are always used by us. It seems to impr

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-12 Thread Simon Riggs
On 14 November 2013 12:09, KONDO Mitsumasa wrote: > For your information of effect of this patch, I got results of pgbench which > are > in-memory-size database and out-memory-size database, and postgresql.conf > settings are always used by us. It seems to improve performance to a better. > And

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-12 Thread KONDO Mitsumasa
(2013/12/12 9:30), Claudio Freire wrote: On Wed, Dec 11, 2013 at 3:14 AM, KONDO Mitsumasa wrote: enable_readahead=os|fadvise with os = on, fadvise = off Hmm. fadvise is method and is not a purpose. So I consider another idea of this GUC. Yeah, I was thinking of opening the door for reada

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-11 Thread Claudio Freire
On Wed, Dec 11, 2013 at 3:14 AM, KONDO Mitsumasa wrote: > >> enable_readahead=os|fadvise >> >> with os = on, fadvise = off > > Hmm. fadvise is method and is not a purpose. So I consider another idea of > this GUC. Yeah, I was thinking of opening the door for readahead=aio, but whatever clearer t

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-10 Thread KONDO Mitsumasa
(2013/12/10 22:55), Claudio Freire wrote: On Tue, Dec 10, 2013 at 5:03 AM, KONDO Mitsumasa wrote: I revise this patch and re-run performance test, it can work collectry in Linux and no complile wanings. I add GUC about enable_kernel_readahead option in new version. When this GUC is on(default),

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-10 Thread Claudio Freire
On Tue, Dec 10, 2013 at 5:03 AM, KONDO Mitsumasa wrote: > I revise this patch and re-run performance test, it can work collectry in > Linux and no complile wanings. I add GUC about enable_kernel_readahead > option in new version. When this GUC is on(default), it works in > POSIX_FADV_NORMAL which

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-12-09 Thread KONDO Mitsumasa
Hi, I revise this patch and re-run performance test, it can work collectry in Linux and no complile wanings. I add GUC about enable_kernel_readahead option in new version. When this GUC is on(default), it works in POSIX_FADV_NORMAL which is general readahead in OS. And when it is off, it works

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-17 Thread KONDO Mitsumasa
(2013/11/18 11:25), Claudio Freire wrote: On Sun, Nov 17, 2013 at 11:02 PM, KONDO Mitsumasa wrote: However, my patch is on the way and needed to more improvement. I am going to add method of controlling readahead by GUC, for user can freely select readahed parameter in their transactions. Ra

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-17 Thread Claudio Freire
On Sun, Nov 17, 2013 at 11:02 PM, KONDO Mitsumasa wrote: >>> However, my patch is on the way and needed to more improvement. I am >>> going >>> to add method of controlling readahead by GUC, for user can freely select >>> readahed parameter in their transactions. >> >> >> Rather, I'd try to avoid

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-17 Thread KONDO Mitsumasa
(2013/11/15 13:48), Claudio Freire wrote: On Thu, Nov 14, 2013 at 11:13 PM, KONDO Mitsumasa I use CentOS 6.4 which kernel version is 2.6.32-358.23.2.el6.x86_64 in this test. That's close to the kernel version I was using, so you should see the same effect. OK. You proposed readahead maximum pa

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-15 Thread Peter Eisentraut
On 11/14/13, 7:09 AM, KONDO Mitsumasa wrote: > I create a patch that is improvement of disk-read and OS file caches. It can > optimize kernel readahead parameter using buffer access strategy and > posix_fadvice() in various disk-read situations. Various compiler warnings: tablecmds.c: In function

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Claudio Freire
On Thu, Nov 14, 2013 at 11:13 PM, KONDO Mitsumasa wrote: > Hi Claudio, > > > (2013/11/14 22:53), Claudio Freire wrote: >> >> On Thu, Nov 14, 2013 at 9:09 AM, KONDO Mitsumasa >> wrote: >>> >>> I create a patch that is improvement of disk-read and OS file caches. It >>> can >>> optimize kernel read

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread KONDO Mitsumasa
(2013/11/15 11:17), Peter Geoghegan wrote: On Thu, Nov 14, 2013 at 6:18 PM, KONDO Mitsumasa wrote: I will fix it. Could you tell me your Mac OS version and gcc version? I have only mac book air with Maverick OS(10.9). I have an idea that Mac OSX doesn't have posix_fadvise at all. Didn't you u

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Peter Geoghegan
On Thu, Nov 14, 2013 at 6:18 PM, KONDO Mitsumasa wrote: > I will fix it. Could you tell me your Mac OS version and gcc version? I have > only mac book air with Maverick OS(10.9). I have an idea that Mac OSX doesn't have posix_fadvise at all. Didn't you use the relevant macros so that the code at

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread KONDO Mitsumasa
(2013/11/15 2:03), Fujii Masao wrote: On Thu, Nov 14, 2013 at 9:09 PM, KONDO Mitsumasa wrote: Hi, I create a patch that is improvement of disk-read and OS file caches. It can optimize kernel readahead parameter using buffer access strategy and posix_fadvice() in various disk-read situations.

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread KONDO Mitsumasa
Hi Claudio, (2013/11/14 22:53), Claudio Freire wrote: On Thu, Nov 14, 2013 at 9:09 AM, KONDO Mitsumasa wrote: I create a patch that is improvement of disk-read and OS file caches. It can optimize kernel readahead parameter using buffer access strategy and posix_fadvice() in various disk-read s

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Fujii Masao
On Thu, Nov 14, 2013 at 9:09 PM, KONDO Mitsumasa wrote: > Hi, > > I create a patch that is improvement of disk-read and OS file caches. It can > optimize kernel readahead parameter using buffer access strategy and > posix_fadvice() in various disk-read situations. When I compiled the HEAD code wi

Re: [HACKERS] Optimize kernel readahead using buffer access strategy

2013-11-14 Thread Claudio Freire
On Thu, Nov 14, 2013 at 9:09 AM, KONDO Mitsumasa wrote: > I create a patch that is improvement of disk-read and OS file caches. It can > optimize kernel readahead parameter using buffer access strategy and > posix_fadvice() in various disk-read situations. > > In general OS, readahead parameter wa