On 08/25/2014 12:49 AM, johnlumby wrote:
On 08/19/14 18:27, Heikki Linnakangas wrote:
Please write the patch without atomic CAS operation. Just use a spinlock.
Umm, this is a new criticism I think.
Yeah. Be prepared that new issues will crop up as the patch gets slimmer
and easier to revi
Thanks for the replies and thoughts.
On 08/19/14 18:27, Heikki Linnakangas wrote:
On 08/20/2014 12:17 AM, John Lumby wrote:
I am attaching a new version of the patch for consideration in the
current commit fest.
Thanks for working on this!
Relative to the one I submitted on 25 June in
bay17
On Tue, Aug 19, 2014 at 7:27 PM, Heikki Linnakangas
wrote:
> Also, please split prefetching of regular index scans into a separate patch.
> It's orthogonal to doing async I/O; we could prefetch regular index scans
> with posix_fadvise too, and AIO should be useful for prefetching other
> stuff.
T
On 08/20/2014 12:17 AM, John Lumby wrote:
I am attaching a new version of the patch for consideration in the current
commit fest.
Thanks for working on this!
Relative to the one I submitted on 25 June in
bay175-w412ff89303686022a9f16aa3...@phx.gbl
the method for handling aio completion usin
My cut'n'pasting failed me at one point corrected below.
> discussion about what is the difference between a synchronous read
> versus an asynchronous read as far as non-originator waiting on it is
> concerned.
>
> I thought a bit more about this. There are currently two differences,
> one of
On 2014-06-26 00:08:48 +0300, Heikki Linnakangas wrote:
> LWLocks are implemented with semaphores, so if you can increment a semaphore
> in the signal handler / callback thread, then in theory you should be able
> to release a LWLock.
I don't think that's a convincing argument even if semop et al
On 06/25/2014 09:20 PM, Claudio Freire wrote:
On Tue, Jun 24, 2014 at 12:08 PM, John Lumby wrote:
The question is, if you receive the notification of the I/O completion
using a signal or a thread, is it safe to release the lwlock from the
signal handler or a separate thread?
In the forthcomin
On Tue, Jun 24, 2014 at 12:08 PM, John Lumby wrote:
>> The question is, if you receive the notification of the I/O completion
>> using a signal or a thread, is it safe to release the lwlock from the
>> signal handler or a separate thread?
>
> In the forthcoming new version of the patch that uses
On 06/24/2014 06:08 PM, John Lumby wrote:
The question is, if you receive the notification of the I/O completion
using a signal or a thread, is it safe to release the lwlock from the
signal handler or a separate thread?
In the forthcoming new version of the patch that uses sigevent,
the origin
Thanks Heikki,
> Date: Tue, 24 Jun 2014 17:02:38 +0300
> From: hlinnakan...@vmware.com
> To: johnlu...@hotmail.com; st...@mit.edu
> CC: klaussfre...@gmail.com; pgsql-hackers@postgresql.org
> Subject: Re: Extended Prefetching using Asynchronous IO - proposal
On 06/24/2014 04:29 PM, John Lumby wrote:
On Mon, Jun 23, 2014 at 2:43 PM, John Lumby wrote:
It is when some *other* backend gets there first with the ReadBuffer that
things are a bit trickier. The current version of the patch did polling for
that case
but that drew criticism, and so an immine
> From: st...@mit.edu
> Date: Mon, 23 Jun 2014 16:04:50 -0700
> Subject: Re: Extended Prefetching using Asynchronous IO - proposal and patch
> To: johnlu...@hotmail.com
> CC: klaussfre...@gmail.com; hlinnakan...@vmware.com;
> pgsql-hackers@postgresql.org
On Mon, Jun 23, 2014 at 2:43 PM, John Lumby wrote:
> It is when some *other* backend gets there first with the ReadBuffer that
> things are a bit trickier.The current version of the patch did polling
> for that case
> but that drew criticism,and so an imminent new version of the patch
> u
> Date: Thu, 19 Jun 2014 15:43:44 -0300
> Subject: Re: Extended Prefetching using Asynchronous IO - proposal and patch
> From: klaussfre...@gmail.com
> To: st...@mit.edu
> CC: hlinnakan...@vmware.com; johnlu...@hotmail.com;
> pgsql-hackers@postgresql.org
On 6/20/14, 5:12 PM, John Lumby wrote:
I also appreciate it is not easy to review the patch.
There are really 4 (or maybe 5) parts :
. async io (librt functions)
. buffer management (allocating, locking and pinning etc)
. scanners making prefetch calls
. statist
Fri, 20 Jun 2014 04:21:19 +0900
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal
> and patch
> From: masao.fu...@gmail.com
> To: johnlu...@hotmail.com
> CC: pgsql-hackers@postgresql.org; klaussfre...@gmail.com
>
> On Mon, Jun 9, 2014 at 11
On Mon, Jun 9, 2014 at 11:12 AM, johnlumby wrote:
> updated version of patch compatible with git head of 140608,
> (adjusted proc oid and a couple of minor fixes)
Compilation of patched version on MacOS failed. The error messages were
gcc -O0 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclarat
On Thu, Jun 19, 2014 at 2:49 PM, Greg Stark wrote:
> I don't think the threaded implementation on Linux is the one to use
> though. I find this *super* confusing but the kernel definitely
> supports aio syscalls, glibc also has a threaded implementation it
> uses if run on a kernel that doesn't im
On Wed, May 28, 2014 at 2:19 PM, Heikki Linnakangas
wrote:
> How portable is POSIX aio nowadays? Googling around, it still seems that on
> Linux, it's implemented using threads. Does the thread-emulation
> implementation cause problems with the rest of the backend, which assumes
> that there is on
I'm having trouble setting max_async_io_prefetchers in postgresql.conf
It says it cannot be changed.
Is that fixed at initdb time? (sounds like a bad idea if it is)
On Sun, Jun 8, 2014 at 11:12 PM, johnlumby wrote:
> updated version of patch compatible with git head of 140608,
> (adjusted proc
On 06/01/2014 03:44 AM, johnlumby wrote:
If you look at the new patch, you'll see that for the different-pid case,
I still call aio_suspend with a timeout.
As you or Claudio pointed out earlier,it could just as well sleep
for the same timeout,
but the small advantage of calling aio_suspend i
On Sat, May 31, 2014 at 9:44 PM, johnlumby wrote:
> I'll try to do some measuring of performance with:
> a) git head
> b) git head + patch as-is
> c) git head + patch without aio_suspend in foreign processes (just re-read)
> d) git head + patch with a lwlock (or whatever works) instead of aio_susp
On 05/31/14 20:44, johnlumby wrote:
On 05/30/14 09:36, Claudio Freire wrote:
Good point. I have included the guts of your little test program
(modified to do polling) into the existing autoconf test program
that decides on the
#define USE_AIO_ATOMIC_BUILTIN_COMP_SWAP.
See config/c-library.m
On Fri, May 30, 2014 at 4:15 AM, Heikki Linnakangas
wrote:
We don't want polling... And even if we did, calling aio_suspend() in a
way
that's known to be broken, in a loop, is a pretty crappy way of polling.
>>
>>
>> Well, as mentioned earlier, it is not broken. Whether it is
On 05/30/2014 12:53 AM, John Lumby wrote:
Date: Thu, 29 May 2014 18:00:28 -0300
Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal
and patch
From: klaussfre...@gmail.com
To: hlinnakan...@vmware.com
CC: johnlu...@hotmail.com; pgsql-hackers@postgresql.org
On Thu, May 29
On Thu, May 29, 2014 at 7:26 PM, Claudio Freire wrote:
> 1) If it's the same process, wait for the full timeout (no looping).
> If you have to loop (EAGAIN or EINTR - which I just noticed you don't
> check for), that's ok.
Sorry, meant to say just looping on EINTR.
About the style guidelines, no
On Thu, May 29, 2014 at 6:53 PM, John Lumby wrote:
> Well, as mentioned earlier, it is not broken. Whether it is efficient
> I am not sure.
> I have looked at the mutex in aio_suspend that you mentioned and I am not
> quite convinced that, if caller is not the original aio_read process,
> i
> Date: Thu, 29 May 2014 18:00:28 -0300
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal
> and patch
> From: klaussfre...@gmail.com
> To: hlinnakan...@vmware.com
> CC: johnlu...@hotmail.com; pgsql-hackers@postgresql.org
>
> >>>
Hi,
On 2014-05-29 17:53:51 -0400, John Lumby wrote:
> to see how we have done it. And I am attaching corrected version
> of your test program which runs just fine.
It's perfectly fine to not be up to the coding style at this point, but
trying to adhere to it to some degree will make code review
On Thu, May 29, 2014 at 7:11 PM, John Lumby wrote:
>> Nonetheless, getting the next tid out of the index may involve stepping
>> to the next index page, at which point you've lost your interlock
>
> I think we are ok as peeknexttuple (yes bad name, sorry, can change it
> ...
> never advances to
On Thu, May 29, 2014 at 6:56 PM, Tom Lane wrote:
> Claudio Freire writes:
>> On Thu, May 29, 2014 at 6:43 PM, Claudio Freire
>> wrote:
>>> On Thu, May 29, 2014 at 6:19 PM, Tom Lane wrote:
"ampeeknexttuple"? That's a bit scary. It would certainly be unsafe
for non-MVCC snapshots (re
> From: t...@sss.pgh.pa.us
> To: klaussfre...@gmail.com
> CC: hlinnakan...@vmware.com; johnlu...@hotmail.com;
> pgsql-hackers@postgresql.org
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal
> and patch
> Date: Thu, 29 May 2014 17:56:57 -040
Claudio Freire writes:
> On Thu, May 29, 2014 at 6:43 PM, Claudio Freire
> wrote:
>> On Thu, May 29, 2014 at 6:19 PM, Tom Lane wrote:
>>> "ampeeknexttuple"? That's a bit scary. It would certainly be unsafe
>>> for non-MVCC snapshots (read about vacuum vs indexscan interlocks in
>>> nbtree/REA
> Date: Thu, 29 May 2014 18:00:28 -0300
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal
> and patch
> From: klaussfre...@gmail.com
> To: hlinnakan...@vmware.com
> CC: johnlu...@hotmail.com; pgsql-hackers@postgresql.org
>
> On Thu, May 29,
On Thu, May 29, 2014 at 6:43 PM, Claudio Freire wrote:
> On Thu, May 29, 2014 at 6:19 PM, Tom Lane wrote:
>> Claudio Freire writes:
>>> Didn't fix that, but the attached patch does fix regression tests when
>>> scanning over index types other than btree (was invoking elog when the
>>> index am d
On Thu, May 29, 2014 at 6:19 PM, Tom Lane wrote:
> Claudio Freire writes:
>> Didn't fix that, but the attached patch does fix regression tests when
>> scanning over index types other than btree (was invoking elog when the
>> index am didn't have ampeeknexttuple)
>
> "ampeeknexttuple"? That's a b
Claudio Freire writes:
> Didn't fix that, but the attached patch does fix regression tests when
> scanning over index types other than btree (was invoking elog when the
> index am didn't have ampeeknexttuple)
"ampeeknexttuple"? That's a bit scary. It would certainly be unsafe
for non-MVCC snaps
On Thu, May 29, 2014 at 5:39 PM, Heikki Linnakangas
wrote:
> On 05/29/2014 11:34 PM, Claudio Freire wrote:
>>
>> On Thu, May 29, 2014 at 5:23 PM, Heikki Linnakangas
>> wrote:
>>>
>>> On 05/29/2014 04:12 PM, John Lumby wrote:
> On 05/28/2014 11:52 PM, John Lumby wrote:
>
> Th
On 05/29/2014 11:34 PM, Claudio Freire wrote:
On Thu, May 29, 2014 at 5:23 PM, Heikki Linnakangas
wrote:
On 05/29/2014 04:12 PM, John Lumby wrote:
On 05/28/2014 11:52 PM, John Lumby wrote:
The patch seems to assume that you can put the aiocb struct in shared
memory, initiate an asynchronous
; pgsql-hackers@postgresql.org
>>> CC: klaussfre...@gmail.com
>>> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO -
>>> proposal and patch
>>>
>>> On 05/28/2014 11:52 PM, John Lumby wrote:
>>>>
>>>>
>>>
>&
On 05/29/2014 04:12 PM, John Lumby wrote:
Thanks for looking at it!
Date: Thu, 29 May 2014 00:19:33 +0300
From: hlinnakan...@vmware.com
To: johnlu...@hotmail.com; pgsql-hackers@postgresql.org
CC: klaussfre...@gmail.com
Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO
El 28/05/2014 22:12, "Peter Geoghegan" escribió:
>
> On Wed, May 28, 2014 at 5:59 PM, Claudio Freire
wrote:
> > For nestloop, correct me if I'm wrong, but index scan nodes don't have
> > visibility of the next tuple to be searched for.
>
> Nested loop joins are considered a particularly compellin
sounds as though Claudio is on it too).
The area of exactly what the best prefetch strategy should be for
each particular type of scan and context is a good one to work on.
John
> Date: Wed, 28 May 2014 18:12:23 -0700
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - pr
Thanks for looking at it!
> Date: Thu, 29 May 2014 00:19:33 +0300
> From: hlinnakan...@vmware.com
> To: johnlu...@hotmail.com; pgsql-hackers@postgresql.org
> CC: klaussfre...@gmail.com
> Subject: Re: [HACKERS] Extended Prefetching using Asynchronous IO - proposal
> and patch
On Wed, May 28, 2014 at 5:59 PM, Claudio Freire wrote:
> For nestloop, correct me if I'm wrong, but index scan nodes don't have
> visibility of the next tuple to be searched for.
Nested loop joins are considered a particularly compelling case for
prefetching, actually.
--
Peter Geoghegan
--
On Wed, May 28, 2014 at 6:51 PM, Peter Geoghegan wrote:
> Have you thought about things like specialized prefetching for nested
> loop joins?
Currently, such a thing would need some non-trivial changes to the
execution nodes, I believe.
For nestloop, correct me if I'm wrong, but index scan nodes
On Tue, May 27, 2014 at 3:17 PM, John Lumby wrote:
> Below I am pasting the README we have written for this new functionality
> which mentions some of the measurements, advantages (and disadvantages)
> and we welcome all and any comments on this.
I think that this is likely to be a useful area to
On 05/28/2014 11:52 PM, John Lumby wrote:
The patch is attached.
It is based on clone of today's 9.4dev source.
I have noticed that this source is
(not suprisingly) quite a moving target at present,
meaning that this patch becomes stale quite quickly.
So although this copy is fine for reviewing,
48 matches
Mail list logo
