Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-07 Thread Ashutosh Bapat
On Fri, Aug 8, 2014 at 8:54 AM, Kyotaro HORIGUCHI < horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hi, thank you for the comment. > > > Hi Kyotaro, > > I looked at the patches and felt that the approach taken here is too > > intrusive, considering that the feature is only for foreign scans. > > I agree

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-07 Thread Kyotaro HORIGUCHI
Hi, thank you for the comment. > Hi Kyotaro, > I looked at the patches and felt that the approach taken here is too > intrusive, considering that the feature is only for foreign scans. I agree to you premising that it's only for foreign scans but I regard it as an example of parallel execution pl

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-06 Thread Robert Haas
On Tue, Aug 5, 2014 at 7:05 AM, Ashutosh Bapat wrote: > There are quite a few members added to the generic Path, Plan structures, > whose use is is induced only through foreign scans. Each path now stores two > sets of costs, one with parallelism and one without. The parallel values > will make se

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-05 Thread Ashutosh Bapat
Hi Kyotaro, I looked at the patches and felt that the approach taken here is too intrusive, considering that the feature is only for foreign scans. There are quite a few members added to the generic Path, Plan structures, whose use is is induced only through foreign scans. Each path now stores two

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-01 Thread Kyotaro HORIGUCHI
Hello, > Hello, this is the new version which is complete to some extent > of parallelism based on postgres_fdw. > > This compares the costs for parallel and non-parallel execution > and choose parallel one if it is faster by some extent specified > by GUCs. The attached files are, > > 0001_par

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-08-01 Thread Kyotaro HORIGUCHI
Hello, this is the new version which is complete to some extent of parallelism based on postgres_fdw. This compares the costs for parallel and non-parallel execution and choose parallel one if it is faster by some extent specified by GUCs. The attached files are, 0001_parallel_exec_planning_v0.p

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-07-28 Thread Kyotaro HORIGUCHI
Hello, > In order to minimize the impact, what can be done is to execute > fetch_more_data() in asynchronous mode every time, when there only few rows > left to be consumed. So in current code below > 1019 /* > 1020 * Get some more tuples, if we've run out. > 1021 */ > 1022 if (f

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-07-28 Thread Kyotaro HORIGUCHI
Hello, thank you for the comment. > Hi Kyotaro, > fetch_more_rows() always runs "FETCH 100 FROM " on the foreign > server to get the next set of rows. The changes you have made seem to run > only the first FETCHes from all the nodes but not the subsequent ones. The > optimization will be helpful o

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-07-25 Thread Ashutosh Bapat
In order to minimize the impact, what can be done is to execute fetch_more_data() in asynchronous mode every time, when there only few rows left to be consumed. So in current code below 1019 /* 1020 * Get some more tuples, if we've run out. 1021 */ 1022 if (fsstate->next_tuple >=

Re: [HACKERS] Introducing coarse grain parallelism by postgres_fdw.

2014-07-25 Thread Ashutosh Bapat
Hi Kyotaro, fetch_more_rows() always runs "FETCH 100 FROM " on the foreign server to get the next set of rows. The changes you have made seem to run only the first FETCHes from all the nodes but not the subsequent ones. The optimization will be helpful only when there are less than 100 rows per pos