Re: [HACKERS] Parallel Seq Scan

On Wed, Dec 2, 2015 at 12:06 PM, Michael Paquier wrote: > > On Sun, Nov 22, 2015 at 3:25 PM, Amit Kapila wrote: > > On Fri, Nov 20, 2015 at 11:34 PM, Robert Haas wrote: > >> > >> On Thu, Nov 19, 2015 at 11:59 PM, Amit

Re: [HACKERS] Parallel Seq Scan

On Wed, Dec 2, 2015 at 5:45 PM, Amit Kapila wrote: > I am fine with it. I think the further optimizations can be done > separately. Done. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:

Re: [HACKERS] Parallel Seq Scan

On Sun, Nov 22, 2015 at 3:25 PM, Amit Kapila wrote: > On Fri, Nov 20, 2015 at 11:34 PM, Robert Haas wrote: >> >> On Thu, Nov 19, 2015 at 11:59 PM, Amit Kapila >> wrote: >> > Isn't it better to destroy the memory for

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 20, 2015 at 11:34 PM, Robert Haas wrote: > > On Thu, Nov 19, 2015 at 11:59 PM, Amit Kapila wrote: > > Isn't it better to destroy the memory for readers array as that gets > > allocated > > even if there are no workers available for

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 19, 2015 at 11:59 PM, Amit Kapila wrote: > Isn't it better to destroy the memory for readers array as that gets > allocated > even if there are no workers available for execution? > > Attached patch fixes the issue by just destroying readers array. Well, then

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 19, 2015 at 9:29 PM, Robert Haas wrote: > > On Wed, Nov 18, 2015 at 10:41 PM, Amit Kapila wrote: > > I think whats going on here is that when any of the session doesn't > > get any workers, we shutdown the Gather node which internally

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 18, 2015 at 10:41 PM, Amit Kapila wrote: > I think whats going on here is that when any of the session doesn't > get any workers, we shutdown the Gather node which internally destroys > the dynamic shared memory segment as well. However the same is > needed

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 18, 2015 at 12:48 AM, Amit Kapila wrote: >> I suggest that we instead fix ExecParallelFinish() to be idempotent. >> Add a "bool finished" flag to ParallelExecutorInfo and return at once >> if it's already set. Get rid of the exposed >>

Re: [HACKERS] Parallel Seq Scan

On Tue, Nov 17, 2015 at 1:21 AM, Bert wrote: > Hey, > > I've just pulled and compiled the new code. > I'm running a TPC-DS like test on different PostgreSQL installations, but > running (max) 12queries in parallel on a server with 12cores. > I've configured max_parallel_degree

Re: [HACKERS] Parallel Seq Scan

edit: maybe this is more useful? :) (gdb) bt full #0 0x00490b56 in heap_parallelscan_nextpage () No symbol table info available. #1 0x00493fdf in heap_getnext () No symbol table info available. #2 0x005c0733 in SeqNext () No symbol table info available. #3

Re: [HACKERS] Parallel Seq Scan

Hi, this is the backtrace: gdb /var/lib/pgsql/9.6/data/ /var/lib/pgsql/9.6/data/core.7877 GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el7 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you

Re: [HACKERS] Parallel Seq Scan

On Tue, Nov 17, 2015 at 6:52 AM, Bert wrote: > edit: maybe this is more useful? :) Definitely. But if you've built with --enable-debug and not stripped the resulting executable, we ought to get line numbers as well, plus the arguments to each function on the stack. That

Re: [HACKERS] Parallel Seq Scan

On Mon, Nov 16, 2015 at 9:49 PM, Amit Kapila wrote: >> I don't understand this part. >> > > The code in question is as below: > > tqueueReceiveSlot(TupleTableSlot *slot, DestReceiver *self) > > { > .. > > if (tqueue->tupledesc != tupledesc || > > tqueue->remapinfo->natts

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 12, 2015 at 10:23 AM, Amit Kapila wrote: > Thanks for the report. The reason for this problem is that instrumentation > information from workers is getting aggregated multiple times. In > ExecShutdownGatherWorkers(), we call ExecParallelFinish where it >

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 18, 2015 at 12:59 AM, Robert Haas wrote: > > On Thu, Nov 12, 2015 at 10:23 AM, Amit Kapila wrote: > > Thanks for the report. The reason for this problem is that instrumentation > > information from workers is getting aggregated

Re: [HACKERS] Parallel Seq Scan

On Tue, Nov 17, 2015 at 4:51 PM, Bert wrote: > Hey Robert, > > Thank you for the help. As you might (not) know, I'm quite new to the > community, but I'm learning. with the help from people like you. > anyhow, find attached a third attempt to a valid backtrace file. > > This

Re: [HACKERS] Parallel Seq Scan

On Tue, Nov 17, 2015 at 11:22 PM, Robert Haas wrote: > > On Mon, Nov 16, 2015 at 9:49 PM, Amit Kapila wrote: > >> I don't understand this part. > >> > > > > Here the above check always passes as tqueue->tupledesc is not > > set due to which it

Re: [HACKERS] Parallel Seq Scan

Hey Robert, Thank you for the help. As you might (not) know, I'm quite new to the community, but I'm learning. with the help from people like you. anyhow, find attached a third attempt to a valid backtrace file. This run is compiled from commit 5f10b7a604c87fc61a2c20a56552301f74c9bd5f and your

Re: [HACKERS] Parallel Seq Scan

On Mon, Nov 16, 2015 at 4:35 AM, Robert Haas wrote: > > On Fri, Nov 13, 2015 at 10:46 AM, Thom Brown wrote: > >>> And perhaps associated PIDs? > >> > >> Yeah, that can be useful, if others also feel like it is important, I can > >> look into preparing a

Re: [HACKERS] Parallel Seq Scan

2015-11-16 14:17 GMT+01:00 Amit Kapila : > On Mon, Nov 16, 2015 at 4:35 AM, Robert Haas > wrote: > > > > On Fri, Nov 13, 2015 at 10:46 AM, Thom Brown wrote: > > >>> And perhaps associated PIDs? > > >> > > >> Yeah, that can be

Re: [HACKERS] Parallel Seq Scan

On Sat, Nov 14, 2015 at 10:12 PM, Amit Kapila wrote: > On Fri, Nov 13, 2015 at 11:05 PM, Jeff Janes wrote: >> >> On Wed, Nov 11, 2015 at 6:53 AM, Robert Haas >> wrote: >> > >> > I've committed most of this, except for some

Re: [HACKERS] Parallel Seq Scan

Hey, I've just pulled and compiled the new code. I'm running a TPC-DS like test on different PostgreSQL installations, but running (max) 12queries in parallel on a server with 12cores. I've configured max_parallel_degree to 2, and I get messages that backend processes crash. I am running the same

Re: [HACKERS] Parallel Seq Scan

On Mon, Nov 16, 2015 at 2:51 PM, Bert wrote: > I've just pulled and compiled the new code. > I'm running a TPC-DS like test on different PostgreSQL installations, but > running (max) 12queries in parallel on a server with 12cores. > I've configured max_parallel_degree to 2, and

Re: [HACKERS] Parallel Seq Scan

On Mon, Nov 16, 2015 at 7:39 AM, Robert Haas wrote: > On Sun, Nov 15, 2015 at 1:12 AM, Amit Kapila wrote: > > Thanks for the report. > > > > I think main reason of the leak in workers seems to be due the reason > > that one of the buffer used while

Re: [HACKERS] Parallel Seq Scan

On Mon, Nov 16, 2015 at 10:36 PM, Jeff Janes wrote: > > On Sat, Nov 14, 2015 at 10:12 PM, Amit Kapila wrote: > > On Fri, Nov 13, 2015 at 11:05 PM, Jeff Janes wrote: > > > > I think main reason of the leak in workers seems to

Re: [HACKERS] Parallel Seq Scan

On 16/11/15 12:05, Robert Haas wrote: On Fri, Nov 13, 2015 at 10:46 AM, Thom Brown wrote: And perhaps associated PIDs? Yeah, that can be useful, if others also feel like it is important, I can look into preparing a patch for the same. Thanks. Thom, what do you think the

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 13, 2015 at 10:46 AM, Thom Brown wrote: >>> And perhaps associated PIDs? >> >> Yeah, that can be useful, if others also feel like it is important, I can >> look into preparing a patch for the same. > > Thanks. Thom, what do you think the EXPLAIN output should look

Re: [HACKERS] Parallel Seq Scan

On Sun, Nov 15, 2015 at 1:12 AM, Amit Kapila wrote: > Thanks for the report. > > I think main reason of the leak in workers seems to be due the reason > that one of the buffer used while sending tuples (in function > BuildRemapInfo) > from worker to master is not getting

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 13, 2015 at 11:05 PM, Jeff Janes wrote: > > On Wed, Nov 11, 2015 at 6:53 AM, Robert Haas wrote: > > > > I've committed most of this, except for some planner bits that I > > didn't like, and after a bunch of cleanup. Instead, I committed

Re: [HACKERS] Parallel Seq Scan

On 13 November 2015 at 15:22, Amit Kapila wrote: > On Fri, Nov 13, 2015 at 7:59 PM, Thom Brown wrote: >> >> On 13 November 2015 at 13:38, Amit Kapila wrote: >> > On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule >> >

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 13, 2015 at 7:59 PM, Thom Brown wrote: > > On 13 November 2015 at 13:38, Amit Kapila wrote: > > On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule > > wrote: > >> > >> > >> yes - the another little bit unclean in

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 11, 2015 at 6:53 AM, Robert Haas wrote: > > I've committed most of this, except for some planner bits that I > didn't like, and after a bunch of cleanup. Instead, I committed the > consider-parallel-v2.patch with some additional planner bits to make > up for

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 13, 2015 at 9:16 PM, Thom Brown wrote: > > On 13 November 2015 at 15:22, Amit Kapila wrote: > > > > There will be hardly any difference in nodes for each worker and it could > > be very long plan for large number of workers. What kind of

Re: [HACKERS] Parallel Seq Scan

On 13 November 2015 at 13:38, Amit Kapila wrote: > On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule > wrote: >> >> >> yes - the another little bit unclean in EXPLAIN is number of workers. If I >> understand to the behave, the query is processed by

Re: [HACKERS] Parallel Seq Scan

On 13 November 2015 at 03:39, Amit Kapila wrote: > On Thu, Nov 12, 2015 at 9:05 PM, Thom Brown wrote: >> >> On 12 November 2015 at 15:23, Amit Kapila wrote: >> > On Wed, Nov 11, 2015 at 11:29 PM, Pavel Stehule >> >

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 13, 2015 at 10:56 AM, Robert Haas wrote: > > On Thu, Nov 12, 2015 at 10:39 PM, Amit Kapila wrote: > > The number of shared buffers hit could be different across different runs > > because the read sequence of parallel workers can't be

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule wrote: > > yes - the another little bit unclean in EXPLAIN is number of workers. If I > understand to the behave, the query is processed by two processes if > workers in the explain is one. > > You are right and I think

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 13, 2015 at 6:17 PM, Thom Brown wrote: > > > > > The number of shared buffers hit could be different across different runs > > because the read sequence of parallel workers can't be guaranteed, also > > I don't think same is even guaranteed for Seq Scan node, the other

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 12, 2015 at 10:39 PM, Amit Kapila wrote: > The number of shared buffers hit could be different across different runs > because the read sequence of parallel workers can't be guaranteed, also > I don't think same is even guaranteed for Seq Scan node, The

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 12, 2015 at 9:05 PM, Thom Brown wrote: > > On 12 November 2015 at 15:23, Amit Kapila wrote: > > On Wed, Nov 11, 2015 at 11:29 PM, Pavel Stehule > > wrote: > >> > >> Hi > >> > >> I have a first query > >> > >> I

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 11, 2015 at 11:29 PM, Pavel Stehule wrote: > > Hi > > I have a first query > > I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are differen > Thanks for the report. The reason for this problem is that instrumentation information from

Re: [HACKERS] Parallel Seq Scan

On 12 November 2015 at 15:23, Amit Kapila wrote: > On Wed, Nov 11, 2015 at 11:29 PM, Pavel Stehule > wrote: >> >> Hi >> >> I have a first query >> >> I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are >> differen >> > >

Re: [HACKERS] Parallel Seq Scan

2015-11-11 19:03 GMT+01:00 Thom Brown : > On 11 November 2015 at 17:59, Pavel Stehule > wrote: > > Hi > > > > I have a first query > > > > I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are > > differen > > > > postgres=# set

Re: [HACKERS] Parallel Seq Scan

Hi I have a first query I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are differen postgres=# set max_parallel_degree to 4; SET Time: 0.717 ms postgres=# EXPLAIN ANALYZE select count(*) from xxx where a % 10 = 0;

Re: [HACKERS] Parallel Seq Scan

On 11 November 2015 at 17:59, Pavel Stehule wrote: > Hi > > I have a first query > > I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are > differen > > postgres=# set max_parallel_degree to 4; > SET > Time: 0.717 ms > postgres=# EXPLAIN ANALYZE select

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 11, 2015 at 11:53 PM, Robert Haas wrote: > For those following along at home, here's a demo: > > rhaas=# \timing > Timing is on. > rhaas=# select * from pgbench_accounts where filler like '%a%'; > aid | bid | abalance | filler > -+-+--+

Re: [HACKERS] Parallel Seq Scan

On Mon, Nov 9, 2015 at 11:15 AM, Amit Kapila wrote: > Okay, I have updated the patch to make seq scan node parallel aware. > To make that happen we need to have parallel_aware flag both in Plan > as well as Path, so that we can pass that information from Path to Plan. > I

Re: [HACKERS] Parallel Seq Scan

On 11 November 2015 at 14:53, Robert Haas wrote: > On Mon, Nov 9, 2015 at 11:15 AM, Amit Kapila wrote: >> Okay, I have updated the patch to make seq scan node parallel aware. >> To make that happen we need to have parallel_aware flag both in Plan

Re: [HACKERS] Parallel Seq Scan

2015-11-11 16:18 GMT+01:00 Thom Brown : > On 11 November 2015 at 14:53, Robert Haas wrote: > > On Mon, Nov 9, 2015 at 11:15 AM, Amit Kapila > wrote: > >> Okay, I have updated the patch to make seq scan node parallel aware. > >> To

Re: [HACKERS] Parallel Seq Scan

On 2015/11/12 4:26, Robert Haas wrote: > On Wed, Nov 11, 2015 at 12:59 PM, Pavel Stehule > wrote: >> I have a first query >> >> I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are >> differen > > Hmm, I see I was right about people finding more bugs

Re: [HACKERS] Parallel Seq Scan

On 11 November 2015 at 19:51, Thom Brown wrote: > On 11 November 2015 at 19:26, Robert Haas wrote: >> On Wed, Nov 11, 2015 at 12:59 PM, Pavel Stehule >> wrote: >>> I have a first query >>> >>> I looked on EXPLAIN ANALYZE output

Re: [HACKERS] Parallel Seq Scan

2015-11-11 20:26 GMT+01:00 Robert Haas : > On Wed, Nov 11, 2015 at 12:59 PM, Pavel Stehule > wrote: > > I have a first query > > > > I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are > > differen > > Hmm, I see I was right

Re: [HACKERS] Parallel Seq Scan

On 11 November 2015 at 19:26, Robert Haas wrote: > On Wed, Nov 11, 2015 at 12:59 PM, Pavel Stehule > wrote: >> I have a first query >> >> I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are >> differen > > Hmm, I see I was

Re: [HACKERS] Parallel Seq Scan

On Wed, Nov 11, 2015 at 12:59 PM, Pavel Stehule wrote: > I have a first query > > I looked on EXPLAIN ANALYZE output and the numbers of filtered rows are > differen Hmm, I see I was right about people finding more bugs once this was committed. That didn't take long.

Re: [HACKERS] Parallel Seq Scan

On Fri, Nov 6, 2015 at 10:13 AM, Robert Haas wrote: > > On Thu, Nov 5, 2015 at 10:49 PM, Amit Kapila wrote: > > On Thu, Nov 5, 2015 at 11:54 PM, Robert Haas wrote: > >> I was thinking about this idea: > >> > >> 1. Add a

Re: [HACKERS] Parallel Seq Scan

On Sat, Nov 7, 2015 at 4:11 AM, Robert Haas wrote: > > On Fri, Oct 23, 2015 at 9:22 PM, Amit Kapila wrote: > > The base rel's consider_parallel flag won't be percolated to childrels, so > > even > > if we mark base rel as parallel capable, while

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 9:22 PM, Amit Kapila wrote: > The base rel's consider_parallel flag won't be percolated to childrels, so > even > if we mark base rel as parallel capable, while generating the path it won't > be considered. I think we need to find a way to pass on

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 5, 2015 at 12:52 AM, Haribabu Kommi wrote: >> Now instead of displaying Partial Seq Scan, if we just display Seq Scan, >> then it might confuse user, so it is better to add some thing indicating >> parallel node if we want to go this route. > > IMO, the

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 5, 2015 at 10:49 PM, Amit Kapila wrote: > On Thu, Nov 5, 2015 at 11:54 PM, Robert Haas wrote: >> I was thinking about this idea: >> >> 1. Add a parallel_aware flag to each plan. > > Okay, so shall we add it in generic Plan node or to

Re: [HACKERS] Parallel Seq Scan

On Thu, Nov 5, 2015 at 11:54 PM, Robert Haas wrote: > > I was thinking about this idea: > > 1. Add a parallel_aware flag to each plan. > Okay, so shall we add it in generic Plan node or to specific plan nodes like SeqScan, IndexScan, etc. To me, it appears that

Re: [HACKERS] Parallel Seq Scan

On Tue, Nov 3, 2015 at 9:41 PM, Amit Kapila wrote: > On Fri, Oct 23, 2015 at 4:41 PM, Amit Kapila > wrote: >> >> On Fri, Oct 23, 2015 at 10:33 AM, Robert Haas >> wrote: > > Please find the rebased partial seq scan patch

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 4:41 PM, Amit Kapila wrote: > > On Fri, Oct 23, 2015 at 10:33 AM, Robert Haas wrote: Please find the rebased partial seq scan patch attached with this mail. Robert suggested me off list that we should once try to see if we

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 30, 2015 at 11:12 PM, Noah Misch wrote: > On Wed, Oct 28, 2015 at 01:04:12AM +0100, Robert Haas wrote: >> Well, OK. That's not strictly a correctness issue, but here's an >> updated patch along the lines you suggested. > >> Finally, have setup_param_list set a new

Re: [HACKERS] Parallel Seq Scan

On Wed, Oct 28, 2015 at 01:04:12AM +0100, Robert Haas wrote: > Well, OK. That's not strictly a correctness issue, but here's an > updated patch along the lines you suggested. > Finally, have setup_param_list set a new ParamListInfo field, > paramMask, to the parameters actually used in the

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 24, 2015 at 6:31 PM, Noah Misch wrote: > On Sat, Oct 24, 2015 at 07:49:07AM -0400, Robert Haas wrote: >> On Fri, Oct 23, 2015 at 9:38 PM, Noah Misch wrote: >> > Since that specification permits ParamListInfo consumers to ignore >> > paramMask,

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 24, 2015 at 07:49:07AM -0400, Robert Haas wrote: > On Fri, Oct 23, 2015 at 9:38 PM, Noah Misch wrote: > > Since that specification permits ParamListInfo consumers to ignore > > paramMask, > > the plpgsql_param_fetch() change from copy-paramlistinfo-fixes.patch is

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 9:38 PM, Noah Misch wrote: > Since that specification permits ParamListInfo consumers to ignore paramMask, > the plpgsql_param_fetch() change from copy-paramlistinfo-fixes.patch is still > formally required. So why am I not just doing that, then? Seems

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 3:35 AM, Amit Kapila wrote: > Considering parallelism at RelOptInfo level in the way as done in patch, > won't consider the RelOptInfo's for child relations in case of Append node. > Refer build_simple_rel(). Hmm, true, but what can go wrong

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 10:33 AM, Robert Haas wrote: > > + /* > +* We can't finish transaction commit or abort until all of the > +* workers are dead. This means, in particular, that > we can't respond > +* to

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 5:45 PM, Robert Haas wrote: > > On Fri, Oct 23, 2015 at 3:35 AM, Amit Kapila wrote: > > Considering parallelism at RelOptInfo level in the way as done in patch, > > won't consider the RelOptInfo's for child relations in case

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 22, 2015 at 11:59:58PM -0400, Robert Haas wrote: > On Thu, Oct 15, 2015 at 8:23 PM, Noah Misch wrote: > > Agreed. More specifically, I had in mind for copyParamList() to check the > > mask while e.g. ExecEvalParamExtern() would either check nothing or merely > >

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 5:14 AM, Robert Haas wrote: > > On Tue, Oct 13, 2015 at 5:59 PM, Robert Haas wrote: > > - Although the changes in parallelpaths.c are in a good direction, I'm > > pretty sure this is not yet up to scratch. I am less sure

Re: [HACKERS] Parallel Seq Scan

On Tue, Oct 13, 2015 at 5:59 PM, Robert Haas wrote: > - Although the changes in parallelpaths.c are in a good direction, I'm > pretty sure this is not yet up to scratch. I am less sure exactly > what needs to be fixed, so I'll have to give some more thought to > that.

Re: [HACKERS] Parallel Seq Scan

On Tue, Oct 20, 2015 at 3:04 AM, Amit Kapila wrote: > I have rebased the partial seq scan patch based on the above committed > patches. Now for rescanning it reuses the dsm and to achieve that we > need to ensure that workers have been completely shutdown and then >

Re: [HACKERS] Parallel Seq Scan

Robert Haas writes: > On Thu, Oct 15, 2015 at 8:23 PM, Noah Misch wrote: >>> Would it work to define this as "if non-NULL, >>> params lacking a 1-bit may be safely ignored"? Or some other tweak >>> that basically says that you don't need to care about

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 8:23 PM, Noah Misch wrote: > Agreed. More specifically, I had in mind for copyParamList() to check the > mask while e.g. ExecEvalParamExtern() would either check nothing or merely > assert that any mask included the requested parameter. It would be

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 23, 2015 at 12:31 AM, Tom Lane wrote: > BTW, my Salesforce colleagues have been bit^H^H^Hgriping for quite some > time about the performance costs associated with translating between > plpgsql's internal PLpgSQL_datum-array format and the ParamListInfo >

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 16, 2015 at 9:51 AM, Robert Haas wrote: > > On Mon, Oct 5, 2015 at 8:20 AM, Amit Kapila wrote: > > [ new patch for heapam.c changes ] > > There's a second patch attached here as well, parallel-relaunch.patch, > which makes it possible

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 4:54 PM, Robert Haas wrote: > > On Sat, Oct 17, 2015 at 2:44 AM, Amit Kapila wrote: > > I am not denying from that fact, the point I wanted to convey here is that > > the logic guarded by "params != estate->paramLI" in

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 2:41 AM, Robert Haas wrote: > > On Fri, Oct 16, 2015 at 2:29 AM, Amit Kapila wrote: > >> Yeah, but I think the scenario is legitimate. When a query gets run > >> from within PL/pgsql, parallelism is an option, at least as

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 12:06 PM, Noah Misch wrote: > > On Sat, Oct 17, 2015 at 11:00:57AM +0530, Amit Kapila wrote: > > On Sat, Oct 17, 2015 at 6:15 AM, Noah Misch wrote: > > > On Thu, Oct 15, 2015 at 04:30:01PM +0530, Amit Kapila wrote: > > > > On Mon, Oct

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 11:45 AM, Amit Kapila wrote: > > Agreed and on looking at code, I think in below code, if we pass > parallelOK as true for the cases where Portal is non-null, such a > problem could happen. > > > static int > > exec_run_select(PLpgSQL_execstate

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 11:00:57AM +0530, Amit Kapila wrote: > On Sat, Oct 17, 2015 at 6:15 AM, Noah Misch wrote: > > On Thu, Oct 15, 2015 at 04:30:01PM +0530, Amit Kapila wrote: > > > On Mon, Oct 12, 2015 at 9:16 PM, Robert Haas > > > wrote: > > > >

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 2:44 AM, Amit Kapila wrote: > I am not denying from that fact, the point I wanted to convey here is that > the logic guarded by "params != estate->paramLI" in plpgsql_param_fetch > is only needed if cursors are in use otherwise we won't need them

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 2:15 AM, Amit Kapila wrote: > Agreed and on looking at code, I think in below code, if we pass > parallelOK as true for the cases where Portal is non-null, such a > problem could happen. > > and one such case is > > exec_stmt_return_query() > { >

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 16, 2015 at 2:29 AM, Amit Kapila wrote: >> Yeah, but I think the scenario is legitimate. When a query gets run >> from within PL/pgsql, parallelism is an option, at least as we have >> the code today. So if a Gather were present, and the query used a >>

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 6:52 PM, Robert Haas wrote: > > On Thu, Oct 15, 2015 at 7:00 AM, Amit Kapila wrote: > > > > From what I understood by looking at code in this area, I think the check > > params != estate->paramLI and code under it is

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 16, 2015 at 9:51 AM, Robert Haas wrote: > > On Mon, Oct 5, 2015 at 8:20 AM, Amit Kapila wrote: > > [ new patch for heapam.c changes ] > > I went over this in a fair amount of detail and reworked it somewhat. > The result is attached as

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 16, 2015 at 7:42 AM, Amit Kapila wrote: > I think due to above changes it will report sync location on each page > scan, don't we want to report it once at end of scan? I think reporting for each page is correct. Isn't that what the non-parallel case does?

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 11:38 PM, Haribabu Kommi wrote: > Some more tests that failed in similar configuration settings. > 1. Table that is created under a begin statement is not visible in the worker. > 2. permission problem in worker side for set role command. The

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 16, 2015 at 5:31 PM, Robert Haas wrote: > > On Fri, Oct 16, 2015 at 7:42 AM, Amit Kapila wrote: > > I think due to above changes it will report sync location on each page > > scan, don't we want to report it once at end of scan? > > I

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 04:30:01PM +0530, Amit Kapila wrote: > On Mon, Oct 12, 2015 at 9:16 PM, Robert Haas wrote: > > plpgsql_param_fetch() assumes that it can detect whether it's being > > called from copyParamList() by checking whether params != > > estate->paramLI. I

Re: [HACKERS] Parallel Seq Scan

On Sat, Oct 17, 2015 at 6:15 AM, Noah Misch wrote: > > On Thu, Oct 15, 2015 at 04:30:01PM +0530, Amit Kapila wrote: > > On Mon, Oct 12, 2015 at 9:16 PM, Robert Haas wrote: > > > plpgsql_param_fetch() assumes that it can detect whether it's being > > >

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 5:39 PM, Haribabu Kommi wrote: > > > > On Thu, Oct 15, 2015 at 6:32 PM, Amit Kapila wrote: > > On Wed, Oct 14, 2015 at 3:29 AM, Robert Haas wrote: > > I think this got messed up while rebasing on

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 6:32 PM, Amit Kapila wrote: > On Wed, Oct 14, 2015 at 3:29 AM, Robert Haas wrote: > I think this got messed up while rebasing on top of Gather node > changes, but nonetheless, I have changed it such that PartialSeqScan >

Re: [HACKERS] Parallel Seq Scan

On Mon, Oct 12, 2015 at 9:16 PM, Robert Haas wrote: > > On Sun, Oct 11, 2015 at 7:56 PM, Noah Misch wrote: > > I see no mention in this thread of varatt_indirect, but I anticipated > > datumSerialize() reacting to it the same way datumCopy() reacts. If

Re: [HACKERS] Parallel Seq Scan

On Wed, Oct 14, 2015 at 3:29 AM, Robert Haas wrote: > > On Tue, Oct 13, 2015 at 2:45 AM, Amit Kapila wrote: > > Attached is rebased patch for partial seqscan support. > > Review comments: > > - If you're going to pgindent execParallel.c, you need

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 1:51 AM, Noah Misch wrote: > Tests of prm->prmtype and paramLI->paramFetch appear superfluous. Given that > the paramListCopyHook callback would return a complete substitute > ParamListInfo, I wouldn't expect SerializeParamList() to examine the the >

Re: [HACKERS] Parallel Seq Scan

On Thu, Oct 15, 2015 at 7:00 AM, Amit Kapila wrote: > On Mon, Oct 12, 2015 at 9:16 PM, Robert Haas wrote: >> On Sun, Oct 11, 2015 at 7:56 PM, Noah Misch wrote: >> > I see no mention in this thread of varatt_indirect, but I

Re: [HACKERS] Parallel Seq Scan

On Fri, Oct 16, 2015 at 2:10 PM, Haribabu Kommi wrote: > On Thu, Oct 15, 2015 at 11:45 PM, Amit Kapila wrote: >> On Thu, Oct 15, 2015 at 5:39 PM, Haribabu Kommi >> wrote: >>> >>> >>> >>> On Thu, Oct 15, 2015 at 6:32

  1   2   3   4   5   >