Re: [HACKERS] assessing parallel-safety

2015-09-16 Thread Robert Haas
On Thu, Sep 3, 2015 at 6:05 AM, Amit Kapila wrote: > On Wed, Jul 15, 2015 at 11:20 AM, Amit Kapila > wrote: >> I went ahead and completed this patch with respect to adding new clause >> in CREATE/ALTER FUNCTION which can allow users to define the >> parallel property for User defined functions. >

Re: [HACKERS] assessing parallel-safety

2015-09-03 Thread Amit Kapila
On Wed, Jul 15, 2015 at 11:20 AM, Amit Kapila wrote: > > > I went ahead and completed this patch with respect to adding new clause > in CREATE/ALTER FUNCTION which can allow users to define the > parallel property for User defined functions. > Attached, find the rebased patch which can be used to

Re: [HACKERS] assessing parallel-safety

2015-07-15 Thread Amit Kapila
On Thu, Jul 16, 2015 at 2:02 AM, Robert Haas wrote: > > exec_stmt_execsql is called by exec_stmt_open and exec_stmt_forc. > Those are cursor operations and thus - I think - parallelism can't be > used there. Right, but it also gets called from exec_stmt where a parallel-safe statement could be pa

Re: [HACKERS] assessing parallel-safety

2015-07-15 Thread Robert Haas
On Wed, Jul 15, 2015 at 1:50 AM, Amit Kapila wrote: > One issue which I think we should fix in this patch as pointed earlier > is, in some cases, Function containing Select stmt won't be able to > use parallel plan even though it is marked as parallel safe. > > create or replace function parallel_

Re: [HACKERS] assessing parallel-safety

2015-07-14 Thread Amit Kapila
On Fri, Jul 3, 2015 at 5:30 PM, Amit Kapila wrote: > > > Attached, find the rebased patch which can be used to review/test latest > version of parallel_seqscan patch which I am going to post in the parallel > seq. scan thread soonish. > I went ahead and completed this patch with respect to adding

Re: [HACKERS] assessing parallel-safety

2015-07-03 Thread Amit Kapila
On Thu, Jul 2, 2015 at 8:49 AM, Amit Kapila wrote: > > On Thu, May 21, 2015 at 10:19 PM, Robert Haas wrote: > > > > On Sat, Mar 21, 2015 at 2:30 PM, Thom Brown wrote: > > > Looks like one of the patches I applied is newer than the one in your list: > > > > > > HEAD Commit-id: 13a10c0ccd984643ef8

Re: [HACKERS] assessing parallel-safety

2015-07-01 Thread Amit Kapila
On Thu, May 21, 2015 at 10:19 PM, Robert Haas wrote: > > On Sat, Mar 21, 2015 at 2:30 PM, Thom Brown wrote: > > Looks like one of the patches I applied is newer than the one in your list: > > > > HEAD Commit-id: 13a10c0ccd984643ef88997ac177da7c4b7e46a6 > > parallel-mode-v9.patch > > assess-parall

Re: [HACKERS] assessing parallel-safety

2015-03-25 Thread Amit Kapila
On Fri, Mar 20, 2015 at 11:37 PM, Robert Haas wrote: > > > That might be a different crash than the first one you showed. But it > looks like the problem here is that the parallel sequential scan patch > is calling CreateParallelContext even though this is just an EXPLAIN > and we're not actually

Re: [HACKERS] assessing parallel-safety

2015-03-25 Thread Amit Kapila
On Sun, Mar 22, 2015 at 12:00 AM, Thom Brown wrote: > > On 21 March 2015 at 14:28, Amit Kapila wrote: >> >> On Fri, Mar 20, 2015 at 7:54 PM, Thom Brown wrote: >> > createdb pgbench >> > pgbench -i -s 200 pgbench >> > >> > CREATE TABLE pgbench_accounts_1 (CHECK (bid = 1)) INHERITS (pgbench_accoun

Re: [HACKERS] assessing parallel-safety

2015-03-21 Thread Thom Brown
On 21 March 2015 at 14:28, Amit Kapila wrote: > On Fri, Mar 20, 2015 at 7:54 PM, Thom Brown wrote: > > createdb pgbench > > pgbench -i -s 200 pgbench > > > > CREATE TABLE pgbench_accounts_1 (CHECK (bid = 1)) INHERITS > (pgbench_accounts); > > ... > > CREATE TABLE pgbench_accounts_200 (CHECK (bid

Re: [HACKERS] assessing parallel-safety

2015-03-21 Thread Amit Kapila
On Fri, Mar 20, 2015 at 7:54 PM, Thom Brown wrote: > > On 20 March 2015 at 13:55, Thom Brown wrote: > > On 20 March 2015 at 13:16, Alvaro Herrera wrote: > >> Thom Brown wrote: > >>> On 18 March 2015 at 16:01, Robert Haas wrote: > >>> > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas wrote: > >>>

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Robert Haas
On Fri, Mar 20, 2015 at 1:24 PM, Thom Brown wrote: >> OK, thanks. That looks like it's probably the fault of parallel seq >> scan patch rather than this one. It would help if you could build >> with debug symbols so that we can see line numbers and arguments. > > Sure. > > Program received signa

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Thom Brown
On 20 March 2015 at 15:25, Robert Haas wrote: > On Fri, Mar 20, 2015 at 11:08 AM, Thom Brown wrote: >> On 20 March 2015 at 15:02, Robert Haas wrote: >>> On Fri, Mar 20, 2015 at 10:24 AM, Thom Brown wrote: 2015-03-20 14:19:31 GMT [4273]: [78-1] user=,db=,client= DEBUG: server process (

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Robert Haas
On Fri, Mar 20, 2015 at 11:08 AM, Thom Brown wrote: > On 20 March 2015 at 15:02, Robert Haas wrote: >> On Fri, Mar 20, 2015 at 10:24 AM, Thom Brown wrote: >>> 2015-03-20 14:19:31 GMT [4273]: [78-1] user=,db=,client= DEBUG: >>> server process (PID 4285) was terminated by signal 11: Segmentation >

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Thom Brown
On 20 March 2015 at 15:02, Robert Haas wrote: > On Fri, Mar 20, 2015 at 10:24 AM, Thom Brown wrote: >> 2015-03-20 14:19:31 GMT [4273]: [78-1] user=,db=,client= DEBUG: >> server process (PID 4285) was terminated by signal 11: Segmentation >> fault > > Any chance you can get us a stack backtrace of

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Robert Haas
On Fri, Mar 20, 2015 at 10:24 AM, Thom Brown wrote: > 2015-03-20 14:19:31 GMT [4273]: [78-1] user=,db=,client= DEBUG: > server process (PID 4285) was terminated by signal 11: Segmentation > fault Any chance you can get us a stack backtrace of this crash? -- Robert Haas EnterpriseDB: http://www.

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Thom Brown
On 20 March 2015 at 13:55, Thom Brown wrote: > On 20 March 2015 at 13:16, Alvaro Herrera wrote: >> Thom Brown wrote: >>> On 18 March 2015 at 16:01, Robert Haas wrote: >>> > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas >>> > wrote: >>> >> On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: >>>

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Thom Brown
On 20 March 2015 at 13:16, Alvaro Herrera wrote: > Thom Brown wrote: >> On 18 March 2015 at 16:01, Robert Haas wrote: >> > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas wrote: >> >> On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: >> >>> Neither that rule, nor its variant downthread, would hur

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Alvaro Herrera
Thom Brown wrote: > On 18 March 2015 at 16:01, Robert Haas wrote: > > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas wrote: > >> On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: > >>> Neither that rule, nor its variant downthread, would hurt operator > >>> authors too > >>> much. To make the p

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Thom Brown
On 18 March 2015 at 16:01, Robert Haas wrote: > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas wrote: >> On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: >>> Neither that rule, nor its variant downthread, would hurt operator authors >>> too >>> much. To make the planner categorically parallel-

Re: [HACKERS] assessing parallel-safety

2015-03-20 Thread Amit Kapila
On Thu, Mar 19, 2015 at 8:53 PM, Amit Kapila wrote: > > On Wed, Mar 18, 2015 at 9:31 PM, Robert Haas wrote: > > > > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas wrote: > > > On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: > > >> Neither that rule, nor its variant downthread, would hurt opera

Re: [HACKERS] assessing parallel-safety

2015-03-19 Thread Amit Kapila
On Wed, Mar 18, 2015 at 9:31 PM, Robert Haas wrote: > > On Tue, Mar 17, 2015 at 9:48 AM, Robert Haas wrote: > > On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: > >> Neither that rule, nor its variant downthread, would hurt operator authors too > >> much. To make the planner categorically par

Re: [HACKERS] assessing parallel-safety

2015-03-19 Thread Amit Kapila
On Thu, Mar 19, 2015 at 7:05 PM, Amit Kapila wrote: > > On Wed, Feb 18, 2015 at 10:53 PM, Robert Haas wrote: > > > > On Sun, Feb 15, 2015 at 6:24 PM, Noah Misch wrote: > > > It's important for parallelism to work under the extended query protocol, and > > > that's nontrivial. exec_parse_message

Re: [HACKERS] assessing parallel-safety

2015-03-19 Thread Amit Kapila
On Wed, Feb 18, 2015 at 10:53 PM, Robert Haas wrote: > > On Sun, Feb 15, 2015 at 6:24 PM, Noah Misch wrote: > > It's important for parallelism to work under the extended query protocol, and > > that's nontrivial. exec_parse_message() sets cursorOptions. > > exec_bind_message() runs the planner.

Re: [HACKERS] assessing parallel-safety

2015-03-17 Thread Robert Haas
On Tue, Mar 17, 2015 at 2:26 AM, Noah Misch wrote: > Neither that rule, nor its variant downthread, would hurt operator authors too > much. To make the planner categorically parallel-safe, though, means limiting > evaluate_function() to parallel-safe functions. That would dramatically slow > sel

Re: [HACKERS] assessing parallel-safety

2015-03-16 Thread Noah Misch
On Mon, Mar 16, 2015 at 02:38:34PM -0400, Robert Haas wrote: > On Sun, Mar 15, 2015 at 2:39 AM, Noah Misch wrote: > > On Thu, Mar 12, 2015 at 11:21:37AM -0400, Robert Haas wrote: > >> On Thu, Feb 19, 2015 at 1:19 AM, Noah Misch wrote: > >> > Rereading my previous message, I failed to make the bot

Re: [HACKERS] assessing parallel-safety

2015-03-16 Thread Tom Lane
Robert Haas writes: > On Mon, Mar 16, 2015 at 2:55 PM, Tom Lane wrote: >> Robert Haas writes: >>> Is there a reason not to make a rule that opclass members must be >>> parallel-safe? I ask because I think it's important that the process >>> of planning a query be categorically parallel-safe. >

Re: [HACKERS] assessing parallel-safety

2015-03-16 Thread Robert Haas
On Mon, Mar 16, 2015 at 2:55 PM, Tom Lane wrote: > Robert Haas writes: >> Is there a reason not to make a rule that opclass members must be >> parallel-safe? I ask because I think it's important that the process >> of planning a query be categorically parallel-safe. > > I'm not seeing the connec

Re: [HACKERS] assessing parallel-safety

2015-03-16 Thread Tom Lane
Robert Haas writes: > Is there a reason not to make a rule that opclass members must be > parallel-safe? I ask because I think it's important that the process > of planning a query be categorically parallel-safe. I'm not seeing the connection between those two statements. The planner doesn't us

Re: [HACKERS] assessing parallel-safety

2015-03-16 Thread Robert Haas
On Sun, Mar 15, 2015 at 2:39 AM, Noah Misch wrote: > On Thu, Mar 12, 2015 at 11:21:37AM -0400, Robert Haas wrote: >> On Thu, Feb 19, 2015 at 1:19 AM, Noah Misch wrote: >> > Rereading my previous message, I failed to make the bottom line clear: I >> > recommend marking eqsel etc. PROPARALLEL_UNSAF

Re: [HACKERS] assessing parallel-safety

2015-03-14 Thread Noah Misch
On Thu, Mar 12, 2015 at 11:21:37AM -0400, Robert Haas wrote: > On Thu, Feb 19, 2015 at 1:19 AM, Noah Misch wrote: > > Rereading my previous message, I failed to make the bottom line clear: I > > recommend marking eqsel etc. PROPARALLEL_UNSAFE but *not* checking an > > estimator's proparallel befor

Re: [HACKERS] assessing parallel-safety

2015-03-12 Thread Robert Haas
[ deferring responses to some points until a later time ] On Thu, Feb 19, 2015 at 1:19 AM, Noah Misch wrote: >> This seems backwards to me. If some hypothetical selectivity >> estimator were PROPARALLEL_UNSAFE, then any operator that uses that >> function would also need to be PROPARALLEL_UNSAFE

Re: [HACKERS] assessing parallel-safety

2015-02-18 Thread Noah Misch
On Wed, Feb 18, 2015 at 12:23:05PM -0500, Robert Haas wrote: > On Sun, Feb 15, 2015 at 6:24 PM, Noah Misch wrote: > > It's important for parallelism to work under the extended query protocol, > > and > > that's nontrivial. exec_parse_message() sets cursorOptions. > > exec_bind_message() runs the

Re: [HACKERS] assessing parallel-safety

2015-02-15 Thread Noah Misch
It's important for parallelism to work under the extended query protocol, and that's nontrivial. exec_parse_message() sets cursorOptions. exec_bind_message() runs the planner. We don't know if a parallel plan is okay before seeing max_rows in exec_execute_message(). On Sun, Feb 15, 2015 at 01:29

Re: [HACKERS] assessing parallel-safety

2015-02-13 Thread Noah Misch
On Fri, Feb 13, 2015 at 05:13:06PM -0500, Robert Haas wrote: > On Fri, Feb 13, 2015 at 12:10 AM, Noah Misch wrote: > > Given your wish to optimize, I recommend first investigating the earlier > > thought to issue eval_const_expressions() once per planner() instead of once > > per subquery_planner(

Re: [HACKERS] assessing parallel-safety

2015-02-13 Thread Robert Haas
On Fri, Feb 13, 2015 at 12:10 AM, Noah Misch wrote: > Given your wish to optimize, I recommend first investigating the earlier > thought to issue eval_const_expressions() once per planner() instead of once > per subquery_planner(). Compared to the parallelModeRequired/parallelModeOK > idea, it wo

Re: [HACKERS] assessing parallel-safety

2015-02-13 Thread Amit Kapila
On Fri, Feb 13, 2015 at 2:40 AM, Robert Haas wrote: > > On Thu, Feb 12, 2015 at 3:52 PM, Amit Kapila wrote: > >> Probably not, because many queries will scan multiple relations, and > >> we want to do all of this work just once per query. > > > > By this, do you mean to say that if there is any p

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Noah Misch
On Thu, Feb 12, 2015 at 07:40:12AM -0500, Robert Haas wrote: > On Thu, Feb 12, 2015 at 12:16 AM, Noah Misch wrote: > > That is a major mark against putting the check in simplify_function(), > > agreed. > > I do see one way to rescue that idea, which is this: put two flags, > parallelModeOK and p

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Robert Haas
On Thu, Feb 12, 2015 at 3:52 PM, Amit Kapila wrote: >> Probably not, because many queries will scan multiple relations, and >> we want to do all of this work just once per query. > > By this, do you mean to say that if there is any parallel-unsafe > expression (function call) in query, then we won

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Amit Kapila
On Thu, Feb 12, 2015 at 10:07 PM, Robert Haas wrote: > > On Thu, Feb 12, 2015 at 6:40 AM, Amit Kapila wrote: > > If we have to go this way, then isn't it better to evaluate the same > > when we are trying to create parallel path (something like in the > > parallel_seq scan patch - create_parallel

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Robert Haas
On Wed, Feb 11, 2015 at 3:21 PM, Robert Haas wrote: > I think we may want a dedicated parallel-safe property for functions > rather than piggybacking on provolatile ... I went through the current contents of pg_proc and tried to assess how much parallel-unsafe stuff we've got. I think there are

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Robert Haas
On Thu, Feb 12, 2015 at 6:40 AM, Amit Kapila wrote: > If we have to go this way, then isn't it better to evaluate the same > when we are trying to create parallel path (something like in the > parallel_seq scan patch - create_parallelscan_paths())? Probably not, because many queries will scan mul

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Robert Haas
On Thu, Feb 12, 2015 at 12:16 AM, Noah Misch wrote: > That is a major mark against putting the check in simplify_function(), agreed. I do see one way to rescue that idea, which is this: put two flags, parallelModeOK and parallelModeRequired, into PlannerGlobal. At the beginning of planning, set

Re: [HACKERS] assessing parallel-safety

2015-02-12 Thread Amit Kapila
On Thu, Feb 12, 2015 at 1:51 AM, Robert Haas wrote: > > > I think we may want a dedicated parallel-safe property for functions > rather than piggybacking on provolatile, but that will probably also > be changeable via ALTER FUNCTION, and stored rules won't get > miraculously updated. So this defi

Re: [HACKERS] assessing parallel-safety

2015-02-11 Thread Noah Misch
On Wed, Feb 11, 2015 at 03:21:12PM -0500, Robert Haas wrote: > On Wed, Feb 11, 2015 at 9:39 AM, Robert Haas wrote: > > This does not seem to work out nicely. The problem here is that > > simplify_function() gets called from eval_const_expressions() which > > gets called from a variety of places,

Re: [HACKERS] assessing parallel-safety

2015-02-11 Thread Robert Haas
On Wed, Feb 11, 2015 at 9:39 AM, Robert Haas wrote: > On Sun, Feb 8, 2015 at 12:28 PM, Robert Haas wrote: >> On Sun, Feb 8, 2015 at 11:31 AM, Noah Misch wrote: >>> On Sat, Feb 07, 2015 at 08:18:55PM -0500, Robert Haas wrote: There are a few problems with this design that I don't immediately

Re: [HACKERS] assessing parallel-safety

2015-02-11 Thread Robert Haas
On Sun, Feb 8, 2015 at 12:28 PM, Robert Haas wrote: > On Sun, Feb 8, 2015 at 11:31 AM, Noah Misch wrote: >> On Sat, Feb 07, 2015 at 08:18:55PM -0500, Robert Haas wrote: >>> There are a few problems with this design that I don't immediately >>> know how to solve: >>> >>> 1. I'm concerned that the

Re: [HACKERS] assessing parallel-safety

2015-02-08 Thread Robert Haas
On Sun, Feb 8, 2015 at 11:31 AM, Noah Misch wrote: > On Sat, Feb 07, 2015 at 08:18:55PM -0500, Robert Haas wrote: >> There are a few problems with this design that I don't immediately >> know how to solve: >> >> 1. I'm concerned that the query-rewrite step could substitute a query >> that is not p

Re: [HACKERS] assessing parallel-safety

2015-02-08 Thread Noah Misch
On Sat, Feb 07, 2015 at 08:18:55PM -0500, Robert Haas wrote: > There are a few problems with this design that I don't immediately > know how to solve: > > 1. I'm concerned that the query-rewrite step could substitute a query > that is not parallel-safe for one that is. The upper Query might > sti

[HACKERS] assessing parallel-safety

2015-02-07 Thread Robert Haas
Amit's parallel sequential scan assumes that we can enter parallel mode when the parallel sequential scan is initialized and exit parallel mode when the scan ends and all the code that runs in between will be happy with that. Unfortunately, that's not necessarily the case. There are two ways it c