Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-11-01 Thread Robert Haas
On Thu, Nov 2, 2017 at 8:01 AM, Craig Ringer wrote: > That forces materialization, and I'm guessing part of Tomas's goal > here is to prevent the need to materialize into a temp table / > tuplestore / etc. I get that, but if you're running a query like "SELECT * FROM

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-11-01 Thread Craig Ringer
On 2 November 2017 at 10:01, Robert Haas wrote: > I think that still leaves a fair number of scenarios to consider, and > the error handling by itself seems pretty thorny. Plus it's kind of a > weird mode and, like Craig, I'm not really sure what it gets you. > Maybe if

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-11-01 Thread Robert Haas
On Wed, Nov 1, 2017 at 3:47 AM, Tomas Vondra wrote: > But maybe there's a simpler option - what if we only allow fetches from > the PARALLEL cursor while the cursor is open? That is, this would work: > > BEGIN; > ... > DECLARE x PARALLEL CURSOR FOR SELECT

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-11-01 Thread Robert Haas
On Wed, Nov 1, 2017 at 7:49 AM, Craig Ringer wrote: > If the client wants to fetch in chunks it can use a portal and limited > size fetches. That shouldn't (?) be parallel-unsafe, since nothing > else can happen in the middle anyway. I believe sending a limited-size fetch

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-31 Thread Craig Ringer
> Now, I agree this is somewhat more limited than I hoped for, but OTOH it > still solves the issue I initially aimed for (processing large query > results in efficient way). I don't quite understand this part. We already send results to the client in a stream unless it's something we have to

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-31 Thread Tomas Vondra
Hi, On 10/20/2017 03:23 PM, Robert Haas wrote: > > ... > > The main points I want to make clearly understood is the current > design relies on (1) functions being labeled correctly and (2) other > dangerous code paths being unreachable because there's nothing that > runs between

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-20 Thread Robert Haas
On Tue, Oct 17, 2017 at 12:06 PM, Tomas Vondra wrote: > In general, it may be acceptable to rely on the elog() checks - which is > pretty much what the FETCH+INSERT+SHARE example I shared in the first > message of this thread does. I agree it's not particularly

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-17 Thread Tomas Vondra
On 10/17/2017 03:16 PM, Robert Haas wrote: > On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra > wrote: >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attached is an >> experimental

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-17 Thread Robert Haas
On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra wrote: > I propose is to add a new cursor option (PARALLEL), which would allow > parallel plans for that particular user-defined cursor. Attached is an > experimental patch doing this (I'm sure there are some loose ends).

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-16 Thread Tomas Vondra
On 10/16/2017 01:13 PM, Amit Kapila wrote: > On Sat, Oct 14, 2017 at 11:44 PM, Tomas Vondra > wrote: >> Hi, >> >> >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attached is an >>

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-16 Thread Amit Kapila
On Sat, Oct 14, 2017 at 11:44 PM, Tomas Vondra wrote: > Hi, > > > I propose is to add a new cursor option (PARALLEL), which would allow > parallel plans for that particular user-defined cursor. Attached is an > experimental patch doing this (I'm sure there are some

[HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-10-14 Thread Tomas Vondra
Hi, One of the existing limitations of parallel query is that cursors generally do not benefit from it [1]. Commit 61c2e1a95f [2] improved the situation for cursors from procedural languages, but unfortunately for user-defined cursors parallelism is still disabled. For many use cases that is