Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-02-09 Thread Robert Haas
On Fri, Feb 9, 2018 at 7:53 AM, Amit Kapila wrote: > I am not saying to allow other things. I am just replying to your > question that why can't we use PQsetSingleRowMode. I mean to say that > one can fetch the data parallelly via the usage of cursors (by having > restrictions like don't allow o

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-02-09 Thread Amit Kapila
On Thu, Feb 8, 2018 at 7:49 PM, Robert Haas wrote: > On Thu, Feb 8, 2018 at 9:04 AM, Amit Kapila wrote: >>> Also, if you're OK with not being able to do anything except fetch >>> from the cursor until you reach the end, then couldn't you just issue >>> the query without the cursor and use PQsetSi

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-02-08 Thread Robert Haas
On Thu, Feb 8, 2018 at 9:04 AM, Amit Kapila wrote: > I guess workers need to wait till leader become active and processes > the error message. So if you kill a worker, it doesn't die but sits there waiting for someone to run a command in the leader? That sounds terrible. >> Also, if you're OK w

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-02-08 Thread Amit Kapila
On Wed, Feb 7, 2018 at 9:47 PM, Robert Haas wrote: > On Mon, Jan 22, 2018 at 7:05 AM, Amit Kapila wrote: >> On error, workers should be terminated. What kind of problem do you >> have in mind? > > Hmm. Yeah, I guess that makes sense. If the only thing you can do is > fetch from the cursor -- a

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-02-07 Thread Robert Haas
On Mon, Jan 22, 2018 at 7:05 AM, Amit Kapila wrote: > On error, workers should be terminated. What kind of problem do you > have in mind? Hmm. Yeah, I guess that makes sense. If the only thing you can do is fetch from the cursor -- and you have to make sure to lock down protocol messages as we

Re: Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-02-07 Thread David Steele
Hi Tomas, On 1/22/18 7:05 AM, Amit Kapila wrote: > On Thu, Jan 18, 2018 at 12:59 AM, Robert Haas wrote: >> On Wed, Jan 17, 2018 at 8:53 AM, Simon Riggs wrote: >> >> Or we could go the other way and try to keep the workers running. I >> don't really like that because it ties down those workers f

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-01-22 Thread Amit Kapila
On Thu, Jan 18, 2018 at 12:59 AM, Robert Haas wrote: > On Wed, Jan 17, 2018 at 8:53 AM, Simon Riggs wrote: > > Or we could go the other way and try to keep the workers running. I > don't really like that because it ties down those workers for > potentially a long period of time, but that might b

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-01-17 Thread Robert Haas
On Wed, Jan 17, 2018 at 8:53 AM, Simon Riggs wrote: > At present, one major use of Cursors is in postgres_fdw. > > In that usage, the query executes and returns all the rows. No other > side execution is possible. True, although foreign tables using postgres_fdw can't be scanned in parallel for o

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-01-17 Thread Simon Riggs
On 2 November 2017 at 01:55, Robert Haas wrote: > On Wed, Nov 1, 2017 at 7:49 AM, Craig Ringer wrote: >> If the client wants to fetch in chunks it can use a portal and limited >> size fetches. That shouldn't (?) be parallel-unsafe, since nothing >> else can happen in the middle anyway. > > I beli

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2018-01-17 Thread Simon Riggs
On 17 October 2017 at 14:16, Robert Haas wrote: > On Sat, Oct 14, 2017 at 2:14 PM, Tomas Vondra > wrote: >> I propose is to add a new cursor option (PARALLEL), which would allow >> parallel plans for that particular user-defined cursor. Attached is an >> experimental patch doing this (I'm sure th

Re: [HACKERS] PATCH: enabling parallel execution for cursors explicitly (experimental)

2017-11-29 Thread Michael Paquier
On Thu, Nov 2, 2017 at 11:35 AM, Robert Haas wrote: > On Thu, Nov 2, 2017 at 8:01 AM, Craig Ringer wrote: >> That forces materialization, and I'm guessing part of Tomas's goal >> here is to prevent the need to materialize into a temp table / >> tuplestore / etc. > > I get that, but if you're runn