Re: [HACKERS] pg_background contrib module proposal

On 2/1/17 22:03, Michael Paquier wrote: > On Fri, Jan 27, 2017 at 11:38 PM, Robert Haas wrote: >> On Fri, Jan 27, 2017 at 9:14 AM, Peter Eisentraut >> wrote: >>> On 1/19/17 12:47 PM, Andrey Borodin wrote: 4. There is some controversy

Re: [HACKERS] pg_background contrib module proposal

On Fri, Jan 27, 2017 at 11:38 PM, Robert Haas wrote: > On Fri, Jan 27, 2017 at 9:14 AM, Peter Eisentraut > wrote: >> On 1/19/17 12:47 PM, Andrey Borodin wrote: >>> 4. There is some controversy on where implemented feature shall be: in >>>

Re: [HACKERS] pg_background contrib module proposal

2017-01-27 19:14 GMT+05:00 Peter Eisentraut : > I suppose we should decide first whether we want pg_background as a > separate extension or rather pursue extending dblink as proposed elsewhere. > > I don't know if pg_background allows any use case that dblink

Re: [HACKERS] pg_background contrib module proposal

On Fri, Jan 27, 2017 at 9:14 AM, Peter Eisentraut wrote: > On 1/19/17 12:47 PM, Andrey Borodin wrote: >> 4. There is some controversy on where implemented feature shall be: in >> separate extension (as in this patch), in db_link, in some PL API, in FDW or >>

Re: [HACKERS] pg_background contrib module proposal

On 1/19/17 12:47 PM, Andrey Borodin wrote: > 4. There is some controversy on where implemented feature shall be: in > separate extension (as in this patch), in db_link, in some PL API, in FDW or > somewhere else. I think that new extension is an appropriate place for the > feature. But I’m not

Re: [HACKERS] pg_background contrib module proposal

The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, failed I’ll summarize here my thoughts as a reviewer on the current

Re: [HACKERS] pg_background contrib module proposal

On Tue, Jan 10, 2017 at 7:09 AM, Jim Nasby wrote: > On 1/9/17 7:22 AM, amul sul wrote: >> >> On Sun, Jan 8, 2017 at 8:50 AM, Jim Nasby >> wrote: >>> >>> >> [skipped...] >>> >>> >>> Oh, hmm. So I guess if you do that when the background process

Re: [HACKERS] pg_background contrib module proposal

On 1/9/17 7:22 AM, amul sul wrote: On Sun, Jan 8, 2017 at 8:50 AM, Jim Nasby wrote: [skipped...] Oh, hmm. So I guess if you do that when the background process is idle it's the same as a close? I think we need some way to safeguard against accidental forkbombs

Re: [HACKERS] pg_background contrib module proposal

On Sun, Jan 8, 2017 at 8:50 AM, Jim Nasby wrote: > [skipped...] > > Oh, hmm. So I guess if you do that when the background process is idle it's > the same as a close? > > I think we need some way to safeguard against accidental forkbombs for cases > where users aren't

Re: [HACKERS] pg_background contrib module proposal

On 12/22/16 4:30 PM, Robert Haas wrote: On Thu, Dec 22, 2016 at 4:41 PM, Jim Nasby wrote: On 12/22/16 4:20 AM, amul sul wrote: • pg_background_detach : This API takes the process id and detach the background process. Stored worker's session is not dropped until this

Re: [HACKERS] pg_background contrib module proposal

On Sat, Jan 7, 2017 at 2:06 PM, Andrew Borodin wrote: > Hi! > > 2017-01-07 11:44 GMT+05:00 amul sul : > >> Changes: >> 1. pg_background_launch renamed to pg_background_start >> 2. pg_background_detach renamed to pg_background_close >> 3. Added new api to

Re: [HACKERS] pg_background contrib module proposal

Hi! 2017-01-07 11:44 GMT+05:00 amul sul : > Changes: > 1. pg_background_launch renamed to pg_background_start > 2. pg_background_detach renamed to pg_background_close > 3. Added new api to display previously launch background worker & its > result count waiting to be read.

Re: [HACKERS] pg_background contrib module proposal

Hi all, Attaching latest pg_background patch for review as per design proposed on 22 Dec '16 with following minor changes in the api. Changes: 1. pg_background_launch renamed to pg_background_start 2. pg_background_detach renamed to pg_background_close 3. Added new api to display previously

Re: [HACKERS] pg_background contrib module proposal

On Thu, Dec 22, 2016 at 4:41 PM, Jim Nasby wrote: > On 12/22/16 4:20 AM, amul sul wrote: >> • pg_background_detach : This API takes the process id and detach the >> background process. Stored worker's session is not dropped until this >> called. > > When I hear "detach"

Re: [HACKERS] pg_background contrib module proposal

On 12/22/16 4:20 AM, amul sul wrote: • pg_background_detach : This API takes the process id and detach the background process. Stored worker's session is not dropped until this called. When I hear "detach" I think that whatever I'm detaching from is going to stick around, which I don't think

Re: [HACKERS] pg_background contrib module proposal

Hi all, As we have discussed previously, we need to rework this patch as a client of Peter Eisentraut's background sessions code[1]. Attaching trial version patch to discussed possible design and api. We could have following APIs : • pg_background_launch : This function start and stores new

Re: [HACKERS] pg_background contrib module proposal

2016-12-21 20:42 GMT+05:00 Robert Haas : > This whole subthread seems like a distraction to me. I find it hard > to believe that this test case would be stable enough to survive the > buildfarm where, don't forget, we have things like > CLOBBER_CACHE_ALWAYS machines where

Re: [HACKERS] pg_background contrib module proposal

On Wed, Dec 21, 2016 at 10:42:18AM -0500, Robert Haas wrote: > On Wed, Dec 21, 2016 at 10:29 AM, David Fetter wrote: > > On Wed, Dec 21, 2016 at 06:31:52PM +0800, Craig Ringer wrote: > >> On 21 December 2016 at 14:26, Andrew Borodin wrote: > >> > >> > I'm

Re: [HACKERS] pg_background contrib module proposal

On Wed, Dec 21, 2016 at 10:29 AM, David Fetter wrote: > On Wed, Dec 21, 2016 at 06:31:52PM +0800, Craig Ringer wrote: >> On 21 December 2016 at 14:26, Andrew Borodin wrote: >> >> > I'm not sure every platform supports microsecond sleeps >> >> Windows at

Re: [HACKERS] pg_background contrib module proposal

On Wed, Dec 21, 2016 at 06:31:52PM +0800, Craig Ringer wrote: > On 21 December 2016 at 14:26, Andrew Borodin wrote: > > > I'm not sure every platform supports microsecond sleeps > > Windows at least doesn't by default, unless that changed in Win2k12 > and Win8 with the

Re: [HACKERS] pg_background contrib module proposal

On 21 December 2016 at 14:26, Andrew Borodin wrote: > I'm not sure every platform supports microsecond sleeps Windows at least doesn't by default, unless that changed in Win2k12 and Win8 with the same platform/kernel improvements that delivered

Re: [HACKERS] pg_background contrib module proposal

2016-12-21 4:55 GMT+05:00 David Fetter : > I see. > > I find the following a little easier to follow, and the sleeps still > work even when very short. Now I know a little more SQL :) Thank you. I'm not sure every platform supports microsecond sleeps. But it will work anyway.

Re: [HACKERS] pg_background contrib module proposal

On Tue, Dec 20, 2016 at 7:32 PM, Simon Riggs wrote: > On 9 December 2016 at 13:00, Robert Haas wrote: >> That might be good, because then we wouldn't have to maintain two >> copies of the code. > > So why are there two things at all? Why is this

Re: [HACKERS] pg_background contrib module proposal

On 9 December 2016 at 13:00, Robert Haas wrote: > That might be good, because then we wouldn't have to maintain two > copies of the code. So why are there two things at all? Why is this being worked on as well as Peter's patch? What will that give us? -- Simon Riggs

Re: [HACKERS] pg_background contrib module proposal

On Mon, Dec 19, 2016 at 09:30:32PM +0500, Andrew Borodin wrote: > 2016-12-19 4:21 GMT+05:00 David Fetter : > > Couldn't it sleep in increments smaller than a second? Like maybe > > milliseconds? Also, it's probably cleaner (or at least more > > comprehensible) to write

Re: [HACKERS] pg_background contrib module proposal

On Tue, Dec 20, 2016 at 11:11:36AM +0530, amul sul wrote: > On Tue, Dec 20, 2016 at 12:21 AM, David Fetter wrote: > > On Thu, Nov 24, 2016 at 09:16:53AM +0530, amul sul wrote: > >> Hi All, > >> > >> I would like to take over pg_background patch and repost for > >> discussion and

Re: [HACKERS] pg_background contrib module proposal

On Tue, Dec 20, 2016 at 12:21 AM, David Fetter wrote: > On Thu, Nov 24, 2016 at 09:16:53AM +0530, amul sul wrote: >> Hi All, >> >> I would like to take over pg_background patch and repost for >> discussion and review. > > This looks great. > > Sadly, it now breaks initdb when

Re: [HACKERS] pg_background contrib module proposal

On Thu, Nov 24, 2016 at 09:16:53AM +0530, amul sul wrote: > Hi All, > > I would like to take over pg_background patch and repost for > discussion and review. This looks great. Sadly, it now breaks initdb when applied atop dd728826c538f000220af98de025c00114ad0631 with: performing post-bootstrap

Re: [HACKERS] pg_background contrib module proposal

2016-12-19 4:21 GMT+05:00 David Fetter : > Couldn't it sleep in increments smaller than a second? Like maybe > milliseconds? Also, it's probably cleaner (or at least more > comprehensible) to write something using format() and dollar quoting > than the line with the double 's.

Re: [HACKERS] pg_background contrib module proposal

On Mon, Dec 12, 2016 at 10:17:24PM +0500, Andrew Borodin wrote: > Hi! > > Just in case you'd like to include sleepsort as a test, here it is > wrapped as a regression test(see attachment). But it has serious > downside: it runs no less than 5 seconds. Couldn't it sleep in increments smaller than

Re: [HACKERS] pg_background contrib module proposal

On 13 Dec. 2016 20:54, "amul sul" wrote: postgres=> select * from pg_background_result(67069) as (x text); ERROR: terminating connection due to administrator command CONTEXT: background worker, pid 67069 postgres=> It'll also want to handle cancellation due to conflict

Re: [HACKERS] pg_background contrib module proposal

On Tue, Dec 13, 2016 at 2:05 PM, Andrew Borodin wrote: > 2016-12-13 12:55 GMT+05:00 amul sul : >> I think background-session code is not that much deviated from >> pg_background code, > It is not derived, though it is not much deviated. background sessions

Re: [HACKERS] pg_background contrib module proposal

2016-12-13 12:55 GMT+05:00 amul sul : > I think background-session code is not that much deviated from > pg_background code, It is not derived, though it is not much deviated. background sessions code do not have detailed exceptions and code comments, but it is doing somewhat

Re: [HACKERS] pg_background contrib module proposal

On Mon, Dec 12, 2016 at 10:47 PM, Andrew Borodin wrote: > Hi! > Thanks a lot for your review. > Just in case you'd like to include sleepsort as a test, here it is > wrapped as a regression test(see attachment). But it has serious > downside: it runs no less than 5 seconds.

Re: [HACKERS] pg_background contrib module proposal

On 13 December 2016 at 01:17, Andrew Borodin wrote: > 6. Cancelation: a way to signal to background query that it's time to > quit gracefully. That at least should be fuss-free. SIGTERM it, and make sure the worker does CHECK_FOR_INTERRUPTS() in regularly-hit places and

Re: [HACKERS] pg_background contrib module proposal

Hi! Just in case you'd like to include sleepsort as a test, here it is wrapped as a regression test(see attachment). But it has serious downside: it runs no less than 5 seconds. Also I'll list here every parallelism feature I managed to imagine. It is not to say that I suggest having some of

Re: [HACKERS] pg_background contrib module proposal

2016-12-09 18:00 GMT+05:00 Robert Haas : > It looks like this could be reworked as a client of Peter Eisentraut's > background sessions code, which I think is also derived from > pg_background: > >

Re: [HACKERS] pg_background contrib module proposal

On Wed, Nov 23, 2016 at 10:46 PM, amul sul wrote: > I would like to take over pg_background patch and repost for > discussion and review. > > Initially Robert Haas has share this for parallelism demonstration[1] > and abandoned later with > summary of open issue[2] with this

Re: [HACKERS] pg_background contrib module proposal

2016-12-09 13:48 GMT+01:00 Andrew Borodin : > I've read the code and now I have more suggestions. > > 1. Easy one. The case of different natts of expected and acutal result > throws same errmsg as the case of wrong types. > I think we should assist the user more here > > if

Re: [HACKERS] pg_background contrib module proposal

I've read the code and now I have more suggestions. 1. Easy one. The case of different natts of expected and acutal result throws same errmsg as the case of wrong types. I think we should assist the user more here if (natts != tupdesc->natts) ereport(ERROR,

Re: [HACKERS] pg_background contrib module proposal

This is simply wonderful! Finaly, I can implement my favorite sleepsort in postgres: create table input as select round(random()*20) x from generate_series(1,5,1); create table output(place int,value int); create sequence s start 1; create table handles as select pg_background_launch('select

[HACKERS] pg_background contrib module proposal

Hi All, I would like to take over pg_background patch and repost for discussion and review. Initially Robert Haas has share this for parallelism demonstration[1] and abandoned later with summary of open issue[2] with this pg_background patch need to be fixed, most of them seems to be addressed