Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-21 Thread Kyotaro HORIGUCHI
At Mon, 19 Dec 2016 12:24:38 -0500, Robert Haas wrote in > Interesting idea. My bet is that nobody cares about dtrace very much. FWIW, I just had an inquiry about system tap for PostgreSQL but he

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Robert Haas
On Mon, Dec 19, 2016 at 6:35 PM, Thomas Munro wrote: > On Tue, Dec 20, 2016 at 11:12 AM, Robert Haas wrote: >> On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro >> wrote: >>> On Sat, Nov 26, 2016 at 1:55 AM, Thomas

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Thomas Munro
On Tue, Dec 20, 2016 at 11:12 AM, Robert Haas wrote: > On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro > wrote: >> On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro >> wrote: >>> Here's a new version to apply on top

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Robert Haas
On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro wrote: > On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro > wrote: >> Here's a new version to apply on top of dsa-v7.patch. > > Here's a version to go with dsa-v8.patch. All right, so I've

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-19 Thread Robert Haas
On Sun, Dec 18, 2016 at 10:33 PM, Thomas Munro wrote: > On Sat, Dec 17, 2016 at 5:41 AM, Robert Haas wrote: >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >>> Thoughts? >> >> Hearing no objections, I've gone

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-18 Thread Thomas Munro
On Sat, Dec 17, 2016 at 5:41 AM, Robert Haas wrote: > On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >> Thoughts? > > Hearing no objections, I've gone ahead and committed this. If that > makes somebody really unhappy I can revert it, but I am

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Alvaro Herrera
Robert Haas wrote: > I am not sure the issue was time so much as the ability to foresee all > the problems we'd want to solve. I think all that movement is okay. It's not like we're breaking things to no purpose. The amount of effort that has to go into making extensions compile with changed

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:36 PM, Alvaro Herrera wrote: > Robert Haas wrote: >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >> > Thoughts? >> >> Hearing no objections, I've gone ahead and committed this. If that >> makes somebody really

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:37 PM, Andres Freund wrote: > Yea, I don't think that's good either. I'm all for evolving APIs when > necessary, but constantly breaking the same API release after release > seems indicative of needing to spend a bit more time on it in the first >

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Andres Freund
On 2016-12-16 12:33:11 -0500, Robert Haas wrote: > On Fri, Dec 16, 2016 at 12:32 PM, Robert Haas wrote: > > On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: > >> On 2016-12-16 11:41:49 -0500, Robert Haas wrote: > >>> On Wed, Dec 14, 2016 at 3:25

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Andres Freund
On 2016-12-16 12:32:49 -0500, Robert Haas wrote: > On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: > > On 2016-12-16 11:41:49 -0500, Robert Haas wrote: > >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > >> > Thoughts? > >> > >> Hearing no

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > > Thoughts? > > Hearing no objections, I've gone ahead and committed this. If that > makes somebody really unhappy I can revert it, but I am betting that > the real story is that nobody cares about

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:32 PM, Robert Haas wrote: > On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: >> On 2016-12-16 11:41:49 -0500, Robert Haas wrote: >>> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >>> >

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Fri, Dec 16, 2016 at 12:28 PM, Andres Freund wrote: > On 2016-12-16 11:41:49 -0500, Robert Haas wrote: >> On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: >> > Thoughts? >> >> Hearing no objections, I've gone ahead and committed this. If that >>

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Andres Freund
On 2016-12-16 11:41:49 -0500, Robert Haas wrote: > On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > > Thoughts? > > Hearing no objections, I've gone ahead and committed this. If that > makes somebody really unhappy I can revert it, but I am betting that > the real

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-16 Thread Robert Haas
On Wed, Dec 14, 2016 at 3:25 PM, Robert Haas wrote: > Thoughts? Hearing no objections, I've gone ahead and committed this. If that makes somebody really unhappy I can revert it, but I am betting that the real story is that nobody cares about preserving T_ID(). -- Robert

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-14 Thread Robert Haas
On Mon, Dec 5, 2016 at 3:12 PM, Robert Haas wrote: > On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro > wrote: >> On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro >> wrote: >>> Here's a new version to apply on top of

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-05 Thread Robert Haas
On Thu, Dec 1, 2016 at 6:35 AM, Thomas Munro wrote: > On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro > wrote: >> Here's a new version to apply on top of dsa-v7.patch. > > Here's a version to go with dsa-v8.patch. Thomas has spent a

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-04 Thread Haribabu Kommi
On Thu, Dec 1, 2016 at 10:35 PM, Thomas Munro wrote: > On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro > wrote: > > Here's a new version to apply on top of dsa-v7.patch. > > Here's a version to go with dsa-v8.patch. Moved to next CF

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-12-01 Thread Thomas Munro
On Sat, Nov 26, 2016 at 1:55 AM, Thomas Munro wrote: > Here's a new version to apply on top of dsa-v7.patch. Here's a version to go with dsa-v8.patch. -- Thomas Munro http://www.enterprisedb.com dsa-area-for-executor-v4.patch Description: Binary data -- Sent

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-25 Thread Thomas Munro
On Fri, Nov 25, 2016 at 4:32 AM, Dilip Kumar wrote: > I have one more question, > > In V1 we were calling dsa_detach in ExecParallelCleanup and in > ParallelQueryMain, but it's removed in v2. > > Any specific reason ? > Does this need to be used differently ? > >

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-24 Thread Dilip Kumar
I have one more question, In V1 we were calling dsa_detach in ExecParallelCleanup and in ParallelQueryMain, but it's removed in v2. Any specific reason ? Does this need to be used differently ? ExecParallelCleanup(ParallelExecutorInfo *pei) { + if (pei->area != NULL) + { +

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-24 Thread Dilip Kumar
On Wed, Nov 23, 2016 at 5:42 PM, Thomas Munro wrote: > ... or we could allow DSA areas to be constructed inside existing > shmem, as in the attached patch which requires dsa_create_in_place, > from the patch at >

Re: [HACKERS] Creating a DSA area to provide work space for parallel execution

2016-11-23 Thread Thomas Munro
On Wed, Oct 5, 2016 at 10:32 AM, Thomas Munro wrote: > One obvious problem is that this patch results in at least *two* DSM > segments being created for every parallel query execution: the main > segment used for parallel execution, and then the initial segment >

[HACKERS] Creating a DSA area to provide work space for parallel execution

2016-10-04 Thread Thomas Munro
Hi hackers, A couple of months ago I proposed dynamic shared areas[1]. DSA areas are dynamically sized shared memory heaps that backends can use to share data, building on top of the existing DSM infrastructure. One target use case for DSA areas is to provide work space for parallel query