Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-21 Thread Tom Lane
Robert Haas writes: > On Mon, Feb 8, 2016 at 2:36 PM, Joshua D. Drake > wrote: >> I have no problem running any test cases you wish on a branch in a loop for >> the next week and reporting back any errors. > Well, what I've done is push into the buildfarm code that will allow > us to do *the mo

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-18 Thread Michael Paquier
On Thu, Feb 18, 2016 at 9:45 PM, Craig Ringer wrote: > That lets you make assertions about replication behaviour. It was built for > BDR and I think we'll need something along those lines in core if/when any > kind of logical replication facilities land, for things like testing > failover slots,

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-18 Thread Craig Ringer
On 18 February 2016 at 20:35, Michael Paquier wrote: > On Thu, Feb 18, 2016 at 5:35 PM, Amit Langote > wrote: > > On 2016/02/18 16:38, Craig Ringer wrote: > >> I should resurrect Abhijit's patch to allow the isolationtester to talk > to > >> multiple servers. We'll want that when we're doing tes

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-18 Thread Michael Paquier
On Thu, Feb 18, 2016 at 5:35 PM, Amit Langote wrote: > On 2016/02/18 16:38, Craig Ringer wrote: >> I should resurrect Abhijit's patch to allow the isolationtester to talk to >> multiple servers. We'll want that when we're doing tests like "assert that >> this change isn't visible on the replica be

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-18 Thread Amit Langote
On 2016/02/18 16:38, Craig Ringer wrote: > I should resurrect Abhijit's patch to allow the isolationtester to talk to > multiple servers. We'll want that when we're doing tests like "assert that > this change isn't visible on the replica before it becomes visible on the > master". (Well, except we

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-17 Thread Craig Ringer
On 9 February 2016 at 03:00, Joshua D. Drake wrote: > > I think this further points to the need for more reviewers and less > feature pushes. There are fundamental features that we could use, this is > one of them. It is certainly more important than say pgLogical or BDR (not > that those aren't

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-17 Thread Jim Nasby
On 2/8/16 4:39 PM, Peter Geoghegan wrote: On Mon, Feb 8, 2016 at 2:35 PM, Andres Freund wrote: I think having a public git tree, that contains the current state, is greatly helpful for that. Just announce that you're going to screw wildly with history, and that you're not going to be terribly c

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Andres Freund
Hi! Thanks for the answer. Sounds good. On 2016-02-08 18:47:18 -0500, Robert Haas wrote: > and if I'd gone out of my way to say "hey, everybody, here's a patch > that you might want to object to" I'm sure I could have found some > volunteers to do just that. But, you know, that's not really what

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 5:27 PM, Andres Freund wrote: >> > contentious issue, a few days after the initial post. Hm. Not sure how >> > you'd react if you weren't the author. >> >> Probably not very well. Do you want me to revert it? > > No. I want(ed) to express that I am not comfortable with how

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
On Mon, Feb 8, 2016 at 2:35 PM, Andres Freund wrote: > I think having a public git tree, that contains the current state, is > greatly helpful for that. Just announce that you're going to screw > wildly with history, and that you're not going to be terribly careful > about commit messages. That m

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Andres Freund
On 2016-02-08 15:18:13 -0500, Robert Haas wrote: > I agree that you had to be pretty deeply involved in that thread to > follow everything that was going on. But it's not entirely fair to > say that it was impossible for anyone else to get involved. Both > Amit and I, mostly Amit, posted directi

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Andres Freund
Hi Robert, On 2016-02-08 13:45:37 -0500, Robert Haas wrote: > > I realize that this stuff has all been brewing long, and that there's > > still a lot to do. So you gotta keep moving. And I'm not sure that > > there's anything wrong or if there's any actually better approach. But > > pushing an unr

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 4:54 PM, Peter Geoghegan wrote: > On Mon, Feb 8, 2016 at 1:45 PM, Robert Haas wrote: >> Far from the negligence that you seem to be implying, I think Amit was >> remarkably diligent about providing these kinds of updates. > > I don't think I remotely implied negligence. Tha

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
On Mon, Feb 8, 2016 at 1:45 PM, Robert Haas wrote: > Far from the negligence that you seem to be implying, I think Amit was > remarkably diligent about providing these kinds of updates. I don't think I remotely implied negligence. That word has very severe connotations (think "criminal negligence

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 4:11 PM, Peter Geoghegan wrote: > All that I wanted to do was look at EXPLAIN ANALYZE output that showed > a parallel seq scan on my laptop, simply because I wanted to see a > cool thing happen. I had to complain about it [1] to get clarification > from you [2]. > > I accept

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 01:11 PM, Peter Geoghegan wrote: On Mon, Feb 8, 2016 at 12:18 PM, Robert Haas wrote: I accept that this might have been a somewhat isolated incident (that I couldn't easily get *at least* a little instant gratification), but it still should be avoided. You've accused me of buryi

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
On Mon, Feb 8, 2016 at 12:18 PM, Robert Haas wrote: > So, there may be a person who knows how to do all of that > work and get it done in a reasonable time frame and also knows how to > make sure that everybody has the opportunity to be as involved in the > process as they want to be and that ther

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 2:48 PM, Peter Geoghegan wrote: > FWIW, I appreciate your candor. However, I think that you could have > done a better job of making things easier for reviewers, even if that > might not have made an enormous difference. I suspect I would have not > been able to get UPSERT d

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Alvaro Herrera
Robert Haas wrote: > Oh: another thing that I would like to do is commit the isolation > tests I wrote for the deadlock detector a while back, which nobody has > reviewed either, though Tom and Alvaro seemed reasonably positive > about the concept. Right now, the deadlock.c part of this patch isn

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 2:36 PM, Joshua D. Drake wrote: > I have no problem running any test cases you wish on a branch in a loop for > the next week and reporting back any errors. > > Where this gets tricky is the tooling itself. For me to be able to do so > (and others really) I need to be able t

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Peter Geoghegan
On Mon, Feb 8, 2016 at 10:45 AM, Robert Haas wrote: > And, by the way, the patch, aside from the deadlock.c portion, was > posted back in October, admittedly without much fanfare, but nobody > reviewed that or any other patch on this thread. If I'd waited for > those reviews to come in, parallel

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 11:24 AM, Robert Haas wrote: On Mon, Feb 8, 2016 at 2:00 PM, Joshua D. Drake wrote: If I am off base, please feel free to yell Latin at me again but isn't this exactly what different trees are for in Git? Would it be possible to say: Robert says, "Hey pull XYZ, run ABC tests. The

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Tom Lane
Robert Haas writes: > Oh: another thing that I would like to do is commit the isolation > tests I wrote for the deadlock detector a while back, which nobody has > reviewed either, though Tom and Alvaro seemed reasonably positive > about the concept. Possibly the reason that wasn't reviewed is tha

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 2:00 PM, Joshua D. Drake wrote: > If I am off base, please feel free to yell Latin at me again but isn't this > exactly what different trees are for in Git? Would it be possible to say: > > Robert says, "Hey pull XYZ, run ABC tests. They are what the parallelism > fixes do"?

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Joshua D. Drake
On 02/08/2016 10:45 AM, Robert Haas wrote: On Mon, Feb 8, 2016 at 10:17 AM, Andres Freund wrote: On 2016-02-02 15:41:45 -0500, Robert Haas wrote: I realize that this stuff has all been brewing long, and that there's still a lot to do. So you gotta keep moving. And I'm not sure that there's a

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Robert Haas
On Mon, Feb 8, 2016 at 10:17 AM, Andres Freund wrote: > On 2016-02-02 15:41:45 -0500, Robert Haas wrote: >> group-locking-v1.patch is a vastly improved version of the group >> locking patch that we discussed, uh, extensively last year. I realize >> that there was a lot of doubt about this approac

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-08 Thread Andres Freund
On 2016-02-02 15:41:45 -0500, Robert Haas wrote: > group-locking-v1.patch is a vastly improved version of the group > locking patch that we discussed, uh, extensively last year. I realize > that there was a lot of doubt about this approach, but I still believe > it's the right approach, I have put

Re: [HACKERS] a raft of parallelism-related bug fixes

2016-02-02 Thread Robert Haas
On Mon, Oct 19, 2015 at 12:02 PM, Robert Haas wrote: > On Sat, Oct 17, 2015 at 9:16 PM, Andrew Dunstan wrote: >> If all that is required is a #define, like CLOBBER_CACHE_ALWAYS, then no >> special buildfarm support is required - you would just add that to the >> animal's config file, more or less

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-11-06 Thread Robert Haas
On Mon, Nov 2, 2015 at 9:29 PM, Robert Haas wrote: > On Wed, Oct 28, 2015 at 10:23 AM, Robert Haas wrote: >> On Sun, Oct 18, 2015 at 12:17 AM, Robert Haas wrote: So reviewing patch 13 isn't possible without prior knowledge. >>> >>> The basic question for patch 13 is whether ephemeral record

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-11-02 Thread Robert Haas
On Wed, Oct 28, 2015 at 10:23 AM, Robert Haas wrote: > On Sun, Oct 18, 2015 at 12:17 AM, Robert Haas wrote: >>> So reviewing patch 13 isn't possible without prior knowledge. >> >> The basic question for patch 13 is whether ephemeral record types can >> occur in executor tuples in any contexts tha

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-28 Thread Robert Haas
On Sun, Oct 18, 2015 at 12:17 AM, Robert Haas wrote: >> So reviewing patch 13 isn't possible without prior knowledge. > > The basic question for patch 13 is whether ephemeral record types can > occur in executor tuples in any contexts that I haven't identified. I > know that a tuple table slot ca

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-22 Thread Robert Haas
On Wed, Oct 21, 2015 at 9:04 AM, Amit Langote wrote: > ... node *need* not be parallel aware? Yes, thanks. Committed that way. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-22 Thread Robert Haas
On Tue, Oct 20, 2015 at 6:12 PM, Simon Riggs wrote: > Not on your case in a big way, just noting the need for change there. Yes, I appreciate your attitude. I think we are on the same wavelength. > I'll help as well, but if you could start with enough basics to allow me to > ask questions that

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-21 Thread Amit Langote
On Wednesday, 21 October 2015, Amit Kapila wrote: > On Tue, Oct 20, 2015 at 8:16 PM, Robert Haas > wrote: > > > > On Sat, Oct 17, 2015 at 6:17 PM, Robert Haas > wrote: > > > It's good to have your perspective on how this can be improved, and > > > I'm definitely willing to write more documentat

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-21 Thread Amit Kapila
On Tue, Oct 20, 2015 at 8:16 PM, Robert Haas wrote: > > On Sat, Oct 17, 2015 at 6:17 PM, Robert Haas wrote: > > It's good to have your perspective on how this can be improved, and > > I'm definitely willing to write more documentation. Any lack in that > > area is probably due to being too close

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-20 Thread Simon Riggs
On 17 October 2015 at 18:17, Robert Haas wrote: > It's good to have your perspective on how this can be improved, and > I'm definitely willing to write more documentation. Any lack in that > area is probably due to being too close to the subject area, having > spent several years on parallelism

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-20 Thread Robert Haas
On Sat, Oct 17, 2015 at 6:17 PM, Robert Haas wrote: > It's good to have your perspective on how this can be improved, and > I'm definitely willing to write more documentation. Any lack in that > area is probably due to being too close to the subject area, having > spent several years on paralleli

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-19 Thread Robert Haas
On Sat, Oct 17, 2015 at 9:16 PM, Andrew Dunstan wrote: > If all that is required is a #define, like CLOBBER_CACHE_ALWAYS, then no > special buildfarm support is required - you would just add that to the > animal's config file, more or less like this: > > config_env => > { > CPPF

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-17 Thread Andrew Dunstan
On 10/17/2015 06:17 PM, Robert Haas wrote: However, I'm pretty sure that we don't want to switch the *entire* buildfarm to using lots of unnecessary parallelism. What we might be able to do is have some critters that people spin up for this precise purpose. Just like we currently have CLOBB

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-17 Thread Stephen Frost
* Noah Misch (n...@leadboat.com) wrote: > On Sat, Oct 17, 2015 at 06:17:37PM -0400, Robert Haas wrote: > > people are willing to run critters in that mode, I will be happy - > > more than happy, really - to put the test code into committable form, > > guarded by a #define, and away we go. > > I wo

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-17 Thread Noah Misch
On Sat, Oct 17, 2015 at 06:17:37PM -0400, Robert Haas wrote: > One idea that I think would provide > *excellent* test coverage is to take the test code included on this > thread and run it on the buildfarm. The idea of the code is to > basically run the regression test suite with every parallel-el

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-17 Thread Robert Haas
On Sat, Oct 17, 2015 at 9:16 AM, Simon Riggs wrote: > From reading this my understanding is that there isn't a test suite included > with this commit? Right. The patches on the thread contain code that can be used for testing, but the committed code does not itself include test coverage. I welco

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-17 Thread Simon Riggs
On 12 October 2015 at 18:04, Robert Haas wrote: > My recent commit of the Gather executor node has made it relatively > simple to write code that does an end-to-end test of all of the > parallelism-relate commits which have thus far gone into the tree. > I've been wanting to help here for a whi

Re: [HACKERS] a raft of parallelism-related bug fixes

2015-10-16 Thread Robert Haas
On Mon, Oct 12, 2015 at 1:04 PM, Robert Haas wrote: > Attached are 14 patches. Patches #1-#4 are > essential for testing purposes but are not proposed for commit, > although some of the code they contain may eventually become part of > other patches which are proposed for commit. Patches #5-#12