On Tue, May 05, 2015 at 01:05:38PM -0400, Robert Haas wrote:
On Sat, May 2, 2015 at 12:35 PM, Noah Misch n...@leadboat.com wrote:
Perhaps, rather than model it as master M waiting on worker list W1|W2|W3,
model it with queue-nonempty and queue-nonfull events, one pair per queue.
That comment
On Thu, May 7, 2015 at 3:09 AM, Noah Misch n...@leadboat.com wrote:
Each individual queue has only a single reader and a single writer.
In your example, there would be three queues, not just one. Of
course, one could design a new shared memory data structure
representing a collection of
On Sat, May 2, 2015 at 12:35 PM, Noah Misch n...@leadboat.com wrote:
Perhaps, rather than model it as master M waiting on worker list W1|W2|W3,
model it with queue-nonempty and queue-nonfull events, one pair per queue.
Each individual queue has only a single reader and a single writer.
In your
On Tue, Apr 21, 2015 at 02:08:00PM -0400, Robert Haas wrote:
On Tue, Mar 24, 2015 at 3:26 PM, Andres Freund and...@2ndquadrant.com wrote:
[proposal: teach deadlock detector about parallel master waiting on workers]
deadlock.c is far from simple, and at least I don't find the control
flow to
On Wed, Apr 29, 2015 at 12:23 PM, Robert Haas robertmh...@gmail.com wrote:
So, I think it makes sense to split up this patch in two. There's no
real debate, AFAICS, about anything in the patch other than the
heavyweight locking stuff. So I'd like to go ahead and commit the
rest. That's
On Sun, Apr 26, 2015 at 2:03 PM, Euler Taveira eu...@timbira.com.br wrote:
On 19-03-2015 15:13, Robert Haas wrote:
On Wed, Mar 18, 2015 at 5:36 PM, Andres Freund and...@2ndquadrant.com
wrote:
Reading the README first, the rest later. So you can comment on my
comments, while I actually look
On Sun, Apr 26, 2015 at 9:57 PM, Amit Langote
langote_amit...@lab.ntt.co.jp wrote:
On 2015-04-22 AM 04:14, Robert Haas wrote:
We should check IsParallelWorker() for operations that are allowed in
the master during parallel mode, but not allowed in the workers - e.g.
the master can scan its own
On 19-03-2015 15:13, Robert Haas wrote:
On Wed, Mar 18, 2015 at 5:36 PM, Andres Freund and...@2ndquadrant.com wrote:
Reading the README first, the rest later. So you can comment on my
comments, while I actually look at the code. Parallelism, yay!
I'm also looking at this piece of code and
On 2015-04-22 AM 04:14, Robert Haas wrote:
We should check IsParallelWorker() for operations that are allowed in
the master during parallel mode, but not allowed in the workers - e.g.
the master can scan its own temporary relations, but its workers
can't. We should check IsInParallelMode()
On Tue, Apr 21, 2015 at 11:38 PM, Robert Haas robertmh...@gmail.com wrote:
After thinking about it a bit more, I realized that even if we settle
on some solution to that problem, there's another issues: the
wait-edges created by this system don't really have the same semantics
as regular
On Tue, Mar 24, 2015 at 3:26 PM, Andres Freund and...@2ndquadrant.com wrote:
You'd need some kind of
API that says pretend I'm waiting for this lock, but don't really
wait for it, and you'd need to be darn sure that you removed yourself
from the wait queue again before doing any other
On Mon, Apr 20, 2015 at 6:49 PM, Peter Geoghegan p...@heroku.com wrote:
I see that you're using git format-patch to generate this. But the
patch is only patch 1/4. Is that intentional? Where are the other
pieces?
I think that the parallel seqscan patch, and the assessing parallel
safety
On Tue, Mar 24, 2015 at 11:56 PM, Alvaro Herrera
alvhe...@2ndquadrant.com wrote:
Well, it's not actually the same message. They're all a bit
different. Or mostly all of them. And the variable part is not a
command name, as in the PreventTransactionChain() case, so it would
affect
On Thu, Mar 19, 2015 at 11:13 AM, Robert Haas robertmh...@gmail.com wrote:
Here is yet another version of this patch. In addition to the fixes
mentioned above, this version includes some minor rebasing around
recent commits, and also better handling of the case where we discover
that we
On Tue, Mar 24, 2015 at 3:26 PM, Andres Freund and...@2ndquadrant.com wrote:
+ - The currently active user ID and security context. Note that this is
+the fourth user ID we restore: the initial step of binding to the
correct
+database also involves restoring the authenticated
Robert Haas wrote:
On Wed, Mar 18, 2015 at 7:10 PM, Andres Freund and...@2ndquadrant.com wrote:
+ /*
+ * For now, parallel operations are required to be strictly
read-only.
+ * Unlike heap_update() and heap_delete(), an insert should never
create
+ * a combo CID,
On 2015-03-19 14:13:59 -0400, Robert Haas wrote:
On Wed, Mar 18, 2015 at 5:36 PM, Andres Freund and...@2ndquadrant.com wrote:
Reading the README first, the rest later. So you can comment on my
comments, while I actually look at the code. Parallelism, yay!
Sorry, we don't support parallelism
Andres Freund and...@2ndquadrant.com writes:
Also: Man, trying to understand the guts of deadlock.c only made me
understand how *friggin* expensive deadlock checking is. I'm really
rather surprised that it only infrequently causes problems.
The reason for that is that we only run deadlock
On 2015-03-18 12:02:07 -0400, Robert Haas wrote:
diff --git a/src/backend/access/heap/heapam.c
b/src/backend/access/heap/heapam.c
index cb6f8a3..173f0ba 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2234,6 +2234,17 @@ static HeapTuple
Hi,
Reading the README first, the rest later. So you can comment on my
comments, while I actually look at the code. Parallelism, yay!
On 2015-03-18 12:02:07 -0400, Robert Haas wrote:
+Instead, we take a more pragmatic approach: we try to make as many of the
+operations that are safe outside of
On Sun, Feb 15, 2015 at 11:48 AM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Feb 13, 2015 at 2:22 AM, Michael Paquier
michael.paqu...@gmail.com wrote:
On Thu, Feb 12, 2015 at 3:59 AM, Robert Haas robertmh...@gmail.com
wrote:
We're not seeing eye to eye here yet, since I don't accept
On Fri, Mar 6, 2015 at 7:01 AM, Amit Kapila amit.kapil...@gmail.com wrote:
Today, while testing parallel_seqscan patch, I encountered one
intermittent issue (it hangs in below function) and I have question
related to below function.
+void
+WaitForParallelWorkersToFinish(ParallelContext
On 2015-02-11 13:59:04 -0500, Robert Haas wrote:
On Tue, Feb 10, 2015 at 8:45 PM, Andres Freund and...@2ndquadrant.com wrote:
The only reason I'd like it to be active is because that'd *prohibit*
doing the crazier stuff. There seems little reason to not da it under
the additional protection
On Thu, Feb 12, 2015 at 3:59 AM, Robert Haas robertmh...@gmail.com wrote:
We're not seeing eye to eye here yet, since I don't accept your
example scenario and you don't accept mine. Let's keep discussing.
Meanwhile, here's an updated patch.
A lot of cool activity is showing up here, so
On Wed, Feb 11, 2015 at 1:59 PM, Robert Haas robertmh...@gmail.com wrote:
I'm not sure what you mean by the severity of the lock. How about
marking the locks that the worker inherited from the parallel master
and throwing an error if it tries to lock one of those objects in a
mode that it
On 2015-02-10 11:49:58 -0500, Robert Haas wrote:
On Sat, Feb 7, 2015 at 7:20 PM, Andres Freund and...@2ndquadrant.com wrote:
* Don't like CreateParallelContextForExtension much as a function name -
I don't think we normally equate the fact that code is located in a
loadable library with
On Sat, Feb 7, 2015 at 7:20 PM, Andres Freund and...@2ndquadrant.com wrote:
Observations:
* Some tailing whitespace in the readme. Very nice otherwise.
Fixed. Thanks.
* Don't like CreateParallelContextForExtension much as a function name -
I don't think we normally equate the fact that
Hi,
On 2015-02-06 22:43:21 -0500, Robert Haas wrote:
Here's v4, with that fixed and a few more tweaks.
If you attached files generated with 'git format-patch' I could directly
apply then with the commit message and such. All at once if it's
mutliple patches, as individual commits. On
On Wed, Feb 4, 2015 at 9:40 PM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Jan 30, 2015 at 12:08 PM, Robert Haas robertmh...@gmail.com
wrote:
Here's a new version. Andres mentioned previously that he thought it
would be a good idea to commit the addition of
On 1/30/15 11:08 AM, Robert Haas wrote:
The final patch attached her (parallel-dummy-v2.patch) has been
updated slightly to incorporate some prefetching logic. It's still
just demo code and is not intended for commit. I'm not sure whether
the prefetching logic can actually be made to improve
On Wed, Jan 21, 2015 at 2:11 AM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 20, 2015 at 9:52 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 20, 2015 at 9:41 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
It seems [WaitForBackgroundWorkerShutdown] has possibility to wait
On Wed, Jan 21, 2015 at 6:35 PM, Robert Haas robertmh...@gmail.com wrote:
On Wed, Jan 21, 2015 at 2:11 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
On Tue, Jan 20, 2015 at 9:52 PM, Robert Haas robertmh...@gmail.com
wrote:
On Tue, Jan 20, 2015 at 9:41 AM, Amit Kapila amit.kapil...@gmail.com
On Sat, Jan 17, 2015 at 3:40 AM, Robert Haas robertmh...@gmail.com wrote:
New patch attached. I'm going to take the risk of calling this v1
(previous versions have been 0.x), since I've now done something about
the heavyweight locking issue, as well as fixed the message-looping
bug Amit
On Tue, Jan 20, 2015 at 9:41 AM, Amit Kapila amit.kapil...@gmail.com wrote:
It seems [WaitForBackgroundWorkerShutdown] has possibility to wait forever.
Assume one of the worker is not able to start (not able to attach
to shared memory or some other reason), then status returned by
On Tue, Jan 20, 2015 at 9:52 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 20, 2015 at 9:41 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
It seems [WaitForBackgroundWorkerShutdown] has possibility to wait
forever.
Assume one of the worker is not able to start (not able to attach
On 2015-01-05 11:27:57 -0500, Robert Haas wrote:
On Sat, Jan 3, 2015 at 7:31 PM, Andres Freund and...@2ndquadrant.com wrote:
* I wonder if parallel contexts shouldn't be tracked via resowners
That is a good question. I confess that I'm somewhat fuzzy about
which things should be tracked
On Thu, Jan 15, 2015 at 9:09 AM, Amit Kapila amit.kapil...@gmail.com wrote:
On Thu, Jan 15, 2015 at 6:52 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Jan 15, 2015 at 7:00 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
+HandleParallelMessages(void)
+{
..
..
+ for (i = 0; i
On Thu, Jan 15, 2015 at 7:00 AM, Amit Kapila amit.kapil...@gmail.com wrote:
On Tue, Jan 13, 2015 at 1:33 AM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Jan 8, 2015 at 6:52 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
+ seg = dsm_attach(DatumGetInt32(main_arg));
Here, I think
On Tue, Jan 13, 2015 at 1:33 AM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Jan 8, 2015 at 6:52 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
+ seg = dsm_attach(DatumGetInt32(main_arg));
Here, I think DatumGetUInt32() needs to be used instead of
DatumGetInt32() as the segment
On Thu, Jan 15, 2015 at 6:52 PM, Robert Haas robertmh...@gmail.com wrote:
On Thu, Jan 15, 2015 at 7:00 AM, Amit Kapila amit.kapil...@gmail.com
wrote:
+HandleParallelMessages(void)
+{
..
..
+ for (i = 0; i pcxt-nworkers; ++i)
+ {
+ /*
+ * Read messages for as long as we have an
On Wed, Jan 7, 2015 at 3:54 PM, Jim Nasby jim.na...@bluetreble.com wrote:
Agreed, I was more concerned with calls to nextval(), which don't seem to be
prevented in parallel mode?
It looks prevented:
/*
* Forbid this during parallel operation because, to make it work,
* the
On Thu, Jan 8, 2015 at 6:52 AM, Amit Kapila amit.kapil...@gmail.com wrote:
+ seg = dsm_attach(DatumGetInt32(main_arg));
Here, I think DatumGetUInt32() needs to be used instead of
DatumGetInt32() as the segment handle is uint32.
OK, I'll change that in the next version.
--
Robert Haas
On Wed, Jan 7, 2015 at 11:03 PM, Robert Haas robertmh...@gmail.com wrote:
I have little doubt that this version is still afflicted with various
bugs, and the heavyweight locking issue remains to be dealt with, but
on the whole I think this is headed in the right direction.
On 6 January 2015 at 21:37, Simon Riggs si...@2ndquadrant.com wrote:
I get it now and agree
Yes, very much.
Should we copy both the top-level frame and the current subxid?
Hot Standby links subxids directly to the top-level, so this would be similar.
If we copied both, we wouldn't need to
On Tue, Jan 6, 2015 at 4:37 PM, Simon Riggs si...@2ndquadrant.com wrote:
So when you say Only the top frame of the transaction state stack is
copied you don't mean the top, you mean the bottom (the latest
subxact)? Which then becomes the top in the parallel worker? OK...
The item most recently
On 1/7/15, 9:39 AM, Robert Haas wrote:
sequence.c: Is it safe to read a sequence value in a parallel worker if the
cache_value is 1?
No, because the sequence cache isn't synchronized between the workers.
Maybe it would be safe if cache_value == 1, but there's not much
use-case: how often are
On Tue, Jan 6, 2015 at 9:37 PM, Jim Nasby jim.na...@bluetreble.com wrote:
CreateParallelContext(): Does it actually make sense to have nworkers=0?
ISTM that's a bogus case.
I'm not sure whether we'll ever use the zero-worker case in
production, but I've certainly found it useful for
On 7 January 2015 at 13:11, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 6, 2015 at 4:37 PM, Simon Riggs si...@2ndquadrant.com wrote:
So when you say Only the top frame of the transaction state stack is
copied you don't mean the top, you mean the bottom (the latest
subxact)? Which then
On 6 January 2015 at 16:33, Robert Haas robertmh...@gmail.com wrote:
These comments don’t have any explanation or justification
OK, I rewrote them. Hopefully it's better now.
Thanks for new version and answers.
* Transaction stuff strikes me as overcomplicated and error prone.
Given there
On Tue, Jan 6, 2015 at 3:04 PM, Simon Riggs si...@2ndquadrant.com wrote:
If you can explain it in more detail in comments and README, I may
agree. At present, I don't get it and it makes me nervous.
The comment says
Only the top frame of the transaction state stack is copied to a parallel
On 6 January 2015 at 21:01, Robert Haas robertmh...@gmail.com wrote:
On Tue, Jan 6, 2015 at 3:04 PM, Simon Riggs si...@2ndquadrant.com wrote:
If you can explain it in more detail in comments and README, I may
agree. At present, I don't get it and it makes me nervous.
The comment says
Only
On 1/6/15, 10:33 AM, Robert Haas wrote:
Entrypints?
Already noted by Andres; fixed in the attached version.
Perhaps we only parallelize while drinking beer... ;)
CreateParallelContext(): Does it actually make sense to have nworkers=0? ISTM
that's a bogus case. Also, since the number of
On Sat, Jan 3, 2015 at 7:31 PM, Andres Freund and...@2ndquadrant.com wrote:
A couple remarks:
* Shouldn't this provide more infrastructure to deal with the fact that
we might get less parallel workers than we had planned for?
Maybe, but I'm not really sure what that should look like. My
Robert Haas robertmh...@gmail.com writes:
On Sat, Jan 3, 2015 at 7:31 PM, Andres Freund and...@2ndquadrant.com wrote:
* I wonder if parallel contexts shouldn't be tracked via resowners
That is a good question. I confess that I'm somewhat fuzzy about
which things should be tracked via the
On 22 December 2014 at 19:14, Robert Haas robertmh...@gmail.com wrote:
Here is another new version, with lots of bugs fixed.
An initial blind review, independent of other comments already made on thread.
OK src/backend/access/heap/heapam.c
heapam.c prohibitions on update and delete look fine
On Mon, Jan 5, 2015 at 9:57 PM, Robert Haas robertmh...@gmail.com wrote:
On Sat, Jan 3, 2015 at 7:31 PM, Andres Freund and...@2ndquadrant.com
wrote:
* Doesn't the restriction in GetSerializableTransactionSnapshotInt()
apply for repeatable read just the same?
Yeah. I'm not sure whether
On Fri, Jan 2, 2015 at 9:04 AM, Amit Kapila amit.kapil...@gmail.com wrote:
While working on parallel seq-scan patch to adapt this framework, I
noticed few things and have questions regrading the same.
1.
Currently parallel worker just attaches to error queue, for tuple queue
do you expect it
On 2014-12-22 14:14:31 -0500, Robert Haas wrote:
On Fri, Dec 19, 2014 at 8:59 AM, Robert Haas robertmh...@gmail.com wrote:
And here is a new version.
Here is another new version, with lots of bugs fixed.
A couple remarks:
* Shouldn't this provide more infrastructure to deal with the fact
On Thu, Dec 18, 2014 at 1:23 AM, Robert Haas robertmh...@gmail.com wrote:
In the meantime, I had a good chat with Heikki on IM yesterday which
gave me some new ideas on how to fix up the transaction handling in
here, which I am working on implementing. So hopefully I will have
that by then.
On Thu, Dec 18, 2014 at 4:53 AM, Robert Haas robertmh...@gmail.com wrote:
On Wed, Dec 17, 2014 at 9:16 AM, Simon Riggs si...@2ndquadrant.com wrote:
On 12 December 2014 at 22:52, Robert Haas robertmh...@gmail.com wrote:
I would be remiss if I failed to mention that this patch includes work
by
On Wed, Dec 17, 2014 at 2:53 PM, Robert Haas robertmh...@gmail.com wrote:
In the meantime, I had a good chat with Heikki on IM yesterday which
gave me some new ideas on how to fix up the transaction handling in
here, which I am working on implementing. So hopefully I will have
that by then.
On 12 December 2014 at 22:52, Robert Haas robertmh...@gmail.com wrote:
I would be remiss if I failed to mention that this patch includes work
by my colleagues Amit Kapila, Rushabh Lathia, and Jeevan Chalke, as
well as my former colleague Noah Misch; and that it would not have
been possible
On Wed, Dec 17, 2014 at 9:16 AM, Simon Riggs si...@2ndquadrant.com wrote:
On 12 December 2014 at 22:52, Robert Haas robertmh...@gmail.com wrote:
I would be remiss if I failed to mention that this patch includes work
by my colleagues Amit Kapila, Rushabh Lathia, and Jeevan Chalke, as
well as
Attached is a patch that adds two new concepts: parallel mode, and
parallel contexts. The idea of this code is to provide a framework
for specific parallel things that you want to do, such as parallel
sequential scan or parallel sort. When you're in parallel mode,
certain operations - like DDL,
64 matches
Mail list logo
