Andrew Dunstan wrote:
Stefan Kaltenbrunner wrote:
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
Um, FKs could conflict with each other too, so that by itself isn't
gonna fix anything.
Good point. Looks like we'll need to make a list of can't run in
parallel
Philip Warner wrote:
Andrew Dunstan wrote:
Unfortunately, it quite possibly would. You would not be able to build
two indexes on the same table in parallel, even though they wouldn't
have conflicting locks.
I suppose so, but:
1. By the same logic it might speed things up; it might
Andrew Dunstan wrote:
Andrew Dunstan wrote:
this works better but there is something fishy still - using the same
dump file I get a proper restore using pg_restore normally. If I
however use -m for a parallel one I only get parts (in this case only
243 of the 709 tables) of the
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
pg_restore: [archiver (db)] could not execute query: ERROR: deadlock
detected
DETAIL: Process 18100 waits for AccessExclusiveLock on relation
1460818342 of database 1460815284; blocked by process 18103.
Process 18103 waits for
Tom Lane wrote:
Stefan Kaltenbrunner [EMAIL PROTECTED] writes:
pg_restore: [archiver (db)] could not execute query: ERROR: deadlock
detected
DETAIL: Process 18100 waits for AccessExclusiveLock on relation
1460818342 of database 1460815284; blocked by process 18103.
Process 18103 waits
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
Hmm, I'll bet the restore code doesn't realize that this can't run in
parallel with index creation on either table ...
Yeah. Of course, it's never needed to bother with stuff like that till now.
The very simple fix is probably to run
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
Hmm, I'll bet the restore code doesn't realize that this can't run in
parallel with index creation on either table ...
Yeah. Of course, it's never needed to bother with stuff like that till now.
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
Um, FKs could conflict with each other too, so that by itself isn't
gonna fix anything.
Good point. Looks like we'll need to make a list of can't run in
parallel with items as well as strict dependencies.
Yeah, I was just thinking
Le lundi 29 septembre 2008, Tom Lane a écrit :
* Extend the archive format to provide some indication that restoring
this object requires exclusive access to these dependencies.
* Hardwire knowledge into pg_restore that certain types of objects
require exclusive access to their dependencies.
Dimitri Fontaine wrote:
Le lundi 29 septembre 2008, Tom Lane a écrit :
* Extend the archive format to provide some indication that restoring
this object requires exclusive access to these dependencies.
* Hardwire knowledge into pg_restore that certain types of objects
require exclusive
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
Um, FKs could conflict with each other too, so that by itself isn't
gonna fix anything.
Good point. Looks like we'll need to make a list of can't run in
parallel with items as well as strict dependencies.
Yeah, I was
Stefan Kaltenbrunner wrote:
Tom Lane wrote:
Andrew Dunstan [EMAIL PROTECTED] writes:
Tom Lane wrote:
Um, FKs could conflict with each other too, so that by itself isn't
gonna fix anything.
Good point. Looks like we'll need to make a list of can't run in
parallel with items as well as
+ if (strcmp(te-desc,CONSTRAINT) == 0
||
+ strcmp(te-desc,FK
CONSTRAINT) == 0 ||
+ strcmp(te-desc,CHECK
CONSTRAINT) == 0 ||
+
Philip Warner wrote:
+ if (strcmp(te-desc,CONSTRAINT) == 0
||
+ strcmp(te-desc,FK
CONSTRAINT) == 0 ||
+ strcmp(te-desc,CHECK
CONSTRAINT) == 0 ||
+
Andrew Dunstan wrote:
Unfortunately, it quite possibly would. You would not be able to build
two indexes on the same table in parallel, even though they wouldn't
have conflicting locks.
I suppose so, but:
1. By the same logic it might speed things up; it might build two
completely separate
Andrew Dunstan wrote:
this works better but there is something fishy still - using the same
dump file I get a proper restore using pg_restore normally. If I
however use -m for a parallel one I only get parts (in this case only
243 of the 709 tables) of the database restored ...
Andrew Dunstan wrote:
Attached is my WIP patch for parallel pg_restore. It's still very rough,
but seems to work.
Anyone who can test this with highend equipment would be helping some.
tried playing with this(on a 22Gb compressed dump using 4 connections)
but it does not seem to work at
Hi,
As I'm interested in this topic, I thought I'd take a look at the
patch. I have no capability to test it on high end hardware but did
some basic testing on my workstation and basic review of the patch.
I somehow had the impression that instead of creating a new connection
for each restore
Russell Smith wrote:
Hi,
As I'm interested in this topic, I thought I'd take a look at the
patch. I have no capability to test it on high end hardware but did
some basic testing on my workstation and basic review of the patch.
I somehow had the impression that instead of creating a new
Stefan Kaltenbrunner wrote:
Andrew Dunstan wrote:
Attached is my WIP patch for parallel pg_restore. It's still very
rough, but seems to work.
Anyone who can test this with highend equipment would be helping some.
tried playing with this(on a 22Gb compressed dump using 4 connections)
Andrew Dunstan wrote:
Do we know why we experience tuple concurrently updated errors if we
spawn thread too fast?
No. That's an open item.
Okay, I'll see if I can have a little more of a look into it. No
promises as the restore the restore isn't playing nicely.
the memory context is
Russell Smith wrote:
I'm sorry, I meant processes there. I'm aware there are no threads.
But my feeling was that when you forked with open files you got all of
the open file properties, including positions, and as you dupped the
descriptor, you share all that it's pointing to with every
This version of the patch should fix the shared file descriptor bug
Russell Smith noticed. It also disables the 1/2 second sleep between
forks, so the performance on a small db (regression) is vastly improved.
cheers
andrew
Index: pg_backup.h
Andrew Dunstan wrote:
This version of the patch should fix the shared file descriptor bug
Russell Smith noticed. It also disables the 1/2 second sleep between
forks, so the performance on a small db (regression) is vastly improved.
this works better but there is something fishy still -
Stefan Kaltenbrunner wrote:
Andrew Dunstan wrote:
This version of the patch should fix the shared file descriptor bug
Russell Smith noticed. It also disables the 1/2 second sleep between
forks, so the performance on a small db (regression) is vastly improved.
this works better but there
On Fri, 26 Sep 2008 17:10:44 -0400
Andrew Dunstan [EMAIL PROTECTED] wrote:
Yes, there are several funny things going on, including some stuff
with dependencies. I'll have a new patch tomorrow with luck. Thanks
for testing.
O.k. I took at look at the patch itself and although I don't
Joshua Drake wrote:
On Fri, 26 Sep 2008 17:10:44 -0400
Andrew Dunstan [EMAIL PROTECTED] wrote:
Yes, there are several funny things going on, including some stuff
with dependencies. I'll have a new patch tomorrow with luck. Thanks
for testing.
O.k. I took at look at the patch itself
27 matches
Mail list logo
