On Fri, Jul 5, 2013 at 11:27 AM, Andres Freund and...@2ndquadrant.com wrote:
Hi Robert,
On 2013-07-02 09:31:23 -0400, Robert Haas wrote:
I have a few ideas for getting rid of the remaining uses of
SnapshotNow that I'd like to throw out there:
Is your current plan to get rid of SnapshotNow
Hi Robert,
On 2013-07-02 09:31:23 -0400, Robert Haas wrote:
I have a few ideas for getting rid of the remaining uses of
SnapshotNow that I'd like to throw out there:
Is your current plan to get rid of SnapshotNow entirely? I am wonder
because the changeset extraction needs to care and how the
On 2013-07-01 15:02:41 -0400, Robert Haas wrote:
* These are updated by GetSnapshotData. We initialize them this way
@@ -177,6 +188,9 @@ GetTransactionSnapshot(void)
else
CurrentSnapshot =
GetSnapshotData(CurrentSnapshotData);
+ /*
On Tue, Jul 2, 2013 at 9:02 AM, Andres Freund and...@2ndquadrant.com wrote:
Some things that might be worth changing when committing:
* Could you add a Assert(!RelationHasSysCache(relid)) to
RelationInvalidatesSnapshotsOnly? It's not unlikely that it will be
missed by the next person
On 2013-07-02 09:31:23 -0400, Robert Haas wrote:
On Tue, Jul 2, 2013 at 9:02 AM, Andres Freund and...@2ndquadrant.com wrote:
Some things that might be worth changing when committing:
* Could you add a Assert(!RelationHasSysCache(relid)) to
RelationInvalidatesSnapshotsOnly? It's not
On Tue, Jul 2, 2013 at 9:52 AM, Andres Freund and...@2ndquadrant.com wrote:
* possibly paranoid, but I'd add a Assert to heap_beginscan_catalog or
GetCatalogSnapshot ensuring we're dealing with a catalog relation. The
consistency mechanisms in GetCatalogSnapshot() only work for those, so
On 2013-07-02 10:38:17 -0400, Robert Haas wrote:
On Tue, Jul 2, 2013 at 9:52 AM, Andres Freund and...@2ndquadrant.com wrote:
* possibly paranoid, but I'd add a Assert to heap_beginscan_catalog or
GetCatalogSnapshot ensuring we're dealing with a catalog relation. The
consistency
On 02.07.2013 18:24, Andres Freund wrote:
I've quickly verified that it indeed uses them. I wish I hadn't. Brrr. I
can't even guess what that should do from the surrounding code/function
names. Except that it looks broken under concurrency as long as
SnapshotNow is used (because the query's
On 2013-06-28 23:14:23 -0400, Robert Haas wrote:
Here's a further update of this patch. In this version, I added some
mechanism to send a new kind of sinval message that is sent when a
catalog without catcaches is updated; it doesn't apply to all
catalogs, just to whichever ones we want to
On Mon, Jul 1, 2013 at 10:04 AM, Andres Freund and...@2ndquadrant.com wrote:
This is really cool stuff.
Thanks.
I have to say, if the thing that primarily suffers is pretty extreme DDL
in extreme situations I am not really worried. Anybody running anything
close to the territory of such
Robert Haas escribió:
All right, so here's a patch that does something along those lines.
We have to always take a new snapshot when somebody scans a catalog
that has no syscache, because there won't be any invalidation messages
to work off of in that case. The only catalog in that category
On Mon, Jun 17, 2013 at 8:12 AM, Andres Freund and...@2ndquadrant.com wrote:
So, the biggest issue with the patch seems to be performance worries. I
tried to create a worst case scenario:
postgres (patched and HEAD) running with:
-c shared_buffers=4GB \
-c max_connections=2000 \
-c
On 2013-06-20 09:45:26 -0400, Robert Haas wrote:
With that setup one can create quite a noticeable overhead for the mvcc
patch (best of 5):
master-optimize:
tps = 1261.629474 (including connections establishing)
tps = 15121.648834 (excluding connections establishing)
dev-optimize:
On Thu, Jun 20, 2013 at 10:35 AM, Andres Freund and...@2ndquadrant.com wrote:
But I'm still on the fence about whether this is really a valid test.
I think it shows that we need to be careful and do further performance
evaluations and/or alleviate the pain by making things cheaper (say, a
ddl
On 2013-06-20 10:58:59 -0400, Robert Haas wrote:
On Thu, Jun 20, 2013 at 10:35 AM, Andres Freund and...@2ndquadrant.com
wrote:
But I'm still on the fence about whether this is really a valid test.
I think it shows that we need to be careful and do further performance
evaluations and/or
On 2013-06-03 14:57:12 -0400, Robert Haas wrote:
On Thu, May 30, 2013 at 1:39 AM, Michael Paquier
michael.paqu...@gmail.com wrote:
+1.
Here's a more serious patch for MVCC catalog access. This one
involves more data copying than the last one, I think, because the
previous version did not
On 2013-06-06 12:49:14 -0400, Robert Haas wrote:
On Thu, Jun 6, 2013 at 5:30 AM, Andres Freund and...@2ndquadrant.com wrote:
+ * XXX: Now that we have MVCC catalog access, the reasoning above is no
longer
+ * true. Are there other good reasons to hard-code this, or should we
revisit
On Thu, Jun 6, 2013 at 2:48 PM, Jim Nasby j...@nasby.net wrote:
On 6/5/13 3:49 PM, Robert Haas wrote:
Now, I did find a couple that I thought should probably stick with
SnapshotNow, specifically pgrowlocks and pgstattuple.
FWIW, I've often wished for a way to make all stat access
On 2013-06-05 18:56:28 -0400, Tom Lane wrote:
Robert Haas robertmh...@gmail.com writes:
Now, I did find a couple that I thought should probably stick with
SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just
gathering statistical information, so there's no harm in having the
Hi Robert,
Took a quick look through the patch to understand what your current
revision is actually doing and to facilitate thinking about possible
pain points.
Here are the notes I made during my reading:
On 2013-06-03 14:57:12 -0400, Robert Haas wrote:
+++ b/src/backend/catalog/catalog.c
@@
On Thu, Jun 6, 2013 at 5:30 AM, Andres Freund and...@2ndquadrant.com wrote:
+ * XXX: Now that we have MVCC catalog access, the reasoning above is no
longer
+ * true. Are there other good reasons to hard-code this, or should we
revisit
+ * that decision?
We could just the function by
On 6/5/13 3:49 PM, Robert Haas wrote:
Now, I did find a couple that I thought should probably stick with
SnapshotNow, specifically pgrowlocks and pgstattuple.
FWIW, I've often wished for a way to make all stat access transactional, across all the
stats views. Perhaps that couldn't be done by
On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote:
We've had a number of discussions about the evils of SnapshotNow. As
far as I can tell, nobody likes it and everybody wants it gone, but
there is concern about the performance impact.
I was always under the impression
On 06/05/2013 04:28 PM, Greg Stark wrote:
On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote:
We've had a number of discussions about the evils of SnapshotNow. As
far as I can tell, nobody likes it and everybody wants it gone, but
there is concern about the performance
On 2013-06-05 15:28:09 +0100, Greg Stark wrote:
On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote:
We've had a number of discussions about the evils of SnapshotNow. As
far as I can tell, nobody likes it and everybody wants it gone, but
there is concern about the
Andres Freund and...@2ndquadrant.com writes:
On 2013-06-05 15:28:09 +0100, Greg Stark wrote:
I thought there were many call sites that were specifically depending
on seeing dirty reads to avoid race conditions with other backends --
which probably just narrowed the race condition or created
On 2013-06-05 11:35:58 -0400, Tom Lane wrote:
Andres Freund and...@2ndquadrant.com writes:
On 2013-06-05 15:28:09 +0100, Greg Stark wrote:
I thought there were many call sites that were specifically depending
on seeing dirty reads to avoid race conditions with other backends --
which
On Wed, Jun 5, 2013 at 10:28 AM, Greg Stark st...@mit.edu wrote:
On Wed, May 22, 2013 at 3:18 AM, Robert Haas robertmh...@gmail.com wrote:
We've had a number of discussions about the evils of SnapshotNow. As
far as I can tell, nobody likes it and everybody wants it gone, but
there is concern
Robert Haas robertmh...@gmail.com writes:
Now, I did find a couple that I thought should probably stick with
SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just
gathering statistical information, so there's no harm in having the
snapshot change part-way through the scan, and
On Wed, Jun 5, 2013 at 6:56 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
Now, I did find a couple that I thought should probably stick with
SnapshotNow, specifically pgrowlocks and pgstattuple. Those are just
gathering statistical information, so there's no
On Tue, Jun 4, 2013 at 3:57 AM, Robert Haas robertmh...@gmail.com wrote:
On Thu, May 30, 2013 at 1:39 AM, Michael Paquier
michael.paqu...@gmail.com wrote:
+1.
Here's a more serious patch for MVCC catalog access. This one
involves more data copying than the last one, I think, because the
On Tue, May 28, 2013 at 10:39 PM, Robert Haas robertmh...@gmail.com wrote:
IMHO, we should press forward with this approach. Considering that
these are pretty extreme test cases, I'm inclined to view the
performance loss as acceptable. We've never really viewed DDL as
something that needs
On Sun, May 26, 2013 at 9:10 PM, Michael Paquier
michael.paqu...@gmail.com wrote:
Perhaps we see little difference in performance because PGPROC has been
separated into PGPROC and PGXACT, reducing lock contention with getting
snapshot data?
By the way, I grabbed a 32-core machine and did some
Perhaps we see little difference in performance because PGPROC has been
separated into PGPROC and PGXACT, reducing lock contention with getting
snapshot data?
By the way, I grabbed a 32-core machine and did some more performance tests
with some open connections with XIDs assigned using pg_cxn v2
On Wed, May 22, 2013 at 11:11 PM, Andres Freund and...@2ndquadrant.com wrote:
Make that actually having acquired an xid. We skip a large part of the
work if a transaction doesn't yet have one afair. I don't think the mere
presence of 600 idle connections without an xid in contrast to just
On Tue, May 21, 2013 at 10:18 PM, Robert Haas robertmh...@gmail.com wrote:
[ MVCC catalog access seems to be pretty cheap ]
In discussions today, Tom Lane suggested testing the time to start up
a backend and run a simple query such as SELECT 2+2 in the absence
of a relcache file.
I did this and
On 2013-05-22 22:51:13 -0400, Robert Haas wrote:
In discussions today, Tom Lane suggested testing the time to start up
a backend and run a simple query such as SELECT 2+2 in the absence
of a relcache file.
I did this and can't measure any overhead as a result of MVCC catalog
access. I tried
On Wed, May 22, 2013 at 11:02 PM, Andres Freund and...@2ndquadrant.com wrote:
On 2013-05-22 22:51:13 -0400, Robert Haas wrote:
In discussions today, Tom Lane suggested testing the time to start up
a backend and run a simple query such as SELECT 2+2 in the absence
of a relcache file.
I did
On 2013-05-22 23:05:40 -0400, Robert Haas wrote:
On Wed, May 22, 2013 at 11:02 PM, Andres Freund and...@2ndquadrant.com
wrote:
On 2013-05-22 22:51:13 -0400, Robert Haas wrote:
In discussions today, Tom Lane suggested testing the time to start up
a backend and run a simple query such as
We've had a number of discussions about the evils of SnapshotNow. As
far as I can tell, nobody likes it and everybody wants it gone, but
there is concern about the performance impact. I decided to do some
testing to measure the impact. I was pleasantly surprised by the
results.
The attached
40 matches
Mail list logo
