On Thursday, September 27, 2012 9:12 AM Noah Misch wrote:
On Mon, Sep 24, 2012 at 10:57:02AM +, Amit kapila wrote:
Rebased version of patch based on latest code.
I like the direction you're taking with this patch; the gains are
striking,
especially considering the isolation of the
On Fri, Aug 10, 2012 at 1:25 AM, Amit Kapila amit.kap...@huawei.com wrote:
I think the property that recovery only needs to worry about each
block individually is one that we want to preserve. Supporting this
optimizating only when full_page_writes=off seems ugly,
I think recovery needs to
On Thursday, August 30, 2012 11:23 PM Robert Haas
[mailto:robertmh...@gmail.com] wrote:
On Fri, Aug 10, 2012 at 1:25 AM, Amit Kapila amit.kap...@huawei.com wrote:
I think the property that recovery only needs to worry about each
block individually is one that we want to preserve. Supporting
On August 27, 2012 7:00 PM Amit Kapila wrote:
On August 27, 2012 5:58 PM Heikki Linnakangas wrote:
On 27.08.2012 15:18, Amit kapila wrote:
I have implemented the WAL Reduction Patch for the case of HOT Update as
Let's do it for HOT updates only. Simon Robert made good arguments on
why this
On 27.08.2012 15:18, Amit kapila wrote:
I have implemented the WAL Reduction Patch for the case of HOT Update as
pointed out by Simon and Robert. In this patch it only goes for Optimized WAL
in case of HOT Update with other restrictions same as in previous patch.
The performance numbers for
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Monday, August 27, 2012 5:58 PM
To: Amit kapila
On 27.08.2012 15:18, Amit kapila wrote:
I have implemented the WAL Reduction Patch for the case of HOT Update as
pointed out by Simon and Robert. In this patch it only goes
On Wed, Aug 22, 2012 at 07:38:33PM +0530, Amit Kapila wrote:
I had made sure no full_page_write happens by making checkpoint interval and
checkpoints segments longer.
Original code - 1.8GModified code - 1.1G Diff - 63% reduction, incase of
fill factor 100.
Original code - 1.6G
From: Bruce Momjian [mailto:br...@momjian.us]
Sent: Friday, August 24, 2012 2:12 AM
On Wed, Aug 22, 2012 at 07:38:33PM +0530, Amit Kapila wrote:
I had made sure no full_page_write happens by making checkpoint interval
and
checkpoints segments longer.
Original code - 1.8GModified
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Amit Kapila
Sent: Wednesday, August 22, 2012 8:34 AM
From: Jesper Krogh [mailto:jes...@krogh.cc]
Sent: Wednesday, August 22, 2012 1:13 AM
On 21/08/12 16:57, Amit kapila wrote:
Test results:
1.
From: Jesper Krogh [mailto:jes...@krogh.cc]
Sent: Wednesday, August 22, 2012 1:13 AM
On 21/08/12 16:57, Amit kapila wrote:
Test results:
1. The pgbench test run for 10min.
2. The test reult is for modified pgbench (such that total row size is
1800 and updated columns are of length 300)
On 3 August 2012 12:46, Amit kapila amit.kap...@huawei.com wrote:
Frame the new tuple from old tuple and WAL record:
Sounds good.
I'd suggest we do this only when the saving is large enough for
benefit, rather than do this every time.
You don't mention whether or not the old and the new tuple
From: Simon Riggs [mailto:si...@2ndquadrant.com]
Sent: Thursday, August 09, 2012 12:36 PM
On 3 August 2012 12:46, Amit kapila amit.kap...@huawei.com wrote:
Frame the new tuple from old tuple and WAL record:
Sounds good.
Thanks.
I'd suggest we do this only when the saving is large enough
On 9 August 2012 09:49, Amit Kapila amit.kap...@huawei.com wrote:
I'd suggest we do this only when the saving is large enough for
benefit, rather than do this every time.
Do you mean to say that when length of updated values of tuple is less
than some threshold(1/3 or 2/3, etc..) value of
On 09.08.2012 12:18, Simon Riggs wrote:
On 9 August 2012 09:49, Amit Kapilaamit.kap...@huawei.com wrote:
WAL reduction is done for the case even when old and new are on different
data blocks as well.
That makes me feel nervous. I doubt the marginal gain is worth it.
Most updates don't
On 9 August 2012 11:30, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
On 09.08.2012 12:18, Simon Riggs wrote:
On 9 August 2012 09:49, Amit Kapilaamit.kap...@huawei.com wrote:
WAL reduction is done for the case even when old and new are on
different
data blocks as well.
From: pgsql-hackers-ow...@postgresql.org
[mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Simon Riggs
Sent: Thursday, August 09, 2012 2:49 PM
On 9 August 2012 09:49, Amit Kapila amit.kap...@huawei.com wrote:
I'd suggest we do this only when the saving is large enough for
benefit, rather
On 09.08.2012 14:11, Simon Riggs wrote:
Given the marginal gain because of a low percentage of cross-block
updates, I'm not keen. Low percentage because HOT tries hard to keep
things on same block - even for non-HOT updates (which is the case,
even though it sounds weird).
That depends
On 9 August 2012 12:17, Amit Kapila amit.kap...@huawei.com wrote:
This optimization is to reduce the amount of WAL and definitely adding
anything extra will have some impact.
Of course. The question is How much impact?. Each tweak has
progressively less and less gain. This isn't a binary
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Thursday, August 09, 2012 4:59 PM
On 09.08.2012 14:11, Simon Riggs wrote:
But then again, full-page writes cover that too. There
will be a full-page image of the old block in the WAL anyway.
Right, but we're planning
On 09.08.2012 15:56, Amit Kapila wrote:
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Thursday, August 09, 2012 4:59 PM
On 09.08.2012 14:11, Simon Riggs wrote:
But then again, full-page writes cover that too. There
will be a full-page image of the old block in the
From: Simon Riggs [mailto:si...@2ndquadrant.com]
Sent: Thursday, August 09, 2012 5:29 PM
On 9 August 2012 12:17, Amit Kapila amit.kap...@huawei.com wrote:
This optimization is to reduce the amount of WAL and definitely adding
anything extra will have some impact.
Of course. The question is
On Thu, Aug 9, 2012 at 9:09 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I meant corruption caused by anything, like disk failure, bugs, cosmic rays,
etc. The point is that currently the WAL record contains all the information
required to reconstruct the old tuple. With a
On 09.08.2012 19:39, Robert Haas wrote:
On Thu, Aug 9, 2012 at 9:09 AM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
I meant corruption caused by anything, like disk failure, bugs, cosmic rays,
etc. The point is that currently the WAL record contains all the information
On Thu, Aug 9, 2012 at 12:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So suppose that the following sequence of events occurs:
1. Tuple A on page 1 is updated. The new version, tuple B, is placed on
page 2.
2. The table is vacuumed, removing tuple A.
3. Page 1 is
From: Robert Haas [mailto:robertmh...@gmail.com]
Sent: Thursday, August 09, 2012 11:18 PM
On Thu, Aug 9, 2012 at 12:43 PM, Heikki Linnakangas
heikki.linnakan...@enterprisedb.com wrote:
So suppose that the following sequence of events occurs:
1. Tuple A on page 1 is updated. The new version,
On 06.08.2012 06:10, Amit Kapila wrote:
Currently the solution for fixed length columns cannot handle the case of
variable length columns and NULLS. The reason is for fixed length columns
there is no need of diff technology between old and new tuple, however for
other cases it will be required.
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Monday, August 06, 2012 2:32 PM
To: Amit Kapila
Cc: 'Bruce Momjian'; pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] [WIP] Performance Improvement by reducing WAL for
Update Operation
On 06.08.2012 06:10, Amit Kapila
From: Bruce Momjian [mailto:br...@momjian.us]
Sent: Saturday, August 04, 2012 8:06 PM
On Sat, Aug 4, 2012 at 05:21:06PM +0300, Heikki Linnakangas wrote:
On 04.08.2012 11:01, Amit Kapila wrote:
Missed one point which needs to be handled is pg_upgrade
I don't think there's anything to do for
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Saturday, August 04, 2012 1:33 AM
On 03.08.2012 14:46, Amit kapila wrote:
Currently the change is done only for fixed length columns for simple
tables and the tuple should not contain NULLS.
This is a Proof of concept,
Subject: Re: [HACKERS] [WIP] Performance Improvement by reducing WAL for
Update Operation
From: Heikki Linnakangas [mailto:heikki.linnakan...@enterprisedb.com]
Sent: Saturday, August 04, 2012 1:33 AM
On 03.08.2012 14:46, Amit kapila wrote:
Currently the change is done only for fixed length columns
On 04.08.2012 11:01, Amit Kapila wrote:
Missed one point which needs to be handled is pg_upgrade
I don't think there's anything to do for pg_upgrade. This doesn't change
the on-disk data format, just the WAL format, and pg_upgrade isn't
sensitive to WAL format changes.
--
Heikki
On Sat, Aug 4, 2012 at 05:21:06PM +0300, Heikki Linnakangas wrote:
On 04.08.2012 11:01, Amit Kapila wrote:
Missed one point which needs to be handled is pg_upgrade
I don't think there's anything to do for pg_upgrade. This doesn't
change the on-disk data format, just the WAL format, and
, August 03, 2012 5:17 PM
To: pgsql-hackers@postgresql.org
Subject: [HACKERS] [WIP] Performance Improvement by reducing WAL for Update
Operation
Problem statement:
---
Reducing wal size for an update operation for performance improvement.
Advantages
On 03.08.2012 14:46, Amit kapila wrote:
Currently the change is done only for fixed length columns for simple tables
and the tuple should not contain NULLS.
This is a Proof of concept, the design and implementation needs to be changed
based on final design required for handling other
34 matches
Mail list logo
