Hi all,
I examined the effect of block-hole compression patch.
I compared the amounts of writing of the WAL data of CVS(7/11) and
8.0.3. (The amount of the WAL data writing was measured by the number of
executions of the write() function in XLogWrite().)
And, I measured the size of the hole.
On Sat, Jun 04, 2005 at 11:46:07AM -0400, Tom Lane wrote:
I've completed a test run for this (it's essentially MySQL's sql-bench
done immediately after initdb). What I get is:
CVS tip of 6/1: ending WAL offset = 0/A364A780 = 2741282688 bytes written
CVS tip of 6/2: ending WAL offset =
I'd say that's an improvement worth having, especially considering that
it requires no net expenditure of CPU time. But the table is certainly
still open to discuss more complicated approaches.
If it's not hard to hack in as a test, it'd be interesting to see what
additional gains a
Tom Lane wrote:
In the XLogInsert(), it makes two kinds of logs, "whole buffer(page)
log" and "partial buffer log", isn't it? Is it only "who buffer
log"
to generate a log with "hole"?
Right.
I see.
I think, it is important to reduce the necessities to write
Hello all,
I am interested in how to "Compress WAL entries".
Then, I study the source now, and read this discussion.
There are some questions.
1.
In the XLogInsert(), it makes two kinds of logs, "whole buffer(page)
log" and "partial buffer log", isn't it? Is it only
-Original Message-
From: Tom Lane [mailto:[EMAIL PROTECTED]
Sent: 04 June 2005 16:46
To: Mark Cave-Ayland (External)
Cc: pgsql-hackers@postgresql.org
Subject: Re: Quick-and-dirty compression for WAL backup blocks
(cut)
I've completed a test run for this (it's essentially MySQL's
Junji TERAMOTO [EMAIL PROTECTED] writes:
In the XLogInsert(), it makes two kinds of logs, whole buffer(page)
log and partial buffer log, isn't it? Is it only who buffer log
to generate a log with hole?
Right.
Tom Lane wrote:
The overhead needed is only 2 bytes to show the number of
bytes
On Mon, 6 Jun 2005, Tom Lane wrote:
Junji TERAMOTO [EMAIL PROTECTED] writes:
In whole buffer log, there is a page header that includes offset of
hole (lower and upper). If we use that information, we don't need
any overhead, do we?
No, because the WAL code cannot assume that all pages
Heikki Linnakangas [EMAIL PROTECTED] writes:
Vacuum doesn't zero out the free space between lower and upper,
It does now ;-)
How about adding a flag to XLogRecData to indicate if the space between
pd_lower and pd_upper is meaningful or not? The XLogInsert caller probably
knows that. That
On Mon, 6 Jun 2005, Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
Vacuum doesn't zero out the free space between lower and upper,
It does now ;-)
Oh :). Does it affect vacuum performance?
How about adding a flag to XLogRecData to indicate if the space between
pd_lower and
Heikki Linnakangas [EMAIL PROTECTED] writes:
On Mon, 6 Jun 2005, Tom Lane wrote:
Heikki Linnakangas [EMAIL PROTECTED] writes:
Vacuum doesn't zero out the free space between lower and upper,
It does now ;-)
Oh :). Does it affect vacuum performance?
I haven't tried to measure it ... but
Mark Cave-Ayland [EMAIL PROTECTED] writes:
A run-length compressor would be reasonably quick but I think that the
omit-the-middle-hole approach gets most of the possible win with even
less work.
I can't think that a RLE scheme would be much more expensive than a 'count
the hole' approach
Date: Tue, 31 May 2005 16:26:20 -0400
From: Tom Lane [EMAIL PROTECTED]
To: pgsql-hackers@postgreSQL.org
Subject: Quick-and-dirty compression for WAL backup blocks
Message-ID: [EMAIL PROTECTED]
(cut)
... make the WAL writing logic aware of the layout
of buffer pages --- specifically, to
Mark Cave-Ayland [EMAIL PROTECTED] writes:
I also noticed your comment above that mentioned that compression would be
less effective as the pages became more full. Would changing the loading
factor of database pages have an effect here, as I would have thought that
the WAL would be fsync'd
On Wed, Jun 01, 2005 at 10:12:34AM -0400, Tom Lane wrote:
Mark Cave-Ayland [EMAIL PROTECTED] writes:
I also noticed your comment above that mentioned that compression would be
less effective as the pages became more full. Would changing the loading
factor of database pages have an effect
It seems we are more or less agreed that 32-bit CRC ought to be enough
for WAL; and we also need to make a change to ensure that backup blocks
are positively linked to their parent WAL record, as I noted earlier
today. So as long as we have to mess with the WAL record format, I was
wondering what
On Tue, 2005-05-31 at 16:26 -0400, Tom Lane wrote:
The TODO item that comes to mind immediately is Compress WAL entries.
A more concrete version of this is: examine the page to see if the
pd_lower field is between SizeOfPageHeaderData and BLCKSZ, and if so
whether there is a run of consecutive
Simon Riggs [EMAIL PROTECTED] writes:
Is this a change that would be backpatched as you suggested previously?
I don't think we can backpatch any of these items, since they involve
changes in the on-disk file format. I was thinking of them as CVS HEAD
changes only.
18 matches
Mail list logo
