On Tue, 2010-12-21 at 17:42 -0500, Merlin Moncure wrote:
*) is there community interest in a full patch that fills in the
missing details not implemented here?
You're thinking seems sound to me. We now have all-visible flags, fewer
xids, much better clog concurrency. Avoiding hint bits would
On 22.12.2010 15:21, Simon Riggs wrote:
On Tue, 2010-12-21 at 17:42 -0500, Merlin Moncure wrote:
*) is there community interest in a full patch that fills in the
missing details not implemented here?
You're thinking seems sound to me. We now have all-visible flags, fewer
xids, much better
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
I would vote to put this into 9.1 as a non-default option at restart,
opening the door to other features which hint bits are frustrating.
People can then choose between those features and the power of hint
bits. I think many
On 22.12.2010 15:59, Simon Riggs wrote:
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only hint bits are
set. That way you get the benefit of hint bits for
On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
On 22.12.2010 15:59, Simon Riggs wrote:
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only
On Wed, Dec 22, 2010 at 9:52 AM, Simon Riggs si...@2ndquadrant.com wrote:
I think its important for Postgres to implement this in the same release
as sync rep.
i.e. never, at the rate sync rep has been progressing for the last few months?
--
Robert Haas
EnterpriseDB:
On 22.12.2010 16:52, Simon Riggs wrote:
On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
On 22.12.2010 15:59, Simon Riggs wrote:
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but
On Wed, Dec 22, 2010 at 9:52 AM, Simon Riggs si...@2ndquadrant.com wrote:
So what you suggest works only if we restrict CRC checking to blocks
incoming to the buffer cache, but leaves us unable to do CRC checks on
blocks once in the buffer cache. Since many blocks stay in cache almost
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
On 22.12.2010 16:52, Simon Riggs wrote:
On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
On 22.12.2010 15:59, Simon Riggs wrote:
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
My gut feeling is that a
On 22.12.2010 17:31, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
Do you envision that the CRC is calculated at every update, or only when
a page is written out from the buffer cache?
At every update, so there is a clear assertion that the CRC matches the
On 22.12.2010 17:31, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
There's plenty of stuff in memory that's not covered by an
application-level CRC. That's what ECC RAM is for.
http://www.google.com/research/pubs/archive/35162.pdf
Google research shows that
Aidan Van Dyk ai...@highrise.ca writes:
With this statement, you just moved the goal posts on the checksumming
ideas. In fact, you didn't just move the goal posts, you picked the
ball up and teleported it to another stadium.
What he said. I can't imagine that anyone will be interested in any
On Wed, 2010-12-22 at 10:45 -0500, Tom Lane wrote:
Aidan Van Dyk ai...@highrise.ca writes:
With this statement, you just moved the goal posts on the checksumming
ideas. In fact, you didn't just move the goal posts, you picked the
ball up and teleported it to another stadium.
What he
On Wed, Dec 22, 2010 at 10:52 AM, Simon Riggs si...@2ndquadrant.com wrote:
I'm sure it will take a little while for everybody to understand why a
full CRC implementation is both necessary and now possible. Paradigm
shifts of thought do seem like teleports, but they can be beneficial.
But
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only hint bits are
set. That way you get the benefit of hint bits for tuples that are
frequently accessed
On Wed, 2010-12-22 at 17:42 +0200, Heikki Linnakangas wrote:
On 22.12.2010 17:31, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
There's plenty of stuff in memory that's not covered by an
application-level CRC. That's what ECC RAM is for.
On Wed, Dec 22, 2010 at 10:55 AM, Aidan Van Dyk ai...@highrise.ca wrote:
On Wed, Dec 22, 2010 at 10:52 AM, Simon Riggs si...@2ndquadrant.com wrote:
I'm sure it will take a little while for everybody to understand why a
full CRC implementation is both necessary and now possible. Paradigm
Merlin Moncure mmonc...@gmail.com writes:
well, simon's point that hint bits complicate checksum may nor may not
be the case, but no hint bits = less i/o = less checksumming (unless
you checksum around the hint bits).
I think you're optimistically assuming the extra clog accesses don't
cost
On Wed, Dec 22, 2010 at 10:59 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only hint bits are
set. That way you
On Wed, Dec 22, 2010 at 11:06 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Merlin Moncure mmonc...@gmail.com writes:
well, simon's point that hint bits complicate checksum may nor may not
be the case, but no hint bits = less i/o = less checksumming (unless
you checksum around the hint bits).
I
On Wed, Dec 22, 2010 at 11:12 AM, Merlin Moncure mmonc...@gmail.com wrote:
On Wed, Dec 22, 2010 at 11:06 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Merlin Moncure mmonc...@gmail.com writes:
well, simon's point that hint bits complicate checksum may nor may not
be the case, but no hint bits = less
On Wed, 2010-12-22 at 10:59 -0500, Tom Lane wrote:
Heikki Linnakangas heikki.linnakan...@enterprisedb.com writes:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only hint bits are
set. That way you get the benefit
Merlin Moncure mmonc...@gmail.com writes:
I'm going to do lots more testing over the holidays. I'm fishing for
ideas on good ways to flesh things out more.
Based on the analogy to past bufmgr contention problems, I'd suggest
going back through the archives to look for the test cases associated
On Wed, Dec 22, 2010 at 04:00:30PM +, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:42 +0200, Heikki Linnakangas wrote:
On 22.12.2010 17:31, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
There's plenty of stuff in memory that's not covered by an
On 22.12.2010 18:12, Merlin Moncure wrote:
On Wed, Dec 22, 2010 at 11:06 AM, Tom Lanet...@sss.pgh.pa.us wrote:
Merlin Moncuremmonc...@gmail.com writes:
well, simon's point that hint bits complicate checksum may nor may not
be the case, but no hint bits = less i/o = less checksumming (unless
right -- see the attached clog_stress.sql above. It creates a script
that inserts records in blocks of 1, deletes half of them, and
vacuums. Neither the execution of the script nor a seq scan following
its execution showed an interesting performance difference (which I am
arbitrarily
On 23/12/10 05:06, Merlin Moncure wrote:
On Wed, Dec 22, 2010 at 10:59 AM, Tom Lanet...@sss.pgh.pa.us wrote:
Heikki Linnakangasheikki.linnakan...@enterprisedb.com writes:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty
Certainly having a choice about configuring them would be a good
addition in itself, e.g for data warehousing use the hint bits can be a
considerable impediment so the *ability* to not have them would be a
huge advantage.
Would need to be a restart option, no?
Regarding the contention
Josh Berkus j...@agliodbs.com writes:
Regarding the contention which Tom expects: the extra load on the CLOG
would be 100% reads, no? If it's *all* reads, why would we have any
more contention than we have now?
Read involves sharelock which still causes contention. Those bufmgr
contention
Josh Berkus j...@agliodbs.com writes: I might be able to test
on some client workloads. We'll see; currently I lack the
harness to simulate a high level of client contention. We're
pretty successful in doing that with Tsung, even against large
clusters of plproxy nodes.
On 23/12/10 10:54, Tom Lane wrote:
Josh Berkusj...@agliodbs.com writes:
Regarding the contention which Tom expects: the extra load on the CLOG
would be 100% reads, no? If it's *all* reads, why would we have any
more contention than we have now?
Read involves sharelock which still causes
On Wed, 2010-12-22 at 22:08 +0200, Heikki Linnakangas wrote:
On 22.12.2010 18:12, Merlin Moncure wrote:
On Wed, Dec 22, 2010 at 11:06 AM, Tom Lanet...@sss.pgh.pa.us wrote:
Merlin Moncuremmonc...@gmail.com writes:
well, simon's point that hint bits complicate checksum may nor may not
be
On Wed, Dec 22, 2010 at 4:54 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Josh Berkus j...@agliodbs.com writes:
Regarding the contention which Tom expects: the extra load on the CLOG
would be 100% reads, no? If it's *all* reads, why would we have any
more contention than we have now?
Read
Merlin Moncure mmonc...@gmail.com wrote:
*) what's a good way to stress the clog severely? I'd like to pick
a degenerate case to get a better idea of the way things stand
without them.
The worst I can think of is a large database with a 90/10 mix of
reads to writes -- all short
On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits based on compile
switch. It's not complete, for example it's not reconciling some
assumptions in heapam.c that hint bits have been set in various
routines. However, it mostly passes regression and I
On 22/12/10 13:05, Mark Kirkwood wrote:
On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits based on compile
switch. It's not complete, for example it's not reconciling some
assumptions in heapam.c that hint bits have been set in various
routines.
On Tue, Dec 21, 2010 at 7:06 PM, Mark Kirkwood
mark.kirkw...@catalyst.net.nz wrote:
On 22/12/10 13:05, Mark Kirkwood wrote:
On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits based on compile
switch. It's not complete, for example it's not
On Tue, Dec 21, 2010 at 7:20 PM, Merlin Moncure mmonc...@gmail.com wrote:
On Tue, Dec 21, 2010 at 7:06 PM, Mark Kirkwood
mark.kirkw...@catalyst.net.nz wrote:
On 22/12/10 13:05, Mark Kirkwood wrote:
On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits
Merlin Moncure mmonc...@gmail.com writes:
Attached is an incomplete patch disabling hint bits based on compile
switch. ...
So far, at least doing pgbench runs and another test designed to
exercise clog lookups, the performance loss of always doing full
lookup hasn't materialized.
The
On Tue, Dec 21, 2010 at 7:45 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Merlin Moncure mmonc...@gmail.com writes:
Attached is an incomplete patch disabling hint bits based on compile
switch. ...
So far, at least doing pgbench runs and another test designed to
exercise clog lookups, the
On 22/12/10 13:56, Merlin Moncure wrote:
On Tue, Dec 21, 2010 at 7:45 PM, Tom Lanet...@sss.pgh.pa.us wrote:
@Mark: apparently the cvs server is behind git and there are some
recent changes to heapam.c that need more attention. I need to get
git going on my box, but try changing this:
On 22.12.2010 02:56, Merlin Moncure wrote:
On Tue, Dec 21, 2010 at 7:45 PM, Tom Lanet...@sss.pgh.pa.us wrote:
Merlin Moncuremmonc...@gmail.com writes:
Attached is an incomplete patch disabling hint bits based on compile
switch. ...
So far, at least doing pgbench runs and another test
42 matches
Mail list logo
