On Wed, Dec 22, 2010 at 4:54 PM, Tom Lane wrote:
> Josh Berkus writes:
>> Regarding the contention which Tom expects: the extra load on the CLOG
>> would be 100% reads, no? If it's *all* reads, why would we have any
>> more contention than we have now?
>
> Read involves sharelock which still cau
On Wed, 2010-12-22 at 22:08 +0200, Heikki Linnakangas wrote:
> On 22.12.2010 18:12, Merlin Moncure wrote:
> > On Wed, Dec 22, 2010 at 11:06 AM, Tom Lane wrote:
> >> Merlin Moncure writes:
> >>> well, simon's point that hint bits complicate checksum may nor may not
> >>> be the case, but no hint b
On 23/12/10 10:54, Tom Lane wrote:
Josh Berkus writes:
Regarding the contention which Tom expects: the extra load on the CLOG
would be 100% reads, no? If it's *all* reads, why would we have any
more contention than we have now?
Read involves sharelock which still causes contention. Those buf
Josh Berkus writes: > I might be able to test
on some client workloads. We'll see; currently > I lack the
harness to simulate a high level of client contention. We're
pretty successful in doing that with Tsung, even against large
clusters of plproxy nodes. http://tsung.erlang-projects.org
Josh Berkus writes:
> Regarding the contention which Tom expects: the extra load on the CLOG
> would be 100% reads, no? If it's *all* reads, why would we have any
> more contention than we have now?
Read involves sharelock which still causes contention. Those bufmgr
contention storms we saw bef
> Certainly having a choice about configuring them would be a good
> addition in itself, e.g for data warehousing use the hint bits can be a
> considerable impediment so the *ability* to not have them would be a
> huge advantage.
Would need to be a restart option, no?
Regarding the contention w
On 23/12/10 05:06, Merlin Moncure wrote:
On Wed, Dec 22, 2010 at 10:59 AM, Tom Lane wrote:
Heikki Linnakangas writes:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only hint bits are
set. That way you get the benefit
> right -- see the attached clog_stress.sql above. It creates a script
> that inserts records in blocks of 1, deletes half of them, and
> vacuums. Neither the execution of the script nor a seq scan following
> its execution showed an interesting performance difference (which I am
> arbitrari
On 22.12.2010 18:12, Merlin Moncure wrote:
On Wed, Dec 22, 2010 at 11:06 AM, Tom Lane wrote:
Merlin Moncure writes:
well, simon's point that hint bits complicate checksum may nor may not
be the case, but no hint bits = less i/o = less checksumming (unless
you checksum around the hint bits).
On Wed, Dec 22, 2010 at 04:00:30PM +, Simon Riggs wrote:
> On Wed, 2010-12-22 at 17:42 +0200, Heikki Linnakangas wrote:
> > On 22.12.2010 17:31, Simon Riggs wrote:
> > > On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
> > >> There's plenty of stuff in memory that's not covered by a
Merlin Moncure writes:
> I'm going to do lots more testing over the holidays. I'm fishing for
> ideas on good ways to flesh things out more.
Based on the analogy to past bufmgr contention problems, I'd suggest
going back through the archives to look for the test cases associated
with context swa
On Wed, 2010-12-22 at 10:59 -0500, Tom Lane wrote:
> Heikki Linnakangas writes:
> > My gut feeling is that a reasonable compromise is to set hint bits like
> > we do today, but don't mark the page as dirty when only hint bits are
> > set. That way you get the benefit of hint bits for tuples that
On Wed, Dec 22, 2010 at 11:12 AM, Merlin Moncure wrote:
> On Wed, Dec 22, 2010 at 11:06 AM, Tom Lane wrote:
>> Merlin Moncure writes:
>>> well, simon's point that hint bits complicate checksum may nor may not
>>> be the case, but no hint bits = less i/o = less checksumming (unless
>>> you checks
On Wed, Dec 22, 2010 at 11:06 AM, Tom Lane wrote:
> Merlin Moncure writes:
>> well, simon's point that hint bits complicate checksum may nor may not
>> be the case, but no hint bits = less i/o = less checksumming (unless
>> you checksum around the hint bits).
>
> I think you're optimistically ass
On Wed, Dec 22, 2010 at 10:59 AM, Tom Lane wrote:
> Heikki Linnakangas writes:
>> My gut feeling is that a reasonable compromise is to set hint bits like
>> we do today, but don't mark the page as dirty when only hint bits are
>> set. That way you get the benefit of hint bits for tuples that are
Merlin Moncure writes:
> well, simon's point that hint bits complicate checksum may nor may not
> be the case, but no hint bits = less i/o = less checksumming (unless
> you checksum around the hint bits).
I think you're optimistically assuming the extra clog accesses don't
cost any I/O.
On Wed, Dec 22, 2010 at 10:55 AM, Aidan Van Dyk wrote:
> On Wed, Dec 22, 2010 at 10:52 AM, Simon Riggs wrote:
>
>> I'm sure it will take a little while for everybody to understand why a
>> full CRC implementation is both necessary and now possible. Paradigm
>> shifts of thought do seem like telep
On Wed, 2010-12-22 at 17:42 +0200, Heikki Linnakangas wrote:
> On 22.12.2010 17:31, Simon Riggs wrote:
> > On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
> >> There's plenty of stuff in memory that's not covered by an
> >> application-level CRC. That's what ECC RAM is for.
> >
> > htt
Heikki Linnakangas writes:
> My gut feeling is that a reasonable compromise is to set hint bits like
> we do today, but don't mark the page as dirty when only hint bits are
> set. That way you get the benefit of hint bits for tuples that are
> frequently accessed and stay in buffer cache. But y
On Wed, Dec 22, 2010 at 10:52 AM, Simon Riggs wrote:
> I'm sure it will take a little while for everybody to understand why a
> full CRC implementation is both necessary and now possible. Paradigm
> shifts of thought do seem like teleports, but they can be beneficial.
But please don't deny the r
On Wed, 2010-12-22 at 10:45 -0500, Tom Lane wrote:
> Aidan Van Dyk writes:
> > With this statement, you just moved the goal posts on the checksumming
> > ideas. In fact, you didn't just move the goal posts, you picked the
> > ball up and teleported it to another stadium.
>
> What he said. I can
Aidan Van Dyk writes:
> With this statement, you just moved the goal posts on the checksumming
> ideas. In fact, you didn't just move the goal posts, you picked the
> ball up and teleported it to another stadium.
What he said. I can't imagine that anyone will be interested in any
case other tha
On 22.12.2010 17:31, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
There's plenty of stuff in memory that's not covered by an
application-level CRC. That's what ECC RAM is for.
http://www.google.com/research/pubs/archive/35162.pdf
Google research shows that e
On 22.12.2010 17:31, Simon Riggs wrote:
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
Do you envision that the CRC is calculated at every update, or only when
a page is written out from the buffer cache?
At every update, so there is a clear assertion that the CRC matches the
blo
On Wed, 2010-12-22 at 17:01 +0200, Heikki Linnakangas wrote:
> On 22.12.2010 16:52, Simon Riggs wrote:
> > On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
> >> On 22.12.2010 15:59, Simon Riggs wrote:
> >>> On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
> My gut feeli
On Wed, Dec 22, 2010 at 9:52 AM, Simon Riggs wrote:
> So what you suggest works only if we restrict CRC checking to blocks
> incoming to the buffer cache, but leaves us unable to do CRC checks on
> blocks once in the buffer cache. Since many blocks stay in cache almost
> constantly, we're left wi
On 22.12.2010 16:52, Simon Riggs wrote:
On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
On 22.12.2010 15:59, Simon Riggs wrote:
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don
On Wed, Dec 22, 2010 at 9:52 AM, Simon Riggs wrote:
> I think its important for Postgres to implement this in the same release
> as sync rep.
i.e. never, at the rate sync rep has been progressing for the last few months?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Po
On Wed, 2010-12-22 at 16:22 +0200, Heikki Linnakangas wrote:
> On 22.12.2010 15:59, Simon Riggs wrote:
> > On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
> >> My gut feeling is that a reasonable compromise is to set hint bits like
> >> we do today, but don't mark the page as dirty whe
On 22.12.2010 15:59, Simon Riggs wrote:
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
My gut feeling is that a reasonable compromise is to set hint bits like
we do today, but don't mark the page as dirty when only hint bits are
set. That way you get the benefit of hint bits for tu
On Wed, 2010-12-22 at 15:30 +0200, Heikki Linnakangas wrote:
> > I would vote to put this into 9.1 as a non-default option at restart,
> > opening the door to other features which hint bits are frustrating.
> > People can then choose between those features and the "power of hint
> > bits". I think
On 22.12.2010 15:21, Simon Riggs wrote:
On Tue, 2010-12-21 at 17:42 -0500, Merlin Moncure wrote:
*) is there community interest in a full patch that fills in the
missing details not implemented here?
You're thinking seems sound to me. We now have all-visible flags, fewer
xids, much better clo
On Tue, 2010-12-21 at 17:42 -0500, Merlin Moncure wrote:
> *) is there community interest in a full patch that fills in the
> missing details not implemented here?
You're thinking seems sound to me. We now have all-visible flags, fewer
xids, much better clog concurrency. Avoiding hint bits would
On 22.12.2010 02:56, Merlin Moncure wrote:
On Tue, Dec 21, 2010 at 7:45 PM, Tom Lane wrote:
Merlin Moncure writes:
Attached is an incomplete patch disabling hint bits based on compile
switch. ...
So far, at least doing pgbench runs and another test designed to
exercise clog lookups, the perfo
On 22/12/10 13:56, Merlin Moncure wrote:
On Tue, Dec 21, 2010 at 7:45 PM, Tom Lane wrote:
@Mark: apparently the cvs server is behind git and there are some
recent changes to heapam.c that need more attention. I need to get
git going on my box, but try changing this:
if ((tuple->t_info
On Tue, Dec 21, 2010 at 7:45 PM, Tom Lane wrote:
> Merlin Moncure writes:
>> Attached is an incomplete patch disabling hint bits based on compile
>> switch. ...
>> So far, at least doing pgbench runs and another test designed to
>> exercise clog lookups, the performance loss of always doing full
Merlin Moncure writes:
> Attached is an incomplete patch disabling hint bits based on compile
> switch. ...
> So far, at least doing pgbench runs and another test designed to
> exercise clog lookups, the performance loss of always doing full
> lookup hasn't materialized.
The standard pgbench test
On Tue, Dec 21, 2010 at 7:20 PM, Merlin Moncure wrote:
> On Tue, Dec 21, 2010 at 7:06 PM, Mark Kirkwood
> wrote:
>> On 22/12/10 13:05, Mark Kirkwood wrote:
>>>
>>> On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits based on compile
switch.
On Tue, Dec 21, 2010 at 7:06 PM, Mark Kirkwood
wrote:
> On 22/12/10 13:05, Mark Kirkwood wrote:
>>
>> On 22/12/10 11:42, Merlin Moncure wrote:
>>>
>>> Attached is an incomplete patch disabling hint bits based on compile
>>> switch. It's not complete, for example it's not reconciling some
>>> assu
On 22/12/10 13:05, Mark Kirkwood wrote:
On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits based on compile
switch. It's not complete, for example it's not reconciling some
assumptions in heapam.c that hint bits have been set in various
routines. Howev
On 22/12/10 11:42, Merlin Moncure wrote:
Attached is an incomplete patch disabling hint bits based on compile
switch. It's not complete, for example it's not reconciling some
assumptions in heapam.c that hint bits have been set in various
routines. However, it mostly passes regression and I dee
Merlin Moncure wrote:
> *) what's a good way to stress the clog severely? I'd like to pick
> a degenerate case to get a better idea of the way things stand
> without them.
The worst I can think of is a large database with a 90/10 mix of
reads to writes -- all short transactions. Maybe someone
42 matches
Mail list logo
