On Wed, Jul 18, 2012 at 5:17 PM, Tom Lane t...@sss.pgh.pa.us wrote:
I've been chewing on this issue some more, and no longer like my
previous proposal, which was
... What I'm thinking about
is reducing the hash key to just RelFileNodeBackend + ForkNumber,
so that there's one hashtable entry
Robert Haas robertmh...@gmail.com writes:
Seems a bit complex, but it might be worth it. Keep in mind that I
eventually want to be able to make an unlogged table logged or a visca
versa, which will probably entail unlinking just the init fork (for
the logged - unlogged direction).
Well, as
On Thu, Jul 19, 2012 at 10:09 AM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
Seems a bit complex, but it might be worth it. Keep in mind that I
eventually want to be able to make an unlogged table logged or a visca
versa, which will probably entail unlinking
Robert Haas robertmh...@gmail.com writes:
On Thu, Jul 19, 2012 at 10:09 AM, Tom Lane t...@sss.pgh.pa.us wrote:
What about checking just the immediately previous entry? This would
at least fix the problem for bulk-load situations, and the cost ought
to be about negligible compared to acquiring
On Thu, Jul 19, 2012 at 2:57 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Thu, Jul 19, 2012 at 10:09 AM, Tom Lane t...@sss.pgh.pa.us wrote:
What about checking just the immediately previous entry? This would
at least fix the problem for bulk-load
Robert Haas robertmh...@gmail.com writes:
What if we change the hash table to have RelFileNode as the key and an
array of MAX_FORKNUM bitmapsets as the value? Then when you get a
forget request, you can just zap all the sets to empty.
Hm ... the only argument I can really make against that is
Robert Haas robertmh...@gmail.com writes:
On Mon, Jul 16, 2012 at 12:36 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Well, that argument is exactly why the code is designed the way it is...
but we are now finding out that sending useless fsync requests isn't as
cheap as all that.
I agree, but I
On 07/16/2012 02:39 PM, Robert Haas wrote:
Unfortunately, there are lots of important operations (like bulk
loading, SELECT * FROM bigtable, and VACUUM notverybigtable) that
inevitably end up writing out their own dirty buffers. And even when
the background writer does write something, it's not
On Sun, Jul 15, 2012 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote:
I think what we ought to do is bite the bullet and refactor the
representation of the pendingOps table. What I'm thinking about
is reducing the hash key to just RelFileNodeBackend + ForkNumber,
so that there's one hashtable
Robert Haas robertmh...@gmail.com writes:
On Sun, Jul 15, 2012 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Yeah, you have a point there. It's not real clear that switching fsync
from off to on is an operation that we can make any guarantees about,
short of executing something like the code
On Mon, Jul 16, 2012 at 12:08 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Sun, Jul 15, 2012 at 2:29 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Yeah, you have a point there. It's not real clear that switching fsync
from off to on is an operation that we can
Robert Haas robertmh...@gmail.com writes:
On Mon, Jul 16, 2012 at 12:08 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Uh, that's exactly what's under discussion. Not sending useless fsync
requests when fsync is off is just one part of it; a part that happens
to be quite useful for some test
On Mon, Jul 16, 2012 at 12:36 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
On Mon, Jul 16, 2012 at 12:08 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Uh, that's exactly what's under discussion. Not sending useless fsync
requests when fsync is off is just one part
Robert Haas robertmh...@gmail.com writes:
In my view, the elephant in the room here is that it's dramatically
inefficient for every backend to send an fsync request on every block
write.
Yeah. This was better before the decision was taken to separate
bgwriter from checkpointer; before that,
On Mon, Jul 16, 2012 at 12:57 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
In my view, the elephant in the room here is that it's dramatically
inefficient for every backend to send an fsync request on every block
write.
Yeah. This was better before the
Robert Haas robertmh...@gmail.com writes:
Unfortunately, there are lots of important operations (like bulk
loading, SELECT * FROM bigtable, and VACUUM notverybigtable) that
inevitably end up writing out their own dirty buffers. And even when
the background writer does write something, it's
Robert Haas robertmh...@gmail.com writes:
At any rate, I'm somewhat less convinced that the split was a good
idea than I was when we did it, mostly because we haven't really gone
anywhere with it subsequently.
BTW, while we are on the subject: hasn't this split completely broken
the statistics
On Mon, Jul 16, 2012 at 3:18 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
At any rate, I'm somewhat less convinced that the split was a good
idea than I was when we did it, mostly because we haven't really gone
anywhere with it subsequently.
BTW, while we
Robert Haas robertmh...@gmail.com writes:
Yes, it seems to have done just that. The comment for
ForwardFsyncRequest is a few bricks short of a load too:
...
Line 2 seems to have been mechanically changed from background
writer to checkpointer, but of course it should still say
background
Jeff Janes jeff.ja...@gmail.com writes:
On Thu, Jul 12, 2012 at 9:55 PM, Jeff Janes jeff.ja...@gmail.com wrote:
The topic was poor performance when truncating lots of small tables
repeatedly on test environments with fsync=off.
On Thu, Jul 12, 2012 at 6:00 PM, Jeff Janes jeff.ja...@gmail.com
... btw, in the penny wise and pound foolish department, I observe that
smgrdounlink calls mdunlink separately for each possibly existing fork
of a relation to be dropped. That means we are queuing a separate fsync
queue entry for each fork, and could immediately save a factor of four
in
On 07/16/2012 02:29 AM, Tom Lane wrote:
Yeah, you have a point there. It's not real clear that switching fsync
from off to on is an operation that we can make any guarantees about,
short of executing something like the code recently added to initdb
to force-sync the entire PGDATA tree.
Craig Ringer ring...@ringerc.id.au writes:
On 07/16/2012 02:29 AM, Tom Lane wrote:
Yeah, you have a point there. It's not real clear that switching fsync
from off to on is an operation that we can make any guarantees about,
short of executing something like the code recently added to initdb
On 07/16/2012 09:37 AM, Tom Lane wrote:
There's one way that doesn't have any housekeeping cost to Pg. It's
pretty bad manners if there's anybody other than Pg on the system though:
sync()
Yeah, I thought about that: if we could document that issuing a manual
sync after turning fsync on
On Thu, Jul 12, 2012 at 9:55 PM, Jeff Janes jeff.ja...@gmail.com wrote:
I've moved this thread from performance to hackers.
The topic was poor performance when truncating lots of small tables
repeatedly on test environments with fsync=off.
On Thu, Jul 12, 2012 at 6:00 PM, Jeff Janes
I've moved this thread from performance to hackers.
The topic was poor performance when truncating lots of small tables
repeatedly on test environments with fsync=off.
On Thu, Jul 12, 2012 at 6:00 PM, Jeff Janes jeff.ja...@gmail.com wrote:
I think the problem is in the Fsync Absorption queue.
26 matches
Mail list logo
