Josh Berkus wrote:
Was it really all that bad? IIRC we replaced ARC with the current clock
sweep due to patent concerns. (Maybe there were performance concerns as
well, I don't remember).
Yeah, that was why the patent was frustrating. Performance was poor and
we were planning on
On 03/24/2011 03:36 PM, Jim Nasby wrote:
On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
Robert Haasrobertmh...@gmail.com writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good.
On Fri, Mar 25, 2011 at 8:07 AM, Gurjeet Singh singh.gurj...@gmail.com wrote:
On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com
On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com
wrote:
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
On Mar 25, 2011, at 10:07 AM, Gurjeet Singh wrote:
On Tue, Mar 22, 2011 at 3:53 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Mar 18, 2011
On Thu, Mar 24, 2011 at 7:51 PM, Greg Stark gsst...@mit.edu wrote:
On Thu, Mar 24, 2011 at 11:33 PM, Jeff Janes jeff.ja...@gmail.com wrote:
I tried under the circumstances I thought were mostly likely to show a
time difference, and I was unable to detect a reliable difference in
timing between
On Mar 25, 2011, at 11:58 AM, Jim Nasby j...@nasby.net wrote:
Related to that... after talking to Greg Smith at PGEast last night, he felt
it would be very valuable just to profile how much time is being spent
waiting/holding the freelist lock in a real environment. I'm going to see if
we
On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
Robert Haas robertmh...@gmail.com writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good.
Why not? AIUI the free list is only for buffers that
Jim Nasby j...@nasby.net Thursday 24 March 2011 20:36:48
On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
Robert Haas robertmh...@gmail.com writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too
On Wed, Mar 23, 2011 at 6:12 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Robert Haas robertmh...@gmail.com writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good.
Why not? AIUI the free list is
On Thu, Mar 24, 2011 at 8:59 PM, Robert Haas robertmh...@gmail.com wrote:
It seems at least plausible that buffer allocation could be
significantly faster if it need only pop the head of a list, rather
than scanning until it finds a suitable candidate. Moving as much
buffer allocation work as
On Thu, Mar 24, 2011 at 5:34 PM, Greg Stark gsst...@mit.edu wrote:
On Thu, Mar 24, 2011 at 8:59 PM, Robert Haas robertmh...@gmail.com wrote:
It seems at least plausible that buffer allocation could be
significantly faster if it need only pop the head of a list, rather
than scanning until it
Robert Haas robertmh...@gmail.com Thursday 24 March 2011 22:41:19
On Thu, Mar 24, 2011 at 5:34 PM, Greg Stark gsst...@mit.edu wrote:
On Thu, Mar 24, 2011 at 8:59 PM, Robert Haas robertmh...@gmail.com
wrote:
It seems at least plausible that buffer allocation could be
significantly faster if
On Thu, Mar 24, 2011 at 12:36 PM, Jim Nasby j...@nasby.net wrote:
On Mar 23, 2011, at 5:12 PM, Tom Lane wrote:
Robert Haas robertmh...@gmail.com writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
On Thu, Mar 24, 2011 at 11:33 PM, Jeff Janes jeff.ja...@gmail.com wrote:
I tried under the circumstances I thought were mostly likely to show a
time difference, and I was unable to detect a reliable difference in
timing between free list and clock sweep.
It strikes me that it shouldn't be
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote:
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com
wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the
Merlin Moncure mmonc...@gmail.com Tuesday 22 March 2011 23:06:02
On Tue, Mar 22, 2011 at 4:28 PM, Radosław Smogura
rsmog...@softperience.eu wrote:
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote:
On
On Mar 22, 2011, at 2:53 PM, Robert Haas wrote:
On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Maybe the
On Wed, Mar 23, 2011 at 1:53 PM, Jim Nasby j...@nasby.net wrote:
When we started using 192G servers we tried switching our largest OLTP
database (would have been about 1.2TB at the time) from 8GB shared buffers to
28GB. Performance went down enough to notice; I don't have any solid metrics,
On Wed, Mar 23, 2011 at 8:00 PM, Robert Haas robertmh...@gmail.com wrote:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good. I wonder if the background writer shouldn't be trying to
maintain the
Greg Stark gsst...@mit.edu Wednesday 23 March 2011 21:30:04
On Wed, Mar 23, 2011 at 8:00 PM, Robert Haas robertmh...@gmail.com wrote:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good. I
Robert Haas robertmh...@gmail.com writes:
It looks like the only way anything can ever get put on the free list
right now is if a relation or database is dropped. That doesn't seem
too good.
Why not? AIUI the free list is only for buffers that are totally dead,
ie contain no info that's
Hi, hackers.
I am interested in this discussion!
So I surveyed current buffer algorithms around other software. I share about it.
(sorry, it is easy survey..)
CLOCK-PRO and LIRS are popular in current buffer algorithms in my easy survey.
Their algorithms are same author that is Song Jiang.
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Maybe the thing to focus on first is the oft-discussed benchmark
farm (similar to the build farm), with a good mix of loads, so
that
On Fri, Mar 18, 2011 at 8:14 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
rsmogura rsmog...@softperience.eu wrote:
Yes, there is some change, and I looked at this more carefully, as
my performance results wasn't such as I expected. I found PG uses
BufferAccessStrategy to do sequence
On 03/22/2011 12:47 PM, Jeff Janes wrote:
Maybe the thing to focus on first is the oft-discussed benchmark
farm (similar to the build farm), with a good mix of loads, so
that the impact of changes can be better tracked for multiple
workloads on a variety of platforms and configurations.
On Tue, Mar 22, 2011 at 11:24 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Fri, Mar 18, 2011 at 9:19 AM, Robert Haas robertmh...@gmail.com wrote:
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Maybe the thing to focus on first is the oft-discussed
On Tue, 2011-03-22 at 15:53 -0400, Robert Haas wrote:
To be honest, I'm mostly just reporting what I've heard Greg Smith say
on this topic. I don't have any machine with that kind of RAM.
I thought we had a machine for hackers who want to do performance
testing. Mark?
--
Devrim GÜNDÜZ
Radek,
I have implemented initial concept of 2nd level cache. Idea is to keep some
segments of shared memory for special buffers (e.g. indices) to prevent
overwrite those by other operations. I added those functionality to nbtree
index scan.
The problem with any special buffering of database
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote:
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com
wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the
On Tue, Mar 22, 2011 at 4:28 PM, Radosław Smogura
rsmog...@softperience.eu wrote:
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote:
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com
wrote:
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus j...@agliodbs.com wrote:
To take the opposite approach... has anyone looked at having the OS just
manage all caching for us? Something like MMAPed shared buffers? Even if we
find the issue with large shared buffers, we still can't dedicate serious
On Mon, 21 Mar 2011 10:24:22 +, Greg Stark wrote:
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus j...@agliodbs.com
wrote:
To take the opposite approach... has anyone looked at having the OS
just manage all caching for us? Something like MMAPed shared buffers?
Even if we find the issue with
On Mon, Mar 21, 2011 at 5:24 AM, Greg Stark gsst...@mit.edu wrote:
On Fri, Mar 18, 2011 at 11:55 PM, Josh Berkus j...@agliodbs.com wrote:
To take the opposite approach... has anyone looked at having the OS just
manage all caching for us? Something like MMAPed shared buffers? Even if we
find
On 21.03.2011 17:54, Merlin Moncure wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the problem with mmap was not being able to detect other
processes
(http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html)
compatibility issues (possibly
On 3/21/11 3:24 AM, Greg Stark wrote:
2-level caches work well for a variety of applications.
I think 2-level caches with simple heuristics like pin all the
indexes is unlikely to be helpful. At least it won't optimize the
average case and I think that's been proven. It might be helpful for
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the problem with mmap was not being able to detect other
processes
(http://www.mail-archive.com/pgsql-general@postgresql.org/msg122301.html)
On Mon, Mar 21, 2011 at 4:47 PM, Josh Berkus j...@agliodbs.com wrote:
You're missing my point ... Postgres already *has* a 2-level cache:
shared_buffers and the FS cache. Anything we add to that will be adding
levels.
I don't think those two levels are interesting -- they don't interact
Excerpts from Josh Berkus's message of lun mar 21 13:47:21 -0300 2011:
We already did that, actually, when we implemented ARC: effectively gave
PostgreSQL a 3-level cache. The results were not very good, although
the algorithm could be at fault there.
Was it really all that bad? IIRC we
Was it really all that bad? IIRC we replaced ARC with the current clock
sweep due to patent concerns. (Maybe there were performance concerns as
well, I don't remember).
Yeah, that was why the patent was frustrating. Performance was poor and
we were planning on replacing ARC in 8.2 anyway.
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote:
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the problem with mmap was not being able to detect other
processes
Merlin Moncure mmonc...@gmail.com Monday 21 March 2011 20:58:16
On Mon, Mar 21, 2011 at 2:08 PM, Greg Stark gsst...@mit.edu wrote:
On Mon, Mar 21, 2011 at 3:54 PM, Merlin Moncure mmonc...@gmail.com
wrote:
Can't you make just one large mapping and lock it in 8k regions? I
thought the
On Thu, 17 Mar 2011 16:02:18 -0500, Kevin Grittner wrote:
Rados*aw Smogurarsmog...@softperience.eu wrote:
I have implemented initial concept of 2nd level cache. Idea is to
keep some segments of shared memory for special buffers (e.g.
indices) to prevent overwrite those by other operations. I
rsmogura rsmog...@softperience.eu wrote:
Yes, there is some change, and I looked at this more carefully, as
my performance results wasn't such as I expected. I found PG uses
BufferAccessStrategy to do sequence scans, so my test query took
only 32 buffers from pool and didn't overwritten
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
Maybe the thing to focus on first is the oft-discussed benchmark
farm (similar to the build farm), with a good mix of loads, so
that the impact of changes can be better tracked for multiple
workloads on a
Excerpts from rsmogura's message of vie mar 18 11:57:48 -0300 2011:
Actually idea of this patch was like this:
Some operations requires many buffers, PG uses clock sweep to get
next free buffer, so it may overwrite index buffer. From point of view
of good database design We should use
On Mar 18, 2011, at 11:19 AM, Robert Haas wrote:
On Fri, Mar 18, 2011 at 11:14 AM, Kevin Grittner
kevin.gritt...@wicourts.gov wrote:
A related area that could use some looking at is why performance tops
out at shared_buffers ~8GB and starts to fall thereafter. InnoDB can
apparently handle
On Fri, Mar 18, 2011 at 2:15 PM, Jim Nasby j...@nasby.net wrote:
+1
To take the opposite approach... has anyone looked at having the OS just
manage all caching for us? Something like MMAPed shared buffers? Even if we
find the issue with large shared buffers, we still can't dedicate serious
Kevin Grittner kevin.gritt...@wicourts.gov Thursday 17 March 2011 22:02:18
Rados*aw Smogurarsmog...@softperience.eu wrote:
I have implemented initial concept of 2nd level cache. Idea is to
keep some segments of shared memory for special buffers (e.g.
indices) to prevent overwrite those by
On 3/18/11 11:15 AM, Jim Nasby wrote:
To take the opposite approach... has anyone looked at having the OS just
manage all caching for us? Something like MMAPed shared buffers? Even if we
find the issue with large shared buffers, we still can't dedicate serious
amounts of memory to them
Hi,
I have implemented initial concept of 2nd level cache. Idea is to keep some
segments of shared memory for special buffers (e.g. indices) to prevent
overwrite those by other operations. I added those functionality to nbtree
index scan.
I tested this with doing index scan, seq read, drop
Rados*aw Smogurarsmog...@softperience.eu wrote:
I have implemented initial concept of 2nd level cache. Idea is to
keep some segments of shared memory for special buffers (e.g.
indices) to prevent overwrite those by other operations. I added
those functionality to nbtree index scan.
I
52 matches
Mail list logo
