On Fri, Oct 14, 2011 at 4:29 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Right. I think this one falls into my class #2, ie, we have no idea how
to implement it usefully. Doesn't (necessarily) mean that the core
concept is without merit.
Hm. given that we have an implementation I wouldn't say we
Greg Stark st...@mit.edu writes:
On Fri, Oct 14, 2011 at 4:29 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Right. I think this one falls into my class #2, ie, we have no idea how
to implement it usefully. Doesn't (necessarily) mean that the core
concept is without merit.
Hm. given that we have
2011/10/14 Bruce Momjian br...@momjian.us:
Should this be marked as TODO?
I suppose TODO items *are* wanted and so working on them should remove
the pain to convince people here to accept the feature, aren't they ?
---
On 14.10.2011 11:44, Cédric Villemain wrote:
2011/10/14 Bruce Momjianbr...@momjian.us:
Should this be marked as TODO?
I suppose TODO items *are* wanted and so working on them should remove
the pain to convince people here to accept the feature, aren't they ?
I don't think this is
=?ISO-8859-1?Q?C=E9dric_Villemain?= cedric.villemain.deb...@gmail.com writes:
2011/10/14 Bruce Momjian br...@momjian.us:
Should this be marked as TODO?
I suppose TODO items *are* wanted and so working on them should remove
the pain to convince people here to accept the feature, aren't they ?
Tom Lane wrote:
=?ISO-8859-1?Q?C=E9dric_Villemain?= cedric.villemain.deb...@gmail.com
writes:
2011/10/14 Bruce Momjian br...@momjian.us:
Should this be marked as TODO?
I suppose TODO items *are* wanted and so working on them should remove
the pain to convince people here to accept the
Excerpts from Bruce Momjian's message of vie oct 14 11:56:22 -0300 2011:
Tom Lane wrote:
=?ISO-8859-1?Q?C=E9dric_Villemain?= cedric.villemain.deb...@gmail.com
writes:
2011/10/14 Bruce Momjian br...@momjian.us:
Should this be marked as TODO?
I suppose TODO items *are* wanted and
Alvaro Herrera wrote:
Excerpts from Bruce Momjian's message of vie oct 14 11:56:22 -0300 2011:
Tom Lane wrote:
=?ISO-8859-1?Q?C=E9dric_Villemain?= cedric.villemain.deb...@gmail.com
writes:
2011/10/14 Bruce Momjian br...@momjian.us:
Should this be marked as TODO?
I
On Fri, Oct 14, 2011 at 11:12 AM, Bruce Momjian br...@momjian.us wrote:
OK. But if we are pretty sure we don't want something, e.g. hibernate,
we shouldn't add it.
Fair enough, but I'm not even slightly sure that we don't want that.
I think having prewarming utilities available as contrib
Alvaro Herrera alvhe...@commandprompt.com writes:
Excerpts from Bruce Momjian's message of vie oct 14 11:56:22 -0300 2011:
Tom Lane wrote:
There is plenty of stuff in the TODO list for which there is no
consensus.
Uh, we should probably remove those then. Can you think of any?
Unless
Excerpts from Bruce Momjian's message of vie oct 14 12:12:22 -0300 2011:
Alvaro Herrera wrote:
The guideline, last I checked, was that before getting into coding any
item from the TODO list, the prospective hacker should check previous
discussions and initiate a new one on this list to
Robert Haas robertmh...@gmail.com writes:
On Fri, Oct 14, 2011 at 11:12 AM, Bruce Momjian br...@momjian.us wrote:
OK. But if we are pretty sure we don't want something, e.g. hibernate,
we shouldn't add it.
Fair enough, but I'm not even slightly sure that we don't want that.
I think having
Alvaro Herrera wrote:
Excerpts from Bruce Momjian's message of vie oct 14 12:12:22 -0300 2011:
Alvaro Herrera wrote:
The guideline, last I checked, was that before getting into coding any
item from the TODO list, the prospective hacker should check previous
discussions and
Should this be marked as TODO?
---
Mitsuru IWASAKI wrote:
Hi,
On 05/07/2011 03:32 AM, Mitsuru IWASAKI wrote:
For 1, I've just finish my work. The latest patch is available at:
On 06/05/2011 08:50 AM, Mitsuru IWASAKI wrote:
It seems that I don't have enough time to complete this work.
You don't need to keep cc'ing me, and I'm very happy if postgres to be
the first DBMS which support buffer cache hibernation feature.
Thanks for submitting the patch, and we'll see
Hi,
On 05/07/2011 03:32 AM, Mitsuru IWASAKI wrote:
For 1, I've just finish my work. The latest patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110507.patch
Reminder here--we can't accept code based on it being published to a
Yeah, I'm pretty well convinced this whole approach is a dead end.
Priming the OS buffer cache seems way more useful. I also think
saving the blocks to be read rather than the actual blocks makes a lot
more sense.
Well, his proposal works on any platforms PostgreSQL supports. On the
other
2011/6/1 Tatsuo Ishii is...@postgresql.org:
Yeah, I'm pretty well convinced this whole approach is a dead end.
Priming the OS buffer cache seems way more useful. I also think
saving the blocks to be read rather than the actual blocks makes a lot
more sense.
Well, his proposal works on any
On 06/01/2011 03:03 AM, Tatsuo Ishii wrote:
Also I really want to see the performance comparison between these two
approaches in the real world database.
Well, tell me how big of a performance improvement you want PgFincore to
win by, and I'll construct a benchmark where it does that. If
On Sun, May 15, 2011 at 11:19 AM, Robert Haas robertmh...@gmail.com wrote:
I don't think there's any need for this to get data into
shared_buffers at all. Getting it into the OS cache oughta be plenty
sufficient, no?
ISTM that a very simple approach here would be to save the contents of
On Wed, Jun 1, 2011 at 11:58 AM, Jeff Janes jeff.ja...@gmail.com wrote:
On Sun, May 15, 2011 at 11:19 AM, Robert Haas robertmh...@gmail.com wrote:
I don't think there's any need for this to get data into
shared_buffers at all. Getting it into the OS cache oughta be plenty
sufficient, no?
On Wed, Jun 1, 2011 at 8:58 AM, Jeff Janes jeff.ja...@gmail.com wrote:
In the latter case, wouldn't we just trigger the same inefficient
scattered read of the data that normal database operation would
trigger, taking about the same amount of time to reach cache-warmth?
If you have a system
On 05/07/2011 03:32 AM, Mitsuru IWASAKI wrote:
For 1, I've just finish my work. The latest patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110507.patch
Reminder here--we can't accept code based on it being published to a web
page.
On Fri, May 6, 2011 at 5:31 PM, Greg Smith g...@2ndquadrant.com wrote:
I think that all the complexity with CRCs etc. is unlikely to lead anywhere
too, and those two issues are not completely unrelated. The simplest,
safest thing here is the right way to approach this, not the most
2011/5/15 Robert Haas robertmh...@gmail.com:
On Fri, May 6, 2011 at 5:31 PM, Greg Smith g...@2ndquadrant.com wrote:
I think that all the complexity with CRCs etc. is unlikely to lead anywhere
too, and those two issues are not completely unrelated. The simplest,
safest thing here is the right
Hi,
We can't accept patches just based on a pointer to a web site. Please
e-mail this to the mailing list so that it can be considered a
submission under the project's licensing terms.
I hope this would be committable and the final version.
PostgreSQL has high standards for code
Hi,
I'd suggest doing this as an extension module. All the changes to
existing server code seem superficial.
It sounds interesting. I'll try it later.
Are there any good examples for extension module?
Thanks
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
Mitsuru IWASAKI wrote:
Are there any good examples for extension module?
Browse the subdirectories of contrib.
-Kevin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Hi,
Sorry, I missed these messages because I didn't subscribe to this list.
# I've just subscribed temporary
I think that all the complexity with CRCs etc. is unlikely to lead anywhere
too, and those two issues are not completely unrelated. The simplest,
safest thing here is the right way
Mitsuru IWASAKI wrote:
the patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110508.patch
We can't accept patches just based on a pointer to a web site. Please
e-mail this to the mailing list so that it can be considered a
On 08.05.2011 07:58, Mitsuru IWASAKI wrote:
I'll do more testing tomorrow, and hopefully finalize my patch.
Done! the patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110508.patch
I'd suggest doing this as an extension module. All the
On Sat, May 7, 2011 at 3:32 AM, Mitsuru IWASAKI iwas...@jp.freebsd.org wrote:
I have one more day for working on this, but I may give up...
I think this is an interesting line of inquiry, but if you were hoping
to get something committable in a couple of days, you had unrealistic
expectations...
Hi, folks!
I'll do more testing tomorrow, and hopefully finalize my patch.
Done! the patch is available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110508.patch
I hope this would be committable and the final version.
Major changes from the
Hi,
I revised the patch against HEAD, it's available at:
http://people.freebsd.org/~iwasaki/postgres/buffer-cache-hibernation-postgresql-20110506.patch
Implemented hibernation file validations:
- comparison with pg_control
At shutdown:
pg_control state should be DB_SHUTDOWNED.
At startup:
On 05/05/2011 05:06 AM, Mitsuru IWASAKI wrote:
In summary, PgFincore's target is File System Buffer Cache, Buffer
Cache Hibernation's target is DB Buffer Cache(shared buffers).
Right. The thing to realize is that shared_buffers is becoming a
smaller fraction of the total RAM used by the
On Fri, May 6, 2011 at 5:31 PM, Greg Smith g...@2ndquadrant.com wrote:
On 05/05/2011 05:06 AM, Mitsuru IWASAKI wrote:
In summary, PgFincore's target is File System Buffer Cache, Buffer
Cache Hibernation's target is DB Buffer Cache(shared buffers).
Right. The thing to realize is that
Hi, thanks for your comments!
I'm glad to discuss about this topic.
* pgfadv_WILLNEED
* pgfadv_WILLNEED_snapshot
The former ask to load each segment of a relation *but* the kernel can
decide to not do that or load only part of each segment. (so it is not
as brutal as cat file /dev/null
Josh Berkus j...@agliodbs.com writes:
I thought that Dimitri had already implemented this using Fincore. It's
linux-only, but that should work well enough to test the general concept.
Actually, Cédric did, and I have a clone of his repository where I did
some debian packaging of it.
2011/5/4 Josh Berkus j...@agliodbs.com:
All,
I thought that Dimitri had already implemented this using Fincore. It's
linux-only, but that should work well enough to test the general concept.
Harald provided me some pointers at pgday in Stuttgart to make it work
with windows but ... hum I
Hi, thanks for good suggestions.
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data structure into hibernation files just before shutdown, and
loading them at startup, postgres can start operations with the saved
buffer cache as the same condition as just
Hi,
I think that PgFincore (http://pgfoundry.org/projects/pgfincore/)
provides similar functionality. Are you familiar with that? If so,
could you contrast your approach with that one?
I'm not familiar with PgFincore at all sorry, but I got source code
and documents and read through them
2011/5/5 Mitsuru IWASAKI iwas...@jp.freebsd.org:
Hi,
I think that PgFincore (http://pgfoundry.org/projects/pgfincore/)
provides similar functionality. Are you familiar with that? If so,
could you contrast your approach with that one?
I'm not familiar with PgFincore at all sorry, but I got
On 05/04/2011 10:10 AM, Mitsuru IWASAKI wrote:
Hi,
I am working on new feature `Buffer Cache Hibernation' which enables
postgres to keep higher cache hit ratio even just started.
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data structure into hibernation files
On Wed, May 4, 2011 at 3:10 PM, Mitsuru IWASAKI iwas...@jp.freebsd.org wrote:
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data structure into hibernation files just before shutdown, and
loading them at startup, postgres can start operations with the saved
buffer
Mitsuru IWASAKI iwas...@jp.freebsd.org writes:
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data structure into hibernation files just before shutdown, and
loading them at startup, postgres can start operations with the saved
buffer cache as the same condition as
Excerpts from Tom Lane's message of mié may 04 12:44:36 -0300 2011:
This seems like a lot of complication for rather dubious gain. What
happens when the DBA changes the shared_buffers setting, for instance?
How do you protect against the cached buffers getting out-of-sync with
the actual
2011/5/4 Greg Stark gsst...@mit.edu:
On Wed, May 4, 2011 at 3:10 PM, Mitsuru IWASAKI iwas...@jp.freebsd.org
wrote:
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data structure into hibernation files just before shutdown, and
loading them at startup, postgres can
On Wed, May 4, 2011 at 4:44 PM, Tom Lane t...@sss.pgh.pa.us wrote:
Do you have
any proof that writing out a few GB of buffers and then reading them
back in is actually much cheaper than letting the database re-read the
data from the disk files?
I believe he's just writing out the meta data.
Alvaro Herrera wrote:
As for gain, I have heard of test setups requiring hours of runtime in
order to prime the buffer cache.
And production ones too. I have multiple customers where a server
restart is almost a planned multi-hour downtime. The system may be back
up, but for a couple of
On Wed, May 4, 2011 at 7:10 AM, Mitsuru IWASAKI iwas...@jp.freebsd.org wrote:
Hi,
I am working on new feature `Buffer Cache Hibernation' which enables
postgres to keep higher cache hit ratio even just started.
Postgres usually starts with ZERO buffer cache. By saving the buffer
cache data
All,
I thought that Dimitri had already implemented this using Fincore. It's
linux-only, but that should work well enough to test the general concept.
--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make
51 matches
Mail list logo
