Re: [HACKERS] per table random-page-cost?

2009-11-09 Thread Bruce Momjian
Robert Haas wrote: On Thu, Oct 22, 2009 at 2:28 PM, Josh Berkus j...@agliodbs.com wrote: All, Wouldn't per *tablespace* costs make more sense? --Josh Yes, we already had several votes in favor of that approach. See upthread. Added to TODO: Allow per-tablespace

Re: [HACKERS] per table random-page-cost?

2009-10-26 Thread Cédric Villemain
Le samedi 24 octobre 2009 01:04:19, Josh Berkus a écrit : Cedric, ase is a table containing 29 GB of bytea in a database of 52 GB. Every row on the 29GB table is grab only few times. And it will just renew OS cache memory every time (the server have only 8GB of ram). So when I remove

Re: [HACKERS] per table random-page-cost?

2009-10-24 Thread Stefan Moeding
Hi! Josh Berkus writes: Now, if we had an OS which could be convinced to handle caching differently for different physical devices, then I could see wanting this setting to be per-tablespace. For example, it would make a lot of sense not to FS-cache any data which is on a ramdisk or

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Cédric Villemain
Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le lundi 19 octobre 2009 23:27:20, Greg Stark a écrit : On Mon, Oct 19, 2009 at

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Robert Haas
On Fri, Oct 23, 2009 at 5:23 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote:

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Cédric Villemain
Le vendredi 23 octobre 2009 14:23:09, Robert Haas a écrit : On Fri, Oct 23, 2009 at 5:23 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le vendredi 23 octobre 2009 01:08:15, Joshua D. Drake a écrit : On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at

Re: [HACKERS] per table random-page-cost?

2009-10-23 Thread Josh Berkus
Cedric, ase is a table containing 29 GB of bytea in a database of 52 GB. Every row on the 29GB table is grab only few times. And it will just renew OS cache memory every time (the server have only 8GB of ram). So when I remove this table (not the index) from the OS cache memory, I keep

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Cédric Villemain
Le mardi 20 octobre 2009 06:30:26, Greg Smith a écrit : On Mon, 19 Oct 2009, Jeff Davis wrote: On Mon, 2009-10-19 at 21:22 -0500, Kevin Grittner wrote: I'd bet accounts receivable applications often hit that. (Most payments on recent billings; a sprinkling on older ones.) I'm sure there

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Cédric Villemain
Le lundi 19 octobre 2009 23:14:40, Robert Haas a écrit : On Mon, Oct 19, 2009 at 5:08 PM, marcin mank marcin.m...@gmail.com wrote: Currently random_page_cost is a GUC. I propose that this could be set per-table. I think this is a good idea for widely-wanted planner hints. This way You

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Cédric Villemain
Le lundi 19 octobre 2009 23:27:20, Greg Stark a écrit : On Mon, Oct 19, 2009 at 2:08 PM, marcin mank marcin.m...@gmail.com wrote: Currently random_page_cost is a GUC. I propose that this could be set per-table. Or per-tablespace. Yes, I think there are a class of GUCs which describe the

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Robert Haas
On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le lundi 19 octobre 2009 23:27:20, Greg Stark a écrit : On Mon, Oct 19, 2009 at 2:08 PM, marcin mank marcin.m...@gmail.com wrote: Currently random_page_cost is a GUC. I propose that this could be set

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Greg Stark
On Thu, Oct 22, 2009 at 8:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: You can have situation where you don't want some tables go to OS memory I don't think this is a configuration we want to cater for. The sysadmin shouldn't be required to understand the i/o pattern of postgres.

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Josh Berkus
All, Wouldn't per *tablespace* costs make more sense? --Josh -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Kevin Grittner
Greg Stark gsst...@mit.edu wrote: There is another use case which perhaps needs to be addressed: if the user has some queries which are very latency sensitive and others which are not latency sensitive. Yes. Some products allow you to create a named cache and bind particular objects to it.

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Robert Haas
On Thu, Oct 22, 2009 at 2:28 PM, Josh Berkus j...@agliodbs.com wrote: All, Wouldn't per *tablespace* costs make more sense? --Josh Yes, we already had several votes in favor of that approach. See upthread. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Greg Stark
Well I think we need sone way to accomplish the same high level goal of guaranteeing response times for latency-critical queries. However my point is that cache policy is an internal implementation detail we don't want to expose in a user interface. -- Greg On 2009-10-22, at 11:41 AM,

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Joshua D. Drake
On Thu, 2009-10-22 at 11:33 -0400, Robert Haas wrote: On Thu, Oct 22, 2009 at 11:16 AM, Cédric Villemain cedric.villem...@dalibo.com wrote: Le lundi 19 octobre 2009 23:27:20, Greg Stark a écrit : On Mon, Oct 19, 2009 at 2:08 PM, marcin mank marcin.m...@gmail.com wrote: Currently

Re: [HACKERS] per table random-page-cost?

2009-10-22 Thread Alvaro Herrera
Greg Stark escribió: There is another use case which perhaps needs to be addressed: if the user has some queries which are very latency sensitive and others which are not latency sensitive. In that case it might be very important to keep the pages of data used by the high priority queries in

Re: [HACKERS] per table random-page-cost?

2009-10-20 Thread marcin mank
On Tue, Oct 20, 2009 at 1:21 AM, Tom Lane t...@sss.pgh.pa.us wrote: If the parameter is defined as the chance that a page is in cache there is very real physical meaning to it. We have no such parameter... What a simple person like me would think would work is: - call the parameter

Re: [HACKERS] per table random-page-cost?

2009-10-20 Thread Kevin Grittner
Greg Smith gsm...@gregsmith.com wrote: The unfortunate reality of accounts receivable is that reports run to list people who owe one money happen much more often than posting payments into the system does. How often do you have to print a list of past due accounts? I've generally seen that

Re: [HACKERS] per table random-page-cost?

2009-10-20 Thread Greg Smith
On Tue, 20 Oct 2009, Kevin Grittner wrote: How often do you have to print a list of past due accounts? I've generally seen that done weekly or monthly, in the same places that there are many people standing full time in payment windows just to collect money from those lining up to pay. This

[HACKERS] per table random-page-cost?

2009-10-19 Thread marcin mank
Currently random_page_cost is a GUC. I propose that this could be set per-table. I think this is a good idea for widely-wanted planner hints. This way You can say I do NOT want this table to be index-scanned, because I know it is not cached by setting it`s random_page_cost to a large value (an

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Robert Haas
On Mon, Oct 19, 2009 at 5:08 PM, marcin mank marcin.m...@gmail.com wrote: Currently random_page_cost is a GUC. I propose that this could be set per-table. I think this is a good idea for widely-wanted planner hints. This way You can say I do NOT want this table to be index-scanned, because I

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Greg Stark
On Mon, Oct 19, 2009 at 2:08 PM, marcin mank marcin.m...@gmail.com wrote: Currently random_page_cost is a GUC. I propose that this could be set per-table. Or per-tablespace. Yes, I think there are a class of GUCs which describe the physical attributes of the storage system which should be

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Kevin Grittner
Robert Haas robertmh...@gmail.com wrote: I've been wondering if it might make sense to have a random_page_cost and seq_page_cost setting for each TABLESPACE, to compensate for the fact that different media might be faster or slower, and a percent-cached setting for each table over top of

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread marcin mank
I've been thinking about this a bit, too.  I've been wondering if it might make sense to have a random_page_cost and seq_page_cost setting for each TABLESPACE, to compensate for the fact that different media might be faster or slower, and a percent-cached setting for each table over top of

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread marcin mank
I thought about making it per-table***space***, but realistically I -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Tom Lane
marcin mank marcin.m...@gmail.com writes: I thought about making it per-table, but realistically I think most people don`t use tablespaces now. I would not want to be telling people to be able to hint the planner to (not) index-scan the table, You must move it to a separate tablespace.

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Andrew Dunstan
marcin mank wrote: I've been thinking about this a bit, too. I've been wondering if it might make sense to have a random_page_cost and seq_page_cost setting for each TABLESPACE, to compensate for the fact that different media might be faster or slower, and a percent-cached setting for each

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread marcin mank
This proposal is just hints by the back door, ISTM. As Tom says, there is a justification for having it on tablespaces but not on individual tables. If the parameter is defined as the chance that a page is in cache there is very real physical meaning to it. And this is per-table, not

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Tom Lane
marcin mank marcin.m...@gmail.com writes: This proposal is just hints by the back door, ISTM. As Tom says, there is a justification for having it on tablespaces but not on individual tables. If the parameter is defined as the chance that a page is in cache there is very real physical meaning

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Greg Stark
On Mon, Oct 19, 2009 at 4:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: marcin mank marcin.m...@gmail.com writes: This proposal is just hints by the back door, ISTM. As Tom says, there is a justification for having it on tablespaces but not on individual tables. If the parameter is defined as the

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Robert Haas
On Mon, Oct 19, 2009 at 5:54 PM, Kevin Grittner kevin.gritt...@wicourts.gov wrote: Robert Haas robertmh...@gmail.com wrote: I've been wondering if it might make sense to have a random_page_cost and seq_page_cost setting for each TABLESPACE, to compensate for the fact that different media

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Jeff Davis
On Mon, 2009-10-19 at 16:39 -0700, Greg Stark wrote: But the long-term strategy here I think is to actually have some way to measure the real cache hit rate on a per-table basis. Whether it's by timing i/o operations, programmatic access to dtrace, or some other kind of os interface, if we

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Jeff Janes
On Mon, Oct 19, 2009 at 4:29 PM, Greg Stark gsst...@mit.edu wrote: On Mon, Oct 19, 2009 at 4:21 PM, Tom Lane t...@sss.pgh.pa.us wrote: marcin mank marcin.m...@gmail.com writes: This proposal is just hints by the back door, ISTM. As Tom says, there is a justification for having it on

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Kevin Grittner
Jeff Davis pg...@j-davis.com wrote: what kind of scenario would involve a stable 90% cache hit ratio for a table? I'd bet accounts receivable applications often hit that. (Most payments on recent billings; a sprinkling on older ones.) I'm sure there are others. -Kevin -- Sent via

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Jeff Davis
On Mon, 2009-10-19 at 21:22 -0500, Kevin Grittner wrote: I'd bet accounts receivable applications often hit that. (Most payments on recent billings; a sprinkling on older ones.) I'm sure there are others. You worded the examples in terms of writes (I think), and we're talking about read

Re: [HACKERS] per table random-page-cost?

2009-10-19 Thread Greg Smith
On Mon, 19 Oct 2009, Jeff Davis wrote: On Mon, 2009-10-19 at 21:22 -0500, Kevin Grittner wrote: I'd bet accounts receivable applications often hit that. (Most payments on recent billings; a sprinkling on older ones.) I'm sure there are others. You worded the examples in terms of writes (I