Re: [HACKERS] Free-space-map management thoughts

2003-03-02 Thread Stephen Marshall
Tom Lane wrote: Stephen Marshall [EMAIL PROTECTED] writes: 2. The histogram concept is a neat idea, but I think some reorganization of the page information might make it unnecessary. Currently the FSM pages are sorted by BlockNumber. This was particularly useful for adding

Re: [HACKERS] Free-space-map management thoughts

2003-03-02 Thread Stephen Marshall
Tom, I'm happy to see your attentions turning back to the FSM. I like the design, but I do have a few suggestions, particularly about how to handle oversubscription of the FSM. 1. When the FSM is oversubscribed and one is trying to decide which pages to keep, remember that page info is

Re: [HACKERS] Free-space-map management thoughts

2003-02-27 Thread Tom Lane
Stephen Marshall [EMAIL PROTECTED] writes: 1. When the FSM is oversubscribed and one is trying to decide which pages to keep, remember that page info is stored in groups of CHUNK_SIZE pages, where CHUNK_SIZE is current 32. Right, oversubscription would actually need to be measured in chunks

Re: [HACKERS] Free-space-map management thoughts

2003-02-27 Thread Tom Lane
Robert Treat [EMAIL PROTECTED] writes: Now that indexes are getting some reporting, my understanding is an index would report fewer pages overall than it's associated table, but those pages would be completely empty. However, given that they don't reported non-empty pages, the percentage of

Re: [HACKERS] Free-space-map management thoughts

2003-02-27 Thread Robert Treat
On Thu, 2003-02-27 at 11:00, Tom Lane wrote: Robert Treat [EMAIL PROTECTED] writes: Now that indexes are getting some reporting, my understanding is an index would report fewer pages overall than it's associated table, but those pages would be completely empty. However, given that they

Re: [HACKERS] Free-space-map management thoughts

2003-02-27 Thread Tom Lane
Robert Treat [EMAIL PROTECTED] writes: I think I was thinking that a given table will always report more pages than an index on that table, since tables can report 50% empty pages while indexes only report 100% empty pages. This would cause tables to generally be favored over indexes, even

Re: [HACKERS] Free-space-map management thoughts

2003-02-27 Thread Tom Lane
I wrote: Stephen Marshall [EMAIL PROTECTED] writes: If we sort the page info by available space, we could then use binary search to find space thresholds when we are handling oversubscription. The list-of-chunks storage layout provides only limited traction for searching anyway, and none

Re: [HACKERS] Free-space-map management thoughts

2003-02-27 Thread Tom Lane
Stephen Marshall [EMAIL PROTECTED] writes: If I understand the concept correctly, the histogram will only be calculated when MultiRecordFreeSpace is called AND the FSM is oversubscribed. However, when it is called, we will need to calculate a histogram for, and potentially trim data from,

[HACKERS] Free-space-map management thoughts

2003-02-26 Thread Tom Lane
I've been thinking about improving the algorithm that the free space map (FSM) uses to decide what to store when it's not got enough shared memory to keep track of everything. The present design uses a dynamically adjusted threshold for each relation, throwing away pages whose free space amount

Re: [HACKERS] Free-space-map management thoughts

2003-02-26 Thread Matthew T. O'Connor
PS: Another idea I'm toying with is to dump out the FSM contents at postmaster shutdown and reload them at restart, so that the FSM doesn't have to start from ground zero on every restart cycle. But that's independent of the management algorithm... Correct me if I'm wrong, but the FSM is

Re: [HACKERS] Free-space-map management thoughts

2003-02-26 Thread Tom Lane
Matthew T. O'Connor [EMAIL PROTECTED] writes: Correct me if I'm wrong, but the FSM is only populated by vacuum, so there is no FSM information for any given table / database until it's vacuumed, in a long running production enviornment this may not be that important, but it could result in a