Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Hannu Krosing
Ühel kenal päeval, L, 2006-06-24 kell 19:36, kirjutas Bruce Momjian: Hannu Krosing wrote: ?hel kenal p?eval, R, 2006-06-23 kell 13:08, kirjutas Tom Lane: Bottom line: there's still lots of low-hanging fruit. Why are people feeling that we need to abandon or massively complicate our

Re: [HACKERS] [CORE] GPL Source and Copyright Questions

2006-06-25 Thread Michael Meskes
On Sat, Jun 24, 2006 at 09:45:45PM -0400, Bruce Momjian wrote: Michael, I saw your patch stating that the copyright was assigned to PGDG. However, once that happens, we are of the policy to remove copyrights to individual users because it confuses things. Therefore, I have updated your

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Mark Woodward
On 6/24/2006 9:23 AM, Mark Woodward wrote: On Sat, 24 Jun 2006, Mark Woodward wrote: I'm probably mistaken, but aren't there already forward references in tuples to later versions? If so, I'm only sugesting reversing the order and referencing the latest version. I thought I understood

Re: [HACKERS] Buffer for inner and outer table

2006-06-25 Thread Daniel Xavier de Sousa
Hi Álvaro,Thank you about your answer. I thought that Postgres could management the space for outer and inner tables. Because, some articles has showed how this make the difference.Alvaro please, And about count of pages that postgres read on query

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
Hannu Krosing wrote: Maybe we could start from reusing the index tuples which point to invisible tuples ? The index is not MVCC anyway, so maybe it is easier to do in-place replacement there ? This probably has the same obstacles which have prevented us from removing those in

Re: [HACKERS] vacuum row?

2006-06-25 Thread Jonah H. Harris
On 6/24/06, Mark Woodward [EMAIL PROTECTED] wrote: I originally suggested a methodology for preserving MVCC and everyone is confusing it as update in place, this isnot what I intended. Actually, you should've presented your idea as performing MVCC the way Firebird does... the idea is basically

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Jan Wieck
On 6/25/2006 6:52 AM, Mark Woodward wrote: On 6/24/2006 9:23 AM, Mark Woodward wrote: On Sat, 24 Jun 2006, Mark Woodward wrote: I'm probably mistaken, but aren't there already forward references in tuples to later versions? If so, I'm only sugesting reversing the order and referencing the

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Jan Wieck
On 6/25/2006 12:27 PM, Bruce Momjian wrote: Hannu Krosing wrote: Maybe we could start from reusing the index tuples which point to invisible tuples ? The index is not MVCC anyway, so maybe it is easier to do in-place replacement there ? This probably has the same obstacles which

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Jan Wieck
On 6/24/2006 4:10 PM, Hannu Krosing wrote: Ühel kenal päeval, L, 2006-06-24 kell 15:44, kirjutas Jan Wieck: That fixes the symptom, not the problem. The problem is performance steadily degrades over time. No, you got it backwards. The performance degradation is the symptom. The

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Heikki Linnakangas
On Sat, 24 Jun 2006, Bruce Momjian wrote: Because having them be on the same page is the only way you can update the page item pointer so when you recycle the row, you the indexes are now pointing to the new version. Pages look like:

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
Jan Wieck wrote: Sure, but index reuse seems a lot easier, as there is nothing additional to remember or clean out when doing it. Yes, seems so. TODO added: * Reuse index tuples that point to heap tuples that are not visible to anyone? When reusing a heap tuple you

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
Heikki Linnakangas wrote: On Sat, 24 Jun 2006, Bruce Momjian wrote: Because having them be on the same page is the only way you can update the page item pointer so when you recycle the row, you the indexes are now pointing to the new version. Pages look like:

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Jan Wieck
On 6/25/2006 2:24 PM, Bruce Momjian wrote: Jan Wieck wrote: Sure, but index reuse seems a lot easier, as there is nothing additional to remember or clean out when doing it. Yes, seems so. TODO added: * Reuse index tuples that point to heap tuples that are not visible to

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Hannu Krosing
Ühel kenal päeval, P, 2006-06-25 kell 14:24, kirjutas Bruce Momjian: Jan Wieck wrote: Sure, but index reuse seems a lot easier, as there is nothing additional to remember or clean out when doing it. Yes, seems so. TODO added: * Reuse index tuples that point to heap tuples

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Hannu Krosing
Ühel kenal päeval, P, 2006-06-25 kell 06:52, kirjutas Mark Woodward: I'm not sure why vacuum can't run similarly to the way it does now. What do you mean ? Currently vacuum runs a three-step process 1) runs a full scan over heap and collects all dead tuple ctids from heap 2) run full scans

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
Jan Wieck wrote: An update that results in all the same values of every indexed column of a known deleted invisible tuple. This reused tuple can by definition not be the one currently updated. So unless it is a table without a primary key, this assumes that at least 3 versions of the

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
Hannu Krosing wrote: ?hel kenal p?eval, P, 2006-06-25 kell 14:24, kirjutas Bruce Momjian: Jan Wieck wrote: Sure, but index reuse seems a lot easier, as there is nothing additional to remember or clean out when doing it. Yes, seems so. TODO added: * Reuse

Re: [HACKERS] xlog viewer proposal

2006-06-25 Thread Diogo Biazus
Alright, I'm working on a fast prototype using the SRF.On 6/23/06, Simon Riggs [EMAIL PROTECTED] wrote: On Fri, 2006-06-23 at 10:59 -0300, Diogo Biazus wrote: On 6/23/06, Simon Riggs [EMAIL PROTECTED] wrote: - give more flexibility for managing the xlogs remotely Not sure what you mean. - I

[HACKERS] Table clustering idea

2006-06-25 Thread Dawid Kuroczko
There is a well known command called CLUSTER which organizes tablein specified index's order. It has a drawback, that new tuples added arenot in this order. Last night I had idea which could be interesting, I hope. The idea is to make use of 'histogram_bounds' collected statistical data.Instead of

Re: [HACKERS] Table clustering idea

2006-06-25 Thread Luke Lonergan
Dawid, Other idea than using histogram_bounds would be using the position of key inside the index to determine the ideal place of row inside the table and find the closest free spot there. This would be of course much more precise and wouldn't rely on statistic. This is generally known

Re: [HACKERS] Table clustering idea

2006-06-25 Thread Josh Berkus
Luke, Other DBMS have index organized tables that can use either hash or btree organizations, both of which have their uses.  We are planning to implement btree organized tables sometime - anyone else interested in this idea? Search the archives. It's been dicussed on this list at least

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
bruce wrote: Why three? I explained using only two heap tuples: [item1]...[tuple1] becomes on UPDATE: -- [item1]...[tuple1][tuple2] - on another UPDATE, if tuple1 is no longer visible: --

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Jan Wieck
On 6/25/2006 5:18 PM, Bruce Momjian wrote: Jan Wieck wrote: An update that results in all the same values of every indexed column of a known deleted invisible tuple. This reused tuple can by definition not be the one currently updated. So unless it is a table without a primary key, this

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Bruce Momjian
Jan Wieck wrote: [item1]...[tuple1] becomes on UPDATE: -- [item1]...[tuple1][tuple2] - on another UPDATE, if tuple1 is no longer visible: -- [item1]...[tuple1][tuple2]

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Jan Wieck
On 6/25/2006 10:12 PM, Bruce Momjian wrote: When you are using the update chaining, you can't mark that index row as dead because it actually points to more than one row on the page, some are non-visible, some are visible. Back up the truck ... you mean in the current code base we have heap

Re: [HACKERS] vacuum, performance, and MVCC

2006-06-25 Thread Alvaro Herrera
Jan Wieck wrote: On 6/25/2006 10:12 PM, Bruce Momjian wrote: When you are using the update chaining, you can't mark that index row as dead because it actually points to more than one row on the page, some are non-visible, some are visible. Back up the truck ... you mean in the current code