Heikki Linnakangas napsal(a):
Zdenek Kotala wrote:
Heikki Linnakangas napsal(a):
The patch seems to be missing the new htup.c file.
I'm sorry. I attached new version which is synchronized with current
head. I would like to say more comments as well.
1) The patch contains also changes which was discussed during July
commit fest. - PageGetTempPage modification suggested by Tom
- another hash.h backward compatible cleanup
It might be a good idea to split that into a separate patch. The sheer
size of this patch is quite daunting, even though the bulk of it is
straightforward search&replace.
Yes, I will do it.
2) I add tuplimits.h header file which contains tuple limits for
different access method. It is not finished yet, but idea is to keep
all limits in one file and easily add limits for different page layout
version - for example replace static computing with dynamic based on
relation (maxtuplesize could be store in pg_class for each relation).
I need this header also because I fallen in a cycle in header dependency.
3) I already sent Page API performance result in
http://archives.postgresql.org/pgsql-hackers/2008-08/msg00398.php
I replaced call sequence PagetGetItemId, PageGetItemId with
PageGetIndexTuple and PageGetHeapTuple function. It is main difference
in this patch. PAgeGetHeap Tuple fills t_ver in HeapTuple to identify
correct tupleheader version.
It would be good to mention that PageAPI (and tuple API)
implementation is only prototype without any performance optimization.
You mentioned 5% performance degradation in that thread. What test case
was that? What would be a worst-case scanario, and how bad is it?
Paul van den Bogaart tested long run OLTP workload on it. He used iGen test.
5% is a pretty hefty price, especially when it's paid by not only
upgraded installations, but also freshly initialized clusters. I think
you'll need to pursue those performance optimizations.
5% is worst scenario. Current version is not optimized. It is written for easy
debugging and (D)tracing. Pageheaders structures are very similar and we can
easily remove switches for most of attributes and replace function with macros
or inline function.
4) This patch contains more topics for decision. First is general if
this approach is acceptable.
I don't like the invasiveness of this approach. It's pretty invasive
already, and ISTM you'll need similar switch-case handling of all data
types that have changed the internal representation as well.
I agree in general. But for example new page API is not so invasive and by my
opinion it should be implemented (with or without multiversion support), because
it cleans a code. HeapTuple processing is easy too, but unfortunately it
requires lot of modifications on many places. I has wonder how many pieces of
code access directly to HeapTupleHeader and does not use HeapTuple data
structure. I think we should make a conclusion what is recommended usage of
HeapTupleHeader and HeapTuple. Most of changes in a code is like replacing
HeapTupleHeaderGetXmax(tuple->t_data) with HeapTupleGetXmax(tuple) and so on. I
think it should be cleanup anyway.
You mentioned data types, but it is not a problem. You can easily extend data
type attribute about version information and call correct in/out functions. Or
use different Oid for new data type version. There are more possible easy
solutions for data types. And for conversion You can use ALTER TABLE command.
Main idea is keep data in all format in a relation. This approach should use
also for integer/float datetime problem.
We've talked about this before, so you'll remember that I favor teh
approach is to convert the page format, page at a time, when the pages
are read in. I grant you that there's non-trivial issues with that as
well, like if the converted data takes more space and don't fit in the
page anymore.
I like conversion on read too, because it is easy but there are more problems.
The non-fit page is one them. Others problems are with indexes. For example
hash index stores bitmap into page and it is not mentioned anywhere. Only hash
am knows what page contains this kind of data. It is probably impossible to
convert this page during a reading. :(
I wonder if we could go with some sort of a hybrid approach? Convert the
whole page when it's read in, but if it doesn't fit, fall back to
tricks like loosening the alignment requirements on platforms that can
handle non-aligned data, or support a special truncated page header,
without pd_tli and pd_prune_xid fields. Just a thought, not sure how
feasible those particular tricks are, but something along those lines..
OK, I have backup idea :-). Stay tuned :-)
All in all, though. I find it a bit hard to see the big picture. For
upgrade-in-place, what are all the pieces that we need? To keep this
concrete, let's focus on PG 8.2 -> PG 8.3 (or are you focusing on PG 8.3
-> 8.4? That's fine with me as well, but let's pick one) and forget
about hypothetical changes that might occur in a future version. I can see:
1. Handling page layout changes (pd_prune_xid, pd_flags)
2. Handling tuple header changes (infomask2, HOT bits, combocid)
2.5 + composite data type
3. Handling changes in data type representation (packed varlens)
3.5 Data types generally (cidr/inet)
4. Toast chunk size
4.5 general MaxTupleSize for each different AM
5. Catalogs
6. AM methods
After putting all those together, how large a patch are we talking
about, and what's the performance penalty then? How much of all that
needs to be in core, and how much can live in a pgfoundry project or an
extra binary in src/bin or contrib? I realize that none of us have a
crystal ball, and one has to start somewhere, but I feel uneasy
committing to an approach until we have a full plan.
Unfortunately, I'm still in analyzing phase. Presented patch is prototype of one
possible approach. I hit lot of problems and I don't have still answers on all
of them. I'm going to update wiki page to share all these information.
At this moment, I think that I can implement offline heap conversion (8.2->8.4)
and all indexed will be reindex. It is what we can have for 8.4. Online
conversion has lot of problems which we are not able to answer at this moment.
Zdenek
--
Zdenek Kotala Sun Microsystems
Prague, Czech Republic http://sun.com/postgresql
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
