[HACKERS] [Patch] Space reservation (pgupgrade)

2008-12-12 Thread Zdenek Kotala
I attached patch which add capability to reserve space on page for future upgrade. It is mandatory for future in-place upgrade implementation. This patch contains basic infrastructure not preupgrade script itself. I'm going to send WIP preupgrade script today in separate mail. This patch conta

Re: [HACKERS] visibility maps

2008-12-11 Thread Zdenek Kotala
Pavan Deolasee napsal(a): On Thu, Dec 11, 2008 at 5:01 PM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: IIRC, Memory reading/writing is atomic operation. Only one CPU(hw thread) can access to the same memory address(es)* in same time*. The question is how compiler compile C code to assembler

Re: [HACKERS] visibility maps

2008-12-11 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Pavan Deolasee wrote: /* * We don't need to lock the page, as we're only looking at a single bit. */ result = (map[mapByte] & (1 << mapBit)) ? true : false; Isn't this a dangerous assumption to make ? I am not so sure that even a bit can be read

Re: [HACKERS] Debug crash

2008-12-11 Thread Zdenek Kotala
Alif Isophoqote napsal(a): Hi, I am developing a backend module for postgresql. For some queries, postgresql crashes: segmentation fault (most probably due to my own code). How to get the function name & line number where it crashes? At first it depends on your OS. On unixes you should get cor

Re: [HACKERS] cvs head initdb hangs on unixware

2008-12-10 Thread Zdenek Kotala
Tom Lane napsal(a): [EMAIL PROTECTED] writes: On Wed, 10 Dec 2008, Heikki Linnakangas wrote: BTW, why does this work on warthog buildfarm member? Different compiler version? it's configured with --enable-debug. Maybe run_build.pl should run twice, onece with --enable-debug once without. N

Re: [HACKERS] cvs head initdb hangs on unixware

2008-12-09 Thread Zdenek Kotala
[EMAIL PROTECTED] napsal(a): I first misread your mail, and added only the first fprintf , while I was uploading a 400M initdb.log, I went back to add the second one. Guess what! with the fprintf .. descending node... in place, everything goes well. The optimizer definitly does something we

Re: [HACKERS] Versioned mo file installation

2008-12-05 Thread Zdenek Kotala
Solaris currently stores messages into separate directories like /usr/postgres/8.3/share/locale which solves multiversion problem, but it brings another problem, because gettext on solaris needs extra directory infrastructure. I think debians patch is good approach. Zdenek Pet

Re: [HACKERS] [patch] pg_upgrade script for 8.3->8.4

2008-12-05 Thread Zdenek Kotala
Greg Smith napsal(a): On Thu, 4 Dec 2008, Zdenek Kotala wrote: 1) Keep relfileid of toast file same. It is important because toast pointer contains relfileid. Currently script creates fake files with same number to protect postgresql to create new relation with this refileid. It works but by

Re: [HACKERS] [patch] pg_upgrade script for 8.3->8.4

2008-12-04 Thread Zdenek Kotala
Zdenek Kotala napsal(a): Hi all, I attached pg_upgrade.sh script patch which works now for 8.3->8.4. It is contrib module in contrib/pg_upgrade directory. Just make/make install and it works. I forget to mention that default datetime format is different now. Please, use same datet

[HACKERS] [patch] pg_upgrade script for 8.3->8.4

2008-12-04 Thread Zdenek Kotala
Hi all, I attached pg_upgrade.sh script patch which works now for 8.3->8.4. It is contrib module in contrib/pg_upgrade directory. Just make/make install and it works. There are two changes from previous 8.1->8.2. 1) pg_largeobject is also upgraded 2) added check for dropped column And now ho

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Zdenek Kotala
Gregory Stark napsal(a): How about adding a special syntax for CREATE TABLE which indicates to include a dropped column in that position? Then pg_dump could have a -X option to include those columns as placeholders. Something like: CREATE TABLE foo ( col1 integer, NULL COLUMN, col2 integer )

Re: [HACKERS] In-place upgrade: catalog side

2008-12-04 Thread Zdenek Kotala
Greg Smith napsal(a): On Wed, 3 Dec 2008, Zdenek Kotala wrote: It works fine for 8.3->8.4 too, but I'm working on cleanup and fixing bugs. I hope that I will send updated version to community today. That would be great. It didn't feel like you were quite done with it yet. I&

Re: [HACKERS] In-place upgrade: catalog side

2008-12-03 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Greg Smith napsal(a): -There are 10 TODO items listed for the pg_migrator project, most or all of which look like should be squashed before this is really complete. Any chance somebody (Korry?) has an improved version of this floating around

Re: [HACKERS] In-place upgrade: catalog side

2008-12-02 Thread Zdenek Kotala
Greg Smith napsal(a): The main thing you'll find there is a ksh script that handles most of the upgrade, presuming there's no page format changes. It looks like it was originally aimed at 8.1->8.2 upgrades (easy as long as you don't use INET/CIDR in your database) and still has some hard-cod

Re: [HACKERS] cvs head initdb hangs on unixware

2008-12-02 Thread Zdenek Kotala
Could you generate a core and send a stacktrace? kill SIGABRT should do that. Zdenek [EMAIL PROTECTED] napsal(a): Hi all, cvs head configured without --enable-debug hang in initdb while making check. warthog doesn't exhibit it because it's configured with debug. when it hangs, po

Re: [HACKERS] Thread safety

2008-11-27 Thread Zdenek Kotala
Andrew Chernow napsal(a): It would probably be useful to nail down the supported platforms, have a list somewhere of the oldest ones: oldest solaris, hpux, irix, aix, sco, etc... I ran into a few --enable-thread-safety problems, Magnus fixed the cygwin build already. hpux 10.20 and solari

Re: [HACKERS] [WIP] In-place upgrade

2008-11-27 Thread Zdenek Kotala
Robert Haas napsal(a): 1. htup and bufpage API clean up 2. HeapTuple version extension + code cleanup 3. In-place online upgrade 4. Extending pg_class info + more flexible TOAST chunk size big thanks for your review. I think #1 is still partially valid, because it contains general cleanups, but

Re: [HACKERS] [WIP] In-place upgrade

2008-11-26 Thread Zdenek Kotala
Alvaro Herrera napsal(a): Robert Haas escribió: With respect to #4, I know that Alvaro submitted a draft patch, but I'm not clear on whether that needs to be reviewed, because: - I'm not sure whether it's close enough to being finished for a review to be a good use of time. - I'm not sure how

Re: [HACKERS] [WIP] In-place upgrade

2008-11-26 Thread Zdenek Kotala
Robert, big thanks for your review. I think #1 is still partially valid, because it contains general cleanups, but part of it is not necessary now. #2, #3 and #4 you can move to return with feedback section. Thanks Zdenek Robert Haas napsal(a): Zdenek - I am a bit murky on

Re: [HACKERS] Minor race-condition problem during database startup

2008-11-24 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Zdenek Kotala napsal(a): By my opinion there is not reason to retry kill call, because it fails only in situation if process does not exist or caller does not have permission to send a signal. If I think more about it, tha

Re: [HACKERS] Minor race-condition problem during database startup

2008-11-24 Thread Zdenek Kotala
Zdenek Kotala napsal(a): Tom Lane napsal(a): What seems to have happened is that the bgwriter didn't get as far as the first line of BackgroundWriterMain before the client backend tried to issue a checkpoint request. This is obviously a pretty minor issue, but it still seems worth fixin

Re: [HACKERS] Minor race-condition problem during database startup

2008-11-24 Thread Zdenek Kotala
Tom Lane napsal(a): What seems to have happened is that the bgwriter didn't get as far as the first line of BackgroundWriterMain before the client backend tried to issue a checkpoint request. This is obviously a pretty minor issue, but it still seems worth fixing. We could either try to make su

Re: [HACKERS] Cool hack with recursive queries

2008-11-21 Thread Zdenek Kotala
Alvaro Herrera napsal(a): David Fetter escribió: On Wed, Nov 19, 2008 at 10:21:06PM -, David Rowley wrote: Gregory Stark wrote: So based on Graeme Job's T-SQL hack over at thedailywtf.com I adapted the T-SQL code to Postgres and got this. Thought others might find it amusing. That's pret

Re: [HACKERS] pg_upgrade: How to deal with toast

2008-11-20 Thread Zdenek Kotala
Bruce Momjian napsal(a): Zdenek Kotala wrote: Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Heikki Linnakangas napsal(a): Perhaps we should just add the new attid attribute to the toast table, but mark it as nullable? Hmm, It seems to me as a good idea. No, it'

Re: [HACKERS] pg_upgrade: How to deal with toast

2008-11-20 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Heikki Linnakangas napsal(a): Perhaps we should just add the new attid attribute to the toast table, but mark it as nullable? Hmm, It seems to me as a good idea. No, it's a really horrid idea. Nullable attributes

Re: [HACKERS] pg_upgrade: How to deal with toast

2008-11-20 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): It the new attribute is added to the end, the old tuples will be compatible as is. If there's no null bitmap (or it's shorter than expectd), attributes missing from a tuple are implicitly NULL. That's how we support ALTER TABLE ADD COLUMN without rewriting all the

Re: [HACKERS] pg_upgrade: How to deal with toast

2008-11-20 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: The question is if we should do toast modification now to avoid potential future problems or if we will solve it when any on-disk format change requires it? Perhaps we should just add the new attid attribute to the toast table, but mark it

Re: [HACKERS] pg_upgrade: How to deal with toast

2008-11-19 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: 2) data type is unknown Unfortunately, in low function is no clue what data type is really stored in a chunks. Do we need to know? We haven't changed the on-disk format of any data types between 8.3 and 8.4, have we? Yeah, there

[HACKERS] pg_upgrade: How to deal with toast

2008-11-19 Thread Zdenek Kotala
We are now in discussion about toast table upgrading. I try to collect ideas and figure out how it should work and where are problems. Overview: - A few weeks ago we made a decision to use convert on read. We already made a decision how to solve problem with overflow data after conversi

[HACKERS] Re: toast by chunk-end (was Re: PG_PAGE_LAYOUT_VERSION 5 - time for change)

2008-11-18 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Just a very quick look on your patch. See my comments: ... 2) PG_PAGE_LAYOUT_VERSION should be bump The patch doesn't change the page layout AFAICS. It is good question what is and what is not page layout. I think that

Re: [HACKERS] Re: toast by chunk-end (was Re: PG_PAGE_LAYOUT_VERSION 5 - time for change)

2008-11-18 Thread Zdenek Kotala
Alvaro Herrera napsal(a): Zdenek Kotala wrote: If I'm thinking more, it is not probably CATALOG_VERSION_NO as well. Because toast table is created on demand. It is not in BKI. It's not catversion in the sense that there's no catalog change, but it certainly requires a catvers

[HACKERS] Re: toast by chunk-end (was Re: PG_PAGE_LAYOUT_VERSION 5 - time for change)

2008-11-18 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Just a very quick look on your patch. See my comments: ... 2) PG_PAGE_LAYOUT_VERSION should be bump The patch doesn't change the page layout AFAICS. It is good question what i

[HACKERS] Re: toast by chunk-end (was Re: PG_PAGE_LAYOUT_VERSION 5 - time for change)

2008-11-18 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Just a very quick look on your patch. See my comments: ... 2) PG_PAGE_LAYOUT_VERSION should be bump The patch doesn't change the page layout AFAICS. It is good question what i

[HACKERS] Re: toast by chunk-end (was Re: PG_PAGE_LAYOUT_VERSION 5 - time for change)

2008-11-18 Thread Zdenek Kotala
Alvaro Herrera napsal(a): Zdenek Kotala wrote: Alvaro Herrera napsal(a): Heikki Linnakangas wrote: Hmm, you're right. I think it can be made to work by storing the *end* offset of each chunk. To find the chunk containing offset X, search for the first chunk with end_offset > X.

Re: [HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-11-17 Thread Zdenek Kotala
Alvaro Herrera napsal(a): Heikki Linnakangas wrote: Hmm, you're right. I think it can be made to work by storing the *end* offset of each chunk. To find the chunk containing offset X, search for the first chunk with end_offset > X. FWIW I'm trying to do this. So far I've managed to make t

Re: [HACKERS] pg_upgrade project status

2008-11-11 Thread Zdenek Kotala
Bruce Momjian napsal(a): Zdenek Kotala wrote: Bruce Momjian napsal(a): Zdenek Kotala wrote: 1) Data does not fit on the new page. It will be solve by pre-upgrade check which reserve space on each page, before upgrade. Rather than specifying free space as an amount, I was thinking of

Re: [HACKERS] pg_upgrade project status

2008-11-11 Thread Zdenek Kotala
Bruce Momjian napsal(a): Zdenek Kotala wrote: In the last week community made decision about pg_upgrade project and its implementation. I would like to try summarize this conclusion and I add other topic which should be finished for 8.4. Convert on read has been selected as a good way

[HACKERS] pg_upgrade project status

2008-11-10 Thread Zdenek Kotala
In the last week community made decision about pg_upgrade project and its implementation. I would like to try summarize this conclusion and I add other topic which should be finished for 8.4. Convert on read has been selected as a good way, because it is not invasive and does not limit fresh d

Re: [HACKERS] WIP: Page space reservation (pgupgrade)

2008-11-10 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Attached patch allows to setup storage parameter for space reservation. What is the point of this? We don't need it for 8.3->8.4, we aren't going to back-patch such a thing into 8.2 or before (certainl

Re: [HACKERS] WIP: Page space reservation (pgupgrade)

2008-11-10 Thread Zdenek Kotala
Robert Haas napsal(a): Let's suppose, for example, that in 8.5 we decide to change some type that is presently 16 bits to 32 bits, or 8 bits to 16 bits, etc. This will make some tuples bigger, and potentially much bigger, but since it presumably won't be a commonly used data-type, most tuples w

Re: [HACKERS] [WIP] In-place upgrade

2008-11-10 Thread Zdenek Kotala
Decibel! napsal(a): Unless I'm mistaken, there are only two cases we care about for additional space: per-page and per-tuple. Yes. And maybe special space indexes could be extended, but it is covered in per-page setting. Those requirements could also vary for different types of pg_class o

Re: [HACKERS] WIP: Page space reservation (pgupgrade)

2008-11-09 Thread Zdenek Kotala
Jonah H. Harris napsal(a): On Sun, Nov 9, 2008 at 7:55 PM, Decibel! <[EMAIL PROTECTED]> wrote: On Nov 8, 2008, at 8:35 PM, Jonah H. Harris wrote: That's my question. Why is this needed at all? I suspect this is to deal with needing to reserve space in a cluster that you're planning on upgradi

[HACKERS] WIP: Page space reservation (pgupgrade)

2008-11-08 Thread Zdenek Kotala
criteria. Is it correct or I'm wrong? Thanks for your comments. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql diff -Nrc pgsql_spacereserve.9e46c188067f/src/backend/access/common/relopti

Re: [HACKERS] [WIP] In-place upgrade

2008-11-07 Thread Zdenek Kotala
ump and import which lost all extended information. Thanks Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [WIP] In-place upgrade

2008-11-07 Thread Zdenek Kotala
be sure if it is possible. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [WIP] In-place upgrade

2008-11-07 Thread Zdenek Kotala
de. I prefer to have this information in pg_class. It is accessible by SQL commands. pg_class should also contains information about last checked page to prevent repeatable check on very large tables. The tool could run concurrently with normal activity, so you could just let it run

Re: [HACKERS] [WIP] In-place upgrade

2008-11-06 Thread Zdenek Kotala
ntioned tuple and page conversion is basic and same for all upgrade method and it should be done first. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] [WIP] In-place upgrade

2008-11-05 Thread Zdenek Kotala
Martijn van Oosterhout napsal(a): On Wed, Nov 05, 2008 at 03:04:42PM +0100, Zdenek Kotala wrote: Greg Stark napsal(a): It is exceptional case between V3 and V4 and only on heap, because you save in varlena. But between V4 and V5 we will lost another 4 bytes in a page header -> page hea

Re: [HACKERS] [WIP] In-place upgrade

2008-11-05 Thread Zdenek Kotala
Greg Stark napsal(a): I don't think this really qualifies as "in place upgrade" since it would mean creating a whole second copy of all your data. And it's only online got read-only queries too. I think we need a way to upgrade the pages in place and deal with any overflow data as exceptional

Re: [HACKERS] [WIP] In-place upgrade

2008-11-05 Thread Zdenek Kotala
Tom Lane napsal(a): I concur that I don't want to see this patch adding more than the absolute unavoidable minimum of overhead for data that meets the "current" layout definition. I'm disturbed by the proposal to stick overhead into tuple header access, for example. OK. I agree that it is ov

Re: [HACKERS] [WIP] In-place upgrade

2008-11-05 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: We've talked about this many times before, so I'm sure you know what my opinion is. Let me phrase it one more time: 1. You *will* need a function to convert a page from old format to new format. We do want to get rid of the

Re: [HACKERS] [PATCH] Extending pg_class info + more flexible TOAST chunk size

2008-11-04 Thread Zdenek Kotala
, or is this 8.5 material at this point? Yes, I plan to do it. 8.5 is too late for this change. Thanks Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] [PATCH] PageGetTempPage cleanup

2008-11-04 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: I attach patch which cleans up code around PageGetTempPage. These changes were discussed here: http://archives.postgresql.org/pgsql-hackers/2008-08/msg00102.php Applied with a minor change: instead of inventing

Re: [HACKERS] [WIP] In-place upgrade

2008-11-04 Thread Zdenek Kotala
me way with V4 without any overhead (they will be small overhead with allocating and free heaptupledata in some places - mostly vacuum). Only multi version access will be driven on page basis. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http

Re: [HACKERS] [WIP] In-place upgrade

2008-11-04 Thread Zdenek Kotala
corner case it could requires converts all index when you request one record. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] [WIP] In-place upgrade

2008-11-03 Thread Zdenek Kotala
mments. Try snapshot link. I hope that it will work. Zdenek PS: I'm sorry about response time, but I'm on training this week. -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] How to parse Datum

2008-11-03 Thread Zdenek Kotala
, BUPT www.bupt.edu.cn -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-11-02 Thread Zdenek Kotala
re complexity now for converting chunk from old to the new version. But it add a benefit - for example vacuum can remove data from dropped columns and so on. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pg

[HACKERS] PostgreSQL in-place upgrade Git repository is ready to play

2008-10-31 Thread Zdenek Kotala
need also pg_upgrade.sh http://src.opensolaris.org/source/xref/sfw/usr/src/cmd/postgres/postgresql-upgrade/ I will soon put pg_upgrade.sh into git as well. Any feedback is welcome thanks Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http

Re: [HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-10-31 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: 1) HeapTupleHeader modification typedef struct HeapTupleFields { TransactionId t_xmin; /* inserting xact ID */ TransactionId t_xmax; /* deleting or locking xact ID */

Re: [HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-10-31 Thread Zdenek Kotala
Gregory Stark napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: 3) TOAST modification a) TOAST table per attribute b) replace chunk id with offset+variable chunk size c) add column identification into first chunk Thats all. I think infomask/infomask2 shuffle flag should b

[HACKERS] PG_PAGE_LAYOUT_VERSION 5 - time for change

2008-10-30 Thread Zdenek Kotala
Thats all. I think infomask/infomask2 shuffle flag should be done. TOAST modification complicates in-place upgrade. Comments other ideas? Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via

[HACKERS] [PATCH] HeapTuple version extension + code cleanup

2008-10-30 Thread Zdenek Kotala
This patch requires patch: http://archives.postgresql.org/message-id/[EMAIL PROTECTED] It add t_ver information into HeapTuple structure and add DatumGetHeapTuple function which allows to reduce code on many places. Zdenek -- Zdenek Kotala Sun Microsystems

Re: [HACKERS] Block-level CRC checks

2008-10-30 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: By the way, do you need CRC as a first page member? Is it for future development like CLOG integration into buffers? Why not put it on the end as and mark it as a special? It will reduce space requirement when CRC is not e

Re: [HACKERS] Block-level CRC checks

2008-10-30 Thread Zdenek Kotala
Alvaro Herrera napsal(a): Zdenek Kotala wrote: Alvaro Herrera napsal(a): Simon Riggs wrote: But perhaps writing a single WAL record if you scan whole page and set all bits at once. Then it makes sense in some cases. So this is what I ended up doing; attached. Please, DO NOT MOVE position

Re: [HACKERS] Block-level CRC checks

2008-10-30 Thread Zdenek Kotala
Jonah H. Harris napsal(a): On Thu, Oct 30, 2008 at 10:33 AM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: Please, DO NOT MOVE position of page version in PageHeader structure! And PG_PAGE_LAYOUT_VERSION should be bump to 5. Umm, any in-place upgrade should be capable of handling changes

Re: [HACKERS] Block-level CRC checks

2008-10-30 Thread Zdenek Kotala
version in PageHeader structure! And PG_PAGE_LAYOUT_VERSION should be bump to 5. Thanks Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Question about GetAttributeByNum(Name) ExecQual.c

2008-10-30 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: GetAttributeByNum has first parameter as HeapTupleHeader, but most functions use Datum and after that they call DatumGetHeapTupleHeader. The difference is that DatumGetHeapTupleHeader performs detoast on tuple(row type).

[HACKERS] Question about GetAttributeByNum(Name) ExecQual.c

2008-10-30 Thread Zdenek Kotala
? thanks Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Any reason to have heap_(de)formtuple?

2008-10-28 Thread Zdenek Kotala
Kris Jurka napsal(a): On Tue, 28 Oct 2008, Zdenek Kotala wrote: Kris Jurka napsal(a): Here's a patch that changes everything over to the the new API and implements the old API by calling the new API. It seems to me OK. I have only one comment. I prefer to pfree allocated memor

Re: [HACKERS] Any reason to have heap_(de)formtuple?

2008-10-28 Thread Zdenek Kotala
e the new API and implements the old API by calling the new API. It seems to me OK. I have only one comment. I prefer to pfree allocated memory for temporary nulls array. I think that caller could call old API many times without memory context cleanup. Zdenek -- Zdenek K

Re: [HACKERS] Any reason to have heap_(de)formtuple?

2008-10-24 Thread Zdenek Kotala
ago, got bored, and forgot about it. If people do want this conversion done while keeping the old interface around, I can track down that patch, update it and finish it up for the next CommitFest. Yes, Please. thanks -- Zdenek Kotala Sun Microsystems Prague

Re: [HACKERS] Any reason to have heap_(de)formtuple?

2008-10-23 Thread Zdenek Kotala
Alvaro Herrera napsal(a): Zdenek Kotala wrote: Tom Lane napsal(a): Well, aside from the gruntwork needed to convert all the core code that still uses the old APIs, there's the prospect of breaking extension modules that still use the old APIs. It's kind of annoying to have two

Re: [HACKERS] Any reason to have heap_(de)formtuple?

2008-10-23 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Currently in heaptuple.c we have duplicated code. heap_deformtuple and heap_formtuple are mark as a obsolete interface. Is any reason to have still them? I know that they are still used on many places, but is there any stopp

[HACKERS] Any reason to have heap_(de)formtuple?

2008-10-23 Thread Zdenek Kotala
Currently in heaptuple.c we have duplicated code. heap_deformtuple and heap_formtuple are mark as a obsolete interface. Is any reason to have still them? I know that they are still used on many places, but is there any stopper to keep these function alive? Zdenek -- Zdenek

[HACKERS] HEAD is broken - transam.c

2008-10-20 Thread Zdenek Kotala
dt=2008-10-20%2020:06:01 Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Problem to get the tuple of a table

2008-10-15 Thread Zdenek Kotala
Zhe He napsal(a): I'm currently implement a new operation in Postgres. If I want to get all tuples of a single attribute table to create an array defined by myself, how can I do that? Look into heapam.c and pgstattuple contrib module is good example. Zdenek -- Zdenek K

Re: [HACKERS] There's some sort of race condition with the new FSM stuff

2008-10-14 Thread Zdenek Kotala
Tom Lane napsal(a): Heikki Linnakangas <[EMAIL PROTECTED]> writes: I still wonder, though, why we're seeing the error consistently on kudu, and not on any other animal. Perhaps the forknum field that's left uninitialized gets a different value there than on other platforms. Hmm ... AFAICS thi

Re: [HACKERS] Year 2038 Bug?

2008-10-13 Thread Zdenek Kotala
(But we do have that regression test check in there to make sure.) Unfortunately, you are not correct here :( see: http://bugs.opensolaris.org/view_bug.do?bug_id=4246033 Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postg

Re: [HACKERS] Year 2038 Bug?

2008-10-13 Thread Zdenek Kotala
David E. Wheeler napsal(a): On Oct 13, 2008, at 11:22, Zdenek Kotala wrote: PostgreSQL 8.4 uses 64bit data type for time. But if you use system timezone then you can get in trouble if system does not support 64bit zic files. I've never noticed a problem with the TZinfo database that

Re: [HACKERS] Year 2038 Bug?

2008-10-13 Thread Zdenek Kotala
type for time. But if you use system timezone then you can get in trouble if system does not support 64bit zic files. Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-h

Re: [HACKERS] pg_upgrade: convert on read is dead end

2008-10-13 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Heikki Linnakangas napsal(a): Besides, the "read all formats" approach wouldn't really avoid it either. If you passed a toasted datum to a function, when the function needs to detoast it, detoast_datum still wouldn't kno

Re: [HACKERS] [PATCH] Extending pg_class info + more flexible TOAST chunk size

2008-10-13 Thread Zdenek Kotala
hough :-). Yes, no change please :-) Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mai

Re: [HACKERS] [PATCH] Extending pg_class info + more flexible TOAST chunk size

2008-10-13 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: The problem what I need to solve is how to handle different TOAST chunk size which becomes with upgrade. if you insert new record into TOAST table it will be created on the new page which has different max chunk size, because it depends on

Re: [HACKERS] pg_upgrade: convert on read is dead end

2008-10-13 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: PostgreSQL stores all table's external data in one TOAST table and stored data does not contains any clue about datatype. When postgreSQL reads TOSTEed value then there is not possible detect what type it is and perform varlena conversi

Re: [HACKERS] [PATCH] Extending pg_class info + more flexible TOAST chunk size

2008-10-13 Thread Zdenek Kotala
ITAGAKI Takahiro napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> wrote: Bacause MAX_TOAST_CHUNK_SIZE is related on page layout version I need have toast chunk size more flexible. I agree that flexible toast threshold is useful, but I have several questions in your implementations. relblo

Re: [HACKERS] [PATCH] Extending pg_class info + more flexible TOAST chunk size

2008-10-13 Thread Zdenek Kotala
Tom Lane napsal(a): Zdenek Kotala <[EMAIL PROTECTED]> writes: Bacause MAX_TOAST_CHUNK_SIZE is related on page layout version I need have toast chunk size more flexible. Attached patch add three new columns into pg_class relblocksize - which is always BLCKSZ. I put it there for fu

[HACKERS] [PATCH] Extending pg_class info + more flexible TOAST chunk size

2008-10-12 Thread Zdenek Kotala
to track information for each page version). Additional work could be add CRATE TABLE ... WITH chunksize= Patch requires increase control file version and catalog version. thanks for your comments Zdenek -- Zdenek Kotala Sun Microsystems Pragu

[HACKERS] pg_upgrade: convert on read is dead end

2008-10-11 Thread Zdenek Kotala
ution and only "read all formats..." solution is right one. Comments, ideas? Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] [WIP] Reduce alignment requirements on 64-bit systems.

2008-10-09 Thread Zdenek Kotala
Ryan Bradetich napsal(a): Hello Zdenek, Hello Ryan, On Wed, Oct 8, 2008 at 10:59 PM, Zdenek Kotala <[EMAIL PROTECTED]> wrote: Just a quick look. At first point. Your change introduces new page layout version. Which is not acceptable from my point of view for 8.4 (it add I would l

Re: [HACKERS] [WIP] Reduce alignment requirements on 64-bit systems.

2008-10-08 Thread Zdenek Kotala
eliminate the impact for this patch on 32-bit systems, which would not gain any benefit from this patch. Feedback welcome! Thanks, - Ryan -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http

[HACKERS] doubts about toast_flatten_tuple_attribute/heap_form_tuple

2008-10-07 Thread Zdenek Kotala
I'm trying write tuple conversion function and now I'm hit problem with composite data types. Composite data types can contain inner composite data types. Is there any limit? And I found that heap_form_tuple calls toast_flatten_tuple_attribute only one level composite type. Is it bug?

Re: [HACKERS] FSM rewrite committed, loose ends

2008-10-02 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: Heikki Linnakangas napsal(a): The FSM is not updated during WAL replay. That means that after crash recovery, the FSM won't be completely up-to-date, but at roughly the state it was at last checkpoint. In a warm stand-by, the FSM

Re: [HACKERS] Block-level CRC checks

2008-10-01 Thread Zdenek Kotala
Alvaro Herrera napsal(a): This code would be run-time or compile-time configurable. I'm not absolutely sure which yet; the problem with run-time is what to do if the user restarts the server with the setting flipped. It would have almost no impact on users who don't enable it. I prefer runti

[HACKERS] [PATCH] PageGetTempPage cleanup

2008-09-30 Thread Zdenek Kotala
I attach patch which cleans up code around PageGetTempPage. These changes were discussed here: http://archives.postgresql.org/pgsql-hackers/2008-08/msg00102.php Zdenek -- Zdenek Kotala Sun Microsystems Prague, Czech Republic http://sun.com/postgresql diff

Re: [HACKERS] FSM rewrite committed, loose ends

2008-09-30 Thread Zdenek Kotala
ed to think when the FSM should be updated during WAL replay. Probably not after every record, because of the overhead, but certainly more often than never. What's about after a page write during a WAL replay? Zdenek -- Zdenek Kotala Sun Microsystems Pra

Re: [HACKERS] [REVIEW] Prototype: In-place upgrade v02

2008-09-30 Thread Zdenek Kotala
Abbas napsal(a): On Mon, 2008-09-29 at 14:42 +0200, Zdenek Kotala wrote: Do I have to perform performance tests too? Yes, please. My colleague tested it and got 5% performance drop, but it was not complete version and I tested full patch on Friday and It was surprise for me ... I got

[HACKERS] pg_upgrade performance test

2008-09-29 Thread Zdenek Kotala
. And good to mention that it is not optimized version. It would be good if somebody will run different performance test on it and verify my results. I used iGen OLTP test with 60 concurrent users and run it for 30minutes. Zdenek -- Zdenek Kotala Sun Microsystems

Re: [HACKERS] [REVIEW] Prototype: In-place upgrade v02

2008-09-29 Thread Zdenek Kotala
esult also from others. thanks Zdenek Regards Abbas On Fri, 2008-09-19 at 14:28 +0200, Zdenek Kotala wrote: thanks Abbas napsal(a): Even with that a hunk failed for bufpage.c, but I applied that part manually to move on. Regards Abbas On Thu, 2008-09-18 at 12:17 +0200, Zde

Re: [HACKERS] FSM, now without WAL-logging

2008-09-26 Thread Zdenek Kotala
Heikki Linnakangas napsal(a): Zdenek Kotala wrote: I've also tried various pgbench tests, on a RAM disk and otherwise, as well as the "table population" test I ran earlier, and don't see any difference in performance. I think performance is OK Zdenek

<    1   2   3   4   5   6   7   8   >