Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-26 Thread Zdenek . Kotala
Andrew Dunstan wrote: Zdenek Kotala wrote: Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-26 Thread Merlin Moncure
On 9/24/07, Gregory Stark [EMAIL PROTECTED] wrote: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse the weight and

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-26 Thread Tom Lane
Merlin Moncure [EMAIL PROTECTED] writes: would this break any application pulling a numeric field as binary over the protocol? No. regards, tom lane ---(end of broadcast)--- TIP 7: You can help support the PostgreSQL

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-25 Thread Zdenek Kotala
Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse the weight and

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-25 Thread Andrew Dunstan
Zdenek Kotala wrote: Tom Lane wrote: Gregory Stark [EMAIL PROTECTED] writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-24 Thread Gregory Stark
We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse the weight and sign/dscale so we could do this sometime without

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-24 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: We previously discussed compressing the numeric data type for small values: http://archives.postgresql.org/pgsql-hackers/2007-06/msg00715.php We didn't do this for 8.3 but in any case Tom did suggest we ought to reverse the weight and sign/dscale so we

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-09-24 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: Tom Lane [EMAIL PROTECTED] writes: Gregory Stark [EMAIL PROTECTED] writes: I think we also should move the NumericData and declaration to numeric.c and make the Numeric type an opaque pointer for the rest of the source tree. I don't agree with that; we

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-07-17 Thread Bruce Momjian
This has been saved for the 8.4 release: http://momjian.postgresql.org/cgi-bin/pgpatches_hold --- Simon Riggs wrote: We've changed the on-disk database format in 8.3, so we have an opportunity to change other

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-27 Thread Josh Berkus
Tom, That sounds good, but there are corner cases where it wouldn't work --- consider a page containing a single maximum-length tuple. Certainly any mature upgrade-in-place tool will require a checker which you run first which determines if you have a prohibitive corner case. Besides, I

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-22 Thread Zdenek Kotala
Heikki Linnakangas wrote: Since we're discussing upgrades, let me summarize the discussions we had over dinner in Ottawa for the benefit of all: Thanks for summary. As before, someone just needs to step up and do it. I'm now working on proposal. I hope that it will ready soon.

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-21 Thread Zdenek Kotala
Andrew Sullivan wrote: On Wed, Jun 20, 2007 at 12:34:21PM -0400, Robert Treat wrote: FWIW pg_migrator is a pretty good swing at an in-place upgrade tool for 8.1-8.2. Unfortunately until the PGDG decides that in-place upgrade is a constraint their willing to place on development, I see them a

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-21 Thread Heikki Linnakangas
Since we're discussing upgrades, let me summarize the discussions we had over dinner in Ottawa for the benefit of all: * pg_migrator is a sound approach to handling catalog changes. * Handling any page format change that doesn't grow the space taken by data is straightforward. Note that all

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-21 Thread Tom Lane
Heikki Linnakangas [EMAIL PROTECTED] writes: * Page format changes that grow data size are problematic, because there can be pages that can't be expanded to the new format because there's not enough space. However, it would be possible to write a pre-upgrade program to look for the

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-20 Thread Robert Treat
On Tuesday 19 June 2007 10:15, Tom Lane wrote: Zdenek Kotala [EMAIL PROTECTED] writes: I'm little bit confused when we introduce new page layout version? I expect that new version become with changes with pageheader, tuple header or data encoding (varlen/TOAST ...). But in case when there

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-20 Thread Andrew Sullivan
On Wed, Jun 20, 2007 at 12:34:21PM -0400, Robert Treat wrote: FWIW pg_migrator is a pretty good swing at an in-place upgrade tool for 8.1-8.2. Unfortunately until the PGDG decides that in-place upgrade is a constraint their willing to place on development, I see them a good chicken/egg

What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-19 Thread Zdenek Kotala
Simon Riggs wrote: The objections to applying this patch originally were: 1. it changes on-disk format (we've done this, so argument is void) I'm little bit confused when we introduce new page layout version? I expect that new version become with changes with pageheader, tuple header or

Re: What does Page Layout version mean? (Was: Re: [HACKERS] Reducing NUMERIC size for 8.3)

2007-06-19 Thread Tom Lane
Zdenek Kotala [EMAIL PROTECTED] writes: I'm little bit confused when we introduce new page layout version? I expect that new version become with changes with pageheader, tuple header or data encoding (varlen/TOAST ...). But in case when there is new data type internal implementation, there

[HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
We've changed the on-disk database format in 8.3, so we have an opportunity to change other things also. There is a patch thats been on the patch queue for some time called numeric508, submitted Dec 2005; I've updated this patch now for 8.3 to remove bit rot (an hour's work). This is posted to

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: We've changed the on-disk database format in 8.3, so we have an opportunity to change other things also. There is a patch thats been on the patch queue for some time called numeric508, submitted Dec 2005; I thought that idea had been rejected long since as

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Andreas Pflug
Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store 508 digits. If 508 digits are not enough, are1000 digits be sufficient? Both limits appear quite arbitrary to me.

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 16:56 +0200, Andreas Pflug wrote: Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store 508 digits. If 508 digits are not enough, are1000

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 10:54 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: We've changed the on-disk database format in 8.3, so we have an opportunity to change other things also. There is a patch thats been on the patch queue for some time called numeric508, submitted Dec

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Michael Paesold
Andreas Pflug wrote: Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store 508 digits. If 508 digits are not enough, are1000 digits be sufficient? Both limits appear

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Andreas Pflug [EMAIL PROTECTED] writes: Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store 508 digits. If 508 digits are not enough, are1000 digits be sufficient? Both

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Andreas Pflug
Tom Lane wrote: Andreas Pflug [EMAIL PROTECTED] writes: Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store 508 digits. If 508 digits are not enough,

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 11:32 -0400, Tom Lane wrote: Andreas Pflug [EMAIL PROTECTED] writes: Simon Riggs wrote: The objections to applying this patch originally were: 2. it would restrict number of digits to 508 and there are allegedly some people that want to store 508 digits. If 508

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Andreas Pflug [EMAIL PROTECTED] writes: Tom Lane wrote: In any case, no capability is lost, unlike the original proposal; and this would be much less invasive than the original patch since there's no need to play tricks with the content of the digit array. I wonder if the currently waiting

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 17:49 +0200, Andreas Pflug wrote: I wonder if the currently waiting patch isn't Good Enough for 999. % of use cases, and all others can use numeric instead of numeric(1000,800) or so. Especially since there are many patches waiting that do need further

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: Since this is your idea, would you like to do this, or should I? Go for it. I'm not actually convinced this is worth spending time on, as Greg Stark's 1-byte-varlena patch already saved more for typical numerics than this will.

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 11:55 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Since this is your idea, would you like to do this, or should I? Go for it. OK I'm not actually convinced this is worth spending time on, as Greg Stark's 1-byte-varlena patch already saved more for

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Gregory Stark
Tom Lane [EMAIL PROTECTED] writes: I had a thought though: it's possible to reduce the header overhead for typical-size numbers without giving up the ability to store large ones. This is because the POS/NEG/NAN sign possibilities leave one unused bit pattern. Hence: I had a whack and doing

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Gregory Stark [EMAIL PROTECTED] writes: Anything shorter than the shortest possible numeric representation can implicitly be interpreted as some alternate compact representation. I already had a patch that stored small integers in a single NumericDigit without any numeric header at all.

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
I wrote: Gregory Stark [EMAIL PROTECTED] writes: Anything shorter than the shortest possible numeric representation can implicitly be interpreted as some alternate compact representation. I already had a patch that stored small integers in a single NumericDigit without any numeric header at

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Simon Riggs
On Mon, 2007-06-18 at 12:44 -0400, Tom Lane wrote: I wrote: Gregory Stark [EMAIL PROTECTED] writes: Anything shorter than the shortest possible numeric representation can implicitly be interpreted as some alternate compact representation. I already had a patch that stored small integers

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes: I think representing zero as compactly as possible is worth the trouble, Either of these proposals can do that in 2 bytes. regards, tom lane ---(end of broadcast)--- TIP 4: Have you

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Bruce Momjian
Simon Riggs wrote: On Mon, 2007-06-18 at 11:55 -0400, Tom Lane wrote: Simon Riggs [EMAIL PROTECTED] writes: Since this is your idea, would you like to do this, or should I? Go for it. OK I'm not actually convinced this is worth spending time on, as Greg Stark's 1-byte-varlena

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes: Would someone please explain why we are considering this so far past features freeze, and who suggtested that the 8.3-8.4 upgrade being a binary upgrade was anything more than a pipe dream? Well, Greg had left further squeezing of numerics as an open

Re: [HACKERS] Reducing NUMERIC size for 8.3

2007-06-18 Thread Gregory Stark
Bruce Momjian [EMAIL PROTECTED] writes: Would someone please explain why we are considering this so far past features freeze, and who suggtested that the 8.3-8.4 upgrade being a binary upgrade was anything more than a pipe dream? Simon just updated a patch he had originally submitted over a