Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas robertmh...@gmail.com wrote: The smallest value for precision which requires 2 numeric_digits is always 2; and the required number of numeric_digits increases by 1 each time the number of base-10 digits

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Robert Haas
On Wed, Aug 4, 2010 at 11:05 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas robertmh...@gmail.com wrote: The smallest value for precision which requires 2 numeric_digits is always 2; and the required number of

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: *scratches head* One of those tests uses and the other uses = Which one is wrong? Well, neither, since NUMERIC(0,0) is disallowed. However, it's probably not the place of these functions to assume that, so I'd suggest treating equality as valid.

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-04 Thread Robert Haas
On Wed, Aug 4, 2010 at 12:55 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: *scratches head* One of those tests uses and the other uses = Which one is wrong? Well, neither, since NUMERIC(0,0) is disallowed.  However, it's probably not the place of these

Re: [HACKERS] reducing NUMERIC size for 9.1, take two

2010-08-03 Thread Brendan Jurd
On 31 July 2010 07:58, Robert Haas robertmh...@gmail.com wrote: Here's a second version of the main patch, in which I have attempted to respond to Tom's concerns/suggestions. (There is still a small, side issue with numeric_maximum_size() which I plan to fix, but this patch is the good

Re: [HACKERS] reducing NUMERIC size for 9.1, take two

2010-08-03 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Here's a second version of the main patch, in which I have attempted to respond to Tom's concerns/suggestions. This version looks fine to me. regards, tom lane -- Sent via pgsql-hackers mailing list

Re: [HACKERS] reducing NUMERIC size for 9.1, take two

2010-08-03 Thread Robert Haas
On Tue, Aug 3, 2010 at 6:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Here's a second version of the main patch, in which I have attempted to respond to Tom's concerns/suggestions. This version looks fine to me. Excellent. Committed. -- Robert Haas

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-08-03 Thread Robert Haas
On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, Jul 30, 2010 at 2:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes:  Maybe something like this, obviously with a suitable comment which I haven't written yet:    

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Robert Haas
On Fri, Jul 30, 2010 at 1:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: But, looking at it a bit more carefully, isn't the maximum-size logic for numeric rather bogus? Perhaps, but I think you're confused on at least one point. numeric(2,1) has to be

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Fri, Jul 30, 2010 at 1:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Perhaps, but I think you're confused on at least one point. numeric(2,1) has to be able to hold 2 decimal digits, not 2 NumericDigits (which'd actually be 8 decimal digits given the

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Maybe something like this, obviously with a suitable comment which I haven't written yet: numeric_digits = (precision + 6) / 4; return (numeric_digits * sizeof(int16)) + NUMERIC_HDRSZ; This is OK for the base-10K case, but there's still

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Tom Lane
[ forgot to answer this part ] Robert Haas robertmh...@gmail.com writes: Another question here is whether we should just fix this in CVS HEAD, or whether there's any reason to back-patch it. Agreed, fixing it in HEAD seems sufficient. regards, tom lane -- Sent via

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Robert Haas
On Fri, Jul 30, 2010 at 9:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jul 30, 2010 at 1:13 AM, Tom Lane t...@sss.pgh.pa.us wrote: Perhaps, but I think you're confused on at least one point. numeric(2,1) has to be able to hold 2 decimal digits,

[HACKERS] reducing NUMERIC size for 9.1, take two

2010-07-30 Thread Robert Haas
Here's a second version of the main patch, in which I have attempted to respond to Tom's concerns/suggestions. (There is still a small, side issue with numeric_maximum_size() which I plan to fix, but this patch is the good stuff.) -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-30 Thread Robert Haas
On Fri, Jul 30, 2010 at 2:08 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes:  Maybe something like this, obviously with a suitable comment which I haven't written yet:     numeric_digits = (precision + 6) / 4;     return (numeric_digits * sizeof(int16))

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Wed, Jul 28, 2010 at 3:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't like the way you did that either (specifically, not the kluge in NUMERIC_DIGITS()).  It would probably work

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Wed, Jul 28, 2010 at 3:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: No, you can do something like this: typedef union numeric { uint16 word1; numeric_short short; numeric_longlong; } numeric; That doesn't quite work

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 1:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: Yeah, you would need an additional layer of struct to represent the numeric with a length word in front of it.  I think this is not necessarily bad because it would perhaps open the door to working directly with

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Thu, Jul 29, 2010 at 1:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: On-disk is what I'm thinking about.  Right now, a NaN's first word is all dscale except the sign bits.  You're proposing to change that but I think it's unnecessary to do so. *Where*

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 4:37 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: On Thu, Jul 29, 2010 at 1:20 PM, Tom Lane t...@sss.pgh.pa.us wrote: On-disk is what I'm thinking about.  Right now, a NaN's first word is all dscale except the sign bits.  You're

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: OK. I think you're misinterpreting the point of that comment, which may mean that it needs some clarification. By the two byte header format is also used, I think I really meant the header (and in fact the entire value) is just 2 bytes. Really, the

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 5:03 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: OK.  I think you're misinterpreting the point of that comment, which may mean that it needs some clarification.  By the two byte header format is also used, I think I really meant the

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: Did you look at the patch to move the numeric stuff out of the .h file that I attached a few emails back? If that looks OK, I can commit it and then redo the rest of this along the lines we've discussed. A couple of thoughts: * It'd be good to get

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Robert Haas
On Thu, Jul 29, 2010 at 5:35 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: Did you look at the patch to move the numeric stuff out of the .h file that I attached a few emails back?  If that looks OK, I can commit it and then redo the rest of this along the

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-29 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: But, looking at it a bit more carefully, isn't the maximum-size logic for numeric rather bogus? Perhaps, but I think you're confused on at least one point. numeric(2,1) has to be able to hold 2 decimal digits, not 2 NumericDigits (which'd actually be 8

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-28 Thread Tom Lane
[ gradually catching up on email ] Robert Haas robertmh...@gmail.com writes: On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: I don't like the way you did that either (specifically, not the kluge in NUMERIC_DIGITS()).  It would probably work better if you declared two

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-20 Thread Robert Haas
On Fri, Jul 16, 2010 at 2:39 PM, Tom Lane t...@sss.pgh.pa.us wrote: Robert Haas robertmh...@gmail.com writes: I'm not entirely happy with the way I handled the variable-length struct, although I don't think it's horrible, either. I'm willing to rework it if someone has a better idea. I don't

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Brendan Jurd
On 16 July 2010 03:47, Robert Haas robertmh...@gmail.com wrote: On Jul 15, 2010, at 11:58 AM, Brendan Jurd dire...@gmail.com wrote: I dropped one thousand numerics with value zero into a table and checked the on-disk size of the relation with your patch and on a stock 8.4 instance.  In both

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Richard Huxton
On 16/07/10 13:44, Brendan Jurd wrote: pg_column_size() did return the results I was expecting. pg_column_size(0::numeric) is 8 bytes on 8.4 and it's 6 bytes on HEAD with your patch. At this scale we should be seeing around 2 million bytes saved, but instead the tables are identical. Is

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Brendan Jurd
On 16 July 2010 22:51, Richard Huxton d...@archonet.com wrote: On 16/07/10 13:44, Brendan Jurd wrote: At this scale we should be seeing around 2 million bytes saved, but instead the tables are identical.  Is there some kind of disconnect in how the new short numeric is making it to the disk,

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Thom Brown
On 16 July 2010 14:14, Brendan Jurd dire...@gmail.com wrote: On 16 July 2010 22:51, Richard Huxton d...@archonet.com wrote: On 16/07/10 13:44, Brendan Jurd wrote: At this scale we should be seeing around 2 million bytes saved, but instead the tables are identical.  Is there some kind of

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread David E. Wheeler
On Jul 16, 2010, at 6:17 AM, Thom Brown wrote: Joy! :) Nice patch Robert. Indeed. What are the implications for pg_upgrade? Will a database with values created before the patch continue to work after the patch has been applied (as happened with the new hstore in 9.0), or will pg_upgrade

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Bruce Momjian
David E. Wheeler wrote: On Jul 16, 2010, at 6:17 AM, Thom Brown wrote: Joy! :) Nice patch Robert. Indeed. What are the implications for pg_upgrade? Will a database with values created before the patch continue to work after the patch has been applied (as happened with the new hstore

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread David E. Wheeler
On Jul 16, 2010, at 9:04 AM, Bruce Momjian wrote: What are the implications for pg_upgrade? Will a database with values created before the patch continue to work after the patch has been applied (as happened with the new hstore in 9.0), or will pg_upgrade need to be taught how to upgrade the

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Hitoshi Harada
2010/7/16 Brendan Jurd dire...@gmail.com: On 16 July 2010 03:47, Robert Haas robertmh...@gmail.com wrote: You might also look at testing with pg_column_size(). pg_column_size() did return the results I was expecting. pg_column_size(0::numeric) is 8 bytes on 8.4 and it's 6 bytes on HEAD with

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-16 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: I'm not entirely happy with the way I handled the variable-length struct, although I don't think it's horrible, either. I'm willing to rework it if someone has a better idea. I don't like the way you did that either (specifically, not the kluge in

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-15 Thread Brendan Jurd
On 10 July 2010 00:58, Robert Haas robertmh...@gmail.com wrote: EnterpriseDB asked me to develop the attached patch to reduce the on-disk size of numeric and to submit it for inclusion in PG 9.1. After searching the archives, I found a possible design for this by Tom Lane based on an earlier

Re: [HACKERS] reducing NUMERIC size for 9.1

2010-07-15 Thread Robert Haas
On Jul 15, 2010, at 11:58 AM, Brendan Jurd dire...@gmail.com wrote: On 10 July 2010 00:58, Robert Haas robertmh...@gmail.com wrote: EnterpriseDB asked me to develop the attached patch to reduce the on-disk size of numeric and to submit it for inclusion in PG 9.1. After searching the archives,

[HACKERS] reducing NUMERIC size for 9.1

2010-07-09 Thread Robert Haas
EnterpriseDB asked me to develop the attached patch to reduce the on-disk size of numeric and to submit it for inclusion in PG 9.1. After searching the archives, I found a possible design for this by Tom Lane based on an earlier proposal by Simon Riggs.