port/snprintf.c (was Re: [PATCHES] Numeric 508 datatype)

2005-12-04 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 OK, snprintf.c fixed.  I added a 'stream' and outlen parameter to all
 the calls, and cleaned up the switch() statement that was outputing
 twice.  When we were outputing just to a string, it didn't matter, but
 now that we are also outputting to a stream, it does.

I found a whole bunch more problems than this :-(.  I've committed a
cleaned-up version that seems to work correctly in a simple standalone
testbed, but it'd be a good idea to exercise it inside PG as well.
Can you try regression tests and the factorial() problem on CVS tip?

The problems are sufficiently bad that it might be a good idea to
backport the fixes into 8.0 and before as well --- but I note that
the ABI is different (pg_snprintf vs snprintf, etc) so this requires
a bit of investigation rather than just committing the file as-is.

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: port/snprintf.c (was Re: [PATCHES] Numeric 508 datatype)

2005-12-04 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 The problems are sufficiently bad that it might be a good idea to
 backport the fixes into 8.0 and before as well --- but I note that
 the ABI is different (pg_snprintf vs snprintf, etc) so this requires
 a bit of investigation rather than just committing the file as-is.

 Not as many 8.0.X platforms used *printf because we didn't test %$ for
 its use on that release, so my bet is that very few platforms would be
 using it.

Hm.  One of the main problems I found was incorrect results for
LONGLONG_MIN (-2^63).  I'm rather tempted to add a test case for
that to the int8 regression test and see if any platforms fail ;-)

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: port/snprintf.c (was Re: [PATCHES] Numeric 508 datatype)

2005-12-04 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  OK, snprintf.c fixed.  I added a 'stream' and outlen parameter to all
  the calls, and cleaned up the switch() statement that was outputing
  twice.  When we were outputing just to a string, it didn't matter, but
  now that we are also outputting to a stream, it does.
 
 I found a whole bunch more problems than this :-(.  I've committed a
 cleaned-up version that seems to work correctly in a simple standalone
 testbed, but it'd be a good idea to exercise it inside PG as well.
 Can you try regression tests and the factorial() problem on CVS tip?

Thanks.  Tested 8.1.1 and CVS tip and all compile, and regression pass. 
I also tested the factorial test and the result looks perfect, thanks!

 The problems are sufficiently bad that it might be a good idea to
 backport the fixes into 8.0 and before as well --- but I note that
 the ABI is different (pg_snprintf vs snprintf, etc) so this requires
 a bit of investigation rather than just committing the file as-is.

Not as many 8.0.X platforms used *printf because we didn't test %$ for
its use on that release, so my bet is that very few platforms would be
using it.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: port/snprintf.c (was Re: [PATCHES] Numeric 508 datatype)

2005-12-04 Thread Tom Lane
I wrote:
 Hm.  One of the main problems I found was incorrect results for
 LONGLONG_MIN (-2^63).  I'm rather tempted to add a test case for
 that to the int8 regression test and see if any platforms fail ;-)

Done ... let me know whether the back branches still pass regression
for you ;-)

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: port/snprintf.c (was Re: [PATCHES] Numeric 508 datatype)

2005-12-04 Thread Bruce Momjian
Tom Lane wrote:
 I wrote:
  Hm.  One of the main problems I found was incorrect results for
  LONGLONG_MIN (-2^63).  I'm rather tempted to add a test case for
  that to the int8 regression test and see if any platforms fail ;-)
 
 Done ... let me know whether the back branches still pass regression
 for you ;-)

I checked back to 7.3 and everything passed. I did a cvs update,
configure, gmake, and regression run for each branch.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: port/snprintf.c (was Re: [PATCHES] Numeric 508 datatype)

2005-12-04 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 Done ... let me know whether the back branches still pass regression
 for you ;-)

 I checked back to 7.3 and everything passed. I did a cvs update,
 configure, gmake, and regression run for each branch.

[ digs a bit deeper... ]  Actually, it appears that that bug didn't
exist before 8.1; it was introduced here:

2005-03-16 22:18  momjian

* src/port/snprintf.c: Factor duplicate snprintf code into
functions.

by an ill-considered removal of an unsigned local variable.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-03 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 (I could have done the struct but that seemed too invasive.)

I think it'd be a lot cleaner with the struct.  Mind if I take another
pass at it?

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PATCHES] Numeric 508 datatype

2005-12-03 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  (I could have done the struct but that seemed too invasive.)
 
 I think it'd be a lot cleaner with the struct.  Mind if I take another
 pass at it?

OK, you want my patch or want me to apply and then you can modify?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-03 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 I think it'd be a lot cleaner with the struct.  Mind if I take another
 pass at it?

 OK, you want my patch or want me to apply and then you can modify?

You sent out your patch already --- I've been working from that.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-03 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  Tom Lane wrote:
  I think it'd be a lot cleaner with the struct.  Mind if I take another
  pass at it?
 
  OK, you want my patch or want me to apply and then you can modify?
 
 You sent out your patch already --- I've been working from that.

OK, thanks.
-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Simon Riggs
On Thu, 2005-12-01 at 23:34 -0500, Bruce Momjian wrote:
 Where are we on this patch?  It is ready for the patch queue?

It's good to be applied, AFAIK.

 Simon Riggs wrote:
  As previously agreed, reviewing this is a 2 stage process:
  1. review/possibly agree OK to commit
  2. check with everybody on GENERAL that the restriction to 508 is
  acceptable
  
  Figure there's no point doing (2) until we agree the proposal/code is
  workable.

I was hoping you'd give me a this looks good enough to apply
thumbs-up, then I'll ask for comments via a Weekly News item.

If I ask for comments and then it is technically rejected we would be
wasting everybody's time.

Best Regards, Simon Riggs


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Simon Riggs wrote:
 
 Now we're into 8.2devel mode, its time to submit the previously
 discussed patch that:
 
 - reduces Numeric storage format by 2 bytes
 - limits scale to +/- 508 decimal places
 
 This is sufficient to allow Numeric to continue to be used as the
 default numeric representation for all numbers in the parser.
 
 Passes: make check on cvstip, as well as some tests not in there.
 
 Code comments explain the new format and consequences.
 
 As previously agreed, reviewing this is a 2 stage process:
 1. review/possibly agree OK to commit
 2. check with everybody on GENERAL that the restriction to 508 is
 acceptable
 
 Figure there's no point doing (2) until we agree the proposal/code is
 workable.

OK, seems all objections have been dealt with so it goes into the patch
queue.  I will ask on 'general'.

The only downside I see is that I can't impress people by doing:

SELECT factorial(4000);

I don't suppose the _impression_ factor is worth two bytes per value. 
Shame.

I suppose people wanting to do such manipulations will have to store the
numbers as text and use a server-side library like perl to do
calculations.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Bruce Momjian wrote:
 OK, seems all objections have been dealt with so it goes into the patch
 queue.  I will ask on 'general'.
 
 The only downside I see is that I can't impress people by doing:
 
   SELECT factorial(4000);
 
 I don't suppose the _impression_ factor is worth two bytes per value. 
 Shame.
 
 I suppose people wanting to do such manipulations will have to store the
 numbers as text and use a server-side library like perl to do
 calculations.

Oops, I was wrong about this.  The patch changes the maximum _specified_
precision:

  /*
!  * Hardcoded precision limit - arbitrary, but must be small enough 
that
!  * dscale values will fit in 14 bits.
   */
! #define NUMERIC_MAX_PRECISION 1000

  /*
   * Internal limits on the scales chosen for calculation results
--- 15,23 
  #define _PG_NUMERIC_H_

  /*
!  * Hardcoded precision limit - maximum that can fit in Numeric storage
   */
! #define NUMERIC_MAX_PRECISION 508

but in fact, our computational precision is 4096, and we silently
overflow for values greater than that:

test= create table test(x numeric);
CREATE TABLE
test= insert into test values (factorial(4000));
INSERT 0 1

The length is 4096 digits, and so is factorial(1) --- clearly wrong.
I now see in the TODO:

* Change NUMERIC to enforce the maximum precision, and increase it

So we are really decreasing the specified precision from 1000 to 508,
and the computational precision from 4096 to 508.  Is there any plan to
fix the silent overflow problem?  Is that in the patch?  I don't see it.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 So we are really decreasing the specified precision from 1000 to 508,
 and the computational precision from 4096 to 508.

The internal computational precision isn't any less, the limit is only
on the result of a function (ie, partial results within one of the
numeric.c routines could still exceed 10^508).  Not sure how much that
distinction matters though.

 Is there any plan to
 fix the silent overflow problem?  Is that in the patch?  I don't see it.

It will get fixed before application ;-)

I haven't reviewed the patch yet; I think the gating factor at this
point is whether anyone protests losing dynamic range in NUMERIC,
and we ought to go ahead and ask that.  After that we can look at the
code more closely.

regards, tom lane

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  So we are really decreasing the specified precision from 1000 to 508,
  and the computational precision from 4096 to 508.
 
 The internal computational precision isn't any less, the limit is only
 on the result of a function (ie, partial results within one of the
 numeric.c routines could still exceed 10^508).  Not sure how much that
 distinction matters though.

Agreed.

  Is there any plan to
  fix the silent overflow problem?  Is that in the patch?  I don't see it.
 
 It will get fixed before application ;-)
 
 I haven't reviewed the patch yet; I think the gating factor at this
 point is whether anyone protests losing dynamic range in NUMERIC,
 and we ought to go ahead and ask that.  After that we can look at the
 code more closely.

I am confused by your use of the term dynamic range.  From what you
say above that we are just moving from 1000 to 508 for storage, and that
computational range would still be 4096?

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 I am confused by your use of the term dynamic range.  From what you
 say above that we are just moving from 1000 to 508 for storage, and that
 computational range would still be 4096?

No, computational range would still be on the order of 10^16G ... in the
computational format, the weight is an int.  The restriction to 1000
digits was never anything but an artificial limit.  (Of course, you
might not have the patience to actually do any arithmetic with that many
digits, but the point is there was a whole lot of headroom before, and
now there won't be.)

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  I am confused by your use of the term dynamic range.  From what you
  say above that we are just moving from 1000 to 508 for storage, and that
  computational range would still be 4096?
 
 No, computational range would still be on the order of 10^16G ... in the
 computational format, the weight is an int.  The restriction to 1000
 digits was never anything but an artificial limit.  (Of course, you
 might not have the patience to actually do any arithmetic with that many
 digits, but the point is there was a whole lot of headroom before, and
 now there won't be.)

Sorry, I am confused.  If our computational range is that high, why does
SELECT factorial(4000) and SELECT factorial(6000) produce the same
number of digits on my screen.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Sorry, I am confused.  If our computational range is that high, why does
 SELECT factorial(4000) and SELECT factorial(6000) produce the same
 number of digits on my screen.

Are you counting correctly?

regression=# select log(factorial(4000));
  log

 12673.2621666764869773
(1 row)

regression=# select log(factorial(6000));
log

 20065.428782473590
(1 row)

regression=# select factorial(4000)
regression-# \g z4000
regression=# select factorial(6000)
regression-# \g z6000
regression=# \q
$ wc z4000 z6000
5 5 38039 z4000
5 5 60215 z6000

The actual representation limit at the moment (with int16 weights in the
storable format) is 10^128K, as you can soon prove with pow():

regression=# select pow(10::numeric, 131071);
 lots o zeroes 
regression=# select pow(10::numeric, 131072);
ERROR:  value overflows numeric format

I don't recall what factorial that might correspond to, but it's
considerably above 6000.

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian

Uh, I tried factorial(4000) and for display I got:


1828801951514065013314743175573919044217377710730439219706452695420895979797317736485037028687048410733644304156928557175467246186154355733394261561795699671674528483159731749881876093748280498041957651294872061055892812978809780062059342953770532674062445388428509174395175674617362378722469436194575929579900114212973360658998073977714697261205048663725936337490404066097966637170254021348800944280342285355946649681316260163459743803577175903394733170076841764779082166891184529324230033414145497801832598218518406552257097392530024582738982919104406782168708871495603501905867399966298798534877747923179195791416504408054878974770308650707120878837624986576073340449414854578367383301715706358194127400849855604080473305196833482408079420964275187538889115296655522397723924887154624810659788321005620558369604778657904771918388054319251513981954296741688447246185021250402225010116433016818588036690180177691461779713104301640395708274734701186772756966064611023656528765138735704190876200697145804692125236821066805337175220605745377557452592208653939853278523841448314026548802309860391087939783218946129582647928430739985554883806198749831633640196211202756086016039171607744078776876219661603702759454887945247605749205543464095883664514960293873244842409803801480566600124415293789831496309554117113888561569494314926134470477513516416560102984058751906208865570183683850791317395702861350821464653600469443279077733978568711404244774089509216727922510660941411716412467443445414001188915966547283773988670739792818897314762082568914041952211779194055311405259158538932388745292324386826830135904886472292289993848482289254307628467614523292519222687689180219788377184005246290896703260524910362136627321135976515358528150143796798116836263053229733971612275184896139539613129329008449214723196703789119820971205922195513915546814704778682373487718946560822811623038853887054357316290622378472322045316639418491798517077275839637525427601452961835674484434498885698840692468550825765131610925966585339561854456154229048295742274725126218799745448031391826295221114381890600683208441558088271228618006589059444108806652992787854634497487158675770983422610936590600627170500972481399444145398522756870626097250230229195799277299921844954715690883242553569256657132515663544931830393317518828986443942138971609142621397646808351809694603734872977984148002699965137870448199866167162949256435040416146886823942144459105175033488395869910405207521329016842673168563837531518918339627724066152933627236730561155418227888673513937454508103826102827706121560330906016404162420051373313654570020033195778785022169191701120746087228523767999431915904806516239580629828294520352271190365024265837525121998240897256117110591539354344189851092414041350690471095275147306485020646304313711859225230369416210263927838134355401958005319886454303447452988456400170827326232488384737716034783363266625792191376014226320576487588079352339155275628179423786752439198868000562094347314076856919423270924641011362547954991593511035427477234345444363663131049963736616549894654981808927164628050422270382221047840626260274801515673778418213162920952956863686193004178633275307643013230819024359711659251635132255111762589194716734375533209349169105739990209660872076631338715165303917875357554203481745199540130159991933352050325711764601050057161153057486693646826752650143102232717628076202428056174355949278906764089530573848907196812258404003966981556247931615658504360478296170497139776495940475135844585691458195718653357320769035589415077664772799415642564195375551772796548609667538432344185537279888780570854092084221993660761154788359777439798490851148031275123759286793224660188593768897291749180357297185650430735063126035278321749629995702200115722386600046772883018963166273463715186812543356173550341233310175600817771447170656517505385258772069067139665478506263898380526394388218036388768999161538645432932116471146392562992204259450889291928261780531925618148311151259088519869787637913607866079920830781433275298468534248595471485354420794089854693167644320115900404712

and did factorial(6000) and see:



Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Both are 4096 characters.

You forgot the part that scrolled off the screen.  Or else your
installation is broken.

I get this for factorial(4000)
 
182880195151406501331474317557391904421737771073043921970645269542089597979731773648503702868704841073364430415692855717546724618615435573339426156179569967167452848315973174988187609374828049804195765129487206105589281297880978006205934295377053267406244538842850917439517567461736237872246943619457592957990011421297336065899807397771469726120504866372593633749040406609796663717025402134880094428034228535594664968131626016345974380357717590339473317007684176477908216689118452932423003341414549780183259821851840655225709739253002458273898291910440678216870887149560350190586739996629879853487774792317919579141650440805487897477030865070712087883762498657607334044941485457836738330171570635819412740084985560408047330519683348240807942096427518753888911529665552239772392488715462481065978832100562055836960477865790477191838805431925151398195429674168844724618502125040222501011643301681858803669018017769146177971310430164039570827473470118677275696606461102365652876513873570!
 
4190876200697145804692125236821066805337175220605745377557452592208653939853278523841448314026548802309860391087939783218946129582647928430739985554883806198749831633640196211202756086016039171607744078776876219661603702759454887945247605749205543464095883664514960293873244842409803801480566600124415293789831496309554117113888561569494314926134470477513516416560102984058751906208865570183683850791317395702861350821464653600469443279077733978568711404244774089509216727922510660941411716412467443445414001188915966547283773988670739792818897314762082568914041952211779194055311405259158538932388745292324386826830135904886472292289993848482289254307628467614523292519222687689180219788377184005246290896703260524910362136627321135976515358528150143796798116836263053229733971612275184896139539613129329008449214723196703789119820971205922195513915546814704778682373487718946560822811623038853887054357316290622378472322045316639418491798517077275839637525427601452961835674484434498885!
 6988406924685508257651316109259665853395618544561542290482957422747251
2621879974544803139182629522111438189060068320844155808827122861800658905944410880665299278785463449748715867577098342261093659060062717050097248139944414539852275687062609725023022919579927729992184495471569088324255356925665713251566354493183039331751882898644394213897160914262139764680835180969460373487297798414800269996513787044819986616716294925643504041614688682394214445910517503348839586991040520752132901684267316856383753151891833962772406615293362723673056115541822788867351393745450810382610282770612156033090601640416242005137331365457002003319577878502216919170112074608722852376799943191590480651623958062982829452035227119036502426583752512199824089725611711059153935434418985109241404135069047109527514730648502064630431371185922523036941621026392783813435540195800531988645430344745298845640017082732623248838473771603478336326662579219137601422632057648758807935233915527562817942378675243919886800056209434731407685691942327092464101136254795499159351103542747723!
 
43454443636631310499637366165498946549818089271646280504222703822210478406262602748015156737784182131629209529568636861930041786332753076430132308190243597116592516351322551117625891947167343755332093491691057399902096608720766313387151653039178753575542034817451995401301599919333520503257117646010500571611530574866936468267526501431022327176280762024280561743559492789067640895305738489071968122584040039669815562479316156585043604782961704971397764959404751358445856914581957186533573207690355894150776647727994156425641953755517727965486096675384323441855372798887805708540920842219936607611547883597774397984908511480312751237592867932246601885937688972917491803572971856504307350631260352783217496299957022001157223866000467728830189631662734637151868125433561735503412333101756008177714471706565175053852587720690671396654785062638983805263943882180363887689991615386454329321164711463925629922042594508892919282617805319256181483111512590885198697876379136078660799208307!
 814332752984685342485954714853544207940898546931676443201159004047

Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Alvaro Herrera
Bruce Momjian wrote:

 I tried your \g test and the file size difference is the length of the
 dashed line in the file, not the number of digits display, which are
 both 4096.  One has 12550 dashes, the other 19950 dashes.

So this is a psql bug?  I can count the correct number of chars with
SPI:


alvherre=# create or replace function factorial_length(int) returns int 
language plphp
alvherre-# as $$ $r = spi_exec(select factorial($args[0]));
alvherre$# $row = spi_fetch_row($r);
alvherre$# return strlen($row['factorial']); $$; 

alvherre=# select factorial_length(4000);
 factorial_length 
--
12674
(1 fila)

alvherre=# select factorial_length(6000);
 factorial_length 
--
20066
(1 fila)

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Alvaro Herrera
Tom Lane wrote:
 Alvaro Herrera [EMAIL PROTECTED] writes:
  So this is a psql bug?
 
 Not here.  Do you see it?

Actually, no.  If I cut'n paste the number from psql to

cat  foo
shift insert

then only 4096 chars are copied.  (Amusingly, I can't add a newline to
^D and close the file.  I must delete one char to do that.)

However if I open vim and paste there, the whole 12000+delta chars are
copied.

So it must be a bug in Bruce's procedure I guess ...

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian

Please try Tom's \g test:

test= select factorial(4000)
test- \g /tmp/x
test= \q
$ wc -c /tmp/x
   20881 /tmp/x

Do you see a number greater than 20881, something like 3?

---

Alvaro Herrera wrote:
 Tom Lane wrote:
  Alvaro Herrera [EMAIL PROTECTED] writes:
   So this is a psql bug?
  
  Not here.  Do you see it?
 
 Actually, no.  If I cut'n paste the number from psql to
 
 cat  foo
 shift insert
 
 then only 4096 chars are copied.  (Amusingly, I can't add a newline to
 ^D and close the file.  I must delete one char to do that.)
 
 However if I open vim and paste there, the whole 12000+delta chars are
 copied.
 
 So it must be a bug in Bruce's procedure I guess ...
 
 -- 
 Alvaro Herrerahttp://www.CommandPrompt.com/
 PostgreSQL Replication, Consulting, Custom Development, 24x7 support
 
 ---(end of broadcast)---
 TIP 3: Have you checked our extensive FAQ?
 
http://www.postgresql.org/docs/faq
 

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 Actually, no.  If I cut'n paste the number from psql to
 cat  foo
 shift insert
 then only 4096 chars are copied.  (Amusingly, I can't add a newline to
 ^D and close the file.  I must delete one char to do that.)

Hmm, cut buffer limitation in X or someplace?  I definitely get the
right number of characters into the file written with \g, and what looks
like a reasonable number of screensful of plain psql output.  If Bruce
is seeing the right number of dashes and the wrong number of data
characters in his \g output then *something* is pretty weird there.

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Alvaro Herrera
Tom Lane wrote:
 Alvaro Herrera [EMAIL PROTECTED] writes:
  Actually, no.  If I cut'n paste the number from psql to
  cat  foo
  shift insert
  then only 4096 chars are copied.  (Amusingly, I can't add a newline to
  ^D and close the file.  I must delete one char to do that.)
 
 Hmm, cut buffer limitation in X or someplace?  I definitely get the
 right number of characters into the file written with \g, and what looks
 like a reasonable number of screensful of plain psql output.  If Bruce
 is seeing the right number of dashes and the wrong number of data
 characters in his \g output then *something* is pretty weird there.

Well, I just tried the \g test and it is correct (12675 digits or so).

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Alvaro Herrera wrote:
 Tom Lane wrote:
  Alvaro Herrera [EMAIL PROTECTED] writes:
   Actually, no.  If I cut'n paste the number from psql to
   cat  foo
   shift insert
   then only 4096 chars are copied.  (Amusingly, I can't add a newline to
   ^D and close the file.  I must delete one char to do that.)
  
  Hmm, cut buffer limitation in X or someplace?  I definitely get the
  right number of characters into the file written with \g, and what looks
  like a reasonable number of screensful of plain psql output.  If Bruce
  is seeing the right number of dashes and the wrong number of data
  characters in his \g output then *something* is pretty weird there.
 
 Well, I just tried the \g test and it is correct (12675 digits or so).

I just tested from a standalone backend:

backend select pow(10::numeric, 131071) + 1

and got 4095 zeros and no trailing '1' (wrong), so it isn't psql, it
must be something in the backend.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 I just tested from a standalone backend:
   backend select pow(10::numeric, 131071) + 1
 and got 4095 zeros and no trailing '1' (wrong), so it isn't psql, it
 must be something in the backend.

If the backend is truncating the result length, I don't see why psql
would decide it needs 12K dashes for the header.  There's something
awfully fishy going on in your machine.

Try something like

regression=# select length((pow(10::numeric, 131071))::text);
 length

 131089
(1 row)

If that gives the right answer then the NUMERIC code is off the hook,
and what you've got is a strange limitation on output column length.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  I just tested from a standalone backend:
  backend select pow(10::numeric, 131071) + 1
  and got 4095 zeros and no trailing '1' (wrong), so it isn't psql, it
  must be something in the backend.
 
 If the backend is truncating the result length, I don't see why psql
 would decide it needs 12K dashes for the header.  There's something
 awfully fishy going on in your machine.
 
 Try something like
 
 regression=# select length((pow(10::numeric, 131071))::text);
  length
 
  131089
 (1 row)
 
 If that gives the right answer then the NUMERIC code is off the hook,
 and what you've got is a strange limitation on output column length.

test= select length((pow(10::numeric, 131071))::text);
 length

 131089
(1 row)

Looks good.  From psql I just tried:

SELECT repeat('x', 4000);

and got 4k x's, and SELECT repeat('x', 8000) returns 8k x's, so that works.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Simon Riggs
On Fri, 2005-12-02 at 14:09 -0500, Bruce Momjian wrote:

 I ran your SELECT pow(10::numeric, 131071), and gain, 4096 0's are
 displayed on my screen.  SELECT pow(10::numeric, 7000) and SELECT
 pow(10::numeric, 1) generate identical displays on my screen.

Are you saying there is a bug with or without my patch?

Can we get the usual release levels/cvstip, ports, etc.

Are you connecting across network/ protocol differences...

[Just out of interest, has anybody ever used a number bigger than 10^20
in an application? My understanding is that the number of atoms in the
universe is around 10^80. Accuracy is needed during calculation, but
much less so during storage.]

Thanks,

Best Regards, Simon Riggs



---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Simon Riggs wrote:
 On Fri, 2005-12-02 at 14:09 -0500, Bruce Momjian wrote:
 
  I ran your SELECT pow(10::numeric, 131071), and gain, 4096 0's are
  displayed on my screen.  SELECT pow(10::numeric, 7000) and SELECT
  pow(10::numeric, 1) generate identical displays on my screen.
 
 Are you saying there is a bug with or without my patch?

Without.

 Can we get the usual release levels/cvstip, ports, etc.

Sorry, BSD/OS 4.3.1, CVS HEAD, no patches.

 Are you connecting across network/ protocol differences...

psql and the backend are on the same machine.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 If that gives the right answer then the NUMERIC code is off the hook,
 and what you've got is a strange limitation on output column length.

   test= select length((pow(10::numeric, 131071))::text);
length
   
131089
   (1 row)

 Looks good.  From psql I just tried:

   SELECT repeat('x', 4000);

 and got 4k x's, and SELECT repeat('x', 8000) returns 8k x's, so that works.

Curiouser and curiouser.  How about if you repeat 4k or 8k '1's?  If the
behavior is different for letters and digits then I'd look at the column
justification logic in psql's printing code.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  Tom Lane wrote:
  If that gives the right answer then the NUMERIC code is off the hook,
  and what you've got is a strange limitation on output column length.
 
  test= select length((pow(10::numeric, 131071))::text);
   length
  
   131089
  (1 row)
 
  Looks good.  From psql I just tried:
 
  SELECT repeat('x', 4000);
 
  and got 4k x's, and SELECT repeat('x', 8000) returns 8k x's, so that works.
 
 Curiouser and curiouser.  How about if you repeat 4k or 8k '1's?  If the

1's print just fine too.

 behavior is different for letters and digits then I'd look at the column
 justification logic in psql's printing code.

Again, I checked on a stand-alone backend and saw the same failures, so
it isn't psql.

Wow, check this out:

test= SELECT CAST (pow(10::numeric, 1) + 1 AS TEXT)

It works fine!  I have all the digits, and the trailing 1.0:

01.

while SELECT pow(10::numeric, 1) fails.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Wow, check this out:
   test= SELECT CAST (pow(10::numeric, 1) + 1 AS TEXT)
 It works fine!  I have all the digits, and the trailing 1.0:
   01.
 while SELECT pow(10::numeric, 1) fails.

That's just about as wacky as can be, because numeric_text() is
implemented on top of numeric_out() ... there's no way that numeric_out
can be delivering the wrong answer if the cast produces the right text.
So somewhere between numeric_out and the delivery to the client,
something's getting confused.  I think it's time you got out your
debugger and started tracing through the backend ...

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Michael Fuhr
On Fri, Dec 02, 2005 at 04:30:54PM -0500, Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  Wow, check this out:
  test= SELECT CAST (pow(10::numeric, 1) + 1 AS TEXT)
  It works fine!  I have all the digits, and the trailing 1.0:
  01.
  while SELECT pow(10::numeric, 1) fails.
 
 That's just about as wacky as can be, because numeric_text() is
 implemented on top of numeric_out() ... there's no way that numeric_out
 can be delivering the wrong answer if the cast produces the right text.
 So somewhere between numeric_out and the delivery to the client,
 something's getting confused.  I think it's time you got out your
 debugger and started tracing through the backend ...

Bruce, have you run a process trace on the backend to see if write()
(or whatever) is writing the correct number of characters?  What
exactly is your output device and how are you connected to the
machine that runs the backend (ssh to a remote box from an xterm,
sitting in front of the box's VT52 serial console, etc.)?

If you run the query that fails in a standalone backend, do you get
something like (typeid = 1700, len = -1, typmod = -1, byval = f)
at the end of the line, or is that part truncated too?

-- 
Michael Fuhr

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Michael Fuhr wrote:
 On Fri, Dec 02, 2005 at 04:30:54PM -0500, Tom Lane wrote:
  Bruce Momjian pgman@candle.pha.pa.us writes:
   Wow, check this out:
 test= SELECT CAST (pow(10::numeric, 1) + 1 AS TEXT)
   It works fine!  I have all the digits, and the trailing 1.0:
 01.
   while SELECT pow(10::numeric, 1) fails.
  
  That's just about as wacky as can be, because numeric_text() is
  implemented on top of numeric_out() ... there's no way that numeric_out
  can be delivering the wrong answer if the cast produces the right text.
  So somewhere between numeric_out and the delivery to the client,
  something's getting confused.  I think it's time you got out your
  debugger and started tracing through the backend ...
 
 Bruce, have you run a process trace on the backend to see if write()
 (or whatever) is writing the correct number of characters?  What
 exactly is your output device and how are you connected to the
 machine that runs the backend (ssh to a remote box from an xterm,
 sitting in front of the box's VT52 serial console, etc.)?
 
 If you run the query that fails in a standalone backend, do you get
 something like (typeid = 1700, len = -1, typmod = -1, byval = f)
 at the end of the line, or is that part truncated too?

I found the cause.  I traced into printf then realized I was not in libc
but port/snprintf.c, and I see 4096 defined for those buffers.  I will
work on a patch to make it dynamic.  At the time I think there was
thought that 4096 was as large as it ever needed to be, but obviously
this was wrong. I think Win32 would see the same failure because it used
port/snprintf.c too.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 I found the cause.  I traced into printf then realized I was not in libc
 but port/snprintf.c, and I see 4096 defined for those buffers.

Uh, how is control getting to snprintf?  I don't see that used either
in numeric.c or in printtup.c.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  I found the cause.  I traced into printf then realized I was not in libc
  but port/snprintf.c, and I see 4096 defined for those buffers.
 
 Uh, how is control getting to snprintf?  I don't see that used either
 in numeric.c or in printtup.c.

I am seeing it in the standalone backend here:

debugtup (slot=0x856e0b0, self=0x84306d0) at printtup.c:548
548 printatt((unsigned) i + 1, typeinfo-attrs[i], 
value);
(gdb) s
printatt (attributeId=1, attributeP=0x856efa4, value=0x857201c 1, '0' 
repeats 199 times...) at printtup.c:480
480 printf(\t%2d: %s%s%s%s\t(typeid = %u, len = %d, typmod 
= %d, byval = %c)\n,

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 Uh, how is control getting to snprintf?  I don't see that used either
 in numeric.c or in printtup.c.

 I am seeing it in the standalone backend here:

   debugtup (slot=0x856e0b0, self=0x84306d0) at printtup.c:548
   548 printatt((unsigned) i + 1, typeinfo-attrs[i], 
 value);

That would only affect a standalone backend, however, not normal
operation.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  Tom Lane wrote:
  Uh, how is control getting to snprintf?  I don't see that used either
  in numeric.c or in printtup.c.
 
  I am seeing it in the standalone backend here:
 
  debugtup (slot=0x856e0b0, self=0x84306d0) at printtup.c:548
  548 printatt((unsigned) i + 1, typeinfo-attrs[i], 
  value);
 
 That would only affect a standalone backend, however, not normal
 operation.

Ah, psql itself is using the same buggy snprintf.c.  That would explain
the long dash line, but short digits.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Bruce Momjian wrote:
  If you run the query that fails in a standalone backend, do you get
  something like (typeid = 1700, len = -1, typmod = -1, byval = f)
  at the end of the line, or is that part truncated too?
 
 I found the cause.  I traced into printf then realized I was not in libc
 but port/snprintf.c, and I see 4096 defined for those buffers.  I will
 work on a patch to make it dynamic.  At the time I think there was
 thought that 4096 was as large as it ever needed to be, but obviously
 this was wrong. I think Win32 would see the same failure because it used
 port/snprintf.c too.

My Win32 8.1 platform is fine so Win32 must not use port/snprintf.c.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 That would only affect a standalone backend, however, not normal
 operation.

 Ah, psql itself is using the same buggy snprintf.c.

Doh.  OK, we gotta fix it then.  But what are you going to do when you
can't malloc enough memory?  You can't ereport in a client environment,
and there's no API for printf to report failure.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  Tom Lane wrote:
  That would only affect a standalone backend, however, not normal
  operation.
 
  Ah, psql itself is using the same buggy snprintf.c.
 
 Doh.  OK, we gotta fix it then.  But what are you going to do when you
 can't malloc enough memory?  You can't ereport in a client environment,
 and there's no API for printf to report failure.

Yep, I am digging through snprintf.c now to try find a solution.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Tom Lane wrote:
 Doh.  OK, we gotta fix it then.  But what are you going to do when you
 can't malloc enough memory?  You can't ereport in a client environment,
 and there's no API for printf to report failure.

 Yep, I am digging through snprintf.c now to try find a solution.

Well, there's always the option of going back to plan B, which is not
using printf for strings that might be long.  AFAICS it would only be
a convenience for psql, since it's not doing any real conversions,
just inserting the right number of spaces.

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Alvaro Herrera [EMAIL PROTECTED] writes:
 So this is a psql bug?

Not here.  Do you see it?

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Tom Lane
Bruce Momjian pgman@candle.pha.pa.us writes:
 Yep, I am digging through snprintf.c now to try find a solution.

The cleanest solution is probably to fix things so that dopr_outch is
aware of whether it's working for sprintf or fprintf, and can dump the
buffer directly to the file when it gets full in the fprintf case.
Its existing API would need to be changed a bit ... maybe pass it a
struct containing what it needs, instead of having all the layers of
code know what to pass.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-12-02 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian pgman@candle.pha.pa.us writes:
  Yep, I am digging through snprintf.c now to try find a solution.
 
 The cleanest solution is probably to fix things so that dopr_outch is
 aware of whether it's working for sprintf or fprintf, and can dump the
 buffer directly to the file when it gets full in the fprintf case.
 Its existing API would need to be changed a bit ... maybe pass it a
 struct containing what it needs, instead of having all the layers of
 code know what to pass.

OK, snprintf.c fixed.  I added a 'stream' and outlen parameter to all
the calls, and cleaned up the switch() statement that was outputing
twice.  When we were outputing just to a string, it didn't matter, but
now that we are also outputting to a stream, it does.

Passed regression and initdb tests, and factorial(4000) works!

(I could have done the struct but that seemed too invasive.)

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073
Index: src/port/snprintf.c
===
RCS file: /cvsroot/pgsql/src/port/snprintf.c,v
retrieving revision 1.29
diff -c -c -r1.29 snprintf.c
*** src/port/snprintf.c 15 Oct 2005 02:49:51 -  1.29
--- src/port/snprintf.c 3 Dec 2005 04:23:00 -
***
*** 64,70 
  
  /*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.29 
2005/10/15 02:49:51 momjian Exp $;*/
  
! static void dopr(char *buffer, const char *format, va_list args, char *end);
  
  /* Prevent recursion */
  #undefvsnprintf
--- 64,70 
  
  /*static char _id[] = $PostgreSQL: pgsql/src/port/snprintf.c,v 1.29 
2005/10/15 02:49:51 momjian Exp $;*/
  
! static int dopr(FILE *stream, char *buffer, const char *format, va_list args, 
char *end);
  
  /* Prevent recursion */
  #undefvsnprintf
***
*** 73,89 
  #undeffprintf
  #undefprintf
  
! int
! pg_vsnprintf(char *str, size_t count, const char *fmt, va_list args)
  {
char   *end;
  
!   str[0] = '\0';
!   end = str + count - 1;
!   dopr(str, fmt, args, end);
!   if (count  0)
end[0] = '\0';
!   return strlen(str);
  }
  
  int
--- 73,100 
  #undeffprintf
  #undefprintf
  
!   
! static int
! pg_fvsnprintf(FILE *stream, char *str, size_t count, const char *fmt, va_list 
args)
  {
char   *end;
+   int len;
  
!   if (str)
!   {
!   str[0] = '\0';
!   end = str + count - 1;
!   }
!   len = dopr(stream, str, fmt, args, end);
!   if (str  count  0)
end[0] = '\0';
!   return len;
! }
! 
! int
! pg_vsnprintf(char *str, size_t count, const char *fmt, va_list args)
! {
!   return pg_fvsnprintf(NULL, str, count, fmt, args);
  }
  
  int
***
*** 93,99 
va_list args;
  
va_start(args, fmt);
!   len = pg_vsnprintf(str, count, fmt, args);
va_end(args);
return len;
  }
--- 104,110 
va_list args;
  
va_start(args, fmt);
!   len = pg_fvsnprintf(NULL, str, count, fmt, args);
va_end(args);
return len;
  }
***
*** 103,115 
  {
int len;
va_list args;
!   charbuffer[4096];
  
va_start(args, fmt);
!   len = pg_vsnprintf(buffer, (size_t) 4096, fmt, args);
va_end(args);
/* limit output to string */
!   StrNCpy(str, buffer, (len + 1  4096) ? len + 1 : 4096);
return len;
  }
  
--- 114,126 
  {
int len;
va_list args;
!   charbuffer[8192];   /* arbitrary limit */
  
va_start(args, fmt);
!   len = pg_fvsnprintf(NULL, buffer, (size_t) 4096, fmt, args);
va_end(args);
/* limit output to string */
!   StrNCpy(str, buffer, (len + 1  8192) ? len + 1 : 8192);
return len;
  }
  
***
*** 118,131 
  {
int len;
va_list args;
-   charbuffer[4096];
-   char   *p;
  
va_start(args, fmt);
!   len = pg_vsnprintf(buffer, (size_t) 4096, fmt, args);
va_end(args);
-   for (p = buffer; *p; p++)
-   putc(*p, stream);
return len;
  }
  
--- 129,138 
  {
int len;
va_list args;
  
va_start(args, fmt);
!   len = pg_fvsnprintf(stream, NULL, 0, fmt, args);
va_end(args);
return len;
  }
  
***
*** 134,166 
  {
int len;
va_list args;
-   charbuffer[4096];
-   char   *p;
  
va_start(args, fmt);
!  

Re: [PATCHES] Numeric 508 datatype

2005-12-01 Thread Bruce Momjian

Where are we on this patch?  It is ready for the patch queue?

---

Simon Riggs wrote:
 
 Now we're into 8.2devel mode, its time to submit the previously
 discussed patch that:
 
 - reduces Numeric storage format by 2 bytes
 - limits scale to +/- 508 decimal places
 
 This is sufficient to allow Numeric to continue to be used as the
 default numeric representation for all numbers in the parser.
 
 Passes: make check on cvstip, as well as some tests not in there.
 
 Code comments explain the new format and consequences.
 
 As previously agreed, reviewing this is a 2 stage process:
 1. review/possibly agree OK to commit
 2. check with everybody on GENERAL that the restriction to 508 is
 acceptable
 
 Figure there's no point doing (2) until we agree the proposal/code is
 workable.
 
 As Atsushi-san point out, there is also come CPU optimization to be done
 on Numeric comparison, and also on other areas such as aggregation. I've
 not done this yet.
 
 Best Regards, Simon Riggs

[ Attachment, skipping... ]

 
 ---(end of broadcast)---
 TIP 2: Don't 'kill -9' the postmaster

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [PATCHES] Numeric 508 datatype

2005-11-20 Thread Bruce Momjian
Andreas Pflug wrote:
 Alvaro Herrera wrote:
  
  Is anybody working or considering to work on pg_upgrade, or is all this
  hypothetical?  Our past history has seen lots of people offering to work
  on pg_upgrade, and none has produced a working version.  Is it fair or
  useful to impose restrictions on development just because it's remotely
  possible that somebody is going to be motivated enough to consider
  producing it?
 
 Depends on the impact the restriction imposes. If 
 stability/scalability/functionality or so is affected, this sounds not 
 tolerable. If it's about not saving two bytes that have been spoiled for 
 ages before, or keeping a backward compatibility type, it appears 
 feasible to me.
 Changing on-disk structures at the start of the 8.2 dev cycle is a 
 guarantee that nobody will implement pg_upgrade for 8.2.

Let's go ahead and apply the patch.  While this change isn't very
significant, I bet there will be other changes in 8.2 where we will want
to change the database for a significant benefit, like reducing the
tuple header by 4 bytes by recompressing the four xid/cid fields back
into three.

-- 
  Bruce Momjian|  http://candle.pha.pa.us
  pgman@candle.pha.pa.us   |  (610) 359-1001
  +  If your life is a hard drive, |  13 Roberts Road
  +  Christ can be your backup.|  Newtown Square, Pennsylvania 19073

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PATCHES] Numeric 508 datatype

2005-11-20 Thread Andrew Dunstan



Bruce Momjian wrote:



Let's go ahead and apply the patch.  While this change isn't very
significant, I bet there will be other changes in 8.2 where we will want
to change the database for a significant benefit, like reducing the
tuple header by 4 bytes by recompressing the four xid/cid fields back
into three.

 



All of which makes me very uncomfortable about doing 64 bit enums. I am 
still trying to think of a better solution.


cheers

andrew

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-11-18 Thread Andreas Pflug

Tom Lane wrote:

Andrew Dunstan [EMAIL PROTECTED] writes:

However, I don't think we can promise never to change the ondisk 
representation of data, nor the page layout. Sometimes an inplace 
upgrade just won't work, ISTM.



We have talked about batching on-disk changes so that they'd only occur
once every few release cycles.  But until we have a pg_upgrade, there is
no reason to adopt such a policy.


IMHO such a policy is a _prerequisite_ for somebody to come up 
implementing pg_upgrade. Why spend time on pg_upgrade if there's no 
policy to support it?


Regards,
Andreas

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PATCHES] Numeric 508 datatype

2005-11-18 Thread Alvaro Herrera
Andreas Pflug wrote:
 Tom Lane wrote:
 Andrew Dunstan [EMAIL PROTECTED] writes:
 
 However, I don't think we can promise never to change the ondisk 
 representation of data, nor the page layout. Sometimes an inplace 
 upgrade just won't work, ISTM.
 
 We have talked about batching on-disk changes so that they'd only occur
 once every few release cycles.  But until we have a pg_upgrade, there is
 no reason to adopt such a policy.
 
 IMHO such a policy is a _prerequisite_ for somebody to come up 
 implementing pg_upgrade. Why spend time on pg_upgrade if there's no 
 policy to support it?

Is anybody working or considering to work on pg_upgrade, or is all this
hypothetical?  Our past history has seen lots of people offering to work
on pg_upgrade, and none has produced a working version.  Is it fair or
useful to impose restrictions on development just because it's remotely
possible that somebody is going to be motivated enough to consider
producing it?

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-11-18 Thread Andreas Pflug

Alvaro Herrera wrote:


Is anybody working or considering to work on pg_upgrade, or is all this
hypothetical?  Our past history has seen lots of people offering to work
on pg_upgrade, and none has produced a working version.  Is it fair or
useful to impose restrictions on development just because it's remotely
possible that somebody is going to be motivated enough to consider
producing it?


Depends on the impact the restriction imposes. If 
stability/scalability/functionality or so is affected, this sounds not 
tolerable. If it's about not saving two bytes that have been spoiled for 
ages before, or keeping a backward compatibility type, it appears 
feasible to me.
Changing on-disk structures at the start of the 8.2 dev cycle is a 
guarantee that nobody will implement pg_upgrade for 8.2.


Regards,
Andreas

---(end of broadcast)---
TIP 4: Have you searched our list archives?

  http://archives.postgresql.org


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Andreas Pflug

Simon Riggs wrote:

Now we're into 8.2devel mode, its time to submit the previously
discussed patch that:

- reduces Numeric storage format by 2 bytes


This makes the often discussed binary upgrade impossible, so I wonder if 
two bytes savings are worth the trouble.


Regards,
Andreas

---(end of broadcast)---
TIP 4: Have you searched our list archives?

  http://archives.postgresql.org


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Tom Lane
Andreas Pflug [EMAIL PROTECTED] writes:
 Simon Riggs wrote:
 Now we're into 8.2devel mode, its time to submit the previously
 discussed patch that:
 - reduces Numeric storage format by 2 bytes

 This makes the often discussed binary upgrade impossible, so I wonder if 
 two bytes savings are worth the trouble.

Unless someone actually steps forward and produces a working pg_upgrade
in the 8.2 timeframe, this objection is moot.

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Andreas Pflug

Tom Lane wrote:

Andreas Pflug [EMAIL PROTECTED] writes:


Simon Riggs wrote:


Now we're into 8.2devel mode, its time to submit the previously
discussed patch that:
- reduces Numeric storage format by 2 bytes



This makes the often discussed binary upgrade impossible, so I wonder if 
two bytes savings are worth the trouble.



Unless someone actually steps forward and produces a working pg_upgrade
in the 8.2 timeframe, this objection is moot.


Hm, so if this patch is applied now, and in 5 months or so somebody 
implements pg_upgrade, this numeric storage patch would be rolled back?
OTOH, an upgrade mechanism that's compatible for future 8.3+ versions 
only seems not too attractive.
A solution might be to keep the current numeric implementation under a 
different name (deprecatednumeric or so), for backward compatibility 
(this should apply to future storage format changes as well).


Regards,
Andreas

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Joshua D. Drake




Hm, so if this patch is applied now, and in 5 months or so somebody 
implements pg_upgrade, this numeric storage patch would be rolled back?
OTOH, an upgrade mechanism that's compatible for future 8.3+ versions 
only seems not too attractive.
With Slony and Replicator I don't really see the need for in place 
upgrades.


Joshua D. Drake


A solution might be to keep the current numeric implementation under a 
different name (deprecatednumeric or so), for backward compatibility 
(this should apply to future storage format changes as well).


Regards,
Andreas

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match



---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Dave Page
 

 -Original Message-
 From: [EMAIL PROTECTED] 
 [mailto:[EMAIL PROTECTED] On Behalf Of 
 Joshua D. Drake
 Sent: 17 November 2005 15:58
 To: Andreas Pflug
 Cc: Tom Lane; Simon Riggs; pgsql-patches@postgresql.org
 Subject: Re: [PATCHES] Numeric 508 datatype
 
 
 
  Hm, so if this patch is applied now, and in 5 months or so somebody 
  implements pg_upgrade, this numeric storage patch would be 
 rolled back?
  OTOH, an upgrade mechanism that's compatible for future 
 8.3+ versions 
  only seems not too attractive.
 With Slony and Replicator I don't really see the need for in place 
 upgrades.

They're not exactly easy to setup for the casual user (well, I assume
replicator isn't but it's not relevant to the majority of our users
anyway).

Regards, Dave.

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Andrew Dunstan



Joshua D. Drake wrote:

With Slony and Replicator I don't really see the need for in place 
upgrades.




Maintaining a replica is hardly a cost-free exercise.

However, I don't think we can promise never to change the ondisk 
representation of data, nor the page layout. Sometimes an inplace 
upgrade just won't work, ISTM.


cheers

andrew

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Tom Lane
Andrew Dunstan [EMAIL PROTECTED] writes:
 However, I don't think we can promise never to change the ondisk 
 representation of data, nor the page layout. Sometimes an inplace 
 upgrade just won't work, ISTM.

We have talked about batching on-disk changes so that they'd only occur
once every few release cycles.  But until we have a pg_upgrade, there is
no reason to adopt such a policy.

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Simon Riggs
On Thu, 2005-11-17 at 11:20 -0500, Tom Lane wrote:
 Andrew Dunstan [EMAIL PROTECTED] writes:
  However, I don't think we can promise never to change the ondisk 
  representation of data, nor the page layout. Sometimes an inplace 
  upgrade just won't work, ISTM.
 
 We have talked about batching on-disk changes so that they'd only occur
 once every few release cycles.  But until we have a pg_upgrade, there is
 no reason to adopt such a policy.

If the patch is accepted technically, in can be applied at any time,
right up to code freeze for this or the next release. It's a fairly
independent patch.

I'd suggest we check it out now, then put it in a holding pen for awhile
to see if an upgrade tool emerges.

Best Regards, Simon Riggs


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Tom Lane
Simon Riggs [EMAIL PROTECTED] writes:
 If the patch is accepted technically, in can be applied at any time,
 right up to code freeze for this or the next release. It's a fairly
 independent patch.

 I'd suggest we check it out now, then put it in a holding pen for awhile
 to see if an upgrade tool emerges.

I'm disinclined to do that unless there's a pretty firm commitment from
someone to work on pg_upgrade in the near future.  Patches that are not
in the tree tend to suffer from code drift; if we wait six months or a
year to apply what you've done then we'll likely be looking at
significantly more work to get it in.  We'd also be losing the direct
and indirect testing that the patch would get were it in the tree over
that length of time.

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [PATCHES] Numeric 508 datatype

2005-11-17 Thread Joshua D. Drake



They're not exactly easy to setup for the casual user (well, I assume
replicator isn't but it's not relevant to the majority of our users
anyway).
  
Well FYI any half way competent person can set up replicator in 30 
minutes or less ;), that said
in place upgrades are a feature that will take up a lot of man power 
EVERY release I would think.


I could be wrong but I would rather see the resources put into other things.

J

Regards, Dave.

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match
  



---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match