It gets even better with blind faith put into "algorithms" : A nice read on the subject can be found in Cathy O'Neil's book:
https://weaponsofmathdestructionbook.com/ On Sun, Aug 13, 2017 at 10:17 AM, Don Guinn <[email protected]> wrote: > What bothers me is that even though the source of data is unreliable, once > run through a computer program people tend to believe that now the results > are very accurate. One particular measure is oil reserves in the USA and > world. We don't even know within a power of 10 what is out there. Do we > include various methods of recovery like secondary, tertiary, various > injection methods in our estimates? What fields being explored now may > produce? We only have rough estimates on the size of fields. Yet oil > companies have to report to the government the value of these reserves to > the dollar. > > On Sat, Aug 12, 2017 at 8:43 PM, Don Kelly <[email protected]> wrote: > > > I think that Don Guinn was referring to the fact that the output results > > from input of n sig figs will not be good to anything better than n sig > > figs. You have pointed out limits in the representation of base 10 > > integers in a base 2 system. In fact, the results of multiplying 2 4 > digit > > numbers might result in an overflow or conversion to float with > truncation. > > > > Don Kelly > > On 2017-08-12 12:19 PM, 'Bo Jacoby' via Programming wrote: > > > >> Don Guinn wrote: "But few things need precision beyond 16 significant > >> digits". Well, just computing the determinant of a 4*4 matrix of > 4-figure > >> integers require 16 digit precision! > >> > >> Den 1:52 lørdag den 12. august 2017 skrev Don Kelly <[email protected] > >: > >> > >> Right on! > >> > >> In most numerical operations on a computer, there is an inherent > >> propagation of errors (in fact Numerical analysis texts spend a lot of > >> effort on ways to reduce such errors) and 16 or more digits don't > >> provide precision greater than that of the input data but do reduce the > >> computational fuzz to an insignificant level. The ideal kit for a > >> student using a hand calculator would be a strip of electrical tape to > >> cover the extra digits. > >> > >> Don Kelly > >> > >> > >> On 2017-08-11 6:33 AM, Don Guinn wrote: > >> > >>> We too often assume that calculations carried out to 16 significant > >>> digits > >>> are accurate when the input may not be known to less than 2 or 3 > >>> significant digits. We can calculate the distance to the moon > accurately > >>> to > >>> the wavelength of visible light in IEEE floating point. We can > calculate > >>> the national debt to the penny. Maybe calculating relativistic effects > >>> on a > >>> satellite orbiting the earth might exceed IEEE floating point. But few > >>> things need precision beyond 16 significant digits. > >>> > >>> On Thu, Aug 10, 2017 at 11:14 PM, Don Kelly <[email protected]> wrote: > >>> > >>> isn't an advantage of APL and J that the person writing a > >>>> program/app/whatever, doesn't have to deal with the distinctions > >>>> between > >>>> integer and damn near integer within the limitations of the computer > >>>> binary resolution?. In most cases this isa good thing because close > >>>> enough > >>>> -given the +/- of data input is sufficient for the idiot box to > decide. > >>>> J > >>>> moves away from C/C++/ and other languages which often seem to be > >>>> emphasizing stuff that Iverson tried to eliminate in APL and J . Muh > of > >>>> that stuff is something that can be handled by the idiot box so that: > >>>> > >>>> Problem-->basic analysis--. coding that fits the analysis rather than > >>>> the > >>>> details( users aim at the essentials rather than the details- "/I > want > >>>> the > >>>> answer and I dont care about what is involved in the background of %,* > >>>> */ > >>>> > >>>> The discussion below deals with representation of numeric values being > >>>> floating point or integer when pushing the limits-IS IT IMPORTANT IN > THE > >>>> REAL WORLD unless you have a Cray in the back bedroom? > >>>> > >>>> Old fart expressing opinions > >>>> > >>>> Don Kelly > >>>> > >>>> On 2017-08-10 6:27 PM, Bill wrote: > >>>> > >>>> I suspect J interpreter didn't has the knowledge that the original > >>>>> string had been 9999999999999999.3 > >>>>> with .3 because what J saw was the floating point result of parsing > by > >>>>> c > >>>>> library. Ieee floating point has 15 to 16 significant digits so that > >>>>> 1e16 > >>>>> and 1e16-1 is the same number. > >>>>> > >>>>> Perhaps one could use long double to parse number on J64. > >>>>> > >>>>> Sent from my iPhone > >>>>> > >>>>> On 10 Aug, 2017, at 3:48 AM, Henry Rich <[email protected]> > wrote: > >>>>> > >>>>> Quite right. > >>>>> > >>>>>> Henry Rich > >>>>>> > >>>>>> On Aug 9, 2017 20:46, "Raul Miller" <[email protected]> wrote: > >>>>>> > >>>>>> Well, since it's encoded as an integer (which I would have noticed > if > >>>>>> > >>>>>>> I had read Bob Therriault's original post more closely), and not > >>>>>>> [like > >>>>>>> I was thinking] a float, I agree that dropping the .3 is better > than > >>>>>>> adding a 1. > >>>>>>> > >>>>>>> That said, I guess we also should not object too loudly if > >>>>>>> 9999999999999999.3 were instead encoded the same as > >>>>>>> 9999999999999999+0.3 gets encoded. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> -- > >>>>>>> Raul > >>>>>>> > >>>>>>> On Wed, Aug 9, 2017 at 3:25 PM, Henry Rich <[email protected]> > >>>>>>> wrote: > >>>>>>> > >>>>>>> Surely integer 999...9 is a better value than 1000...0 . > >>>>>>>> > >>>>>>>> Henry Rich > >>>>>>>> > >>>>>>>> On Aug 9, 2017 18:33, "Raul Miller" <[email protected]> > wrote: > >>>>>>>> > >>>>>>>> It's not a bug, it's an artifact of the 64 bit floating point > >>>>>>>> standard. > >>>>>>>> > >>>>>>>>> 2 ^.9999999999999999 > >>>>>>>>> 53.1508 > >>>>>>>>> > >>>>>>>>> https://en.wikipedia.org/wiki/IEEE_754#Basic_and_interchange > >>>>>>>>> _formats > >>>>>>>>> > >>>>>>>>> The binary64 format has 53 binary digits or 15.95 decimal digits. > >>>>>>>>> This > >>>>>>>>> means ".16#'9' cannot be represented exactly using this format. > >>>>>>>>> > >>>>>>>>> And, we do not use exact representation of large numbers by > default > >>>>>>>>> because that's too slow for large datasets. Put differently, if > you > >>>>>>>>> want exact representation and are willing to take the performance > >>>>>>>>> hit, > >>>>>>>>> you should specify that. For example: ".'x',~16#'9' or > >>>>>>>>> ".'3r10+','x',~16#'9' > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> Raul > >>>>>>>>> > >>>>>>>>> On Wed, Aug 9, 2017 at 12:05 PM, Henry Rich < > [email protected]> > >>>>>>>>> > >>>>>>>>> wrote: > >>>>>>>> This is a bug, since 999...9.3 should become 999...9 rather than > >>>>>>>> > >>>>>>>>> 100...0. > >>>>>>>>> > >>>>>>>>> I'm away from home now, but I think what's happening is this: > >>>>>>>>>> > >>>>>>>>>> 999...9 is converted to integer > >>>>>>>>>> > >>>>>>>>>> . is encountered and turns it to float > >>>>>>>>>> > >>>>>>>>>> It's rounded to the nearest float which is 100...0 > >>>>>>>>>> > >>>>>>>>>> As a final step the JE checks to see if the value is exactly > >>>>>>>>>> integral, > >>>>>>>>>> which it is, and it is converted back to integer. > >>>>>>>>>> > >>>>>>>>>> If you add this to Interpreter/Bugs I'll fix it when i get back. > >>>>>>>>>> > >>>>>>>>>> Henry Rich > >>>>>>>>>> > >>>>>>>>>> On Aug 9, 2017 16:16, "bill lam" <[email protected]> wrote: > >>>>>>>>>> > >>>>>>>>>> I think this is the difference between 32 and 64-bit, > >>>>>>>>>> > >>>>>>>>>> 9!:14'' > >>>>>>>>>> j602/2008-03-03/16:45 > >>>>>>>>>> 3!:0[ 9999999999999999.3 > >>>>>>>>>> 4 > >>>>>>>>>> > >>>>>>>>>> In J32 > >>>>>>>>>> > >>>>>>>>>> a.i. 2 fc 9999999999999999.3 > >>>>>>>>>> 0 128 224 55 121 195 65 67 > >>>>>>>>>> a.i. 2 fc 1e16 > >>>>>>>>>> 0 128 224 55 121 195 65 67 > >>>>>>>>>> > >>>>>>>>>> the number has the same bit pattern as 1e16 (an integer) > >>>>>>>>>> which can be represented as a 64-bit integer. I guess > >>>>>>>>>> J64 is correct since 9999999999999999.3 and 1e16 is the > >>>>>>>>>> same number in ieee fp and J prefers integer to floats, > >>>>>>>>>> eg > >>>>>>>>>> 3!:0 [ 2.0 > >>>>>>>>>> 4 > >>>>>>>>>> > >>>>>>>>>> Ср, 09 авг 2017, robert therriault написал(а): > >>>>>>>>>> > >>>>>>>>>> Hi Pascal, > >>>>>>>>>>> > >>>>>>>>>>> I see the same behaviour in j806 as j805. Do you see something > >>>>>>>>>>> > >>>>>>>>>>> different? > >>>>>>>>>> JVERSION > >>>>>>>>>> > >>>>>>>>>>> Engine: j806/j64avx/darwin > >>>>>>>>>>> Beta-4: commercial/2017-06-27T12:55:06 > >>>>>>>>>>> Library: 8.06.03 > >>>>>>>>>>> Qt IDE: 1.5.3/5.6.2 > >>>>>>>>>>> Platform: Darwin 64 > >>>>>>>>>>> Installer: J806 install > >>>>>>>>>>> InstallPath: /users/bobtherriault/j64-806 > >>>>>>>>>>> Contact: www.jsoftware.com > >>>>>>>>>>> (; datatype) 999999999999999.3 > >>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>> │1e15│floating│ > >>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>> (; datatype) 9999999999999999.3 > >>>>>>>>>>> ┌─────────────────┬───────┐ > >>>>>>>>>>> │10000000000000000│integer│ > >>>>>>>>>>> └─────────────────┴───────┘ > >>>>>>>>>>> (; datatype) 99999999999999999.3 > >>>>>>>>>>> ┌──────────────────┬───────┐ > >>>>>>>>>>> │100000000000000000│integer│ > >>>>>>>>>>> └──────────────────┴───────┘ > >>>>>>>>>>> (; datatype) 999999999999999999.3 > >>>>>>>>>>> ┌───────────────────┬───────┐ > >>>>>>>>>>> │1000000000000000000│integer│ > >>>>>>>>>>> └───────────────────┴───────┘ > >>>>>>>>>>> (; datatype) 9999999999999999999.3 > >>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>> │1e19│floating│ > >>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>> > >>>>>>>>>>> Cheers, bob > >>>>>>>>>>> > >>>>>>>>>>> On Aug 9, 2017, at 7:54 AM, 'Pascal Jasmin' via Programming < > >>>>>>>>>>> [email protected]> wrote: > >>>>>>>>>>> in j806, 9999999999999999.310000000000000000 probably the > >>>>>>>>>>> j805 > >>>>>>>>>>> behaviour is preferred. If only for consistency. But there > may > >>>>>>>>>>> be > >>>>>>>>>>> > >>>>>>>>>> a > >>>>>>>>>> > >>>>>>>>>> good > >>>>>>>>> > >>>>>>>>> reason for change. > >>>>>>>>>> > >>>>>>>>>> From: robert therriault <[email protected]> > >>>>>>>>>>> > >>>>>>>>>>>> To: Programming forum <[email protected]> > >>>>>>>>>>>> Sent: Wednesday, August 9, 2017 10:40 AM > >>>>>>>>>>>> Subject: [Jprogramming] Integer-floating type change for large > >>>>>>>>>>>> > >>>>>>>>>>>> numbers > >>>>>>>>>>> > >>>>>>>>>> in j805 and j806 > >>>>>>>> > >>>>>>>>> I am guessing that the following has something to do with > >>>>>>>>>>> precision of > >>>>>>>>>>> > >>>>>>>>>> large numbers in j805 and is also true for j806. > >>>>>>>> > >>>>>>>>> (; datatype) 999999999999999.3 > >>>>>>>>>>> > >>>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>>> │1e15│floating│ > >>>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>>> (; datatype) 9999999999999999.3 > >>>>>>>>>>>> ┌─────────────────┬───────┐ > >>>>>>>>>>>> │10000000000000000│integer│ > >>>>>>>>>>>> └─────────────────┴───────┘ > >>>>>>>>>>>> (; datatype) 99999999999999999.3 > >>>>>>>>>>>> ┌──────────────────┬───────┐ > >>>>>>>>>>>> │100000000000000000│integer│ > >>>>>>>>>>>> └──────────────────┴───────┘ > >>>>>>>>>>>> (; datatype) 999999999999999999.3 > >>>>>>>>>>>> ┌───────────────────┬───────┐ > >>>>>>>>>>>> │1000000000000000000│integer│ > >>>>>>>>>>>> └───────────────────┴───────┘ > >>>>>>>>>>>> (; datatype) 9999999999999999999.3 > >>>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>>> │1e19│floating│ > >>>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>>> JVERSION > >>>>>>>>>>>> Engine: j805/j64/darwin > >>>>>>>>>>>> Release: commercial/2016-12-11T08:17:56 > >>>>>>>>>>>> Library: 8.05.14 > >>>>>>>>>>>> Qt IDE: 1.5.4/5.6.2 > >>>>>>>>>>>> Platform: Darwin 64 > >>>>>>>>>>>> Installer: J805 install > >>>>>>>>>>>> InstallPath: /applications/j64-805 > >>>>>>>>>>>> Contact: www.jsoftware.com > >>>>>>>>>>>> > >>>>>>>>>>>> Further investigation shows me it was not this way with the 32 > >>>>>>>>>>>> bit > >>>>>>>>>>>> > >>>>>>>>>>>> version of j701, so it may be an artifact of moving to 64 bit? > >>>>>>>>>>> (; datatype) 999999999999999.3 > >>>>>>>>>>> > >>>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>>> │1e15│floating│ > >>>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>>> (; datatype) 9999999999999999.3 > >>>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>>> │1e16│floating│ > >>>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>>> (; datatype) 999999999999999999.3 > >>>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>>> │1e18│floating│ > >>>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>>> (; datatype) 9999999999999999999.3 > >>>>>>>>>>>> ┌────┬────────┐ > >>>>>>>>>>>> │1e19│floating│ > >>>>>>>>>>>> └────┴────────┘ > >>>>>>>>>>>> JVERSION > >>>>>>>>>>>> Engine: j701/2011-01-10/11:25 > >>>>>>>>>>>> Library: 7.01.088 > >>>>>>>>>>>> Platform: Darwin 32 > >>>>>>>>>>>> Installer: j701a_mac_intel.dmg > >>>>>>>>>>>> InstallPath: /Applications/j701 > >>>>>>>>>>>> > >>>>>>>>>>>> Cheers, bob > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>>> ------------------------------------------------------------ > >>>>>>>>>>>> > >>>>>>>>>>>> ---------- > >>>>>>>>>>> > >>>>>>>>>> For information about J forums see http://www.jsoftware.com/ > >>>>>>>>>> > >>>>>>>>>>> forums.htm > >>>>>>>>>>> > >>>>>>>>>>>> ------------------------------------------------------------ > >>>>>>>>>>>> > >>>>>>>>>>>> ---------- > >>>>>>>>>>> > >>>>>>>>>> For information about J forums see http://www.jsoftware.com/ > >>>>>>>>>> > >>>>>>>>>>> forums.htm > >>>>>>>>>>> > >>>>>>>>>> ------------------------------------------------------------ > >>>>>>>>>> ---------- > >>>>>>>>>> > >>>>>>>>> For information about J forums see http://www.jsoftware.com/ > >>>>>>>> > >>>>>>>>> forums.htm > >>>>>>>>>> > >>>>>>>>> -- > >>>>>>>> > >>>>>>>>> regards, > >>>>>>>>>> ==================================================== > >>>>>>>>>> GPG key 1024D/4434BAB3 2008-08-24 > >>>>>>>>>> gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3 > >>>>>>>>>> gpg --keyserver subkeys.pgp.net --armor --export 4434BAB3 > >>>>>>>>>> ------------------------------------------------------------ > >>>>>>>>>> > >>>>>>>>>> ---------- > >>>>>>>>> > >>>>>>>> For information about J forums see http://www.jsoftware.com/ > >>>>>>>> > >>>>>>>>> forums.htm > >>>>>>>>> > >>>>>>>> ------------------------------------------------------------ > >>>>>>>> > >>>>>>>>> ---------- > >>>>>>>>> > >>>>>>>> For information about J forums see http://www.jsoftware.com/ > >>>>>>>> > >>>>>>>>> forums.htm > >>>>>>>>> > >>>>>>>> ------------------------------------------------------------ > >>>>>>>> ---------- > >>>>>>>> > >>>>>>>>> For information about J forums see > http://www.jsoftware.com/forum > >>>>>>>>> s.htm > >>>>>>>>> > >>>>>>>>> ------------------------------------------------------------ > >>>>>>>> ---------- > >>>>>>>> For information about J forums see http://www.jsoftware.com/forum > >>>>>>>> s.htm > >>>>>>>> > >>>>>>>> ------------------------------------------------------------ > >>>>>>> ---------- > >>>>>>> For information about J forums see http://www.jsoftware.com/forum > >>>>>>> s.htm > >>>>>>> > >>>>>>> ------------------------------------------------------------ > >>>>>> ---------- > >>>>>> For information about J forums see http://www.jsoftware.com/forum > >>>>>> s.htm > >>>>>> > >>>>>> ------------------------------------------------------------ > >>>>> ---------- > >>>>> For information about J forums see http://www.jsoftware.com/ > forums.htm > >>>>> > >>>>> ------------------------------------------------------------ > ---------- > >>>> For information about J forums see http://www.jsoftware.com/ > forums.htm > >>>> > >>> ---------------------------------------------------------------------- > >>> For information about J forums see http://www.jsoftware.com/forums.htm > >>> > >> ---------------------------------------------------------------------- > >> For information about J forums see http://www.jsoftware.com/forums.htm > >> > >> ------------------------------------------------------------ > >> ---------- > >> For information about J forums see http://www.jsoftware.com/forums.htm > >> > > > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
