Re: numberformat question

Thank you. I will take a close look at it. Meanwhile I have finetuned my LC script : it returns the same results as the js script, and according to my tests, solves 1000 cubic equations in roughly 0.8 second. Le 2021-03-23 07:14, Andreas Bergendal via use-livecode a écrit : A few years back I

Re: numberformat question

A few years back I needed calculation to a precision of 300 decimals. I found a js solution by -hh in the forum, which solved my need: https://forums.livecode.com/viewtopic.php?f=93=29018=30 It runs in a browser widget, with easy input

Re: numberformat question

Le 2021-03-22 12:14, John Balgenorth via use-livecode a écrit : You might be able to get the info by running your javascript from a shell command. Here is a link that might be of help; https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line

Re: numberformat question

The ingenuity of people on this list never ceases to amaze me. Bob S > On Mar 22, 2021, at 6:41 AM, Craig Newman via use-livecode > wrote: > > Check out this thread in the forum: > > https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985 > >

Re: numberformat question

On 3/22/21 9:39 AM, Sean Cole (Pi) via use-livecode wrote: JS should use exactly the same standard as LC https://www.avioconsulting.com/blog/overcoming-javascript-numeric-precision-issues 2^53, same in JS as LC. IEEE 754 Right. To get any more precision you'll have to stringify the

Re: numberformat question

JS should use exactly the same standard as LC https://www.avioconsulting.com/blog/overcoming-javascript-numeric-precision-issues 2^53, same in JS as LC. IEEE 754 Sean On Mon, 22 Mar 2021 at 16:14, John Balgenorth via use-livecode < use-livecode@lists.runrev.com> wrote: > You might be able to

Re: numberformat question

You might be able to get the info by running your javascript from a shell command. Here is a link that might be of help; https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line JB > On Mar 22, 2021, at 9:05 AM, jbv via use-livecode > wrote:

Re: numberformat question

Le 2021-03-22 12:03, jbv via use-livecode a écrit : Hi Craig, Thank you for the link, but it doesn't help much : bigDivide(-0.007347880794884118999,10) returns 0 , -0.01 Ooops, I just realized that I forgot to add set numberformat to "0.##" at

Re: numberformat question

Hi Craig, Thank you for the link, but it doesn't help much : bigDivide(-0.007347880794884118999,10) returns 0 , -0.01 what I'm actually trying to do is to transpose in LC an algo for solving cubic equations that runs perfectly in javascript. So my solution will finally be to

Re: numberformat question

Check out this thread in the forum: https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985 Craig > On Mar 21, 2021, at 7:34 AM, jbv via use-livecode > wrote: > > Paul, Sean, > > Thank you both

Re: numberformat question

Paul, Sean, Thank you both for your answers, although it doesn't really solve my problem. The most surprising thing is that javascript runs these calculations without problem : (-0.007347880794884118999 / 10) returns -7.347880794884119e-16 (-0.007347880794884118999 /

Re: numberformat question

Livecode can handle floating point numbers up to 2^52. But then calculations would have to fit within that range in a floating point perspective too. So, once you add that ninth zero to the end of your divider, it struggles with the calculation and returns, effectively, number out of range.

Re: numberformat question

I don't have an answer, other than to say there has been a LOT of discussion over the years about the limits of number precision in LiveCode and a number of enhancement requests and 3rd party libraries that have attempted to provide a framework for high precision math. You might search the

Re: numberFormat question

David, The numberformat doesn't affect precision. It only affects output as is shown by on mouseUp set the numberformat to 00 put 0.01*1 into x set the numberformat to 00.00 put x*1 end mouseUp It even only affects the current handler and doesn't affect precision in calling

Re: numberFormat question

On 6/16/2014, 1:13 PM, dfepst...@comcast.net wrote: I am trying to set the numberFormat so that calculation is precise enough for the situation. Since I don't know ahead of time how many decimal places will be used, I wrote a function that I hoped would adjust things as necessary. But when

Re: numberFormat question

Also, if you don’t care about display, just set the *precision* to LC max. Numberformat is as others have posted, really a display function. After you are done, set the numberFormat back to default or “0”. I bit myself in the butt because I had a counter that I was using to append a number to a

Re: numberFormat question

On Tue, Jun 17, 2014 at 4:34 AM, J. Landman Gay jac...@hyperactivesw.com wrote: set the numberformat to 0.## -- add more if you want I think the reason David isn't doing that, and is trying to limit the number of # to the absolute minimum is because of this note in the Dictionary: