Re: numberformat question

Thank you. I will take a close look at it. Meanwhile I have finetuned my LC script : it returns the same results as the js script, and according to my tests, solves 1000 cubic equations in roughly 0.8 second. Le 2021-03-23 07:14, Andreas Bergendal via use-livecode a écrit : A few years back I

Re: numberformat question

A few years back I needed calculation to a precision of 300 decimals. I found a js solution by -hh in the forum, which solved my need: https://forums.livecode.com/viewtopic.php?f=93&t=29018&start=30 It runs in a browser widget, wi

Re: numberformat question

Le 2021-03-22 12:14, John Balgenorth via use-livecode a écrit : You might be able to get the info by running your javascript from a shell command. Here is a link that might be of help; https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line T

Re: numberformat question

The ingenuity of people on this list never ceases to amaze me. Bob S > On Mar 22, 2021, at 6:41 AM, Craig Newman via use-livecode > wrote: > > Check out this thread in the forum: > > https://forums.livecode.com/viewtopic.php?f=7&t=25132&p=130985&hilit=longadd#p130985 > >

Re: numberformat question

On 3/22/21 9:39 AM, Sean Cole (Pi) via use-livecode wrote: JS should use exactly the same standard as LC https://www.avioconsulting.com/blog/overcoming-javascript-numeric-precision-issues 2^53, same in JS as LC. IEEE 754 Right. To get any more precision you'll have to stringify the calculatio

Re: numberformat question

JS should use exactly the same standard as LC https://www.avioconsulting.com/blog/overcoming-javascript-numeric-precision-issues 2^53, same in JS as LC. IEEE 754 Sean On Mon, 22 Mar 2021 at 16:14, John Balgenorth via use-livecode < use-livecode@lists.runrev.com> wrote: > You might be able to ge

Re: numberformat question

You might be able to get the info by running your javascript from a shell command. Here is a link that might be of help; https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line JB > On Mar 22, 2021, at 9:05 AM, jbv via use-livecode > wrote: >

Re: numberformat question

Le 2021-03-22 12:03, jbv via use-livecode a écrit : Hi Craig, Thank you for the link, but it doesn't help much : bigDivide(-0.007347880794884118999,10) returns 0 , -0.01 Ooops, I just realized that I forgot to add set numberformat to "0.##" at the

Re: numberformat question

Hi Craig, Thank you for the link, but it doesn't help much : bigDivide(-0.007347880794884118999,10) returns 0 , -0.01 what I'm actually trying to do is to transpose in LC an algo for solving cubic equations that runs perfectly in javascript. So my solution will finally be to mul

Re: numberformat question

Check out this thread in the forum: https://forums.livecode.com/viewtopic.php?f=7&t=25132&p=130985&hilit=longadd#p130985 Craig > On Mar 21, 2021, at 7:34 AM, jbv via use-livecode > wrote: > > Paul, Sean,

Re: numberformat question

Paul, Sean, Thank you both for your answers, although it doesn't really solve my problem. The most surprising thing is that javascript runs these calculations without problem : (-0.007347880794884118999 / 10) returns -7.347880794884119e-16 (-0.007347880794884118999 / 1

Re: numberformat question

Livecode can handle floating point numbers up to 2^52. But then calculations would have to fit within that range in a floating point perspective too. So, once you add that ninth zero to the end of your divider, it struggles with the calculation and returns, effectively, number out of range. Fracti

Re: numberformat question

I don't have an answer, other than to say there has been a LOT of discussion over the years about the limits of number precision in LiveCode and a number of enhancement requests and 3rd party libraries that have attempted to provide a framework for high precision math. You might search the list

numberformat question

Hi list, I am a bit lost : set numberformat to ".#" put -0.007347880794884118999 / 1000 -- returns -0.07347880794884119 which is ok put -0.007347880794884118999 / 100 -- returns -0.7347880794884 -> also ok put -0.007347880

Re: numberFormat question

On Tue, Jun 17, 2014 at 4:34 AM, J. Landman Gay wrote: > set the numberformat to "0.##" -- add more if you want I think the reason David isn't doing that, and is trying to limit the number of # to the absolute minimum is because of this note in the Dictionary: Note: Since LiveCode

Re: numberFormat question

Also, if you don’t care about display, just set the *precision* to LC max. Numberformat is as others have posted, really a display function. After you are done, set the numberFormat back to default or “0”. I bit myself in the butt because I had a counter that I was using to append a number to a

Re: numberFormat question

On 6/16/2014, 1:13 PM, dfepst...@comcast.net wrote: I am trying to set the numberFormat so that calculation is precise enough for the situation. Since I don't know ahead of time how many decimal places will be used, I wrote a function that I hoped would adjust things as necessary. But when t

Re: numberFormat question

David, The numberformat doesn't affect precision. It only affects output as is shown by on mouseUp set the numberformat to "00" put 0.01*1 into x set the numberformat to "00.00" put x*1 end mouseUp It even only affects the current handler and doesn't affect precision in calling h

numberFormat question

I am trying to set the numberFormat so that calculation is precise enough for the situation.  Since I don't know ahead of time how many decimal places will be used, I wrote a function that I hoped would adjust things as necessary. But when this function is called with parameters m = 1.09131 an