Re: numberformat question

[jbv via use-livecode]

Thank you.
I will take a close look at it.

Meanwhile I have finetuned my LC script : it returns the same
results as the js script, and according to my tests, solves
1000 cubic equations in roughly 0.8 second.


Le 2021-03-23 07:14, Andreas Bergendal via use-livecode a écrit :

A few years back I needed calculation to a precision of 300 decimals.
I found a js solution by -hh in the forum, which solved my need:
https://forums.livecode.com/viewtopic.php?f=93=29018=30


It runs in a browser widget, with easy input from and output to LC
script. I don’t know the precision limit, but it’s way more than 300
IIRC.

Maybe useful here?

/Andreas
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Andreas Bergendal via use-livecode]

A few years back I needed calculation to a precision of 300 decimals. I found a 
js solution by -hh in the forum, which solved my need:
https://forums.livecode.com/viewtopic.php?f=93=29018=30 


It runs in a browser widget, with easy input from and output to LC script. I 
don’t know the precision limit, but it’s way more than 300 IIRC.

Maybe useful here?

/Andreas
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[jbv via use-livecode]

Le 2021-03-22 12:14, John Balgenorth via use-livecode a écrit :

You might be able to get the info by running your javascript
from a shell command.  Here is a link that might be of help;

https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line



Thank you, but if I run javascript, I would prefer to use a
browser objet and revBrowserCallScript...
BTW, as I haven't used browser controls in a long time, am I
forced to use a distant url, or can I point to some html + js
stored in my stack as a field or even a variable ?

Thanks in advance.

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Bob Sneidar via use-livecode]

The ingenuity of people on this list never ceases to amaze me. 

Bob S


> On Mar 22, 2021, at 6:41 AM, Craig Newman via use-livecode 
>  wrote:
> 
> Check out this thread in the forum:
> 
> https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985
>  
> 
> 
> Craig
> 
>> On Mar 21, 2021, at 7:34 AM, jbv via use-livecode 
>>  wrote:
>> 
>> Paul, Sean,
>> 
>> Thank you both for your answers, although it doesn't really solve my problem.
>> The most surprising thing is that javascript runs these calculations without
>> problem :
>> (-0.007347880794884118999 / 10) returns -7.347880794884119e-16
>> (-0.007347880794884118999 / 1) returns -7.347880794884119e-19
>> 
>> Best
>> 
>> 
>> Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :
>>> Livecode can handle floating point numbers up to 2^52. But then
>>> calculations would have to fit within that range in a floating point
>>> perspective too. So, once you add that ninth zero to the end of your
>>> divider, it struggles with the calculation and returns, effectively, number
>>> out of range.
>>> Fractions in FP binary are tremendously complicated. Calculation in it are
>>> even more so.
>>> https://www.h-schmidt.net/FloatConverter/IEEE754.html
>>> On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
>>> use-livecode@lists.runrev.com> wrote:
 I don't have an answer, other than to say there has been a LOT of
 discussion over the years about the limits of number precision in
 LiveCode and a number of enhancement requests and 3rd party libraries
 that have attempted to provide a framework for high precision math. You
 might search the list archives or search the forum on the LC web site
 for past discussions about number precision.
 On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
> Hi list,
> I am a bit lost :
> 
>   set numberformat to ".#"
>   put -0.007347880794884118999 / 1000
> -- returns -0.07347880794884119  which is ok
> 
>   put -0.007347880794884118999 / 100
> -- returns -0.7347880794884 -> also ok
> 
>   put -0.007347880794884118999 / 1
> -- returns -0.0073478807949 -> still ok
> 
>   put -0.007347880794884118999 / 10
> -- returns 0 -> ?
> 
> Any explanations ?
> Thanks in advance.
> Best,
> jbv
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
 ___
 use-livecode mailing list
 use-livecode@lists.runrev.com
 Please visit this url to subscribe, unsubscribe and manage your
 subscription preferences:
 http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription 
>> preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Mark Wieder via use-livecode]

On 3/22/21 9:39 AM, Sean Cole (Pi) via use-livecode wrote:

JS should use exactly the same standard as LC
https://www.avioconsulting.com/blog/overcoming-javascript-numeric-precision-issues

2^53, same in JS as LC. IEEE 754


Right. To get any more precision you'll have to stringify the 
calculations, and Craig has already posted a link to MaxV's excellent 
set of functions for this.


Someday maybe we'll get that decimal math library shoehorned into place 
and then we can look back on these days and laugh.


--
 Mark Wieder
 ahsoftw...@gmail.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Sean Cole (Pi) via use-livecode]

JS should use exactly the same standard as LC
https://www.avioconsulting.com/blog/overcoming-javascript-numeric-precision-issues

2^53, same in JS as LC. IEEE 754

Sean

On Mon, 22 Mar 2021 at 16:14, John Balgenorth via use-livecode <
use-livecode@lists.runrev.com> wrote:

> You might be able to get the info by running your javascript
> from a shell command.  Here is a link that might be of help;
>
>
> https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line
>
> JB
>
>
> > On Mar 22, 2021, at 9:05 AM, jbv via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >
> > Hi Craig,
> >
> > Thank you for the link, but it doesn't help much :
> >
> > bigDivide(-0.007347880794884118999,10)
> > returns 0 , -0.01
> >
> > what I'm actually trying to do is to transpose in LC
> > an algo for solving cubic equations that runs perfectly
> > in javascript.
> > So my solution will finally be to multiply data by 10^6,
> > run the algo, and then divide the results by 10^6 by
> > treating them as strings and moving the dot...
> > It's not really elegant and especially quite slow, as I
> > need to process large amounts of data and am not sure
> > that I'll be able to use LC for that project (which is
> > so far only a prototype).
> >
> > I don't mean to start another endless thread on this topic,
> > but one thing puzzles me though : I understand that LC doesn't
> > use decimal numbers for internal calculations for reasons
> > of speed, but then how come does js perform high precision
> > calculations without any speed issues ?
> >
> > Best
> >
> > Le 2021-03-22 09:41, Craig Newman via use-livecode a écrit :
> >> Check out this thread in the forum:
> >>
> https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985
> >> <
> https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985
> >
> >> Craig
>  On Mar 21, 2021, at 7:34 AM, jbv via use-livecode <
> use-livecode@lists.runrev.com> wrote:
> >>> Paul, Sean,
> >>> Thank you both for your answers, although it doesn't really solve my
> problem.
> >>> The most surprising thing is that javascript runs these calculations
> without
> >>> problem :
> >>> (-0.007347880794884118999 / 10) returns
> -7.347880794884119e-16
> >>> (-0.007347880794884118999 / 1) returns
> -7.347880794884119e-19
> >>> Best
> >>> Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :
>  Livecode can handle floating point numbers up to 2^52. But then
>  calculations would have to fit within that range in a floating point
>  perspective too. So, once you add that ninth zero to the end of your
>  divider, it struggles with the calculation and returns, effectively,
> number
>  out of range.
>  Fractions in FP binary are tremendously complicated. Calculation in
> it are
>  even more so.
>  https://www.h-schmidt.net/FloatConverter/IEEE754.html
>  On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
>  use-livecode@lists.runrev.com> wrote:
> > I don't have an answer, other than to say there has been a LOT of
> > discussion over the years about the limits of number precision in
> > LiveCode and a number of enhancement requests and 3rd party libraries
> > that have attempted to provide a framework for high precision math.
> You
> > might search the list archives or search the forum on the LC web site
> > for past discussions about number precision.
> > On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
> > > Hi list,
> > > I am a bit lost :
> > >
> > >set numberformat to ".#"
> > >put -0.007347880794884118999 / 1000
> > > -- returns -0.07347880794884119  which is ok
> > >
> > >put -0.007347880794884118999 / 100
> > > -- returns -0.7347880794884 -> also ok
> > >
> > >put -0.007347880794884118999 / 1
> > > -- returns -0.0073478807949 -> still ok
> > >
> > >put -0.007347880794884118999 / 10
> > > -- returns 0 -> ?
> > >
> > > Any explanations ?
> > > Thanks in advance.
> > > Best,
> > > jbv
> > >
> > > ___
> > > use-livecode mailing list
> > > use-livecode@lists.runrev.com
> > > Please visit this url to subscribe, unsubscribe and manage your
> > > subscription preferences:
> > > http://lists.runrev.com/mailman/listinfo/use-livecode
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
> >>> ___
> >>> use-livecode mailing list
> >>> use-livecode@lists.runrev.com
> >>> 

Re: numberformat question

[John Balgenorth via use-livecode]

You might be able to get the info by running your javascript
from a shell command.  Here is a link that might be of help;

https://stackoverflow.com/questions/12659778/is-it-possible-to-run-javascript-files-from-the-command-line

JB


> On Mar 22, 2021, at 9:05 AM, jbv via use-livecode 
>  wrote:
> 
> Hi Craig,
> 
> Thank you for the link, but it doesn't help much :
> 
> bigDivide(-0.007347880794884118999,10)
> returns 0 , -0.01
> 
> what I'm actually trying to do is to transpose in LC
> an algo for solving cubic equations that runs perfectly
> in javascript.
> So my solution will finally be to multiply data by 10^6,
> run the algo, and then divide the results by 10^6 by
> treating them as strings and moving the dot...
> It's not really elegant and especially quite slow, as I
> need to process large amounts of data and am not sure
> that I'll be able to use LC for that project (which is
> so far only a prototype).
> 
> I don't mean to start another endless thread on this topic,
> but one thing puzzles me though : I understand that LC doesn't
> use decimal numbers for internal calculations for reasons
> of speed, but then how come does js perform high precision
> calculations without any speed issues ?
> 
> Best
> 
> Le 2021-03-22 09:41, Craig Newman via use-livecode a écrit :
>> Check out this thread in the forum:
>> https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985
>> 
>> Craig
 On Mar 21, 2021, at 7:34 AM, jbv via use-livecode 
  wrote:
>>> Paul, Sean,
>>> Thank you both for your answers, although it doesn't really solve my 
>>> problem.
>>> The most surprising thing is that javascript runs these calculations without
>>> problem :
>>> (-0.007347880794884118999 / 10) returns -7.347880794884119e-16
>>> (-0.007347880794884118999 / 1) returns 
>>> -7.347880794884119e-19
>>> Best
>>> Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :
 Livecode can handle floating point numbers up to 2^52. But then
 calculations would have to fit within that range in a floating point
 perspective too. So, once you add that ninth zero to the end of your
 divider, it struggles with the calculation and returns, effectively, number
 out of range.
 Fractions in FP binary are tremendously complicated. Calculation in it are
 even more so.
 https://www.h-schmidt.net/FloatConverter/IEEE754.html
 On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
 use-livecode@lists.runrev.com> wrote:
> I don't have an answer, other than to say there has been a LOT of
> discussion over the years about the limits of number precision in
> LiveCode and a number of enhancement requests and 3rd party libraries
> that have attempted to provide a framework for high precision math. You
> might search the list archives or search the forum on the LC web site
> for past discussions about number precision.
> On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
> > Hi list,
> > I am a bit lost :
> >
> >set numberformat to ".#"
> >put -0.007347880794884118999 / 1000
> > -- returns -0.07347880794884119  which is ok
> >
> >put -0.007347880794884118999 / 100
> > -- returns -0.7347880794884 -> also ok
> >
> >put -0.007347880794884118999 / 1
> > -- returns -0.0073478807949 -> still ok
> >
> >put -0.007347880794884118999 / 10
> > -- returns 0 -> ?
> >
> > Any explanations ?
> > Thanks in advance.
> > Best,
> > jbv
> >
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> ___
>>> use-livecode mailing list
>>> use-livecode@lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your 
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> ___
>> use-livecode mailing list
>> use-livecode@lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please 

Re: numberformat question

[jbv via use-livecode]

Le 2021-03-22 12:03, jbv via use-livecode a écrit :

Hi Craig,

Thank you for the link, but it doesn't help much :

bigDivide(-0.007347880794884118999,10)
returns 0 , -0.01




Ooops, I just realized that I forgot to add
set numberformat to "0.##"
at the beginning of bigDivide...

now bigDivide(-0.007347880794884118999,10)
0 , -0.00734788079488411899919975

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[jbv via use-livecode]

Hi Craig,

Thank you for the link, but it doesn't help much :

bigDivide(-0.007347880794884118999,10)
returns 0 , -0.01

what I'm actually trying to do is to transpose in LC
an algo for solving cubic equations that runs perfectly
in javascript.
So my solution will finally be to multiply data by 10^6,
run the algo, and then divide the results by 10^6 by
treating them as strings and moving the dot...
It's not really elegant and especially quite slow, as I
need to process large amounts of data and am not sure
that I'll be able to use LC for that project (which is
so far only a prototype).

I don't mean to start another endless thread on this topic,
but one thing puzzles me though : I understand that LC doesn't
use decimal numbers for internal calculations for reasons
of speed, but then how come does js perform high precision
calculations without any speed issues ?

Best

Le 2021-03-22 09:41, Craig Newman via use-livecode a écrit :

Check out this thread in the forum:

https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985


Craig

On Mar 21, 2021, at 7:34 AM, jbv via use-livecode 
 wrote:


Paul, Sean,

Thank you both for your answers, although it doesn't really solve my 
problem.
The most surprising thing is that javascript runs these calculations 
without

problem :
(-0.007347880794884118999 / 10) returns 
-7.347880794884119e-16
(-0.007347880794884118999 / 1) returns 
-7.347880794884119e-19


Best


Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :

Livecode can handle floating point numbers up to 2^52. But then
calculations would have to fit within that range in a floating point
perspective too. So, once you add that ninth zero to the end of your
divider, it struggles with the calculation and returns, effectively, 
number

out of range.
Fractions in FP binary are tremendously complicated. Calculation in 
it are

even more so.
https://www.h-schmidt.net/FloatConverter/IEEE754.html
On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
use-livecode@lists.runrev.com> wrote:

I don't have an answer, other than to say there has been a LOT of
discussion over the years about the limits of number precision in
LiveCode and a number of enhancement requests and 3rd party 
libraries
that have attempted to provide a framework for high precision math. 
You
might search the list archives or search the forum on the LC web 
site

for past discussions about number precision.
On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
> Hi list,
> I am a bit lost :
>
>set numberformat to ".#"
>put -0.007347880794884118999 / 1000
> -- returns -0.07347880794884119  which is ok
>
>put -0.007347880794884118999 / 100
> -- returns -0.7347880794884 -> also ok
>
>put -0.007347880794884118999 / 1
> -- returns -0.0073478807949 -> still ok
>
>put -0.007347880794884118999 / 10
> -- returns 0 -> ?
>
> Any explanations ?
> Thanks in advance.
> Best,
> jbv
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your 
subscription preferences:

http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Craig Newman via use-livecode]

Check out this thread in the forum:

https://forums.livecode.com/viewtopic.php?f=7=25132=130985=longadd#p130985
 


Craig

> On Mar 21, 2021, at 7:34 AM, jbv via use-livecode 
>  wrote:
> 
> Paul, Sean,
> 
> Thank you both for your answers, although it doesn't really solve my problem.
> The most surprising thing is that javascript runs these calculations without
> problem :
> (-0.007347880794884118999 / 10) returns -7.347880794884119e-16
> (-0.007347880794884118999 / 1) returns -7.347880794884119e-19
> 
> Best
> 
> 
> Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :
>> Livecode can handle floating point numbers up to 2^52. But then
>> calculations would have to fit within that range in a floating point
>> perspective too. So, once you add that ninth zero to the end of your
>> divider, it struggles with the calculation and returns, effectively, number
>> out of range.
>> Fractions in FP binary are tremendously complicated. Calculation in it are
>> even more so.
>> https://www.h-schmidt.net/FloatConverter/IEEE754.html
>> On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
>> use-livecode@lists.runrev.com> wrote:
>>> I don't have an answer, other than to say there has been a LOT of
>>> discussion over the years about the limits of number precision in
>>> LiveCode and a number of enhancement requests and 3rd party libraries
>>> that have attempted to provide a framework for high precision math. You
>>> might search the list archives or search the forum on the LC web site
>>> for past discussions about number precision.
>>> On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
>>> > Hi list,
>>> > I am a bit lost :
>>> >
>>> >set numberformat to ".#"
>>> >put -0.007347880794884118999 / 1000
>>> > -- returns -0.07347880794884119  which is ok
>>> >
>>> >put -0.007347880794884118999 / 100
>>> > -- returns -0.7347880794884 -> also ok
>>> >
>>> >put -0.007347880794884118999 / 1
>>> > -- returns -0.0073478807949 -> still ok
>>> >
>>> >put -0.007347880794884118999 / 10
>>> > -- returns 0 -> ?
>>> >
>>> > Any explanations ?
>>> > Thanks in advance.
>>> > Best,
>>> > jbv
>>> >
>>> > ___
>>> > use-livecode mailing list
>>> > use-livecode@lists.runrev.com
>>> > Please visit this url to subscribe, unsubscribe and manage your
>>> > subscription preferences:
>>> > http://lists.runrev.com/mailman/listinfo/use-livecode
>>> ___
>>> use-livecode mailing list
>>> use-livecode@lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your
>>> subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription 
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[jbv via use-livecode]

Paul, Sean,

Thank you both for your answers, although it doesn't really solve my 
problem.
The most surprising thing is that javascript runs these calculations 
without

problem :
(-0.007347880794884118999 / 10) returns 
-7.347880794884119e-16
(-0.007347880794884118999 / 1) returns 
-7.347880794884119e-19


Best


Le 2021-03-21 07:11, Sean Cole (Pi) via use-livecode a écrit :

Livecode can handle floating point numbers up to 2^52. But then
calculations would have to fit within that range in a floating point
perspective too. So, once you add that ninth zero to the end of your
divider, it struggles with the calculation and returns, effectively, 
number

out of range.

Fractions in FP binary are tremendously complicated. Calculation in it 
are

even more so.

https://www.h-schmidt.net/FloatConverter/IEEE754.html


On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
use-livecode@lists.runrev.com> wrote:


I don't have an answer, other than to say there has been a LOT of
discussion over the years about the limits of number precision in
LiveCode and a number of enhancement requests and 3rd party libraries
that have attempted to provide a framework for high precision math. 
You

might search the list archives or search the forum on the LC web site
for past discussions about number precision.

On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
> Hi list,
> I am a bit lost :
>
>set numberformat to ".#"
>put -0.007347880794884118999 / 1000
> -- returns -0.07347880794884119  which is ok
>
>put -0.007347880794884118999 / 100
> -- returns -0.7347880794884 -> also ok
>
>put -0.007347880794884118999 / 1
> -- returns -0.0073478807949 -> still ok
>
>put -0.007347880794884118999 / 10
> -- returns 0 -> ?
>
> Any explanations ?
> Thanks in advance.
> Best,
> jbv
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Sean Cole (Pi) via use-livecode]

Livecode can handle floating point numbers up to 2^52. But then
calculations would have to fit within that range in a floating point
perspective too. So, once you add that ninth zero to the end of your
divider, it struggles with the calculation and returns, effectively, number
out of range.

Fractions in FP binary are tremendously complicated. Calculation in it are
even more so.

https://www.h-schmidt.net/FloatConverter/IEEE754.html


On Sat, 20 Mar 2021 at 20:23, Paul Dupuis via use-livecode <
use-livecode@lists.runrev.com> wrote:

> I don't have an answer, other than to say there has been a LOT of
> discussion over the years about the limits of number precision in
> LiveCode and a number of enhancement requests and 3rd party libraries
> that have attempted to provide a framework for high precision math. You
> might search the list archives or search the forum on the LC web site
> for past discussions about number precision.
>
> On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
> > Hi list,
> > I am a bit lost :
> >
> >set numberformat to ".#"
> >put -0.007347880794884118999 / 1000
> > -- returns -0.07347880794884119  which is ok
> >
> >put -0.007347880794884118999 / 100
> > -- returns -0.7347880794884 -> also ok
> >
> >put -0.007347880794884118999 / 1
> > -- returns -0.0073478807949 -> still ok
> >
> >put -0.007347880794884118999 / 10
> > -- returns 0 -> ?
> >
> > Any explanations ?
> > Thanks in advance.
> > Best,
> > jbv
> >
> > ___
> > use-livecode mailing list
> > use-livecode@lists.runrev.com
> > Please visit this url to subscribe, unsubscribe and manage your
> > subscription preferences:
> > http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> ___
> use-livecode mailing list
> use-livecode@lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
-- 
Pi Digital
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberformat question

[Paul Dupuis via use-livecode]

I don't have an answer, other than to say there has been a LOT of 
discussion over the years about the limits of number precision in 
LiveCode and a number of enhancement requests and 3rd party libraries 
that have attempted to provide a framework for high precision math. You 
might search the list archives or search the forum on the LC web site 
for past discussions about number precision.


On 3/20/2021 3:06 PM, jbv via use-livecode wrote:

Hi list,
I am a bit lost :

   set numberformat to ".#"
   put -0.007347880794884118999 / 1000
-- returns -0.07347880794884119  which is ok

   put -0.007347880794884118999 / 100
-- returns -0.7347880794884 -> also ok

   put -0.007347880794884118999 / 1
-- returns -0.0073478807949 -> still ok

   put -0.007347880794884118999 / 10
-- returns 0 -> ?

Any explanations ?
Thanks in advance.
Best,
jbv

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your 
subscription preferences:

http://lists.runrev.com/mailman/listinfo/use-livecode



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

numberformat question

[jbv via use-livecode]

Hi list,
I am a bit lost :

   set numberformat to ".#"
   put -0.007347880794884118999 / 1000
-- returns -0.07347880794884119  which is ok

   put -0.007347880794884118999 / 100
-- returns -0.7347880794884 -> also ok

   put -0.007347880794884118999 / 1
-- returns -0.0073478807949 -> still ok

   put -0.007347880794884118999 / 10
-- returns 0 -> ?

Any explanations ?
Thanks in advance.
Best,
jbv

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

numberFormat question

[dfepstein]

I am trying to set the numberFormat so that calculation is precise enough for 
the situation.  Since I don't know ahead of time how many decimal places will 
be used, I wrote a function that I hoped would adjust things as necessary. 
But when this function is called with parameters m = 1.09131 and n = .001 
and k = 1, it returns 1.09131 rather than 1.0913101. 
The last couple of lines were added for testing, and the debugger shows that 
the numberFormat is being set correctly but that the truncated value is put 
into hold. 
Can anybody see what is going wrong? 



Many thanks. 



David Epstein 



function preciseEnough m,n,k 
   -- return the value m + k*n 
   -- default numberFormat shows up to 6 decimal places 
   -- If m, n, or k  has more than that precision, this function sets the 
numberFormat with a margin of safety 
   -- before returning the answer 
   put length(m) - offset(.,m) into aPlaces 
   put length(n) - offset(.,n) into bPlaces 
   put length(k) - offset(.,k) into cPlaces 
   put the numberFormat into myString 
   if max(aPlaces,bPlaces,cPlaces) + 4  length(myString) then 
  get myString   
  set the numberFormat to it 
   end if 
   put the numberFormat into nf 
   put m + k*n into hold 
   return hold 
end preciseEnough 
___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberFormat question

[Mark Schonewille]

David,

The numberformat doesn't affect precision. It only affects output as is 
shown by


on mouseUp
   set the numberformat to 00
   put 0.01*1 into x
   set the numberformat to 00.00
   put x*1
end mouseUp

It even only affects the current handler and doesn't affect precision in 
calling handlers:


on mouseUp
   // default is 0.##
   put a1() into x
   put x*1
end mouseUp

function a1
   set the numberformat to 00
   return 0.01*1
end a1

The above script returns 0.01 instead of 00.

I don't think LiveCode allows you to adjust the precision as other 
languages to with e.g. signed and unsigned integers and floating 
precision. You'd have to write special routines for this.


I think there is quite a lot of discussion to find in the archives about 
high precision and floating point numbers.


--
Best regards,

Mark Schonewille

Economy-x-Talk Consulting and Software Engineering
Homepage: http://economy-x-talk.com
Twitter: http://twitter.com/xtalkprogrammer
KvK: 50277553

Installer Maker for LiveCode:
http://qery.us/468

Buy my new book Programming LiveCode for the Real Beginner 
http://qery.us/3fi


LiveCode on Facebook:
https://www.facebook.com/groups/runrev/

On 6/16/2014 20:13, dfepst...@comcast.net wrote:



I am trying to set the numberFormat so that calculation is precise enough for 
the situation.  Since I don't know ahead of time how many decimal places will 
be used, I wrote a function that I hoped would adjust things as necessary.
But when this function is called with parameters m = 1.09131 and n = .001 
and k = 1, it returns 1.09131 rather than 1.0913101.
The last couple of lines were added for testing, and the debugger shows that 
the numberFormat is being set correctly but that the truncated value is put 
into hold.
Can anybody see what is going wrong?



Many thanks.



David Epstein



function preciseEnough m,n,k
-- return the value m + k*n
-- default numberFormat shows up to 6 decimal places
-- If m, n, or k  has more than that precision, this function sets the 
numberFormat with a margin of safety
-- before returning the answer
put length(m) - offset(.,m) into aPlaces
put length(n) - offset(.,n) into bPlaces
put length(k) - offset(.,k) into cPlaces
put the numberFormat into myString
if max(aPlaces,bPlaces,cPlaces) + 4  length(myString) then
   get myString  
   set the numberFormat to it
end if
put the numberFormat into nf
put m + k*n into hold
return hold
end preciseEnough



___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberFormat question

[J. Landman Gay]

On 6/16/2014, 1:13 PM, dfepst...@comcast.net wrote:



I am trying to set the numberFormat so that calculation is precise enough for 
the situation.  Since I don't know ahead of time how many decimal places will 
be used, I wrote a function that I hoped would adjust things as necessary.
But when this function is called with parameters m = 1.09131 and n = .001 
and k = 1, it returns 1.09131 rather than 1.0913101.
The last couple of lines were added for testing, and the debugger shows that 
the numberFormat is being set correctly but that the truncated value is put 
into hold.
Can anybody see what is going wrong?



Many thanks.



David Epstein



function preciseEnough m,n,k
-- return the value m + k*n
-- default numberFormat shows up to 6 decimal places
-- If m, n, or k  has more than that precision, this function sets the 
numberFormat with a margin of safety
-- before returning the answer
put length(m) - offset(.,m) into aPlaces
put length(n) - offset(.,n) into bPlaces
put length(k) - offset(.,k) into cPlaces
put the numberFormat into myString
if max(aPlaces,bPlaces,cPlaces) + 4  length(myString) then
   get myString  
   set the numberFormat to it
end if
put the numberFormat into nf
put m + k*n into hold
return hold
end preciseEnough


As Mark said, numberformat is only retained during the local handler so 
you need to reset it each time the handler runs.


When you use #, it means to include a numerical value in that postion 
only if there is an actual value there; if there is no value then that 
position is ignored. If you use 0 in the numberformat instead, empty 
positions are padded with zeros. So, to get the precision you want, you 
don't need to calculate the number of places/positions, just use the # 
as you are now. Include enough #s to cover your longest anticipated 
number, up to LC limits (I think that's 16 places.)


Since numberformat only affects the display, you need to force the 
calculation from a numerical value to a text value. You can do that by 
simply putting the result of the calculation into a field. If you want 
it in a variable, you can use value() to do that instead.


So your whole handler can be like this:

function preciseEnough m,n,k
  put m + k*n into hold
  set the numberformat to 0.## -- add more if you want
  put value(hold) into hold
  return hold
end preciseEnough

--
Jacqueline Landman Gay | jac...@hyperactivesw.com
HyperActive Software   | http://www.hyperactivesw.com

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberFormat question

[Bob Sneidar]

Also, if you don’t care about display, just set the *precision* to LC max. 
Numberformat is as others have posted, really a display function. After you are 
done, set the numberFormat back to default or “0”. I bit myself in the butt 
because I had a counter that I was using to append a number to a variable name, 
and it hosed my names.  

Bob S


On Jun 16, 2014, at 11:13 , dfepst...@comcast.net dfepst...@comcast.net 
wrote:

 
 
 I am trying to set the numberFormat so that calculation is precise enough for 
 the situation.  Since I don't know ahead of time how many decimal places will 
 be used, I wrote a function that I hoped would adjust things as necessary. 
 But when this function is called with parameters m = 1.09131 and n = .001 
 and k = 1, it returns 1.09131 rather than 1.0913101. 
 The last couple of lines were added for testing, and the debugger shows that 
 the numberFormat is being set correctly but that the truncated value is put 
 into hold. 
 Can anybody see what is going wrong? 
 
 
 
 Many thanks. 
 
 
 
 David Epstein 
 
 
 
 function preciseEnough m,n,k 
-- return the value m + k*n 
-- default numberFormat shows up to 6 decimal places 
-- If m, n, or k  has more than that precision, this function sets the 
 numberFormat with a margin of safety 
-- before returning the answer 
put length(m) - offset(.,m) into aPlaces 
put length(n) - offset(.,n) into bPlaces 
put length(k) - offset(.,k) into cPlaces 
put the numberFormat into myString 
if max(aPlaces,bPlaces,cPlaces) + 4  length(myString) then 
   get myString   
   set the numberFormat to it 
end if 
put the numberFormat into nf 
put m + k*n into hold 
return hold 
 end preciseEnough 
 ___
 use-livecode mailing list
 use-livecode@lists.runrev.com
 Please visit this url to subscribe, unsubscribe and manage your subscription 
 preferences:
 http://lists.runrev.com/mailman/listinfo/use-livecode


___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode

Re: numberFormat question

[Kay C Lan]

On Tue, Jun 17, 2014 at 4:34 AM, J. Landman Gay
jac...@hyperactivesw.com wrote:
   set the numberformat to 0.## -- add more if you want

I think the reason David isn't doing that, and is trying to limit the
number of # to the absolute minimum is because of this note in the
Dictionary:

Note: Since LiveCode does not use decimal numbers for its internal
calculations (for reasons of speed), the decimal representation of a
number is sometimes slightly off the correct number. For example,
10^-1 is equal to 0.1, but is calculated (to eighteen decimal places)
as 0.16. Because of this, setting the numberFormat to
specify many decimal places after the decimal point may produce
unexpected results in a statement that tests for an exact number. To
prevent this, either avoid setting the numberFormat to a value more
precise than you need, or use the abs function instead of the =
operator to test equality:

I think we all know that at least once or twice a year we get a post
to this List questioning why LC is getting it's math wrong and the
reason is as stated above. For anyone whose app requires precision
math that shouldn't be a note, it should be a CAUTION! It's so easy to
have your app work perfectly 99.% of the time... but
then ruin someone's day.

___
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription 
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode