The behavior is explained by Tolerance:
3162277660168428= T^0.5
1
3162277660168427= T^0.5
1
3162277660168427= 3162277660168428
0
--
David Mitchell
On 1/19/2010 11:30, David Mitchell wrote:
> Devon, I agree with you about the issues of float VS extended integer
> precision.
>
> But, I also find the current results odd:
>
> T=:10000000000000306999999999997669x
> T^0.5
> 3162277660168427.5
> <.T^0.5
> 3162277660168428
>
> It seems to be related to Tolerance:
>
> <.!.0 T^0.5
> 3162277660168427
>
> --
> David Mitchell
>
> On 1/19/2010 9:48, Devon McCormick wrote:
>> Extended _integer_ gives integer results. You're not getting the correct
>> floor because the result of T^0.5 is floating point - which has limited
>> precision.
>>
>> Take a look at http://www.jsoftware.com/jwiki/Essays/Square Root for an idea
>> of how to do this calculation to more digits. The examples given don't work
>> with extended integers - you'll have to write your own root finder to do
>> this.
>>
>> On Tue, Jan 19, 2010 at 2:22 AM, Gerry J<[email protected]> wrote:
>>
>>> Hello,
>>>
>>> although i programmed APL for a few years a long time ago, i am new to J.
>>> I am trying to perform extended precision integer arithmetic.
>>>
>>> T, a global has been assigned this value
>>> 10000000000000306999999999997669x
>>> Then i perform this
>>> x=:x:<.T^0.5
>>> x
>>> 3162277660168428
>>>
>>> The correct square root from another source is
>>> 3162277660168427.8729609771283143493714513369669538
>>> so i would expect to get
>>> 316227766016842* 7* as the floor, not 316227766016842 *8*
>>>
>>> from the result obtained, (3162277660168428)^2 is more than T, so it
>>> would appear not to be the correct floor of square root of T.
>>>
>>> How should i code to get 3162277660168427 for<.T^0.5?
>>> (apart from subtracting 1 :-) )
>>>
>>>
>>> I also computed */ for all primes up to 41 then up to 53 and tried to
>>> get the residue of the result for each of the primes involved.
>>> I expected to get zeroes, but for the cases ending at 53 i was getting
>>> some non zero.
>>>
>>> DP=:80, is there some other configuration variable that is relevant or
>>> do i need to code these differently?
>>>
>>> --
>>> Regards,
>>> Gerry J
>>>
>>>
>>> ----------------------------------------------------------------------
>>> For information about J forums see http://www.jsoftware.com/forums.htm
>>>
>>
>>
>>
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
