Devon, I agree with you about the issues of float VS extended integer precision.
But, I also find the current results odd:
T=:10000000000000306999999999997669x
T^0.5
3162277660168427.5
<.T^0.5
3162277660168428
It seems to be related to Tolerance:
<.!.0 T^0.5
3162277660168427
--
David Mitchell
On 1/19/2010 9:48, Devon McCormick wrote:
> Extended _integer_ gives integer results. You're not getting the correct
> floor because the result of T^0.5 is floating point - which has limited
> precision.
>
> Take a look at http://www.jsoftware.com/jwiki/Essays/Square Root for an idea
> of how to do this calculation to more digits. The examples given don't work
> with extended integers - you'll have to write your own root finder to do
> this.
>
> On Tue, Jan 19, 2010 at 2:22 AM, Gerry J<[email protected]> wrote:
>
>> Hello,
>>
>> although i programmed APL for a few years a long time ago, i am new to J.
>> I am trying to perform extended precision integer arithmetic.
>>
>> T, a global has been assigned this value
>> 10000000000000306999999999997669x
>> Then i perform this
>> x=:x:<.T^0.5
>> x
>> 3162277660168428
>>
>> The correct square root from another source is
>> 3162277660168427.8729609771283143493714513369669538
>> so i would expect to get
>> 316227766016842* 7* as the floor, not 316227766016842 *8*
>>
>> from the result obtained, (3162277660168428)^2 is more than T, so it
>> would appear not to be the correct floor of square root of T.
>>
>> How should i code to get 3162277660168427 for<.T^0.5?
>> (apart from subtracting 1 :-) )
>>
>>
>> I also computed */ for all primes up to 41 then up to 53 and tried to
>> get the residue of the result for each of the primes involved.
>> I expected to get zeroes, but for the cases ending at 53 i was getting
>> some non zero.
>>
>> DP=:80, is there some other configuration variable that is relevant or
>> do i need to code these differently?
>>
>> --
>> Regards,
>> Gerry J
>>
>>
>> ----------------------------------------------------------------------
>> For information about J forums see http://www.jsoftware.com/forums.htm
>>
>
>
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
