We have a similar problem with IEEE floats and underflow. All old
programs are written under the assumption that precision is not lost in
division and multiplication operations. All old error estimations are
based on this assumption. All these programs now give random results and
have to be retested and rewritten, or we have to find a way to get an
exception when we have an underflow. /Erling
On 2017-10-15 13:42, Erling Hellenäs wrote:
Is there a result of this factual discussion we could study?
As I see it the programmer is responsible for creating a program that
gives correct results in the domains in which it is specified to give
correct results.
If modulo gives correct results only in a certain domain, this domain
should be clear from the specification.
The programmer who uses it is then responsible for controlling inputs
and outputs, that they are in the correct domains.
However, since this makes the programs much more complicated, we
mostly create programs which verify their inputs and outputs, and if
not in the domains in which the results are correct, we issue an error
or exception.
Then there is a question of compatibility. Programs were created
before the change to 64 bit integers. The programmer verified their
function, but the verification is no longer valid, so these programs
have to be retested and rewritten, or we are stuck with tools and
systems which give the users random results, and which are therefore
not useful any more. /Erling
On 2017-10-14 21:19, Don Guinn wrote:
I think we have had a factual discussion on the molulo problem. We just
haven't found a good solution to the problem. Larger precision only
moves
the problem. It does not fix it. None of the possible solutions
presented
so far seem satisfactory because there are so many ways for them to fail
anyway, if it is assumed that coming up with a residue of zero is really
wrong when the numbers presented to residue exceed known precision
limits.
We learned several good things in this exercise. One, the square root
of an
integer perfect square stays integer. Two, we can't count on
conversion of
integer to float to be exact. Three, solutions to problems must be
realistic. There will almost always be compromises and tradeoffs. We
must
watch answers we get as was done by whoever asked the question in the
first
place. That he asked the question is most important. It now makes us all
aware of some possible problems we can get into when pushing the
limits of
the hardware and software.
As to spinoffs. It is great for people to explore alternative
solutions to
programming language problems, whether J, C or any other language.
But they
are explorations. That doesn't mean that they will stand for the test of
time. Ken and many others spent a lot of careful designing J. It is
important to fully understand their work before trying to "improve"
on what
they did.
On Sat, Oct 14, 2017 at 12:32 PM, Erling Hellenäs
<[email protected]>
wrote:
Hi all !
I think all problems should be put into the bugtracker and that there
should be factual discussions about how to solve them, so that, should
anyone want to finance or merge a solution, they can do so.
If we deny that any problems exist there will be no development.
If we turn all requests for change down there will be no development.
People are cloning J and are willing to supply their patches for
free. If
we don't cooperate with them there will be no development.
Cheers,
Erling Hellenäs
On 2017-10-14 19:14, Don Guinn wrote:
I think that you are making this out to be a big problem. I don't
think it
is. We have much bigger problems with coming up with good solutions to
problems. Scaling is one of the biggest. That J deals with numbers as
numbers, not as integer or float or whatever and does not predefine
limits
on array sizes removes many of the problems found in traditional
programming languages. At the same time it generates other problems
like
double float cannot represent all 64 bit integers resulting in loss of
precision with automatic conversion of integer to float.
Whether the benefits of J's approach outweigh the disadvantages
depends on
whom you ask. But possible solutions like those you suggested are only
partial solutions. We need to watch for things that don't make sense,
whether caused by our design, or the design of the programming
language.
They all have lots of gotchas.
On Sat, Oct 14, 2017 at 9:11 AM, Erling Hellenäs <
[email protected]>
wrote:
Hi all!
We now have an additional proposed solution from Raul, using extended
precision and rationals instead of integers.
Any more proposed solutions?
Opinions about the proposed solutions?
Cheers,
Erling Hellenäs
On 2017-10-13 22:28, Erling Hellenäs wrote:
Hi all!
You moved to 64 bit integer. You can't go back. Now there is a
serious
problem? You have to determine how to solve it?
The simple solution is to move to quad precision floats? Is it
possible
to add support for keeping the integers ? The ability to do all
integer
arithmetic on integers? To stop auto-converting to floats? To
internally
work with quad precision floats in integer arithmetics?
Maybe you could add support for the new IEEE decimal standard? Move
integer arithmetic to them?
Are there other solutions?
Cheers,
Erling Hellenäs
On 2017-10-08 16:54, Don Guinn wrote:
I realize this is stating the obvious, but the loss of precision
is the
result of 64 bit integer support. Previously "upgrading" a
number from
integer to float was exact. Though the residue problem for very
large
numbers still existed, at least it didn't involve loss of
precision.
It's my personal opinion that one should always be careful when
working
around the limits of a system. But what should be done when
things go a
little crazy around those limits? It is unfortunate that IEEE only
implemented indeterminate (_.) when it could have set other
flags in
the
unused bit configuration to indicate things like underflow, but not
zero
or
overflow but not infinity. But they didn't.
A while back J had an option for upgrade to go to rational
instead of
float. It was useful in labs to more easily show interesting
properties
of
numbers. Is that option still around? If so it could be used in
mod as
an
option. But it cannot be always known that the number will
eventually
be
used in mod. And many transcendental verbs must go to float.
Current hardware now supports quad precision float, at least
some do.
If
quad float were used then the loss of precision goes away when
converting
64 bit integer to float. But that doubles the size of float, and
even
though memory is getting huge it's still a concern for big
problems.
Not
to
mention that quad float is probably slower than double float.
And it
may
not be supported on all hardware, similar to the AVX problem.
IBM's PLI has an interesting approach to precision. You told it (in
decimal
digits) the largest numbers you will deal with and the number of
digits
after the decimal. Then it picked the best way to store the numbers
given
available hardware. In J we have 64 bit integers and floats with
maybe
16
significant decimal digits and a tremendous range for exponents.
Most
problems we deal with don't need such big numbers. An argument
many use
against J in that it uses so much memory for small numbers.
Perhaps a
global setting with Foreign Conjunction could give a similar
choice for
J.
I would argue against it saying things like single/double/quad
float or
16/32/64 bit integers, but specify what range and significance
is need
and
let J choose how to handle it. Including totally ignoring it for
some
implementations. Supporting this could make the J engine larger,
but
nobody
seems too concerned with the monstrous size Qt.
Whatever happened with the idea bouncing around of defining a
floating
point of arbitrary size and precision like with extended
integers and
rationals?
And now IEEE has a decimal float standard. Right now it seems
that only
IBM
has implemented it in hardware. But think of all the confusion
we see
when
decimal numbers like 1.1 are not represented exactly in J.
Maybe I rambled a bit. But this all involves problems when, for one
reason
or another, the hardware can't handle needed precision.
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see
http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
