In summary, I see the dilemma as this:
1. If the default stays at 9 digits, existing programs will break
because of external factors such as big files, big numbers, etc. It seems
to me that the program would break eventually regardless of whether 64-bits
existed or not. Fortunately, the fix is simple, just use NUMERIC DIGITS and
it's fixed on whatever hardware you're on.
2. If the default is changed to 18, existing programs may produce
different results, based on the hardware. This time, however, it will not
fail because of external date but rather on changes to the internals of the
ooREXX. The fix for this is also using NUMERIC DIGITS to set the
In either case, the user's script has to be changed.either changed because
external data is bigger than 9 digits can handle or the internals of REXX
has changed the default for some hardward. To me the question seems one of
consistency and how important consistency is. I agree with a previous
participant in this discussion whose background was in IBM as was mine. The
cardinal rule was that you never, ever, altered the user interface when
going from version to version unless there just isn't any other way. I
think that is true, just look at how Microsoft has jerked around users with
Windows and why I refuse move off XP for as long as I can.
Is the greater value in insuring that all programs works consistently no
matter what hardware or that all programs take advantage of the hardware
their on. I don't know the answer but I am leaning toward consistency.
Delaying the release of 4.0.0 is extremely minor compared to making the
wrong decision here.
That's my thoughts,
From: Rony G. Flatscher [mailto:rony.flatsc...@wu-wien.ac.at]
Sent: Friday, February 27, 2009 5:24 AM
To: Open Object Rexx Developer Mailing List
Subject: Re: [Oorexx-devel] [DISCUSS] What should be the default numeric
digits setting for 64-bit.
Mike Cowlishaw wrote:
The processor architecture only comes in to play with values that
referred to as "numbers used directly by Rexx". This is generally the
internal digits setting used internally by bifs/methods, and for
things like "do nnnn" loops. These are the places where a Rexx number
need to be converted into native values. This is distinct from the
current numeric digits setting, which is what governs Rexx arithmetic.
So, for the last 30 years, both of these values have always been "9",
so everything remained in sync. You could still set numeric digits
higher for calculations, but if you tried to use "1000000000" for a
substr position (for example), you would get an error for an "invalid
whole number", since the substr() bif uses numeric digits 9
internally. All calculations that would evaluate as a whole number
under the default digits setting would also produce good values for
the bifs, and non-whole number results would generally be rejected.
The internal digits setting is unaffected by changes in NUMERIC
DIGITS. It always operates with the internal digits setting.
In the current 4.0 codebase, when compiled for 64-bit, the internal
digits setting is 18, which allows you to manipulate values larger
than 9999999999. The default digits setting is also the same, which
means the symmetry from the 32-bit implementation is maintained. The
danger with decoulping these occurs when a bif such as pos() returns a
value that would not be considered a whole number under the current
digits setting (which btw, is a situation you can encounter if you are
running at a setting smaller than 9). If you use that value to
perform a calculation, you're likely to lose digits off the end,
creating a result with a high "astonishment factor".
There has always been that 'astonishment factor' when (typically using
integers) more than 9 digits of precision are required for a calculation
to be exact. That's a consequence of the choice to go for readability --
a default of 9 digits -- rather than a higher precision by default.
In the 16-Bit world this meant that Rexx number values could be larger than
the processor's largest integer that usually would have been used by
assembled and compiled languages (offering external functions), which I
still think has been a very remarkable (and excellent) decision! This also
meant that for all practical purposes using NUMERIC DIGITS was not needed if
interfacing with external functions (receiving large numbers, calculating
with them, supplying them back).
Having 64-bit implementations does not change that; all 64-bit means is
that larger objects may be more common, so the astonishment may happen a
bit more often. The problem is already there in the 32-bit
implementation, so programmers already have to address the issue, so this
is not something that works in 32-bit but does not work in 64-bit.
For example, using 32-bit 3.2.0 (some irrelevant lines & prompts deleted):
28/04/2005 11:41 1,073,692,672 VTS_01_1.VOB
say stream('VTS_01_1.VOB', 'c', 'query size')
That result is more than 9 digits; if one is doing arithmetic on stream
sizes today, in a 32-bit environment, you must already be using a larger
numeric digits. (And I have to suspect that people probably set 12 or 15
digits, so increasing the default to 18 won't override that in any case.)
This is exactly the problem in today's world where we deal with 64-Bit and
shortly in the future with 128-Bit and more. Interfacing with compiled
languages that take advantage of the register sizes exceeds all of a sudden
the Rexx default capability. Getting and calculating with numbers from
external functions all of a sudden cause them to be edited by Rexx, loosing
digits in the process, which in the past did simply not happen.
This is where a real problem starts: Rexx programs all of a sudden stop
working correctly! Enclosed you'll find a small Rexx utility that I wrote in
the OS/2 days 17 years ago which would break up large files into chunks that
did fit on diskettes and generating a copy command to reassamble the chunks
to allow the original file to be recretated. (As strange as it may seem)
That very same program may be still useful in splitting up large files for
CDs or even DVDs! Just take into account multiemedia files (digital - home -
videos). (Believe it or not, that Rexx program was faster on Windows
machines than specialized, compiled programs!)
It was with this little program that I noted for the first time a couple of
years ago, that in todays hardware environment the digits settings of 9 is
becoming a serious bottleneck, causing Rexx programs that used to run for
decades to all of a sudden to fail! I think that is not acceptable and needs
to be solved once and forever! It is probably not feasible/possible to go
after all existing programs and insert a NUMERIC DIGITS 20 (and then with
128-Bit processors and external functions dealing with those sizes NUMERIC
DIGITS 39 and so on) wherever this becomes a necessity (no one would
voluntarily change deployed, tested and running code). Rexx should insulate
the coder from such "environmental" changes as much as possible.
The legibility argument is one that I would understand, but one that would
not really seem to be as important as the problem that a certain category of
Rexx programs start to fail running "all of a sudden". If numbers become too
large, format() is your friend, hence programmers who really need to be able
to read (probably the intermediate) results of operations could achieve that
easily. Also, if format() was more capable in its formatting features, then
any number of any size could be made legible.
Just look at Lee Peedin's decimalFormat.cls - in the incubator -, which to
me indicates a desperate need for that particular functionality in
commercial applications. I thought that that class was going to become part
of the next release of ooRexx due to its importance.
Probably the problem here is that there are plentyful of Rexx programs that
rely on 9 digits (based on assumptions about the environment they got
created years ago) and have the formatting of reports and results dependent
on that environmental setting, which may break (at least visually) when
numeric digits is increased. (But at least they should have a choice, and if
existing programs break due to external functions needing larger numbers for
today's hardware, then I am sure that they will tolerate older reports
breaking the text-based formatting, relying on the numeric 9 digits.)
If short: if the argument is that the 9-digit choice was wrong in the
first place (which is a valid point of view) then the proposal should be
to change the _language_ to use a larger default (18, 20, whatever) -- and
that would then apply to all implementations, whether they run on 16-bit,
32-bit, or 64-bit platforms.
I really do not like the idea that
Might give 0.142857143 on one machine and 0.142857142857142857 on another,
just because one user installed the 32-bit version and another the 64-bit.
This would lead to all kinds of subtle problems of this nature (a toy
if x/3 = 0.333333333 then say 'X is one'
which would work differently depending on which version one had installed.
However, I do like the idea of a directive like
::numeric digits 18
that would apply to a complete program/class. That would be useful anyway
(as would similar for other 'global defaults', such as Trace setting). :-)
Also, having an option to set the interpreter (via a class) independently to
an arbitrarily value of numeric digits would help tremendeously in such
P.S: Simple Rexx program, written under OS/2 which about 14 (!) years after
its inception started to break due to a change in the hardware environment
it was running (file sizes going beyond 2 GB, and actually, only with ooRexx
4.0 *and* setting explicitly NUMERIC DIGITS would it run again reliably;
having a solution on 32- and 64-Bit and any other bitness that would allow
this program to start to run again *unchanged* would be the most welcomed
------------------ cut here ------------------
(c) 1991, Rony G. Flatscher, donated to the public domain
- split any file into given chunks
- show exception handling for CTL-C
- produce a CMD-file to recreate original file
PARSE ARG infile chunks CMD_file /* get filename, chunks in KB, optional
IF infile = '' | infile = '?' | ,
chunks = '' | \DATATYPE(chunks, 'N') | chunks < 1 THEN SIGNAL usage
SIGNAL ON HALT /* jump to label "HALT:" if user presses CTL-C */
chunks = TRUNC(chunks) /* get integer portion of chunks */
/* open input-file */
IF STREAM(infile,"C","OPEN READ") <> 'READY:' THEN
SAY "problem opening '"infile"', aborting ..."
/* calculate number of files needed */
tmp = STREAM(infile, "C", "QUERY SIZE") / (chunks * 1024)
IF tmp // 1 <> 0 THEN tmp = tmp + 1 /* if more than (chunks*1024) bytes add
another file */
IF LENGTH(TRUNC(tmp)) > 3 THEN
SAY "Sorry, cannot produce more than 999 chunk-files, aborting ..."
SAY "("TRUNC(tmp)" chunk-files would be needed)"
/* define chunk-file's stem name */
tmp = FILESPEC('NAME',infile)
IF LASTPOS('.', tmp) > 0 THEN
tmp = SUBSTR(tmp, 1, LASTPOS('.', tmp)-1) /* get letters up to but
without last point */
outfile_stem = FILESPEC('DRIVE', infile) || FILESPEC('PATH', infile) || tmp
outfile_nr = 0
/* produce the chunk-files */
CALL OPEN_OUTFILE /* open next chunk-file */
/* get integer portion multiplied by 1024 */
total_to_read = chunks * 1024
to_read = 16 * 1024 /* read 16KB-blocks for speed reasons
DO WHILE CHARS(infile) > 0 & total_to_read > 0
IF total_to_read > to_read THEN
total_to_read = total_to_read - to_read
to_read = total_to_read
total_to_read = 0
CALL CHAROUT outfile, CHARIN(infile, ,to_read)
tmp = STREAM(outfile, "C", "CLOSE") /* close present chunk-file */
IF CHARS(infile) < 1 THEN LEAVE
/* create CMD-file which produces the original file */
IF CMD_file = '' THEN CMD_file = outfile_stem || 'CMD'
DO /* check whether given CMD-file has the correct extension */
name = FILESPEC('NAME', CMD_file) /* get filename without path */
tmp = LASTPOS('.', name) /* get position of last point, if any
IF tmp <> 0 THEN
IF TRANSLATE(SUBSTR(name, tmp+1)) <> 'CMD' THEN
name = SUBSTR(name, 1, tmp)||CMD /* produce batch-file extension */
ELSE name = name || '.CMD' /* produce batch-file extension */
CMD_file = FILESPEC('DRIVE', CMD_file) || FILESPEC('PATH', CMD_file) ||
IF STREAM(CMD_file, "C", "OPEN WRITE") <> 'READY:' THEN
SAY "Sorry, '"CMD_file"' cannot be created, aborting & cleaning up..."
/* produce the OS/2-copy-statement to recreate original file */
CALL CHAROUT CMD_file, "COPY /B " || outfile_stem || '001'
DO i = 2 TO outfile_nr
CALL CHAROUT CMD_file, '+'outfile_stem||RIGHT(i, 3, '0')
CALL LINEOUT CMD_file, ' '||infile /* target to build */
/* produce the statements for erasing the chunk-files */
CALL LINEOUT CMD_file, '@IF ERRORLEVEL 1 GOTO ERROR'
CALL LINEOUT CMD_file, '@ECHO Erasing chunk-files ...'
DO i = 1 TO outfile_nr
CALL LINEOUT CMD_file, '@ERASE' outfile_stem||RIGHT(i, 3, '0')
CALL LINEOUT CMD_file, '@ERASE' FILESPEC('NAME',CMD_file)
CALL LINEOUT CMD_file, '@GOTO FINISH'
CALL LINEOUT CMD_file, ':ERROR'
CALL LINEOUT CMD_file, '@ECHO Error occurred while copying...'
CALL LINEOUT CMD_file, ':FINISH'
/* end of main procedure */
/* open output-file, if it exists abort, cleaning up all files produced so
outfile_nr = outfile_nr + 1
outfile = outfile_stem||RIGHT(outfile_nr, 3, '0')
IF STREAM(outfile,"C","QUERY EXISTS") <> '' THEN
SAY "Sorry, '"outfile"' exists, aborting & cleaning up..."
outfile_nr = outfile_nr - 1
IF STREAM(outfile, "C", "OPEN WRITE") <> 'READY:' THEN
SAY "Sorry, '"outfile"' cannot be created, aborting & cleaning up..."
outfile_nr = outfile_nr - 1
/* let OS/2 erase the produced output-files */
DO i = 1 TO outfile_nr
"@ERASE" outfile_stem||RIGHT(i, 3, '0') "2>nul"
/* signal-handler, if user presses CTL-C */
SAY "CTL-C was pressed, aborting & cleaning up..."
/* be sure to close open files */
IF outfile <> '' THEN
tmp = STREAM(outfile, "C", "CLOSE")
IF CMD_file <> '' THEN
tmp = STREAM(CMD_file, "C", "CLOSE")
CALL erase_produced_files /* delete all produced files */
"@ERASE" CMD_file "2>nul" /* throw possible error message into
EXIT -1 /* generate an error value of -1 */
SAY "LSPLIT - syntax: LSPLIT filename 1024bytes-chunks [CMD-file]"
SAY "(splits the given file into several new ones, each being 1024 *
SAY " an optional name for the CMD-file which is used to recreate the
SAY " file)"
SAY "KB-chunks: whole number representing KB-chunks"
------------------ cut here ------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
Oorexx-devel mailing list