Mike Cowlishaw wrote:
>> The processor architecture only comes in to play with values that
>> referred to as "numbers used directly by Rexx". This is generally the
>> internal digits setting used internally by bifs/methods, and for
>> things like "do nnnn" loops. These are the places where a Rexx number
>> need to be converted into native values. This is distinct from the
>> current numeric digits setting, which is what governs Rexx arithmetic.
>> So, for the last 30 years, both of these values have always been "9",
>> so everything remained in sync. You could still set numeric digits
>> higher for calculations, but if you tried to use "1000000000" for a
>> substr position (for example), you would get an error for an "invalid
>> whole number", since the substr() bif uses numeric digits 9
>> internally. All calculations that would evaluate as a whole number
>> under the default digits setting would also produce good values for
>> the bifs, and non-whole number results would generally be rejected.
>> The internal digits setting is unaffected by changes in NUMERIC
>> DIGITS. It always operates with the internal digits setting.
>> In the current 4.0 codebase, when compiled for 64-bit, the internal
>> digits setting is 18, which allows you to manipulate values larger
>> than 9999999999. The default digits setting is also the same, which
>> means the symmetry from the 32-bit implementation is maintained. The
>> danger with decoulping these occurs when a bif such as pos() returns a
>> value that would not be considered a whole number under the current
>> digits setting (which btw, is a situation you can encounter if you are
>> running at a setting smaller than 9). If you use that value to
>> perform a calculation, you're likely to lose digits off the end,
>> creating a result with a high "astonishment factor".
> There has always been that 'astonishment factor' when (typically using
> integers) more than 9 digits of precision are required for a calculation
> to be exact. That's a consequence of the choice to go for readability --
> a default of 9 digits -- rather than a higher precision by default.
In the 16-Bit world this meant that Rexx number values could be larger
than the processor's largest integer that usually would have been used
by assembled and compiled languages (offering external functions), which
I still think has been a very remarkable (and excellent) decision! This
also meant that for all practical purposes using NUMERIC DIGITS was not
needed if interfacing with external functions (receiving large numbers,
calculating with them, supplying them back).
> Having 64-bit implementations does not change that; all 64-bit means is
> that larger objects may be more common, so the astonishment may happen a
> bit more often. The problem is already there in the 32-bit
> implementation, so programmers already have to address the issue, so this
> is not something that works in 32-bit but does not work in 64-bit.
> For example, using 32-bit 3.2.0 (some irrelevant lines & prompts deleted):
> dir VTS_01_1.VOB
> 28/04/2005 11:41 1,073,692,672 VTS_01_1.VOB
> say stream('VTS_01_1.VOB', 'c', 'query size')
> That result is more than 9 digits; if one is doing arithmetic on stream
> sizes today, in a 32-bit environment, you must already be using a larger
> numeric digits. (And I have to suspect that people probably set 12 or 15
> digits, so increasing the default to 18 won't override that in any case.)
This is exactly the problem in today's world where we deal with 64-Bit
and shortly in the future with 128-Bit and more. Interfacing with
compiled languages that take advantage of the register sizes exceeds all
of a sudden the Rexx default capability. Getting and calculating with
numbers from external functions all of a sudden cause them to be edited
by Rexx, loosing digits in the process, which in the past did simply not
This is where a real problem starts: Rexx programs all of a sudden stop
working correctly! Enclosed you'll find a small Rexx utility that I
wrote in the OS/2 days 17 years ago which would break up large files
into chunks that did fit on diskettes and generating a copy command to
reassamble the chunks to allow the original file to be recretated. (As
strange as it may seem) That very same program may be still useful in
splitting up large files for CDs or even DVDs! Just take into account
multiemedia files (digital - home - videos). (Believe it or not, that
Rexx program was faster on Windows machines than specialized, compiled
It was with this little program that I noted for the first time a couple
of years ago, that in todays hardware environment the digits settings of
9 is becoming a serious bottleneck, causing Rexx programs that used to
run for decades to all of a sudden to fail! I think that is not
acceptable and needs to be solved once and forever! It is probably not
feasible/possible to go after all existing programs and insert a NUMERIC
DIGITS 20 (and then with 128-Bit processors and external functions
dealing with those sizes NUMERIC DIGITS 39 and so on) wherever this
becomes a necessity (no one would voluntarily change deployed, tested
and running code). Rexx should insulate the coder from such
"environmental" changes as much as possible.
The legibility argument is one that I would understand, but one that
would not really seem to be as important as the problem that a certain
category of Rexx programs start to fail running "all of a sudden". If
numbers become too large, format() is your friend, hence programmers who
really need to be able to read (probably the intermediate) results of
operations could achieve that easily. Also, if format() was more capable
in its formatting features, then any number of any size could be made
Just look at Lee Peedin's decimalFormat.cls - in the incubator -, which
to me indicates a desperate need for that particular functionality in
commercial applications. I thought that that class was going to become
part of the next release of ooRexx due to its importance.
Probably the problem here is that there are plentyful of Rexx programs
that rely on 9 digits (based on assumptions about the environment they
got created years ago) and have the formatting of reports and results
dependent on that environmental setting, which may break (at least
visually) when numeric digits is increased. (But at least they should
have a choice, and if existing programs break due to external functions
needing larger numbers for today's hardware, then I am sure that they
will tolerate older reports breaking the text-based formatting, relying
on the numeric 9 digits.)
> If short: if the argument is that the 9-digit choice was wrong in the
> first place (which is a valid point of view) then the proposal should be
> to change the _language_ to use a larger default (18, 20, whatever) -- and
> that would then apply to all implementations, whether they run on 16-bit,
> 32-bit, or 64-bit platforms.
> I really do not like the idea that
> say 1/7
> Might give 0.142857143 on one machine and 0.142857142857142857 on another,
> just because one user installed the 32-bit version and another the 64-bit.
> This would lead to all kinds of subtle problems of this nature (a toy
> if x/3 = 0.333333333 then say 'X is one'
> which would work differently depending on which version one had installed.
> However, I do like the idea of a directive like
> ::numeric digits 18
> that would apply to a complete program/class. That would be useful anyway
> (as would similar for other 'global defaults', such as Trace setting). :-)
Also, having an option to set the interpreter (via a class)
independently to an arbitrarily value of numeric digits would help
tremendeously in such situations, IMHO.
P.S: Simple Rexx program, written under OS/2 which about 14 (!) years
after its inception started to break due to a change in the hardware
environment it was running (file sizes going beyond 2 GB, and actually,
only with ooRexx 4.0 *and* setting explicitly NUMERIC DIGITS would it
run again reliably; having a solution on 32- and 64-Bit and any other
bitness that would allow this program to start to run again *unchanged*
would be the most welcomed solution) !
------------------ cut here ------------------
(c) 1991, Rony G. Flatscher, donated to the public domain
- split any file into given chunks
- show exception handling for CTL-C
- produce a CMD-file to recreate original file
PARSE ARG infile chunks CMD_file /* get filename, chunks in KB, optional
IF infile = '' | infile = '?' | ,
chunks = '' | \DATATYPE(chunks, 'N') | chunks < 1 THEN SIGNAL usage
SIGNAL ON HALT /* jump to label "HALT:" if user presses CTL-C */
chunks = TRUNC(chunks) /* get integer portion of chunks */
/* open input-file */
IF STREAM(infile,"C","OPEN READ") <> 'READY:' THEN
SAY "problem opening '"infile"', aborting ..."
/* calculate number of files needed */
tmp = STREAM(infile, "C", "QUERY SIZE") / (chunks * 1024)
IF tmp // 1 <> 0 THEN tmp = tmp + 1 /* if more than (chunks*1024) bytes add
another file */
IF LENGTH(TRUNC(tmp)) > 3 THEN
SAY "Sorry, cannot produce more than 999 chunk-files, aborting ..."
SAY "("TRUNC(tmp)" chunk-files would be needed)"
/* define chunk-file's stem name */
tmp = FILESPEC('NAME',infile)
IF LASTPOS('.', tmp) > 0 THEN
tmp = SUBSTR(tmp, 1, LASTPOS('.', tmp)-1) /* get letters up to but without
last point */
outfile_stem = FILESPEC('DRIVE', infile) || FILESPEC('PATH', infile) || tmp ||
outfile_nr = 0
/* produce the chunk-files */
CALL OPEN_OUTFILE /* open next chunk-file */
/* get integer portion multiplied by 1024 */
total_to_read = chunks * 1024
to_read = 16 * 1024 /* read 16KB-blocks for speed reasons */
DO WHILE CHARS(infile) > 0 & total_to_read > 0
IF total_to_read > to_read THEN
total_to_read = total_to_read - to_read
to_read = total_to_read
total_to_read = 0
CALL CHAROUT outfile, CHARIN(infile, ,to_read)
tmp = STREAM(outfile, "C", "CLOSE") /* close present chunk-file */
IF CHARS(infile) < 1 THEN LEAVE
/* create CMD-file which produces the original file */
IF CMD_file = '' THEN CMD_file = outfile_stem || 'CMD'
DO /* check whether given CMD-file has the correct extension */
name = FILESPEC('NAME', CMD_file) /* get filename without path */
tmp = LASTPOS('.', name) /* get position of last point, if any */
IF tmp <> 0 THEN
IF TRANSLATE(SUBSTR(name, tmp+1)) <> 'CMD' THEN
name = SUBSTR(name, 1, tmp)||CMD /* produce batch-file extension */
ELSE name = name || '.CMD' /* produce batch-file extension */
CMD_file = FILESPEC('DRIVE', CMD_file) || FILESPEC('PATH', CMD_file) || name
IF STREAM(CMD_file, "C", "OPEN WRITE") <> 'READY:' THEN
SAY "Sorry, '"CMD_file"' cannot be created, aborting & cleaning up..."
/* produce the OS/2-copy-statement to recreate original file */
CALL CHAROUT CMD_file, "COPY /B " || outfile_stem || '001'
DO i = 2 TO outfile_nr
CALL CHAROUT CMD_file, '+'outfile_stem||RIGHT(i, 3, '0')
CALL LINEOUT CMD_file, ' '||infile /* target to build */
/* produce the statements for erasing the chunk-files */
CALL LINEOUT CMD_file, '@IF ERRORLEVEL 1 GOTO ERROR'
CALL LINEOUT CMD_file, '@ECHO Erasing chunk-files ...'
DO i = 1 TO outfile_nr
CALL LINEOUT CMD_file, '@ERASE' outfile_stem||RIGHT(i, 3, '0')
CALL LINEOUT CMD_file, '@ERASE' FILESPEC('NAME',CMD_file)
CALL LINEOUT CMD_file, '@GOTO FINISH'
CALL LINEOUT CMD_file, ':ERROR'
CALL LINEOUT CMD_file, '@ECHO Error occurred while copying...'
CALL LINEOUT CMD_file, ':FINISH'
/* end of main procedure */
/* open output-file, if it exists abort, cleaning up all files produced so far
outfile_nr = outfile_nr + 1
outfile = outfile_stem||RIGHT(outfile_nr, 3, '0')
IF STREAM(outfile,"C","QUERY EXISTS") <> '' THEN
SAY "Sorry, '"outfile"' exists, aborting & cleaning up..."
outfile_nr = outfile_nr - 1
IF STREAM(outfile, "C", "OPEN WRITE") <> 'READY:' THEN
SAY "Sorry, '"outfile"' cannot be created, aborting & cleaning up..."
outfile_nr = outfile_nr - 1
/* let OS/2 erase the produced output-files */
DO i = 1 TO outfile_nr
"@ERASE" outfile_stem||RIGHT(i, 3, '0') "2>nul"
/* signal-handler, if user presses CTL-C */
SAY "CTL-C was pressed, aborting & cleaning up..."
/* be sure to close open files */
IF outfile <> '' THEN
tmp = STREAM(outfile, "C", "CLOSE")
IF CMD_file <> '' THEN
tmp = STREAM(CMD_file, "C", "CLOSE")
CALL erase_produced_files /* delete all produced files */
"@ERASE" CMD_file "2>nul" /* throw possible error message into NUL-device
EXIT -1 /* generate an error value of -1 */
SAY "LSPLIT - syntax: LSPLIT filename 1024bytes-chunks [CMD-file]"
SAY "(splits the given file into several new ones, each being 1024 * chunks
SAY " an optional name for the CMD-file which is used to recreate the
SAY " file)"
SAY "KB-chunks: whole number representing KB-chunks"
------------------ cut here ------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
Oorexx-devel mailing list