Bonsoir Jean Louis,
WOW!
On 19.07.2026 13:50, Jean Louis Faucher wrote:
On 29 Jun 2026, at 19:30, Jean Louis Faucher <[email protected]> wrote:
The documentation is still in progress. I will create a second release once it
is finalized.
The second release is now available, together with the reference documentation.
https://github.com/jlfaucher/executor5-bulk/releases
It is a toolkit for implementing buit-in functions and built-in methods that
support UTF-8 strings.
May I just ask a few questions, as obviously the implementation stems from a top Unicode and ooRexx
expert?
Using your github location I also have checked out the project such that I can update locally and
look at your _diary.txt how you progress. This also allows me to get access to your scripts and
testscripts
loadPackage OK for extension/stringChunk.cls loadPackage OK for
utilities/indentedStream.cls
loadPackage OK for oorexxshell_clauser.cls loadPackage OK for
extension/std/extensions-std.cls
loadPackage OK for procedural/dispatcher.cls loadPackage OK for
extension/callable_std.cls
loadPackage OK for extension/extender_std.cls loadPackage OK for
extension/stringChunkMatcher.cls loadLibrary OK for rxunixsys loadPackage OK
for ncurses.cls
loadPackage OK for csvStream.cls loadPackage OK for dateparser.cls
loadLibrary OK for hostemu
loadPackage OK for json.cls loadPackage OK for mime.cls loadPackage OK for
rxftp.cls loadLibrary
OK for rxmath loadPackage OK for rxregexp.cls loadPackage OK for
regex/regex.cls loadPackage OK
for smtp.cls loadPackage OK for socket.cls loadPackage OK for
streamsocket.cls loadPackage OK
for yaml.cls loadPackage OK for pipeline/pipe.cls loadPackage OK for
rgf_util2/rgf_util2.rex
loadPackage OK for BSF.CLS loadPackage OK for jdor.cls loadPackage OK for
oorexxshell_queries_std.cls loadPackage OK for pipeline/pipe_std.cls
loadPackage OK for
Rexx.Parser.cls loadPackage OK for Unicode.cls loadPackage OK for
rxunicode.cls
REXX-ooRexx_5.3.0(MT)_64-bit 6.06 16 Jul 2026 Input queue name:
S120ceQ600003a04cc0
-- Unicode Regression Tests
ooRexx>tutor off
--
===============================================================================
-- 2026 July 15
/* [Unicode] utf8proc facade: Add the wrapper function RexxInteger
*codepointEastAsianWidthIsAmbiguous(RexxInteger *rexxCodepoint); // boolean
Examples: */
ooRexx>.RexxUnicodeServices~codepointEastAsianWidthIsAmbiguous(0)= -- 0
0
ooRexx>.RexxUnicodeServices~codepointEastAsianWidthIsAmbiguous("00A1"~x2d)=
-- 1
1
ooRexx>.RexxUnicodeServices~codepointEastAsianWidthIsAmbiguous("2010"~x2d)=
-- 1
1
--
===============================================================================
-- 2026 July 14
/* [Unicode] Update stringEscape to use the shorter notation \xXX when
applicable. Before:
\x{80} Now: \x80 Update escapedHexdigits to use the shorter notation \xXX
when applicable. */
--
===============================================================================
-- 2026 July 13
/* [Unicode] .RexxUnicodeStringIndexer: Update
.RexxUnicodeStringIndexer~init Storage limits can
be specified when creating the indexer; the default is no limit: -
codepointStorageLimit -
graphemeStorageLimit - errorStorageLimit End storage sizes can be specified
when creating the
indexer; the default is 0. - endCodepointStorageSize -
endGraphemeStorageSize -
endErrorStorageSize .RexxUnicodeStringIndexer~init( string, -- Storage limits
codepointStorageLimit = (.RexxInfo~internalMaxNumber), graphemeStorageLimit =
(.RexxInfo~internalMaxNumber), errorStorageLimit =
(.RexxInfo~internalMaxNumber) -- End storage
sizes endCodepointStorageSize = 0, endGraphemeStorageSize = 0,
endErrorStorageSize = 0 ) Add
::attribute codepointStorageLimit get Add ::attribute graphemeStorageLimit
get Add ::attribute
errorStorageLimit get Add ::attribute codepointCount get -- count after full
scan, not impacted
by codepointStorageLimit Add ::attribute graphemeCount get -- count after
full scan, not
impacted by graphemeStorageLimit Add ::attribute errorCount get -- count
after full scan, not
impacted by errorStorageLimit Add ::attribute endCodepointStorageSize get
Add ::attribute
endGraphemeStorageSize get Add ::attribute endErrorStorageSize get Add
::attribute
endCodepointIndexes get Add ::attribute endGraphemeIndexes get Add
::attribute endErrors get
Examples */ ... cut ...
The questions:
* So there are two indexers/iterators, one for CodePoints, one for Graphemes?
* What is the purpose of "storage limits" when using both iterators, why do
they exist?
Next steps (depending on my motivation):
* Backport the RexxTextTransformer class
<https://github.com/jlfaucher/executor5-bulk/blob/main/main/trunk/extensions/unicode/_notes-transformer.txt>
from
Executor, renamed RexxUnicodeStringTransformer.
* Implement a native version
<https://github.com/jlfaucher/executor/blob/master/sandbox/jlf/unicode/AI/Claude%20Pro%20grapheme-indexer-review-question2-2026.03.12.md>
of
RexxUnicodeStringIndexer.
* Implement a native version
<https://github.com/jlfaucher/executor/blob/master/sandbox/jlf/unicode/AI/Claude%20Pro%20text-transformer-architecture-question2c-2026.03.12.md>
of RexxUnicodeStringTransformer.
* Backport the native RexxText class
<https://htmlpreview.github.io/?https://github.com/jlfaucher/executor/blob/master/sandbox/jlf/_changes.html#encoded-strings>
from
Executor (simplified version, without extensions, without an encoding
object, limited to UTF-8).
* Backport the dynamic target
<https://htmlpreview.github.io/?https://github.com/jlfaucher/executor/blob/master/sandbox/jlf/_changes.html#dynamic-target-when-sending-a-message>
from
Executor.
* At this stage, the Unicode architecture is operational. Scripts that start with
"encoding:
utf-8" will automatically use RexxText instances when a string is not
ASCII. RexxText
instances do not yet provide the equivalent of RexxString's methods. By
default, RexxText
forwards messages to RexxString.
Thank you very much for the links, *very* interesting reading, but also technically overwhelming
(too much for me at the moment due to lack of time and knowhow in this domain).
However, I sincerely hope that you remain motivated, please!! :)
The final stage will be to implement the equivalent of RexxString's methods on RexxText, with full
Unicode support. These methods will use grapheme indexes exclusively.
For example, the description of the ~left method in the rexxref documentation
states:
"Returns a string of length length, containing the leftmost length characters of the receiving
string."
This description applies unchanged to RexxText, with character interpreted as
grapheme.
That would be *phantastic* !!
Please keep up your great work!!
---rony
_______________________________________________
Oorexx-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
