Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Moritz Hoffmann]

Well, the Java scripting framework will always use a Reader, which decodes
the input according to the current Charset. I don't see a way to circumvent
this, either by not using a Reader instance or by injecting a custom,
non-adapting Charset. I agree with Rick that this is not a problem that
ooRexx should solve. Instead, I'd base64-encode the scripts, and decode
them before passing them to the interpreter. Base64 will survive any
reasonable Java byte decoding magic.

Cheers,
Moritz

On Tue, Apr 9, 2019 at 3:34 PM Rony G. Flatscher 
wrote:

> On 09.04.2019 15:13, Rick McGuire wrote:
> > And it doesn't make sense to put the onus on solving this on the
> interpreter. If it is desired to
> > store binary code in an inherently text-based interface, then BSF4ooRexx
> should handle this be
> > including a utility to perform the transform and then decoding the
> format before passing it to the
> > interpreter.
>
> Well, please tell IBM, Oracle and the OpenJDK community which devised and
> applied all of the Java
> scripting frameworks in the past twenty years such that they expect
> scripts to be supplied as text
> only, not as binary data.
>
> If a Java program employs e.g. the "javax.script" framework and it is
> supplied the name of a Rexx
> program file, the "javax.script" framework will use some "java.io.Reader"
> to read the script from
> the file. In the case of rexxc-tokenized Rexx code the Reader will
> inadvertently destroy the binary
> data due to possible codepage translations well before BSF4ooRexx receives
> the (then ruined binary)
> script data.
>
> There is nothing, BSF4ooRexx could do here as it exploits/applies and
> depends on the Java scripting
> frameworks, unfortunately.
>
> Hence asking ooRexx to the rescue;  suggesting to add a single step before
> storing the rexxc
> produced binary data by turning the binary data into a hexadecimal string
> and store that lossless
> rendering;  and upon receiving a tokenized Rexx program adding a single
> reverse step to turn the
> hexadecimal string losslessly back into the needed binary form for
> unflatening.
>
> ---rony
>
>
>
>
> ___
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>


-- 
Moritz Hoffmann;
http://antiguru.de/
___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Rick McGuire]

Seriously, did you not not even read me email. This is totally solvable in
BSFooRexx by storing it a non-binary format defined by BSFooRexx that the
BSFooRexx framework can recognize as being a binary format that needs to be
decoded before passing it to the interpreter.  Provide a little utility
that can transform the compiled program into that format before hand.

Rick

On Tue, Apr 9, 2019 at 9:34 AM Rony G. Flatscher 
wrote:

> On 09.04.2019 15:13, Rick McGuire wrote:
> > And it doesn't make sense to put the onus on solving this on the
> interpreter. If it is desired to
> > store binary code in an inherently text-based interface, then BSF4ooRexx
> should handle this be
> > including a utility to perform the transform and then decoding the
> format before passing it to the
> > interpreter.
>
> Well, please tell IBM, Oracle and the OpenJDK community which devised and
> applied all of the Java
> scripting frameworks in the past twenty years such that they expect
> scripts to be supplied as text
> only, not as binary data.
>
> If a Java program employs e.g. the "javax.script" framework and it is
> supplied the name of a Rexx
> program file, the "javax.script" framework will use some "java.io.Reader"
> to read the script from
> the file. In the case of rexxc-tokenized Rexx code the Reader will
> inadvertently destroy the binary
> data due to possible codepage translations well before BSF4ooRexx receives
> the (then ruined binary)
> script data.
>
> There is nothing, BSF4ooRexx could do here as it exploits/applies and
> depends on the Java scripting
> frameworks, unfortunately.
>
> Hence asking ooRexx to the rescue;  suggesting to add a single step before
> storing the rexxc
> produced binary data by turning the binary data into a hexadecimal string
> and store that lossless
> rendering;  and upon receiving a tokenized Rexx program adding a single
> reverse step to turn the
> hexadecimal string losslessly back into the needed binary form for
> unflatening.
>
> ---rony
>
>
>
>
> ___
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Rony G. Flatscher]

On 09.04.2019 14:56, Michael Lueck wrote:
> Rony G. Flatscher wrote:
>>
>> However,  since he started to use "rexxc" to compile/tokenize all of the 
>> Rexx source files that
>> comprise his elaborate application to shelter his code, he has started to 
>> run into abends/errors
>> 13.1. 
>
>
> "sheltering code" is the only reason he chose to compile/tokenize his code... 
> there is no
> requirement to do so when using BSF4ooRexx?

Indeed "sheltering code" is the only (and quite understandable) reason.

BSF4ooRexx depends on the Java scripting frameworks from IBM, Oracle and 
OpenJDK, which expect
script code to be supplied as text and not as binary data (which unfortunately 
causes this problem).

---rony




___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Rick McGuire]

And it doesn't make sense to put the onus on solving this on the
interpreter. If it is desired to store binary code in an inherently
text-based interface, then BSF4ooRexx should handle this be including a
utility to perform the transform and then decoding the format before
passing it to the interpreter.

Rick

On Tue, Apr 9, 2019 at 9:10 AM Michael Lueck 
wrote:

> Greetings Rony,
>
>
> Rony G. Flatscher wrote:
> >
> > However,  since he started to use "rexxc" to compile/tokenize all of the
> Rexx source files that comprise his elaborate application to shelter his
> code, he has started to run into abends/errors 13.1.
>
>
> "sheltering code" is the only reason he chose to compile/tokenize his
> code... there is no requirement to do so when using BSF4ooRexx?
>
> While tokenizing ooRexx code seems appealing, I have seen time after time
> where tokenized ooRexx code fails to cooperate. I personally have
> "staunchly" decided against ever tokenizing any of my ooRexx
> code.
>
> Thus asking if it is a requirement I have somehow missed that "Programs
> utilizing / touching BSF4ooRexx MUST be tokenized."
>
> I am thankful,
>
> --
> Michael Lueck
> Lueck Data Systems
> http://www.lueckdatasystems.com/
>
>
> ___
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Michael Lueck]

Greetings Rony,


Rony G. Flatscher wrote:


However,  since he started to use "rexxc" to compile/tokenize all of the Rexx source files that comprise his elaborate application to shelter his code, he has started to run into abends/errors 13.1. 



"sheltering code" is the only reason he chose to compile/tokenize his code... 
there is no requirement to do so when using BSF4ooRexx?

While tokenizing ooRexx code seems appealing, I have seen time after time where tokenized ooRexx code fails to cooperate. I personally have "staunchly" decided against ever tokenizing any of my ooRexx 
code.


Thus asking if it is a requirement I have somehow missed that "Programs utilizing / 
touching BSF4ooRexx MUST be tokenized."

I am thankful,

--
Michael Lueck
Lueck Data Systems
http://www.lueckdatasystems.com/


___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Rony G. Flatscher]

On 09.04.2019 14:32, Rony G. Flatscher wrote:
... cut ...

> Unfortunately, running rexxc-tokenized/compiled scripts via the Java 
> scripting framework does not
> work as apprehended in my last post and causes the following runtime error:
>
> G:\test\oorexx\rexxc>rexxj test.rexx
> RexxDispatcher.java: Throwable of type 
> 'org.rexxla.bsf.engines.rexx.RexxException' thrown while invoking Rexx:
> getLocalizedMessage(): [BSF4ooRexx/routine/jniRexxRunProgram(), error 9:
> Rexx traceback line not available from the Rexx condition object (Rexx 
> condition may have been caused by a call or message from Java to Rexx)
> Error 3 running  line 0:  Failure during initialization.
> Error 3.903:  Program "G:\test\oorexx\rexxc\test.rexx" cannot be run by 
> this version of the REXX interpreter.]
>
> (The error text within the square brackets gets created in native code from 
> the Rexx condition
> object. The 'getLocalizedMessage():' right before the opening square bracket 
> seems to come from
> ooRexx, it is not used in the native code.)

... cut ...

The "getLocalizedMessage()" does *not* come from ooRexx, it comes from one of 
the Java programs.

---rony

___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

[Oorexx-devel] MacOS warnings

[Rony G. Flatscher]

When creating the ooRexx interpreter from trunk (r11856) on MacOS the following 
(not yet reported)
warnings get generated:

wu114184:oorexx_build rony$ make

... cut ...

[  4%] Building CXX object 
CMakeFiles/rexxapi.dir/rexxapi/client/LocalQueueManager.cpp.o

/Users/rony/dev/oorexx_allura/main/trunk/rexxapi/client/LocalQueueManager.cpp:571:14:
 warning: comparison of two values with different
  enumeration types in switch statement ('ServiceReturn' and 
'ErrorCode') [-Wenum-compare-switch]
case BAD_WAIT_FLAG:
 ^

/Users/rony/dev/oorexx_allura/main/trunk/rexxapi/client/LocalQueueManager.cpp:568:14:
 warning: comparison of two values with different
  enumeration types in switch statement ('ServiceReturn' and 
'ErrorCode') [-Wenum-compare-switch]
case BAD_FIFO_LIFO:
 ^

/Users/rony/dev/oorexx_allura/main/trunk/rexxapi/client/LocalQueueManager.cpp:565:14:
 warning: comparison of two values with different
  enumeration types in switch statement ('ServiceReturn' and 
'ErrorCode') [-Wenum-compare-switch]
case INVALID_QUEUE_NAME:
 ^~
3 warnings generated.
[  4%] Building CXX object 
CMakeFiles/rexxapi.dir/rexxapi/client/LocalRegistrationManager.cpp.o



... cut ...



[ 68%] Building CXX object 
CMakeFiles/rexx.dir/interpreter/platform/unix/SysRexxUtil.cpp.o

/Users/rony/dev/oorexx_allura/main/trunk/interpreter/platform/unix/SysRexxUtil.cpp:446:57:
 warning: format specifies type 'intmax_t'
  (aka 'long') but the argument has type 'off_t' (aka 'long long') 
[-Wformat]
snprintf(fileAttr, sizeof(fileAttr), "%20jd  ", finfo.st_size);
  ~ ^
  %20lld

/Users/rony/dev/oorexx_allura/main/trunk/interpreter/platform/unix/SysRexxUtil.cpp:454:57:
 warning: format specifies type 'intmax_t'
  (aka 'long') but the argument has type 'off_t' (aka 'long long') 
[-Wformat]
snprintf(fileAttr, sizeof(fileAttr), "%10jd  ", finfo.st_size);
  ~ ^
  %10lld

/Users/rony/dev/oorexx_allura/main/trunk/interpreter/platform/unix/SysRexxUtil.cpp:664:14:
 warning: 'sem_init' is deprecated
  [-Wdeprecated-declarations]
rc = sem_init(semdata->handle, 0, 0);
 ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/semaphore.h:55:42:
 note: 
  'sem_init' has been explicitly marked deprecated here
int sem_init(sem_t *, int, unsigned int) __deprecated;
 ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/cdefs.h:176:40:
 note: 
  expanded from macro '__deprecated'
#define __deprecated__attribute__((deprecated))
   ^

/Users/rony/dev/oorexx_allura/main/trunk/interpreter/platform/unix/SysRexxUtil.cpp:729:5:
 warning: 'sem_init' is deprecated
  [-Wdeprecated-declarations]
sem_init(semdata->handle, 1, 0);
^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/semaphore.h:55:42:
 note: 
  'sem_init' has been explicitly marked deprecated here
int sem_init(sem_t *, int, unsigned int) __deprecated;
 ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/cdefs.h:176:40:
 note: 
  expanded from macro '__deprecated'
#define __deprecated__attribute__((deprecated))
   ^

/Users/rony/dev/oorexx_allura/main/trunk/interpreter/platform/unix/SysRexxUtil.cpp:775:13:
 warning: 'sem_destroy' is deprecated
  [-Wdeprecated-declarations]
if (sem_destroy(semdata->handle))
^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/semaphore.h:53:26:
 note: 
  'sem_destroy' has been explicitly marked deprecated here
int sem_destroy(sem_t *) __deprecated;
 ^

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/sys/cdefs.h:176:40:
 note: 
  expanded from macro '__deprecated'
#define __deprecated__attribute__((deprecated))
   ^

/Users/rony/dev/oorexx_allura/main/trunk/interpreter/platform/unix/SysRexxUtil.cpp:889:14:
 warning: 'sem_init' is deprecated
  [-Wdeprecated-declarations]
   

Re: [Oorexx-devel] Ad compiled/tokenized Rexx code (rexxc) ...

[Rony G. Flatscher]

 fixes NewRoutine(), 
LoadPackageFromDate() and also
the Routine's class method newFile() when employed with 
rexxc-tokenized/compiled code, thank you
very much!

---

Unfortunately, running rexxc-tokenized/compiled scripts via the Java scripting 
framework does not
work as apprehended in my last post and causes the following runtime error:

G:\test\oorexx\rexxc>rexxj test.rexx
RexxDispatcher.java: Throwable of type 
'org.rexxla.bsf.engines.rexx.RexxException' thrown while invoking Rexx:
getLocalizedMessage(): [BSF4ooRexx/routine/jniRexxRunProgram(), error 9:
Rexx traceback line not available from the Rexx condition object (Rexx 
condition may have been caused by a call or message from Java to Rexx)
Error 3 running  line 0:  Failure during initialization.
Error 3.903:  Program "G:\test\oorexx\rexxc\test.rexx" cannot be run by 
this version of the REXX interpreter.]

(The error text within the square brackets gets created in native code from the 
Rexx condition
object. The 'getLocalizedMessage():' right before the opening square bracket 
seems to come from
ooRexx, it is not used in the native code.)

The reason for this runtime error with rexxc-tokenized/compiled code is caused 
by the implicitly
employed Java Reader object for reading the content of a file, changing the 
tokenized/compiled
binary data by applying codepage translations. (There are no Java based 
scripting frameworks that
would allow using a Stream for reading binary script data, unfortunately.)

---rony


On 07.04.2019 15:33, Rony G. Flatscher wrote:
> ... cut ...
>
> ---
>
> *Unfortunately*, there is one principal huge problem left with 
> compiled/tokenized Rexx code, that
> is beyond BSF4ooRexx' ability to solve for good: BSF4ooRexx uses two Java 
> scripting frameworks,
> one originating from IBM (DeveloperWorks) which got handed over to the ASF 
> named "BSF" (Bean
> Scripting Framework), one originating from Java itself which got introduced 
> later as the "Java
> scripting framework" in Java 1.6/6.0 (package named "javax.script", the URL 
> to the package
> documentation 
> ).
>
> BSF4ooRexx implements under the name "RexxScript" the javax.script scripting 
> framework, which
> makes it e.g. possible for JavaFX to denote the programming language "rexx" 
> in the FXML files that
> define the GUI. The FXMLLoader class will therefore be able to create an 
> ooRexx interpreter with
> the help of "javax.script" and execute the code in any given files or Rexx 
> code supplied verbaitm
> in event attributes of JavaFX GUI elements.
>
> The Java documentation for the javax.script.Engine interface that must be 
> (and has been)
> implemented can be found here:
> . 
> Executing ooRexx code
> from Java will be done with one of the eval() methods which either expect a 
> /java.lang.String/ or
> a /java.io.Reader/ object that represents the code to execute.
>
> A Java /Reader /will expect plain text to be read/and applies codepage 
> translations/ if necessary!
> This means that a /Reader /object if reading compiled/tokenized Rexx code 
> might transform the
> binary data! With other words, it is quite likely that compiled/tokenized 
> (binary data) Rexx code
> read by Java from a file with a /Reader/ gets destroyed by applying 
> mistakingly code-page
> translations to that binary data! :-((
>
> ---
>
> A maybe feasible solution could be to apply c2x() to the binary data that 
> results when applying
> "rexxc" and store that data right after the eye-catcher string "/**/@REXX" 
> that indicates that the
> remaining data represents the compiled/tokenized version of a Rexx program. 
> Upon loading, that
> hexadecimal string representing the binary data could be changed back to the 
> original binary
> representation with x2c() before processing it.
>
> This way all the Java defined String/Reader based interfaces would keep 
> working and there would be
> no risk whatsoever, that automatically applied codepage translations could 
> ruin/destroy the binary
> data as that data would be turned into plain text by c2x()!
>
> I would file a short RFE to this effect, but wanted to give "the long story" 
> (trying to explain
> the background and why it would be very important for allowing the Java 
> scripting frameworks to be
> applied safely to compiled/tokenized Rexx code as well) here.
>
> ---rony
>

___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel