[Oorexx-devel] A few questions in the context of RexxCreateInterpreter, LIbraries and co-existence with the classic functions ...
While working on moving BSF4Rexx to the 4.0 APIs and at the same time keeping it backwardly compatible with Rexx scripts using it, I have been running into problems for which the cause is not clear to me (and debugging doesn't really help as the JVM takes over the debugging and I cannot get into rexx.exe or BSF4Rexx.dll when exceptions occur, just a log-file created by the JVM). For backward compatibilty it is important to have the functions RxFuncQuery(), RxFuncLoad() remaining operational (at least not causing runtime errors) such that the following code can get executed: if rxFuncQuery(BSF) = 1 then /* BSF() support not loaded yet ? */ do call rxFuncAdd BsfLoadFuncs, BSF4Rexx, BsfLoadFuncs call BsfLoadFuncs call BsfLoadJava end There is a list of thirteen such functions. Question 1: Is there a way to get to the RexxInterpreter instance pointer or RexxThreadContext pointer (which would allow for finding its RexxInterpreter instance) from within an external classic Rexx function? If there was such a function, then it would be possible to let BsfLoadFuncs be implemented in the classic style, but load from the code there the library containing the new typed REXX_ROUTINE functions. This would allow replacing the rest of the external functions by new REXX_TYPEDROUTINEs, which I would prefer over the classic style seeing how easy it is to use the new 4.0 APIs. --- Currently, the classic external functions are in place and the behaviour is the same as with the non-4.00 version, using RexxStart() to run the programs. Trying to take advantage of the new RexxCreateInterpreter() to later let scripts be executed by that instance, the following phenomenon occurs: Calling an external classic Rexx function BsfQueryRegisteredFunctions() will return a stem with the registered external functions. This external function uses RexxVariablePool() to create the stem entries, but has no effect anymore. Removing an unrelated RexxCreateInterpreter() call, fixes the behaviour, i.e. the stem values are created in the caller's context using the RexxVariablePool, as it did before. Question 2: Is this an expected behaviour or a bug? If a bug, is this information already sufficient to research the cause? --- While implementing the RexxCreateInterpreter() and using the options with either LOAD_REQUIRED_LIBRARY or REIGSTER_LIBRARY will cause a crash (again, unfortunately without being able to get there with the VS debugger). The context of this crash is as follows: Java loads BSF4Rexx (so the DLL is loaded already that contains the classic external functions), a Java engine uses a new jni function which calls into the DLL where RexxCreateInterpreter() is executed with either one of the above options in place. Question 3: Is this an expected behaviour or a bug? Question 4: If in one RexxInterpreter instance a library is loaded, are its routines visible to other RexxInstance instances as well ? --- Question 5: Is it possible to call a REXX_TYPED_ROUTINE from another REXX_TYPED_ROUTINE or from a REXX_TYPED_METHOD in native code? If so, how? ---rony -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] A few questions in the context of RexxCreateInterpreter, LIbraries and co-existence with the classic functions ...
On Mon, May 4, 2009 at 11:43 AM, Rony G. Flatscher rony.flatsc...@wu-wien.ac.at wrote: While working on moving BSF4Rexx to the 4.0 APIs and at the same time keeping it backwardly compatible with Rexx scripts using it, I have been running into problems for which the cause is not clear to me (and debugging doesn't really help as the JVM takes over the debugging and I cannot get into rexx.exe or BSF4Rexx.dll when exceptions occur, just a log-file created by the JVM). For backward compatibilty it is important to have the functions RxFuncQuery(), RxFuncLoad() remaining operational (at least not causing runtime errors) such that the following code can get executed: if rxFuncQuery(BSF) = 1 then /* BSF() support not loaded yet ? */ do call rxFuncAdd BsfLoadFuncs, BSF4Rexx, BsfLoadFuncs call BsfLoadFuncs call BsfLoadJava end There is a list of thirteen such functions. Question 1: Is there a way to get to the RexxInterpreter instance pointer or RexxThreadContext pointer (which would allow for finding its RexxInterpreter instance) from within an external classic Rexx function? No there is not. However, if you just add BsfLoadFuncs() to the list of function defined in the BSF4Rexx() package and make it a NOP when called, then things will appear to behave the same way. This is the same approach that's been taken by packages in the interpreter itself that have been converted to the new package format. For example, see what the rxsock library is doing in the interpreter build. If there was such a function, then it would be possible to let BsfLoadFuncs be implemented in the classic style, but load from the code there the library containing the new typed REXX_ROUTINE functions. This would allow replacing the rest of the external functions by new REXX_TYPEDROUTINEs, which I would prefer over the classic style seeing how easy it is to use the new 4.0 APIs. If you implement the suggestion I made above, this will work just fine. The rxmath package is a good example. It still has a registration routine, but all the functions are defined using the typed format. There is no need to register special stubs. --- Currently, the classic external functions are in place and the behaviour is the same as with the non-4.00 version, using RexxStart() to run the programs. Trying to take advantage of the new RexxCreateInterpreter() to later let scripts be executed by that instance, the following phenomenon occurs: Calling an external classic Rexx function BsfQueryRegisteredFunctions() will return a stem with the registered external functions. This external function uses RexxVariablePool() to create the stem entries, but has no effect anymore. Removing an unrelated RexxCreateInterpreter() call, fixes the behaviour, i.e. the stem values are created in the caller's context using the RexxVariablePool, as it did before. There can only be one interpreter instance in control of a thread at a given time. If you call RexxCreateInterpreter in the context of one of those callouts, then all API calls will be handled by the thread context you just created that pushed all of the other stuff down. Question 2: Is this an expected behaviour or a bug? If a bug, is this information already sufficient to research the cause? This is the expected behavior. You're creating a new context associated with that thread. It is in control of all call-backs on that thread until the instance is terminated. To get the non-nested behavior you want, you'll probably need to spin off another thread to create the interpreter instance and keep it alive. Then, to access this instance in a nested fashion, you'll need to use AttachThread()/DetachThread() in pairs. --- While implementing the RexxCreateInterpreter() and using the options with either LOAD_REQUIRED_LIBRARY or REIGSTER_LIBRARY will cause a crash (again, unfortunately without being able to get there with the VS debugger). The context of this crash is as follows: Java loads BSF4Rexx (so the DLL is loaded already that contains the classic external functions), a Java engine uses a new jni function which calls into the DLL where RexxCreateInterpreter() is executed with either one of the above options in place. Question 3: Is this an expected behaviour or a bug? A crash is never the expected behavior. Open a bug, but you'll probably need to give me enough to recreate the problem. Question 4: If in one RexxInterpreter instance a library is loaded, are its routines visible to other RexxInstance instances as well ? Yes, the library packages are global to the process. --- Question 5: Is it possible to call a REXX_TYPED_ROUTINE from another REXX_TYPED_ROUTINE or from a REXX_TYPED_METHOD in native code? If so, how? No, the entry points are only usable through using the Rexx runtime. If you wish to share code, then make the package routines be just stubs that forward to the appropriate routines. The stream library code in the
Re: [Oorexx-devel] A few questions in the context of RexxCreateInterpreter, LIbraries and co-existence with the classic functions ...
Rick McGuire wrote: On Mon, May 4, 2009 at 11:43 AM, Rony G. Flatscher rony.flatsc...@wu-wien.ac.at wrote: While working on moving BSF4Rexx to the 4.0 APIs and at the same time keeping it backwardly compatible with Rexx scripts using it, I have been running into problems for which the cause is not clear to me (and debugging doesn't really help as the JVM takes over the debugging and I cannot get into rexx.exe or BSF4Rexx.dll when exceptions occur, just a log-file created by the JVM). For backward compatibilty it is important to have the functions RxFuncQuery(), RxFuncLoad() remaining operational (at least not causing runtime errors) such that the following code can get executed: if rxFuncQuery(BSF) = 1 then /* BSF() support not loaded yet ? */ do call rxFuncAdd BsfLoadFuncs, BSF4Rexx, BsfLoadFuncs call BsfLoadFuncs call BsfLoadJava end There is a list of thirteen such functions. Question 1: Is there a way to get to the RexxInterpreter instance pointer or RexxThreadContext pointer (which would allow for finding its RexxInterpreter instance) from within an external classic Rexx function? No there is not. However, if you just add BsfLoadFuncs() to the list of function defined in the BSF4Rexx() package and make it a NOP when called, then things will appear to behave the same way. This is the same approach that's been taken by packages in the interpreter itself that have been converted to the new package format. For example, see what the rxsock library is doing in the interpreter build. If there was such a function, then it would be possible to let BsfLoadFuncs be implemented in the classic style, but load from the code there the library containing the new typed REXX_ROUTINE functions. This would allow replacing the rest of the external functions by new REXX_TYPEDROUTINEs, which I would prefer over the classic style seeing how easy it is to use the new 4.0 APIs. If you implement the suggestion I made above, this will work just fine. The rxmath package is a good example. It still has a registration routine, but all the functions are defined using the typed format. There is no need to register special stubs. Great, thanks for this pointer, will look it up! --- Currently, the classic external functions are in place and the behaviour is the same as with the non-4.00 version, using RexxStart() to run the programs. Trying to take advantage of the new RexxCreateInterpreter() to later let scripts be executed by that instance, the following phenomenon occurs: Calling an external classic Rexx function BsfQueryRegisteredFunctions() will return a stem with the registered external functions. This external function uses RexxVariablePool() to create the stem entries, but has no effect anymore. Removing an unrelated RexxCreateInterpreter() call, fixes the behaviour, i.e. the stem values are created in the caller's context using the RexxVariablePool, as it did before. There can only be one interpreter instance in control of a thread at a given time. If you call RexxCreateInterpreter in the context of one of those callouts, then all API calls will be handled by the thread context you just created that pushed all of the other stuff down. This problem should go away when using the new typed routine. Question: How can one create/set a variable in the context of the caller? AFAIK one can only retrieve the existing variables in the caller's context, but not create/set a new one (in this case I would need to create a stem object and set it to the callers context variable pool using the supplied name for the stem). Question 2: Is this an expected behaviour or a bug? If a bug, is this information already sufficient to research the cause? This is the expected behavior. You're creating a new context associated with that thread. It is in control of all call-backs on that thread until the instance is terminated. To get the non-nested behavior you want, you'll probably need to spin off another thread to create the interpreter instance and keep it alive. Then, to access this instance in a nested fashion, you'll need to use AttachThread()/DetachThread() in pairs. Yes, this would be the planned exploitation. Knowing the difference in the mechanics this won't be a problem anymore, thanks again for the explanation! --- While implementing the RexxCreateInterpreter() and using the options with either LOAD_REQUIRED_LIBRARY or REIGSTER_LIBRARY will cause a crash (again, unfortunately without being able to get there with the VS debugger). The context of this crash is as follows: Java loads BSF4Rexx (so the DLL is loaded already that contains the classic external functions), a Java engine uses a new jni function which calls into the DLL where RexxCreateInterpreter() is executed with either one of the above options in place. Question 3: Is this an expected behaviour or a bug?
Re: [Oorexx-devel] A few questions in the context of RexxCreateInterpreter, LIbraries and co-existence with the classic functions ...
On Mon, May 4, 2009 at 12:12 PM, Rony G. Flatscher rony.flatsc...@wu-wien.ac.at wrote: Rick McGuire wrote: On Mon, May 4, 2009 at 11:43 AM, Rony G. Flatscher rony.flatsc...@wu-wien.ac.at wrote: While working on moving BSF4Rexx to the 4.0 APIs and at the same time keeping it backwardly compatible with Rexx scripts using it, I have been running into problems for which the cause is not clear to me (and debugging doesn't really help as the JVM takes over the debugging and I cannot get into rexx.exe or BSF4Rexx.dll when exceptions occur, just a log-file created by the JVM). For backward compatibilty it is important to have the functions RxFuncQuery(), RxFuncLoad() remaining operational (at least not causing runtime errors) such that the following code can get executed: if rxFuncQuery(BSF) = 1 then /* BSF() support not loaded yet ? */ do call rxFuncAdd BsfLoadFuncs, BSF4Rexx, BsfLoadFuncs call BsfLoadFuncs call BsfLoadJava end There is a list of thirteen such functions. Question 1: Is there a way to get to the RexxInterpreter instance pointer or RexxThreadContext pointer (which would allow for finding its RexxInterpreter instance) from within an external classic Rexx function? No there is not. However, if you just add BsfLoadFuncs() to the list of function defined in the BSF4Rexx() package and make it a NOP when called, then things will appear to behave the same way. This is the same approach that's been taken by packages in the interpreter itself that have been converted to the new package format. For example, see what the rxsock library is doing in the interpreter build. If there was such a function, then it would be possible to let BsfLoadFuncs be implemented in the classic style, but load from the code there the library containing the new typed REXX_ROUTINE functions. This would allow replacing the rest of the external functions by new REXX_TYPEDROUTINEs, which I would prefer over the classic style seeing how easy it is to use the new 4.0 APIs. If you implement the suggestion I made above, this will work just fine. The rxmath package is a good example. It still has a registration routine, but all the functions are defined using the typed format. There is no need to register special stubs. Great, thanks for this pointer, will look it up! --- Currently, the classic external functions are in place and the behaviour is the same as with the non-4.00 version, using RexxStart() to run the programs. Trying to take advantage of the new RexxCreateInterpreter() to later let scripts be executed by that instance, the following phenomenon occurs: Calling an external classic Rexx function BsfQueryRegisteredFunctions() will return a stem with the registered external functions. This external function uses RexxVariablePool() to create the stem entries, but has no effect anymore. Removing an unrelated RexxCreateInterpreter() call, fixes the behaviour, i.e. the stem values are created in the caller's context using the RexxVariablePool, as it did before. There can only be one interpreter instance in control of a thread at a given time. If you call RexxCreateInterpreter in the context of one of those callouts, then all API calls will be handled by the thread context you just created that pushed all of the other stuff down. This problem should go away when using the new typed routine. Question: How can one create/set a variable in the context of the caller? If called as a routine or an exit, you can use the SetContextVariable() API to set the variable. In other contexts, this is not permitted. AFAIK one can only retrieve the existing variables in the caller's context, but not create/set a new one (in this case I would need to create a stem object and set it to the callers context variable pool using the supplied name for the stem). There are Get and Set APIs for variables. These only access the top-level variables, so yes, if you wish to set a stem variable to something, you first create the stem object, populate it, then set the stem variable to the corresponding stem object. Question 2: Is this an expected behaviour or a bug? If a bug, is this information already sufficient to research the cause? This is the expected behavior. You're creating a new context associated with that thread. It is in control of all call-backs on that thread until the instance is terminated. To get the non-nested behavior you want, you'll probably need to spin off another thread to create the interpreter instance and keep it alive. Then, to access this instance in a nested fashion, you'll need to use AttachThread()/DetachThread() in pairs. Yes, this would be the planned exploitation. Knowing the difference in the mechanics this won't be a problem anymore, thanks again for the explanation! --- While implementing the RexxCreateInterpreter() and using the options with either LOAD_REQUIRED_LIBRARY or REIGSTER_LIBRARY
