Re: [Oorexx-devel] Ad debugging multithreaded programs, a working test version

2024-01-29 Thread Rony G. Flatscher

On 29.01.2024 11:04, P.O. Jonsson wrote:




Am 28.01.2024 um 17:32 schrieb Rony G. Flatscher :

The creation of the working test version has costed me already much more time 
than budgeted.


Are you aware of any software project that didn’t ? ;-) You forgot the Pi 
factor*



* „Every project takes 3.14 times longer to conclude than planned“


8-))

___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel


Re: [Oorexx-devel] Ad debugging multithreaded programs, a working test version

2024-01-29 Thread P.O. Jonsson


> Am 28.01.2024 um 17:32 schrieb Rony G. Flatscher :
> 
> The creation of the working test version has costed me already much more time 
> than budgeted.

Are you aware of any software project that didn’t ? ;-) You forgot the Pi 
factor*



* „Every project takes 3.14 times longer to conclude than planned“


Hälsningar/Regards/Grüsse,
P.O. Jonsson
oor...@jonases.se

___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel


Re: [Oorexx-devel] Ad debugging multithreaded programs, a working test version

2024-01-28 Thread Rony G. Flatscher

Dear P.O.,

On 28.01.2024 17:22, P.O. Jonsson wrote:
An obvious candidate for testing this might be the test framework for ooRexx. When running the 
tests on Windows (7,8,10,11) it invariably gets stuck at the very end (after a successful test run 
) suggesting a dangling/blocked process somewhere in the framework that prevent the testing to exit.

:)
This only happens when running Windows in a virtual machine, we do not see it when testing on 
native Windows. This gives a possibility to compare.


Feel free to use any of the Windows WMs connected to Jenkins.


Currently I am afraid, that I do not have the necessary time. However, we can do so together once 
the concept and the implementation gets accepted such that investing further time is not "a fond 
perdu".


Personally I have also use cases where the multihtreaded trace will be important (and instrumental 
to even dare to debug one particular complex one). But for that I also need time to create an 
analyzer that would run the program concurrently and once deadlocked use the collected TraceObjects 
and analyze them (e.g. marking method routine traces in which state they are effectively and then 
hopefully becoming able to understand what is happening where, etc.).


The creation of the working test version has costed me already much more time 
than budgeted.

Need now time to finalize papers, grading students, creating the RexxLA symposium presentations and 
much more. So I hope that the "working test version" is acceptable and does not have errors (and if 
so, fixing them not being too time consuming).


Best regards

---rony




___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel


Re: [Oorexx-devel] Ad debugging multithreaded programs, a working test version

2024-01-28 Thread P.O. Jonsson
Dear Rony,

An obvious candidate for testing this might be the test framework for ooRexx. 
When running the tests on Windows (7,8,10,11) it invariably gets stuck at the 
very end (after a successful test run ) suggesting a dangling/blocked process 
somewhere in the framework that prevent the testing to exit.

This only happens when running Windows in a virtual machine, we do not see it 
when testing on native Windows. This gives a possibility to compare.

Feel free to use any of the Windows WMs connected to Jenkins.

Hälsningar/Regards/Grüsse,
P.O. Jonsson
oor...@jonases.se




> Am 28.01.2024 um 16:31 schrieb Rony G. Flatscher :
> 
> At  
>  you will find a portable 
> ooRexx debug version for Windows that includes a test version of the 
> suggested multithreaded trace, implemented in a way that adheres to Rick's 
> idea, such that a Rexx programmer can freely format traces as he or she 
> wishes.
> 
> The above web site includes Rexx test scripts that are only meant to 
> demonstrate and test the current implemtation.
> 
> The readme.txt file includes suggested drafts and is copied to this message 
> at the end.
> 
> Please post any feedback, findings, bugs here.
> 
> ---rony
> 
> readme.txt:
> 
> Multithreaded tracing, TraceObject class
> 
> 
> This readme.txt file accompanies a portable Windows version of ooRexx 5.1beta
> that implements multithreaded trace prefixes and makes it possible to create
> one own's trace line renderings as well as to use the infrastructure for
> profiling (analyzing) the execution of Rexx programs.
> 
> - unzip the ooRexx portable version, go into the directory and run
>   "setupoorexx.cmd" which creates batch files, e.g.:
> 
>   - setenv2rxenv.cmd ... run to change PATH to first point to the
>  portable version of ooRexx; run Rexx programs
>  with "rexx myProgram.rex" such that the
>  operating system looks and finds for rexx.exe
> 
> - there are three test Rexx programs (you can use your own Rexx programs):
> 
>   - test01.rex ... a simple test Rexx program
> 
>   - test02.rex ... a multithreaded test Rexx program to test guarded and
>unguarded method routines
> 
>   - test03.rex ... this is test01.rex with an intenional syntax error at the
>end
> 
> - there are three Rexx programs that exploit the new infrastructure:
> 
>   - p1_traceRunner.rex ... executes the Rexx program (name supplied via the
>command line argument) four times, once with
>.TraceObject~option set to 'N' (normal),
>'S' (standard), 'F' (full), 'P' (profiling mode);
>each run will have the generated TraceObjects
>collected in an array that could be used to analyze
>the runs afterwards
> 
>   - p2_traceInterceptor.rex ... like p1_traceRunner.rex in addition:
>demonstrates how to intercept each TraceObject
> 
>   - p3_traceEditingPrefix.rex ... like p2_traceInterceptor.rex in addition:
>demonstrates how to change the trace line
>information
> 
> To use the infrastructure:
> 
>- unzip the portable ooRexx version, run setupoorexx.cmd, then the 
> generated
>  script setenv2rxenv.cmd and remain in that session to run the above
>  programs (PATH got set in it), e.g.
> 
> rexx p1_traceRunner.rextest01.rex (or any other Rexx program)
> rexx p2_traceInterceptor.rex   test02.rex (or any other Rexx program)
> rexx p3_traceEditingPrefix.rex test03.rex (or any other Rexx program)
> 
> Please note: the test programs are artificial and got created to test and
> demonstrate the infrastructure.
> 
> 
> ---
> 
> What follows are drafts for rexxref.pdf.
> 
> 
> -
> 
> Starting out with the current documentation the nomenclatura is kept the same,
> possibly with additional information meant for the debugging section in
> rexxref.pdf.
> 
> --- draft (rexxref.pdf) - begin ---
> 
> 15.4 Debugging Multithreaded Programs
> 
> ooRexx allows for executing different parts of ooRexx programs on multiple
> threads of execution using different Rexx interpreter within a single
> operating system process.  This makes it difficult and at times even
> impossible to debug such multithreaded programs without any additional
> information about the specific execution context.  Relevant information
> for debugging multithreaded programs may be: the Rexx interpreter instance
> that executes an invocation ("activation") running on which operating
> system thread and whether a lock is held for exclusive access of 

[Oorexx-devel] Ad debugging multithreaded programs, a working test version

2024-01-28 Thread Rony G. Flatscher
At  you will find a portable ooRexx debug version for 
Windows that includes a test version of the suggested multithreaded trace, implemented in a way that 
adheres to Rick's idea, such that a Rexx programmer can freely format traces as he or she wishes.


The above web site includes Rexx test scripts that are only meant to demonstrate and test the 
current implemtation.


The readme.txt file includes suggested drafts and is copied to this message at 
the end.

Please post any feedback, findings, bugs here.

---rony

readme.txt:

   Multithreaded tracing, TraceObject class
   

   This readme.txt file accompanies a portable Windows version of ooRexx 5.1beta
   that implements multithreaded trace prefixes and makes it possible to create
   one own's trace line renderings as well as to use the infrastructure for
   profiling (analyzing) the execution of Rexx programs.

   - unzip the ooRexx portable version, go into the directory and run
  "setupoorexx.cmd" which creates batch files, e.g.:

  - setenv2rxenv.cmd ... run to change PATH to first point to the
 portable version of ooRexx; run Rexx programs
 with "rexx myProgram.rex" such that the
 operating system looks and finds for rexx.exe

   - there are three test Rexx programs (you can use your own Rexx programs):

  - test01.rex ... a simple test Rexx program

  - test02.rex ... a multithreaded test Rexx program to test guarded and
   unguarded method routines

  - test03.rex ... this is test01.rex with an intenional syntax error at the
   end

   - there are three Rexx programs that exploit the new infrastructure:

  - p1_traceRunner.rex ... executes the Rexx program (name supplied via the
   command line argument) four times, once with
   .TraceObject~option set to 'N' (normal),
   'S' (standard), 'F' (full), 'P' (profiling mode);
   each run will have the generated TraceObjects
   collected in an array that could be used to 
analyze
   the runs afterwards

  - p2_traceInterceptor.rex ... like p1_traceRunner.rex in addition:
   demonstrates how to intercept each TraceObject

  - p3_traceEditingPrefix.rex ... like p2_traceInterceptor.rex in addition:
   demonstrates how to change the trace line
   information

   To use the infrastructure:

   - unzip the portable ooRexx version, run setupoorexx.cmd, then the 
generated
 script setenv2rxenv.cmd and remain in that session to run the above
 programs (PATH got set in it), e.g.

rexx p1_traceRunner.rextest01.rex (or any other Rexx 
program)
rexx p2_traceInterceptor.rex   test02.rex (or any other Rexx 
program)
rexx p3_traceEditingPrefix.rex test03.rex (or any other Rexx 
program)

   Please note: the test programs are artificial and got created to test and
   demonstrate the infrastructure.


   ---

   What follows are drafts for rexxref.pdf.


   -

   Starting out with the current documentation the nomenclatura is kept the 
same,
   possibly with additional information meant for the debugging section in
   rexxref.pdf.

   --- draft (rexxref.pdf) - begin ---

15.4 Debugging Multithreaded Programs

ooRexx allows for executing different parts of ooRexx programs on 
multiple
threads of execution using different Rexx interpreter within a single
operating system process.  This makes it difficult and at times even
impossible to debug such multithreaded programs without any additional
information about the specific execution context.  Relevant information
for debugging multithreaded programs may be: the Rexx interpreter 
instance
that executes an invocation ("activation") running on which operating
system thread and whether a lock is held for exclusive access of an
object's attribute ("object variable") pool.

To ease debugging of multithreaded programs the execution context can be
indicated with a "multithreaded trace prefix" ("OPTION") string in the
following form "[R1 T1 I5 G A1 L2 *]", where

  - "R1": "R" stands for Rexx interpreter instance and the number (a
 counter) indicates which of the instances executes the current
 invocation (activation).

  - "T1": "T" stands for (operating system) thread and the number (a
 counter) indicates on which of the currently existing threads the
 activity gets executed

  - "I5": "I"