Re: [Oorexx-devel] A draft for documenting MT tracing (Re: Planning to add multithreaded (concurrent) tracing (Re: RFC for feature request "794 Concurrency request"

[P. O. Jonsson]

Just like Jon I see myself as a user not a developer (of ooRexx) but in rare cases I would appreciate a trace for multiple threads. We have number of test cases that work when executed as-is but that hang or fail when run in the test framework. I have no opinion on what form it should have but if Ricks proposal of a unique identifier of each thread in the normal trace is sufficient that would be less invasive but per definition a trace is “Ugly” in the sense that it is proof of a failure 😣 Von meinem iPhone gesendetAm 12.02.2023 um 18:48 schrieb Sahananda Sahananda :I don't know if it is helpful for me to say this, but just in case.  When creating ooDialog scripts (I think of myself principally as a USER of ooRexx and ooDialog though I also have my users who don't code) I have sometimes had a great need for this kind of trace.Things can hang, and it often isn't clear where they are or the journey they have taken to get there, as you are handling (perhaps simultaneously) messages from Windows GUI.In the past I have had to resort (with Mark Miesfeld or Rick's help) to the C++ tools to get a chance of understanding what is going on - and for a mere user like me - that is far more difficult.I can understand how users could go through life, and never be in a position where they need to look at multiple threads, and I'm glad for them, but should they prevent these tools being available when they are needed.hope that helpsJonOn Sun, 12 Feb 2023 at 14:41, Rony G. Flatscher  wrote:
  

  
  
On 11.02.2023 20:11, Mike Cowlishaw
  wrote:


  
  
  Similar
comments:  this extra information seems to be aimed at
'developers', not users.   A user seeing the proposed output
would have no idea what it means or what it's about.  It's
astonishing, in every sense.
  

Not sure what you mean when writing "developers" or "users" in
  this context. 
Just to be clear about the meanings in this e-mail context:
  "developer" means the "developer of an ooRexx program" and "user"
  means "user of an ooRexx program (probably not a programmer)".
This is about the TRACE instruction that gets usually used for
  debugging when developing programs, users of programs usually do
  not get to see TRACE output (with the single exception that a
  command causing a failure will be traced by default).


  
To me this seems absolutely contrary to the Rexx
principles.   If developers need this kind of information,
cannot this be achieved by other, less visible and less
ugly, means?

There may be a misunderstanding here.

Multithreaded tracing is meant to help the *developers* of an
  ooRexx program to trace ooRexx programs that execute in different
  threads for debugging purposes (there is no need for this in
  classic Rexx). The next context is that the execution environment
  are method routines, i.e. method routines usually defined in
  classes and being executed on behalf of an ooRexx object (as
  classic Rexx has no classes/methods there is no reason to trace
  them).
The ooRexx reference, rexxref.pdf, documents concurrency in
  ooRexx in chapter "12. Concurrency" such that one can expect that
  there are ooRexx programmers who will apply/exploit it.
---

The ooRexx TRACE is modeled after the classic Rexx trace and
  currently does unfortunately not have the ability to gain insight
  of what happens when ooRexx programs execute in a mulithreaded
  fashion.
Take a look at this ooRexx program taking advantage of ooRexx'
  concurrency:

  c1 = .c~new
call syssleep 0.5
c1~m2 -- wake-up m1
say "done"

::class C
::method init
expose s
s = 0
reply

self~m1

::method m1
expose s
s = 1
guard off
say "before guard" -- here, no lock
guard on when s <> 1 -- but here, is locked while waiting...
say "after guard"

::method m2
expose s
s = 2

-- ::options trace a



If you run this program it will hang (run into a deadlock), here
  the output thanks to some say-statements for debugging:

  G:\test\orx\trace>jlf_dl1.rex
before guard


The program just hangs, one must CTL-C to break the running
  program. 

So where is the problem? The current TRACE does not help to shed
  any light on the (concurrent) ooRexx context.
This is where the MT tracing enhancement comes into play,
  following the "Rexx principles" to make the programmers better
  understand what happens and how the Rexx program gets executed.
  After all, this is the motivation that you came up with TRACE,
  right?
So in the post-classic Rexx, ooRexx, allowing for exploiting OO,
  concurrency, messaging etc., TRACE needs to be enhanced
  accordingly to help and ease th

Re: [Oorexx-devel] A draft for documenting MT tracing (Re: Planning to add multithreaded (concurrent) tracing (Re: RFC for feature request "794 Concurrency request"

[Sahananda Sahananda]

I don't know if it is helpful for me to say this, but just in case.

When creating ooDialog scripts (I think of myself principally as a USER of
ooRexx and ooDialog though I also have my users who don't code) I have
sometimes had a great need for this kind of trace.

Things can hang, and it often isn't clear where they are or the journey
they have taken to get there, as you are handling (perhaps simultaneously)
messages from Windows GUI.

In the past I have had to resort (with Mark Miesfeld or Rick's help) to the
C++ tools to get a chance of understanding what is going on - and for a
mere user like me - that is far more difficult.

I can understand how users could go through life, and never be in a
position where they need to look at multiple threads, and I'm glad for
them, but should they prevent these tools being available when they are
needed.

hope that helps

Jon

On Sun, 12 Feb 2023 at 14:41, Rony G. Flatscher 
wrote:

> On 11.02.2023 20:11, Mike Cowlishaw wrote:
>
> Similar comments:  this extra information seems to be aimed at
> 'developers', not users.   A user seeing the proposed output would have no
> idea what it means or what it's about.  It's astonishing, in every sense.
>
> Not sure what you mean when writing "developers" or "users" in this
> context.
>
> Just to be clear about the meanings in this e-mail context: "developer"
> means the "developer of an ooRexx program" and "user" means "user of an
> ooRexx program (probably not a programmer)".
>
> This is about the TRACE instruction that gets usually used for debugging
> when developing programs, users of programs usually do not get to see TRACE
> output (with the single exception that a command causing a failure will be
> traced by default).
>
>
> To me this seems absolutely contrary to the Rexx principles.   If
> developers need this kind of information, cannot this be achieved by other,
> less visible and less ugly, means?
>
> There may be a misunderstanding here.
>
> Multithreaded tracing is meant to help the *developers* of an ooRexx
> program to trace ooRexx programs that execute in different threads for
> debugging purposes (there is no need for this in classic Rexx). The next
> context is that the execution environment are method routines, i.e. method
> routines usually defined in classes and being executed on behalf of an
> ooRexx object (as classic Rexx has no classes/methods there is no reason to
> trace them).
>
> The ooRexx reference, rexxref.pdf, documents concurrency in ooRexx in
> chapter "12. Concurrency" such that one can expect that there are ooRexx
> programmers who will apply/exploit it.
>
> ---
>
> The ooRexx TRACE is modeled after the classic Rexx trace and currently
> does unfortunately not have the ability to gain insight of what happens
> when ooRexx programs execute in a mulithreaded fashion.
>
> Take a look at this ooRexx program taking advantage of ooRexx' concurrency:
>
> c1 = .c~new
> call syssleep 0.5
> c1~m2 -- wake-up m1
> say "done"
>
> ::class C
> ::method init
> expose s
> s = 0
> reply
>
> self~m1
>
> ::method m1
> expose s
> s = 1
> guard off
> say "before guard" -- here, no lock
> guard on when s <> 1 -- but here, is locked while waiting...
> say "after guard"
>
> ::method m2
> expose s
> s = 2
>
> -- ::options trace a
>
>
> If you run this program it will hang (run into a deadlock), here the
> output thanks to some say-statements for debugging:
>
> G:\test\orx\trace>jlf_dl1.rex
> before guard
>
> The program just hangs, one must CTL-C to break the running program.
>
> So where is the problem? The current TRACE does not help to shed any light
> on the (concurrent) ooRexx context.
>
> This is where the MT tracing enhancement comes into play, following the
> "Rexx principles" to make the programmers better understand what happens
> and how the Rexx program gets executed. After all, this is the motivation
> that you came up with TRACE, right?
>
> So in the post-classic Rexx, ooRexx, allowing for exploiting OO,
> concurrency, messaging etc., TRACE needs to be enhanced accordingly to help
> and ease the ooRexx program developer with his work when hitting problems
> because of the complex environment concurrently executing ooRexx programs
> have to relate to. It is about making debugging in this application area
> easier than is currently possible.
>
> To illustrate, firstly here a TRACE A in "classic Rexx style" as currently
> implemented in ooRexx 5.0 (by uncommenting the options directive at the end
> of the above program which will set TRACE A for the entire package, i.e.
> for all routines in that program):
>
> G:\test\orx\trace>jlf_dl1.rex
>  1 *-* c1 = .c~new
>>I> Method "INIT" with scope "C" in package 
> "G:\test\orx\trace\jlf_dl1.rex".
>  8 *-* expose s
>  9 *-* s = 0
> 10 *-* reply
>  2 *-* call syssleep 0.5
>>I> Method "INIT" with scope "C" in package 
> "G:\test\orx\trace\jlf_dl1.rex".
> 12 *-* self~m1
> 

Re: [Oorexx-devel] Ad '!' as trace prefix letter (Re: Planning to add multithreaded (concurrent) tracing (Re: RFC for feature request "794 Concurrency request"

[Rony G. Flatscher]

On 12.02.2023 15:32, Rick McGuire wrote:



On Sun, Feb 12, 2023 at 9:26 AM Rony G. Flatscher  
wrote:

On 11.02.2023 18:06, Rick McGuire wrote:

'!' suppresses the issuing of commands. Generally used for non-destructiive 
test runs. Not
really used much but it takes up the '!' symbol character.


It seems that this is not in the documentation.

It's something that is part of the Mainframe classic Rexx implementations. I was dropped when Mike 
wrote TRL1 and was never implemented on other platforms.


Ah, I see, thank you!

---rony

___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel

Re: [Oorexx-devel] A draft for documenting MT tracing (Re: Planning to add multithreaded (concurrent) tracing (Re: RFC for feature request "794 Concurrency request"

[Rony G. Flatscher]

On 11.02.2023 20:11, Mike Cowlishaw wrote:
Similar comments:  this extra information seems to be aimed at 'developers', not users.   A user 
seeing the proposed output would have no idea what it means or what it's about.  It's astonishing, 
in every sense.


Not sure what you mean when writing "developers" or "users" in this context.

Just to be clear about the meanings in this e-mail context: "developer" means the "developer of an 
ooRexx program" and "user" means "user of an ooRexx program (probably not a programmer)".


This is about the TRACE instruction that gets usually used for debugging when developing programs, 
users of programs usually do not get to see TRACE output (with the single exception that a command 
causing a failure will be traced by default).




To me this seems absolutely contrary to the Rexx principles.   If developers need this kind of 
information, cannot this be achieved by other, less visible and less ugly, means?


There may be a misunderstanding here.

Multithreaded tracing is meant to help the *developers* of an ooRexx program to trace ooRexx 
programs that execute in different threads for debugging purposes (there is no need for this in 
classic Rexx). The next context is that the execution environment are method routines, i.e. method 
routines usually defined in classes and being executed on behalf of an ooRexx object (as classic 
Rexx has no classes/methods there is no reason to trace them).


The ooRexx reference, rexxref.pdf, documents concurrency in ooRexx in chapter "12. Concurrency" such 
that one can expect that there are ooRexx programmers who will apply/exploit it.


---

The ooRexx TRACE is modeled after the classic Rexx trace and currently does unfortunately not have 
the ability to gain insight of what happens when ooRexx programs execute in a mulithreaded fashion.


Take a look at this ooRexx program taking advantage of ooRexx' concurrency:

   c1 = .c~new
   call syssleep 0.5
   c1~m2 -- wake-up m1
   say "done"

   ::class C
   ::method init
expose s
s = 0
reply

self~m1


   ::method m1
expose s
s = 1
guard off
say "before guard" -- here, no lock
guard on when s <> 1 -- but here, is locked while waiting...
say "after guard"

   ::method m2
expose s
s = 2

   -- ::options trace a

If you run this program it will hang (run into a deadlock), here the output thanks to some 
say-statements for debugging:


   G:\test\orx\trace>jlf_dl1.rex
   before guard

The program just hangs, one must CTL-C to break the running program.

So where is the problem? The current TRACE does not help to shed any light on the (concurrent) 
ooRexx context.


This is where the MT tracing enhancement comes into play, following the "Rexx principles" to make 
the programmers better understand what happens and how the Rexx program gets executed. After all, 
this is the motivation that you came up with TRACE, right?


So in the post-classic Rexx, ooRexx, allowing for exploiting OO, concurrency, messaging etc., TRACE 
needs to be enhanced accordingly to help and ease the ooRexx program developer with his work when 
hitting problems because of the complex environment concurrently executing ooRexx programs have to 
relate to. It is about making debugging in this application area easier than is currently possible.


To illustrate, firstly here a TRACE A in "classic Rexx style" as currently implemented in ooRexx 5.0 
(by uncommenting the options directive at the end of the above program which will set TRACE A for 
the entire package, i.e. for all routines in that program):


   G:\test\orx\trace>jlf_dl1.rex
 1 *-* c1 = .c~new
   >I> Method "INIT" with scope "C" in package 
"G:\test\orx\trace\jlf_dl1.rex".
 8 *-* expose s
 9 *-* s = 0
10 *-* reply
 2 *-* call syssleep 0.5
   >I> Method "INIT" with scope "C" in package 
"G:\test\orx\trace\jlf_dl1.rex".
12 *-* self~m1
   >I> Method "M1" with scope "C" in package 
"G:\test\orx\trace\jlf_dl1.rex".
15 *-* expose s
16 *-* s = 1
17 *-* guard off
18 *-* say "before guard" -- here, no lock
   before guard
19 *-* guard on when s <> 1 -- but here, is locked while waiting...
 3 *-* c1~m2 -- wake-up m1
   >I> Method "M2" with scope "C" in package 
"G:\test\orx\trace\jlf_dl1.rex".

Now, if you are not Rick or Erich or any other deeply acquainted ooRexx programmer you have a 
problem: why is this deadlock occurring, or maybe formulated differently: where is the deadlock 
occurring and why?


This is a relative simple concurrency ooRexx program, and still one can see immediately the 
challenge. Of course, over time, if gaining more and more (and more and more time-consuming) 
experience one may become able to eventually find the problem. But using classic TRACE A is not as 
helpful and not as human-centric in such concurrency cases t

Re: [Oorexx-devel] Ad '!' as trace prefix letter (Re: Planning to add multithreaded (concurrent) tracing (Re: RFC for feature request "794 Concurrency request"

[Rick McGuire]

On Sun, Feb 12, 2023 at 9:26 AM Rony G. Flatscher 
wrote:

> On 11.02.2023 18:06, Rick McGuire wrote:
>
> '!' suppresses the issuing of commands. Generally used for
> non-destructiive test runs. Not really used much but it takes up the '!'
> symbol character.
>
> It seems that this is not in the documentation.
>
It's something that is part of the Mainframe classic Rexx implementations.
I was dropped when Mike wrote TRL1 and was never implemented on other
platforms.

Rick



>
> Also trying this as a trace option prefix causes an error:
>
> say "hi! #1"
> trace ?
> "dir"
> trace !
> say "hi! #2"
> "dir"
> say "hi! #3"
>
> ::options trace r
>
> yields as output:
>
> G:\test\orx\trace>test.rex
>  4 *-* trace !
> Error 24 running G:\test\orx\trace\test.rex line 4:  Invalid TRACE request.
> Error 24.1:  TRACE request letter must be one of "ACEFILNOR"; found  "!".
>
> So there seems to be no '!' trace prefix implemented.
>
> Where and how would '!' be used as a prefix letter with the behaviour that
> you describe? Maybe I have missed the related documentation in rexxref.pdf.
>
> ---rony
>
>
> On Sat, Feb 11, 2023 at 12:01 PM Rony G. Flatscher <
> rony.flatsc...@wu.ac.at> wrote:
>
>> Just a quick question: what is the purpose of '!' in TRACE, have not seen
>> documentation about it.
>>
>> In the case '!' is available then this could be used as an MT toggle?
>>
>> ---rony
>> On 09.02.2023 13:40, Rick McGuire wrote:
>>
>> One other thing about the trigger character. The '?' and '!' triggers act
>> as toggles, so issuing "Trace ?" will trigger the interactive debug setting
>> without changing the level of tracing being done. This should also work
>> with whatever is chosen to turn on the multithreaded information.
>>
>> Another possibility would be to automatically add the additional
>> information when more than one thread is active. That way, most users who
>> only work single threaded never see this, but people who are working with
>> multiple threads get the extra information without needing to think about
>> having to change the trace settings. I think I would prefer that rather
>> than the M suffix, which really works quite differently from how ? and !
>> are handled.
>>
>> Rick
>>
>> On Thu, Feb 9, 2023 at 7:21 AM Rony G. Flatscher 
>> wrote:
>>
>>> Thanks for the feedback. Probably putting M as the trailing letter after
>>> the alphabetic letter as Mike suggests is the best option. Omitting the
>>> trailing M would switch back to the simple form. Would that be acceptable
>>> for everyone?
>>>
>>> ---rony
>>>
>>>
>>> On 08.02.2023 21:24, Rick McGuire wrote:
>>>
>>> The special symbol characters "." and "_" are also available as
>>> indicators. I'm a definite -1 to using environment variables and Erich has
>>> also voiced his displeasure about that.
>>>
>>> Another option might be to allow a second keyword following the trace
>>> type that indicates using the expanded form. It should also allow explicit
>>> specification of the simple form too.
>>>
>>> Rick
>>>
>>> On Wed, Feb 8, 2023 at 2:46 PM Mike Cowlishaw 
>>> wrote:
>>>
 I would have put the M after the other letter because it's really a
 subsidiary option.  If it's first it rather 'M'asks the main option?

 Mike

 --
 *From:* Rony G. Flatscher [mailto:rony.flatsc...@wu.ac.at]
 *Sent:* 08 February 2023 14:16
 *To:* oorexx-devel@lists.sourceforge.net
 *Subject:* [Oorexx-devel] Planning to add multithreaded (concurrent)
 tracing (Re: RFC for feature request "794 Concurrency request"

 Coming back to this RFE from 17 months ago which I would like to add to
 trunk. Without it one can hardly use TRACE for debugging multithreaded
 programs in a Rexx-like, i.e. easy manner.

 Currently having tried to incorporate the feedback about too many
 whitespaces between the new columns (Rexx interpreter instance number,
 Thread number, Activity number, reserved object pool).

 There was another idea about making this concurrency/multihreaded trace
 available without a need to define an environment variable
 RXTRACE_CONCURRENCY before starting a Rexx program. This post is about
 ideas of how to activate and deactivate concurrent tracing at runtime
 (either via the TRACE keyword instruction or the TRACE()-BIF) in a manner
 that is intuitive and easy to remember.

 One possibility would be to introduce new alphabetic options, this time
 with two letters by prepending the letter 'M' (for multithreaded as the
 letter c is already used for tracing commands and may therefore be
 irritating) to the existing alphabetic characters, hence defining the
 following semantics:

 *Trace*
 *Option, turn off MT*
 *Option, turn on MT*
 All
 A
 MA
 Command
 C
 MC
 Error
 E
 ME
 Failure
 F
 MF
 Intermediates
 I
 MI
 Labels
 L
 ML
 Normal
 

[Oorexx-devel] Ad '!' as trace prefix letter (Re: Planning to add multithreaded (concurrent) tracing (Re: RFC for feature request "794 Concurrency request"

[Rony G. Flatscher]

On 11.02.2023 18:06, Rick McGuire wrote:
'!' suppresses the issuing of commands. Generally used for non-destructiive test runs. Not really 
used much but it takes up the '!' symbol character.


It seems that this is not in the documentation.

Also trying this as a trace option prefix causes an error:

   say "hi! #1"
   trace ?
   "dir"
   trace !
   say "hi! #2"
   "dir"
   say "hi! #3"

   ::options trace r

yields as output:

   G:\test\orx\trace>test.rex
 4 *-* trace !
   Error 24 running G:\test\orx\trace\test.rex line 4:  Invalid TRACE request.
   Error 24.1:  TRACE request letter must be one of "ACEFILNOR"; found  "!".

So there seems to be no '!' trace prefix implemented.

Where and how would '!' be used as a prefix letter with the behaviour that you describe? Maybe I 
have missed the related documentation in rexxref.pdf.


---rony



On Sat, Feb 11, 2023 at 12:01 PM Rony G. Flatscher  
wrote:

Just a quick question: what is the purpose of '!' in TRACE, have not seen 
documentation about it.

In the case '!' is available then this could be used as an MT toggle?

---rony

On 09.02.2023 13:40, Rick McGuire wrote:

One other thing about the trigger character. The '?' and '!' triggers act 
as toggles, so
issuing "Trace ?" will trigger the interactive debug setting without 
changing the level of
tracing being done. This should also work with whatever is chosen to turn 
on the
multithreaded information.

Another possibility would be to automatically add the additional 
information when more than
one thread is active. That way, most users who only work single threaded 
never see this, but
people who are working with multiple threads get the extra information 
without needing to
think about having to change the trace settings. I think I would prefer 
that rather than the
M suffix, which really works quite differently from how ? and ! are handled.

Rick

On Thu, Feb 9, 2023 at 7:21 AM Rony G. Flatscher  
wrote:

Thanks for the feedback. Probably putting M as the trailing letter 
after the alphabetic
letter as Mike suggests is the best option. Omitting the trailing M 
would switch back to
the simple form. Would that be acceptable for everyone?

---rony


On 08.02.2023 21:24, Rick McGuire wrote:

The special symbol characters "." and "_" are also available as 
indicators. I'm a
definite -1 to using environment variables and Erich has also voiced 
his displeasure
about that.

Another option might be to allow a second keyword following the trace 
type that
indicates using the expanded form. It should also allow explicit 
specification of the
simple form too.

Rick

On Wed, Feb 8, 2023 at 2:46 PM Mike Cowlishaw  
wrote:

I would have put the M after the other letter because it's really a 
subsidiary
option.  If it's first it rather 'M'asks the main option?
Mike



*From:* Rony G. Flatscher [mailto:rony.flatsc...@wu.ac.at]
*Sent:* 08 February 2023 14:16
*To:* oorexx-devel@lists.sourceforge.net
*Subject:* [Oorexx-devel] Planning to add multithreaded 
(concurrent) tracing
(Re: RFC for feature request "794 Concurrency request"

Coming back to this RFE from 17 months ago which I would like 
to add to trunk.
Without it one can hardly use TRACE for debugging multithreaded 
programs in a
Rexx-like, i.e. easy manner.

Currently having tried to incorporate the feedback about too 
many whitespaces
between the new columns (Rexx interpreter instance number, 
Thread number,
Activity number, reserved object pool).

There was another idea about making this 
concurrency/multihreaded trace
available without a need to define an environment variable 
RXTRACE_CONCURRENCY
before starting a Rexx program. This post is about ideas of how 
to activate and
deactivate concurrent tracing at runtime (either via the TRACE 
keyword
instruction or the TRACE()-BIF) in a manner that is intuitive 
and easy to remember.

One possibility would be to introduce new alphabetic options, 
this time with two
letters by prepending the letter 'M' (for multithreaded as the 
letter c is
already used for tracing commands and may therefore be 
irritating) to the
existing alphabetic characters, hence defining the following 
semantics:

*Trace**
*   *Option, turn off MT**
*   *Option, turn on MT**
*
All
A