Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
On 26.02.2023 17:48, Mike Cowlishaw wrote: OK, also needed for the weekend to spend time on this. Thanks! I think I agree that just adding the thread ID is not sufficient for every case (although I think it would help greatly in many cases). +1 Yes, agreeing that the thread ID should be added to the normal trace output (replacing '>' with a blank followed by the thread ID and '>') as it would help greatly in many cases. Trace is part of the language, however, and adding to it in any way needs much care and attention: the formatting (widths, etc.) is as important as any other detail of the language. The trace format took years to evolve, and is wasn't until lots of people were using it that it became clear how to make it both simple and useful. Yes, I realize that and therefore any changes in this area should be done with cautioness that a Rexx user does not get irritated, surprised. Like Chip (I suspect), I'm beginning to think that this is outside the domain of the language and more suited to some external 'specialised tool' such as an IDE (integrated development environment), or just an option to write the trace to a file .. that is, not expected to be seen by humans but more likely processed by a program to be presented in some useful way to the human. As you might have seen, short of an IDE, Jean-Louis has created a Rexx utiltiy program (tracer.rex) that would process this concurrency/extended trace output into a CSV file (that also indicates for each traced statement from where it stems). Loading this CSV file into a spreadsheet allows one to process this trace data according to the user. Of course, the user would need to be aware of the functionality a spreadsheet possesses (not only filtering, but applying dependent formatting and the like). What is important is to become able to get this concurrency/extended trace format at all. Any ooRexx user would only set this trace mode if really needed, as the information (in a multithreaded scenario) may be overwhelming and mostly intended for a post-mortem analysis. (There may be scenarios conceivable that interactive concurrency trace is helpful, but then the ooRexx user would focus on that aspect, so it should not be ruled out totally.) Also if using some form to activate it that is not letter-based (as the idea with postfix exclamation mark '!') would make sure that such an option could only be activated intentionally, not by mistake. In ooRexx there is the .traceOutput monitor (the .Monitor class is often overlooked, but really great for monitoring messages to objects, allowing for intercepting them by placing one own's object as destination instead) which makes it rather easy to fetch the trace output by setting the destination to a stream object (which would cause the trace to be sent/stored in that stream). So one could code something like (untested): /* e.g. in the "prolog" of an ooRexx program (main program) */ s=.stream~new("myExtendedTrace.log")~~open("replace") -- open file (replace existing one) .traceOuptut~destination(s) -- from now on trace output gets written to "myExtendedTrace.log" -- no matter where tracing takes place ... code ... trace r!-- one of the discussed ideas to turn on concurrency/extended trace result ... code ... trace r -- standard trace result ... code ... trace n -- trace normal ... code ... Upon termination one could then inspect and analyze "myExtendedTrace.log". One then could immediately turn such trace log data it into a CSV with the "tracer.rex" utility and proceed with a spreadsheet application to analyze the trace data. Loading such trace data into an editor and using the editor features (for filtering and/or colorizing if available etc.) would be another attractive alternative. Or writing one own's short Rexx scripts to e.g. turn the trace data into html with a CSS that would apply the filtering, color highlighting et.al would become possible with this. Or turning such trace data into XML and devising XSL stylesheets for automatic processing/analysis of such concurrency/extended traces. Or ... But we would need that concurrency/extended trace data for that. ---rony *From:* Rony G. Flatscher [mailto:rony.flatsc...@wu.ac.at] *Sent:* 22 February 2023 17:43 *To:* oorexx-devel@lists.sourceforge.net *Subject:* Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII) Mike, sorry for the long intermission, having been "under water" for different reasons and I wanted to take my time to get back as becoming able to fully understand and
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Absolutely agree .. colour can and does make a huge difference. I think (there may be independent inventions) that I invented syntax colouring in LEXX (see https://en.wikipedia.org/wiki/LEXX_(text_editor) ). But since colour terminals had only just became available in 1985 ... Colouring trace output differently for different threads is definitely something to explore! Mike > > Like Chip (I suspect), I'm beginning to think that this is > > outside the > > domain of the language and more suited to some external > > 'specialised tool' > > or just an option to write the trace to a file .. that is, not > > expected to be seen by humans but more likely processed by > a program > > to be presented in some useful way to the human. > > That sounds like a good idea to me. Having looked at some of > the examples posted in this discussion, it seems to me that > one thing that would make a massive contribution to reading > such traces would be being able to have lines from specific > threads (or even more granular than that) displayed in > different colours. > > A text editor that entirely uses regexes to identify > character strings or lines which should be displayed in > various colours might be a convenient way to achieve that. > > > I still mainly use Kedit for editing rexx source etc. Its > syntax colouring isn't > able to cope with log lines, but the fact that it can be > scripted (in "Kexx") > makes it a winner for me. > > I noticed a while ago that EditPadPro - > https://www.editpadpro.com - does > have regex-based colouring, and have been thinking about > installing that > JUST to use as a file browser, for application debug logs etc. > > > [I used to use an editor named StrongED [under Risc OS] which > had flexible > colouring, able to be used on application-specific log files > etc & once wrote > an enhanced version of a spam-filtering application where > rules were shown > in colours in rule definitions, & corresponding colours were > used when one > looked at the logs showing rules taking actions on emails, > provided a user > was using StrongED and the config files I provided. > > Some parts of the application GUI also optionally used the > same colours - > which went against the prevailing OS standards for how GUI > panes should > look. It was techincally possible to use colours, but no > part of the OS or > any other application I'd ever seen actually did so. I > suppose that if all > applications had done so, each in their own way, without that > being able > to be customised in a way that made sense to each user, > desktops would > have looked awful. But on the other hand, sensible/restrained use of > colour added a lot of extra meaning in that application.) > > -- > Jeremy Nicoll - my opinions are my own. > > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
On Sun, 26 Feb 2023, at 16:48, Mike Cowlishaw wrote: > Like Chip (I suspect), I'm beginning to think that this is outside the > domain of the language and more suited to some external 'specialised tool' > or just an option to write the trace to a file .. that is, not expected > to be > seen by humans but more likely processed by a program to be presented > in some useful way to the human. That sounds like a good idea to me. Having looked at some of the examples posted in this discussion, it seems to me that one thing that would make a massive contribution to reading such traces would be being able to have lines from specific threads (or even more granular than that) displayed in different colours. A text editor that entirely uses regexes to identify character strings or lines which should be displayed in various colours might be a convenient way to achieve that. I still mainly use Kedit for editing rexx source etc. Its syntax colouring isn't able to cope with log lines, but the fact that it can be scripted (in "Kexx") makes it a winner for me. I noticed a while ago that EditPadPro - https://www.editpadpro.com - does have regex-based colouring, and have been thinking about installing that JUST to use as a file browser, for application debug logs etc. [I used to use an editor named StrongED [under Risc OS] which had flexible colouring, able to be used on application-specific log files etc & once wrote an enhanced version of a spam-filtering application where rules were shown in colours in rule definitions, & corresponding colours were used when one looked at the logs showing rules taking actions on emails, provided a user was using StrongED and the config files I provided. Some parts of the application GUI also optionally used the same colours - which went against the prevailing OS standards for how GUI panes should look. It was techincally possible to use colours, but no part of the OS or any other application I'd ever seen actually did so. I suppose that if all applications had done so, each in their own way, without that being able to be customised in a way that made sense to each user, desktops would have looked awful. But on the other hand, sensible/restrained use of colour added a lot of extra meaning in that application.) -- Jeremy Nicoll - my opinions are my own. ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
OK, also needed for the weekend to spend time on this. I think I agree that just adding the thread ID is not sufficient for every case (although I think it would help greatly in many cases). Trace is part of the language, however, and adding to it in any way needs much care and attention: the formatting (widths, etc.) is as important as any other detail of the language. The trace format took years to evolve, and is wasn't until lots of people were using it that it became clear how to make it both simple and useful. Like Chip (I suspect), I'm beginning to think that this is outside the domain of the language and more suited to some external 'specialised tool' such as an IDE (integrated development environment), or just an option to write the trace to a file .. that is, not expected to be seen by humans but more likely processed by a program to be presented in some useful way to the human. Mike _ From: Rony G. Flatscher [mailto:rony.flatsc...@wu.ac.at] Sent: 22 February 2023 17:43 To: oorexx-devel@lists.sourceforge.net Subject: Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII) Mike, sorry for the long intermission, having been "under water" for different reasons and I wanted to take my time to get back as becoming able to fully understand and to debug multi threaded ooRexx programs is really a very important and needed task. On 17.02.2023 16:00, Mike Cowlishaw wrote: Rony, I fully understand the need for thread information. As I mentioned in an earlier post I often add this to (manually-added) tracing in my C programs -- which are true multi-core hardware threads. I also suggested a way that thread IDs could be added to the existing trace output (if requested) with minimal modification of code and documentation. Yes, and I picked up your suggestion and used it to demonstrate various forms of trace output without and with it using a simple ooRexx multithreaded test case. If you look up the program "mt91.rex" and the trace without and with your suggested thread id you will see, that supplying the thread id in that case is not sufficient to understand how the multithreaded parts of the ooRexx program get executed. Also, if one repeats running that multithreaded ooRexx program the sequence of the executed ooRexx statements may be different and differently interleaved. What I don't understand is the need to add lots of other options, wide spacing, and fixed-format layout which simply won't work when the numbers are more than one or two digits. And it seems to me that only the author of this design might guess that "1*" refers to thread locks ... :-). Probably I did a lousy job explaining the problems that may occur when multithreaded ooRexx programs get executed: * repeating execution of multithreaded ooRexx programs do not necessarily repeat the same sequence of traces * the execution of a section of ooRexx code may be interleaved with the execution of another section of the ooRexx code differently from run to run * this poses the principal problem - comparing with classic Rexx programs - of understanding what happens and why * having the thread id available in the trace output may help understand the execution in relative simple cases of multithreading like in the mt01.rex example, but not so for more complex ones like mt91.rex, unfortunately * there is trace information missing about the execution context that is needed to become able to understand what really happens where * if you look up the trace according to your suggestion to add the thread information using the thread id "1" of mt91.rex you will not be able to understand what happens there; the reason being that in thread id "1" there are different activations (invocations) taking place, and worse these activations (invocations) belong to two totally different program runs, to two different Rexx instances, information that is missing from the trace * if however you look up the trace of mt91.rex with the suggested extended trace information you can all of a sudden distinguish the activations (invocations) that occur on e.g. T1 and can see that they are different from each other; as the same program gets run concurrently on two different Rexx instances you get to see which Rexx instance executes which activation (invocation) on which thread and therefore you can distinguish them from each other even on thread T1 and become able to understand which executions belong together and which ones don't; having only the thread number does not allow you to do that at all in this case! So the examples so far have the purpose to demonstrate multi threaed ooRexx programs and the traces that get caused by them, without concurrent trace information, with the thread id as per your suggestion and the sugg
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
As a life-long unabashed fan of Trace, I can see that anyone doing non-trivial multithreaded Rexx programming would need far more information than 'Trace I' can provide. However, the sheer volume of information necessary seems to present no universally acceptable format to interactively deal with such complexity. Could we not define Mike's "basic-multithreading" Trace format as an interactive trace option, and have another mechanism that writes Rony's "multithread-the-world" Trace information to another device/file for formatting as desired? The latter would not be an "interactive" trace per se, but perhaps it would help to narrow the scope of the problem. Not unlike sprinkling a couple of SNAP dumps in your Assembler, in order to determine the point at which you want to take an ABEND dump. -Chip- RIP: Les Kohler ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
in order to become able to understand and to debug multithreaded ooRexx programs as mt91.rex was intended to demonstrate. --- In the end the point is: ooRexx - unlike classic Rexx - is by design a very flexible and multithreaded programming language. The trace keyword instruction in ooRexx has not been updated to support understanding and debugging such mulithreaded ooRexx programs. Adding the thread id by default like you suggest is a first step that allows for understanding and debugging simple multithreaed ooRexx programs. Also, it allows understanding and debugging classic Rexx programs that get dispatched on different threads by a hosting program that runs Rexx programs on different of its threads. However, adding the thread id is not sufficient to understand and to debug powerful/complex multithreaded ooRexx programs. ooRexx programs can be executed in a multithreaed manner by different means: here some of the ooRexx-related means that play a role in a multithreaded ooRexx world: * the REPLY keyword statement (returns and activation/invocation gets carried on on a different thread) * the START message (root class .Object or class .Message) * methods marked as GUARDED or UNGUARDED (lock related) * the GUARD keyword statement (lock related) * Rexx instances executing ooRexx programs in a multithreaded context (dispatched by host applications) --- Of course any other form, idea to supply the necessary information to help understand and debug multithreaded ooRexx programs/applications is welcome and appreciated! However, so far only supplying the thread id in the trace message was suggested which is not enough information, unfortunately. --- Ad format: of course one can use less widths but at this point in time is maybe not as important as understanding/willing to supply the necessary additional information that enables, empowers one to fully understand and debug multi-threaded ooRexx programs. Please do not misunderstand me: I also think that defining the format is very important to help/ease programmers understand the output. So a discussion about the format, sequence, abbreviations is welcome of course. The more improvements the better. --- Context information like R (Rexx instance), T (thread number), A (activation/invocation), V (variable pool), lock-counts with asterisk (*) indicating who owns the lock, are really important for becoming able to understand and to debug multithreaded ooRexx programs. It is o.k. to challenge each such information. And again: if there are ideas about alternative ways enabling ooRexx programmers to quickly understand and to debug multithreaded ooRexx programs, they are welcome. However, so far no such suggestion, idea has been communicated. And as the need for such information is a reality (in my case urgently vis-à-vis my students who start as beginners and at the end of the semester employ multithreaded ooRexx programs with the need to understand and to debug them as easy as possible) I would very much make it available ASAP. However, I am interested in a constructive procedure, if possible. ---rony *From:* Rony [mailto:rony.flatsc...@wu.ac.at] *Sent:* 16 February 2023 21:43 *To:* Open Object Rexx Developer Mailing List *Subject:* Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII) Am 15.02.2023 um 18:57 schrieb Mike Cowlishaw : As for the 'spaced out' case (excerpt below) ... this really would not work for me. I often have 5-9 windows open when I'm programming and these are 80 characters wide so I can minimise overlaps. With the suggested layout this would only work for programs less than ~40 characters wide! Here's how the excerpt looks for me (and this example has very short lines -- most of my programs use 72 or more characters per line for better commentary): ---> mt91.rex_nr_1_via_JSR223 R1 T1 A1 3 *-* t=.Test~new R1 T1 A2 V1 1* 21 *-* say "arrived in:" .context~name arrived in: INIT R1 T1 A2 V1 1* 22 *-* counter=0 R1 T1 A1 >>> "a TEST" R1 T1 A1 4 *-* t~m1 R1 T1 A3 V1 1* 27 *-* counter+=1 -- increase counter R1 T1 A3 V1 1* 28 *-* say "arrived in:" .context~name "before reply" Almost any line of any length will wrap. That's why the trace headers in Rexx are kept as short as feasible. Yes trace has been well thought out and well designed. It seems that you are under the impression that this extra trace information should get added to trace by default? If so, that is not the case. In effect
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Rony, I fully understand the need for thread information. As I mentioned in an earlier post I often add this to (manually-added) tracing in my C programs -- which are true multi-core hardware threads. I also suggested a way that thread IDs could be added to the existing trace output (if requested) with minimal modification of code and documentation. What I don't understand is the need to add lots of other options, wide spacing, and fixed-format layout which simply won't work when the numbers are more than one or two digits. And it seems to me that only the author of this design might guess that "1*" refers to thread locks ... :-). Mike _ From: Rony [mailto:rony.flatsc...@wu.ac.at] Sent: 16 February 2023 21:43 To: Open Object Rexx Developer Mailing List Subject: Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII) Am 15.02.2023 um 18:57 schrieb Mike Cowlishaw : As for the 'spaced out' case (excerpt below) ... this really would not work for me. I often have 5-9 windows open when I'm programming and these are 80 characters wide so I can minimise overlaps. With the suggested layout this would only work for programs less than ~40 characters wide! Here's how the excerpt looks for me (and this example has very short lines -- most of my programs use 72 or more characters per line for better commentary): ---> mt91.rex_nr_1_via_JSR223 R1 T1 A13 *-* t=.Test~new R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name arrived in: INIT R1 T1 A2V1 1* 22 *-* counter=0 R1 T1 A1 >>> "a TEST" R1 T1 A14 *-* t~m1 R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name "before reply" Almost any line of any length will wrap. That's why the trace headers in Rexx are kept as short as feasible. Yes trace has been well thought out and well designed. It seems that you are under the impression that this extra trace information should get added to trace by default? If so, that is not the case. In effect as designed and communicated (maybe badly), you need to activate this extra trace information explicitly which you would do only, if you have an important reason: needing additional information in order to become able to debug multithreaded programs. You would so only, if it helps, if it benefits the programmer in understanding and debugging. Otherwise the option would not be used. You would probably not be a candidate needing this extra information (assuming that you are not employing ooRexx' multithreading) so you would not activate it and everything would remain the same for your specific configured working environment. Nothing changes for you. Those who are in need will be more than happy to get these extra information to save a lot of time and becoming able to analyze and to understand multithreaded problems. Without this extra, multithreaded related trace information it would not be possible the more complex, the more interleaved multithreaded execution takes place. Just look at the trace, trace with thread number and with extended/mulithreading-related trace e.g. with the mt91.rex example in my earlier post: without the multithreaded/extended trace no one could understand what really happens there. This is the core of the problem: trace has not been extended to supply this necessary multithreaded/extended information in case multithreading causes problems and the programmer requests the multithreaded-related/extra information with trace. Adding an unexplained 27 characters on the front of each line makes little sense, "Unexplained", hence "surprising"? Seriously, this extended trace information would get created only if the ooRexx programmer/user requests it explicitly. To be able to request the multithreaded trace one needs to be aware a) it exists and b) the option to get it activated. This is only possible if the programmer/user reads the documentation which will include the explanation of what the extra trace information is about: ,R' is the Rexx instance, ,T' is the thread number, ,A' is the activation (invocation), ,V' is the variable pool, the number column shows the number of requested locks, the asterisk the owner of the lock (that may cause deadlocks for others). The mnemonics make it easy to remember. Everyone who read that or got the explanation for the letters will be able understand what is being presented. So, they get explained and everyone who read the documentation would understand. Also, it is easy to explain (hence also easy to understand), but only if this information is made available. especially as the information is the same on most lines, The
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
I like these suggestions. Jon On Fri, 17 Feb 2023 at 09:14, René Jansen wrote: > One idea here is to no change the options of TRACE at all (they are very > portable over variants). For implementations that have threads, why don’t > we add a > > TRACE THREADS > > before the trace statement. We can have an TRACE THREADS OFF option to > switch back to the regular trace. > > also, a > > TRACE THREAD x > > would just trace a named thread. Assuming we name them, which would be > better than following the OS. > > In this vein, I would very much like a > > TRACE TIME > > which timestamps trace messages (for performance work), combinable with > threads. > > This would have the advantage of keeping TRACE the same on implementations > and add the extra line length when asked for it. > It can also be done in OPTIONS - where the general line should be that > unknown options are just ignored. > > best regards, > > René. > > On 16 Feb 2023, at 22:42, Rony wrote: > > > Am 15.02.2023 um 18:57 schrieb Mike Cowlishaw : > > > As for the 'spaced out' case (excerpt below) ... this really would not > work for me. I often have 5-9 windows open when I'm programming and these > are 80 characters wide so I can minimise overlaps. With the suggested > layout this would only work for programs less than ~40 characters wide! > Here's how the excerpt looks for me (and this example has very short lines > -- most of my programs use 72 or more characters per line for better > commentary): > > ---> mt91.rex_nr_1_via_JSR223 > R1 T1 A13 *-* t=.Test~new > R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name > arrived in: INIT > R1 T1 A2V1 1* 22 *-* counter=0 > R1 T1 A1 >>> "a TEST" > R1 T1 A14 *-* t~m1 > R1 T1 A3V1 1* 27 *-* counter+=1 -- increase > counter > R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name > "before reply" > > Almost any line of any length will wrap. That's why the trace headers in > Rexx are kept as short as feasible. > > Yes trace has been well thought out and well designed. > > It seems that you are under the impression that this extra trace > information should get added to trace by default? If so, that is not the > case. In effect as designed and > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
One idea here is to no change the options of TRACE at all (they are very portable over variants). For implementations that have threads, why don’t we add a TRACE THREADS before the trace statement. We can have an TRACE THREADS OFF option to switch back to the regular trace. also, a TRACE THREAD x would just trace a named thread. Assuming we name them, which would be better than following the OS. In this vein, I would very much like a TRACE TIME which timestamps trace messages (for performance work), combinable with threads. This would have the advantage of keeping TRACE the same on implementations and add the extra line length when asked for it. It can also be done in OPTIONS - where the general line should be that unknown options are just ignored. best regards, René. > On 16 Feb 2023, at 22:42, Rony wrote: > > >> Am 15.02.2023 um 18:57 schrieb Mike Cowlishaw : >> >> As for the 'spaced out' case (excerpt below) ... this really would not work >> for me. I often have 5-9 windows open when I'm programming and these are >> 80 characters wide so I can minimise overlaps. With the suggested layout >> this would only work for programs less than ~40 characters wide! Here's >> how the excerpt looks for me (and this example has very short lines -- most >> of my programs use 72 or more characters per line for better commentary): >> >> ---> mt91.rex_nr_1_via_JSR223 >> R1 T1 A13 *-* t=.Test~new >> R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name >> arrived in: INIT >> R1 T1 A2V1 1* 22 *-* counter=0 >> R1 T1 A1 >>> "a TEST" >> R1 T1 A14 *-* t~m1 >> R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter >> R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name >> "before reply" >> >> Almost any line of any length will wrap. That's why the trace headers in >> Rexx are kept as short as feasible. > Yes trace has been well thought out and well designed. > > It seems that you are under the impression that this extra trace information > should get added to trace by default? If so, that is not the case. In effect > as designed and ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
> Am 15.02.2023 um 18:57 schrieb Mike Cowlishaw : > > As for the 'spaced out' case (excerpt below) ... this really would not work > for me. I often have 5-9 windows open when I'm programming and these are 80 > characters wide so I can minimise overlaps. With the suggested layout this > would only work for programs less than ~40 characters wide! Here's how the > excerpt looks for me (and this example has very short lines -- most of my > programs use 72 or more characters per line for better commentary): > > ---> mt91.rex_nr_1_via_JSR223 > R1 T1 A13 *-* t=.Test~new > R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name > arrived in: INIT > R1 T1 A2V1 1* 22 *-* counter=0 > R1 T1 A1 >>> "a TEST" > R1 T1 A14 *-* t~m1 > R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter > R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name > "before reply" > > Almost any line of any length will wrap. That's why the trace headers in > Rexx are kept as short as feasible. Yes trace has been well thought out and well designed. It seems that you are under the impression that this extra trace information should get added to trace by default? If so, that is not the case. In effect as designed and communicated (maybe badly), you need to activate this extra trace information explicitly which you would do only, if you have an important reason: needing additional information in order to become able to debug multithreaded programs. You would so only, if it helps, if it benefits the programmer in understanding and debugging. Otherwise the option would not be used. You would probably not be a candidate needing this extra information (assuming that you are not employing ooRexx‘ multithreading) so you would not activate it and everything would remain the same for your specific configured working environment. Nothing changes for you. Those who are in need will be more than happy to get these extra information to save a lot of time and becoming able to analyze and to understand multithreaded problems. Without this extra, multithreaded related trace information it would not be possible the more complex, the more interleaved multithreaded execution takes place. Just look at the trace, trace with thread number and with extended/mulithreading-related trace e.g. with the mt91.rex example in my earlier post: without the multithreaded/extended trace no one could understand what really happens there. This is the core of the problem: trace has not been extended to supply this necessary multithreaded/extended information in case multithreading causes problems and the programmer requests the multithreaded-related/extra information with trace. > Adding an unexplained 27 characters on the front of each line makes little > sense, „Unexplained“, hence „surprising“? Seriously, this extended trace information would get created only if the ooRexx programmer/user requests it explicitly. To be able to request the multithreaded trace one needs to be aware a) it exists and b) the option to get it activated. This is only possible if the programmer/user reads the documentation which will include the explanation of what the extra trace information is about: ‚R‘ is the Rexx instance, ‚T‘ is the thread number, ‚A‘ is the activation (invocation), ‚V‘ is the variable pool, the number column shows the number of requested locks, the asterisk the owner of the lock (that may cause deadlocks for others). The mnemonics make it easy to remember. Everyone who read that or got the explanation for the letters will be able understand what is being presented. So, they get explained and everyone who read the documentation would understand. Also, it is easy to explain (hence also easy to understand), but only if this information is made available. > especially as the information is the same on most lines, The information repeats in some columns in these traces. The purpose of this is to become able to spot and analyze immediately lines that deviate, e.g. the Rexx instance number changes, the thread number changes, the activation number changes etc. which is the case when concurrently other parts get executed, interleaving with the trace lines that otherwise have a few columns in common. Without the multithreaded/extra information one is not able to notice that at all! This is exactly the problem in multithreaded programs, the more concurrency the more important to become able to notice that in order to become able to analyze and for that reason you need to carry these information! > and as I mentioned before is not user-friendly (here I mean 'user' as being a > writer of Rexx programs, not someone who runs a Rexx program without looking > at it or caring which language it is written in). This is where we differ: leaving that away to make it ‚user-friendly‘ by y
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Hi, I think Gil talks sense. Most of the time, most or all of the users do not need or want this feature, but when they do, it would be important and should provide all the information they might require. I may be covering old ground here, but as a user I don't do multithreading on a whim. When I first learnt about it, I got excited about the prospect of improving performance, but then, and probably still now, on Windows all the ooRexx threads run on the same core, so if there are any performance advantages, it seems to me that they would be negligible. Apart from a very few exceptions I pretty much only indulge in multi-threading when forced to by the GUI package. Gil's suggestion provokes two responses in me which I offer in the spirit of helpfulness. 1) If one accepts that this could be an external package, then is it really such a leap to sanction having it contained within the interpreter and subject to our regression testing etc., but requiring something from the user to turn it on. Many applications and services honour the concept of a verbose mode. It doesn't have to be called verbose tracing, it could be called advanced multithread tracing mode or some such. 2) If on the other hand, it does become a separate package in the incubator, and special hooks have to be built into the interpreter for it, then should we not perhaps be more ambitious and aim for the hooks that would be needed to provide a future toy like the IBM Object Rexx Workbench. There must have historically been hooks provided for that, and perhaps they are still there. I mention the workbench, because that is the sort of thing that I'm led to believe young programmers value. Having grown up with merely Xedit and a flowchart template as my IDE I did not miss The Workbench too much personally, but in my opinion, providing a workbench with breakpoint and variable modification capabilities as well as auto-complete and context sensitive help would be a good strategic move for rexx (I am aware of the intelliJ offering, but would advocate taking it further). In case this paragraph might read like I'm demanding the development team pull this rabbit out of a hat here and now, that is not what I'm expecting. I just want us to adopt a strategic direction. I have no idea how, if or when we would get there. just my two-pennies/cents worth On Wed, 15 Feb 2023 at 22:45, Gilbert Barmwater wrote: > Not being a user who writes multi-threaded ooRexx programs, I have > remained silent until now. However, it seems to me that there are enough > objections to the proposal that would add this to TRACE that we should > consider alternatives. I appreciate the need for the information and the > work done both by Jean-Louis and Rony but perhaps this is better provided > as a stand alone package in the Incubator similar to ooSQLite. Then those > that need the information that this package supplies could retrieve the > package and add the appropriate ::requires directive(s) to their program. > I understand this will require some redesign and may still need additions > to the interpreter to expose the needed data to an external package but I > think we should consider going this route as I do not see a compromise that > will satisfy everyone. Just my 2 opinion for what its worth. Gil > On 2/15/2023 1:59 PM, Rick McGuire wrote: > > I’m in complete agreement with Mike on this. There are better ways to make > this sort of information available than trying to force fit it In to > trace. > > Rick > > On Wed, Feb 15, 2023 at 12:58 PM Mike Cowlishaw > wrote: > >> Thanks for the multiple examples! >> >> As for the 'spaced out' case (excerpt below) ... this really would not >> work for me. I often have 5-9 windows open when I'm programming and these >> are 80 characters wide so I can minimise overlaps. With the suggested >> layout this would only work for programs less than ~40 characters wide! >> Here's how the excerpt looks for me (and this example has very short lines >> -- most of my programs use 72 or more characters per line for better >> commentary): >> >> ---> mt91.rex_nr_1_via_JSR223 >> R1 T1 A13 *-* t=.Test~new >> R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name >> arrived in: INIT >> R1 T1 A2V1 1* 22 *-* counter=0 >> R1 T1 A1 >>> "a TEST" >> R1 T1 A14 *-* t~m1 >> R1 T1 A3V1 1* 27 *-* counter+=1 -- increase >> counter >> R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name >> "before reply" >> >> Almost any line of any length will wrap. That's why the trace headers in >> Rexx are kept as short as feasible. Adding an unexplained 27 characters on >> the front of each line makes little sense, especially as the information is >> the same on most lines, and as I mentioned before is not user-friendly >> (here I mean 'user' as being a writer of Rexx programs, not someone who >> runs a
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Not being a user who writes multi-threaded ooRexx programs, I have remained silent until now. However, it seems to me that there are enough objections to the proposal that would add this to TRACE that we should consider alternatives. I appreciate the need for the information and the work done both by Jean-Louis and Rony but perhaps this is better provided as a stand alone package in the Incubator similar to ooSQLite. Then those that need the information that this package supplies could retrieve the package and add the appropriate ::requires directive(s) to their program. I understand this will require some redesign and may still need additions to the interpreter to expose the needed data to an external package but I think we should consider going this route as I do not see a compromise that will satisfy everyone. Just my 2 opinion for what its worth. Gil On 2/15/2023 1:59 PM, Rick McGuire wrote: I’m in complete agreement with Mike on this. There are better ways to make this sort of information available than trying to force fit it In to trace. Rick On Wed, Feb 15, 2023 at 12:58 PM Mike Cowlishaw wrote: Thanks for the multiple examples! As for the 'spaced out' case (excerpt below) ... this really would not work for me. I often have 5-9 windows open when I'm programming and these are 80 characters wide so I can minimise overlaps. With the suggested layout this would only work for programs less than ~40 characters wide! Here's how the excerpt looks for me (and this example has very short lines -- most of my programs use 72 or more characters per line for better commentary): ---> mt91.rex_nr_1_via_JSR223 R1 T1 A1 3 *-* t=.Test~new R1 T1 A2 V1 1* 21 *-* say "arrived in:" .context~name arrived in: INIT R1 T1 A2 V1 1* 22 *-* counter=0 R1 T1 A1 >>> "a TEST" R1 T1 A1 4 *-* t~m1 R1 T1 A3 V1 1* 27 *-* counter+=1 -- increase counter R1 T1 A3 V1 1* 28 *-* say "arrived in:" .context~name "before reply" Almost any line of any length will wrap. That's why the trace headers in Rexx are kept as short as feasible. Adding an unexplained 27 characters on the front of each line makes little sense, especially as the information is the same on most lines, and as I mentioned before is not user-friendly (here I mean 'user' as being a writer of Rexx programs, not someone who runs a Rexx program without looking at it or caring which language it is written in). Mike Multithreading trace output activated: ---> mt91.rex_nr_1_via_JSR223 R1 T1 A13 *-* t=.Test~new R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name arrived in: INIT R1 T1 A2V1 1* 22 *-* counter=0 R1 T1 A1 >>> "a TEST" R1 T1 A14 *-* t~m1 R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name "before reply" ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Lurker here... If I understand correctly, these thread numbers would be from the underlying OS (UNIX, Windows, z/OS...)? If so, you should be aware that on z/OS thread numbers are *huge*. What they've done under the covers is obviously to encode other info (TCB address or something?) in the high part of a 64-bit integer, so just looking at a random process on a system here, the thread number in decimal is 1512391438145421314, which in hex is 04FD 1800 0002, so the underlying thread is obviously 2. But that's not an exposed interface, so you'd have to find space for at least 19 digits! But perhaps I misunderstand, and you're mapping OS thread numbers into local-to-REXX ones, which of course can be of a controlled size of your choosing. Tony H. On Wed, 15 Feb 2023 at 14:00, Rick McGuire wrote: > I’m in complete agreement with Mike on this. There are better ways to make > this sort of information available than trying to force fit it In to > trace. > > Rick > > On Wed, Feb 15, 2023 at 12:58 PM Mike Cowlishaw > wrote: > >> Thanks for the multiple examples! >> >> As for the 'spaced out' case (excerpt below) ... this really would not >> work for me. I often have 5-9 windows open when I'm programming and these >> are 80 characters wide so I can minimise overlaps. With the suggested >> layout this would only work for programs less than ~40 characters wide! >> Here's how the excerpt looks for me (and this example has very short lines >> -- most of my programs use 72 or more characters per line for better >> commentary): >> >> ---> mt91.rex_nr_1_via_JSR223 >> R1 T1 A13 *-* t=.Test~new >> R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name >> arrived in: INIT >> R1 T1 A2V1 1* 22 *-* counter=0 >> R1 T1 A1 >>> "a TEST" >> R1 T1 A14 *-* t~m1 >> R1 T1 A3V1 1* 27 *-* counter+=1 -- increase >> counter >> R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name >> "before reply" >> >> Almost any line of any length will wrap. That's why the trace headers in >> Rexx are kept as short as feasible. Adding an unexplained 27 characters on >> the front of each line makes little sense, especially as the information is >> the same on most lines, and as I mentioned before is not user-friendly >> (here I mean 'user' as being a writer of Rexx programs, not someone who >> runs a Rexx program without looking at it or caring which language it is >> written in). >> >> Mike >> >> >> >> >> >> >> >> Multithreading trace output activated: >> >> ---> mt91.rex_nr_1_via_JSR223 >> R1 T1 A13 *-* t=.Test~new >> R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name >> arrived in: INIT >> R1 T1 A2V1 1* 22 *-* counter=0 >> R1 T1 A1 >>> "a TEST" >> R1 T1 A14 *-* t~m1 >> R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter >> R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name >> "before reply" >> >> ___ >> Oorexx-devel mailing list >> Oorexx-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/oorexx-devel >> > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
I’m in complete agreement with Mike on this. There are better ways to make this sort of information available than trying to force fit it In to trace. Rick On Wed, Feb 15, 2023 at 12:58 PM Mike Cowlishaw wrote: > Thanks for the multiple examples! > > As for the 'spaced out' case (excerpt below) ... this really would not > work for me. I often have 5-9 windows open when I'm programming and these > are 80 characters wide so I can minimise overlaps. With the suggested > layout this would only work for programs less than ~40 characters wide! > Here's how the excerpt looks for me (and this example has very short lines > -- most of my programs use 72 or more characters per line for better > commentary): > > ---> mt91.rex_nr_1_via_JSR223 > R1 T1 A13 *-* t=.Test~new > R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name > arrived in: INIT > R1 T1 A2V1 1* 22 *-* counter=0 > R1 T1 A1 >>> "a TEST" > R1 T1 A14 *-* t~m1 > R1 T1 A3V1 1* 27 *-* counter+=1 -- increase > counter > R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name > "before reply" > > Almost any line of any length will wrap. That's why the trace headers in > Rexx are kept as short as feasible. Adding an unexplained 27 characters on > the front of each line makes little sense, especially as the information is > the same on most lines, and as I mentioned before is not user-friendly > (here I mean 'user' as being a writer of Rexx programs, not someone who > runs a Rexx program without looking at it or caring which language it is > written in). > > Mike > > > > > > > > Multithreading trace output activated: > > ---> mt91.rex_nr_1_via_JSR223 > R1 T1 A13 *-* t=.Test~new > R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name > arrived in: INIT > R1 T1 A2V1 1* 22 *-* counter=0 > R1 T1 A1 >>> "a TEST" > R1 T1 A14 *-* t~m1 > R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter > R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name "before > reply" > > ___ > Oorexx-devel mailing list > Oorexx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/oorexx-devel > ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
Re: [Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Thanks for the multiple examples! As for the 'spaced out' case (excerpt below) ... this really would not work for me. I often have 5-9 windows open when I'm programming and these are 80 characters wide so I can minimise overlaps. With the suggested layout this would only work for programs less than ~40 characters wide! Here's how the excerpt looks for me (and this example has very short lines -- most of my programs use 72 or more characters per line for better commentary): ---> mt91.rex_nr_1_via_JSR223 R1 T1 A13 *-* t=.Test~new R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name arrived in: INIT R1 T1 A2V1 1* 22 *-* counter=0 R1 T1 A1 >>> "a TEST" R1 T1 A14 *-* t~m1 R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name "before reply" Almost any line of any length will wrap. That's why the trace headers in Rexx are kept as short as feasible. Adding an unexplained 27 characters on the front of each line makes little sense, especially as the information is the same on most lines, and as I mentioned before is not user-friendly (here I mean 'user' as being a writer of Rexx programs, not someone who runs a Rexx program without looking at it or caring which language it is written in). Mike Multithreading trace output activated: ---> mt91.rex_nr_1_via_JSR223 R1 T1 A13 *-* t=.Test~new R1 T1 A2V1 1* 21 *-* say "arrived in:" .context~name arrived in: INIT R1 T1 A2V1 1* 22 *-* counter=0 R1 T1 A1 >>> "a TEST" R1 T1 A14 *-* t~m1 R1 T1 A3V1 1* 27 *-* counter+=1 -- increase counter R1 T1 A3V1 1* 28 *-* say "arrived in:" .context~name "before reply" ___ Oorexx-devel mailing list Oorexx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/oorexx-devel
[Oorexx-devel] Musings with tracing multithreaded ooRexx programs, mt91.rex: on two Rexx interpreter instances (RII)
Here the code of "mt91.rex", changes (compared to "mt90.rex") in bold, i.e. commenting : say "--->" .context~name trace r t=.Test~new t~m1 t~m2 *-- t~block -- wait for both methods to complete* *t**~**start(**"M1"**) **-- asynchroneous! **t**~**start(**"M2"**) **-- asynchroneous!* *-- t~block -- wait for both methods to complete* t~start("M1") t~start("M2") *-- t~block -- wait for both methods to complete* ::class Test-- a test class ::attribute counter-- an attribute/an object variable ::method init-- constructor expose counter-- must be very first statement trace a say "arrived in:" .context~name counter=0 ::method m1-- guarded method expose counter-- must be very first statement trace a counter+=1 -- increase counter say "arrived in:" .context~name"before reply" call syssleep getSleepTime()-- randomly sleep reply tsk=1 -- to show up in trace call syssleep getSleepTime()-- randomly sleep tsk=2 -- to show up in trace ::method m2unguarded -- unguarded method trace a say "arrived in:" .context~name"before reply" call syssleep getSleepTime()-- randomly sleep reply call syssleep getSleepTime()-- randomly sleep self~counter=self~counter-1 -- decrease counter ::method block-- block caller, if counter<>0 expose counter-- must be very first statement guard on when counter=0 -- attribute used as control variable return 99 ::routine getSleepTime t=random(1,10) return t/1000 --- The following traces occurred from /*mt91.rex *//*running on two different Rexx interpreter instances (RII)!*/ Trace output without thread number (start of the 'mt91.rex' program on its RII in bold, includes invocation sequence number): *---> mt91.rex_nr_1_via_JSR223* 3 *-* t=.Test~new 21 *-* say "arrived in:" .context~name arrived in: INIT 22 *-* counter=0 >>> "a TEST" 4 *-* t~m1 27 *-* counter+=1 -- increase counter 28 *-* say "arrived in:" .context~name "before reply" arrived in: M1 before reply 29 *-* call syssleep getSleepTime()-- randomly sleep 30 *-* reply 5 *-* t~m2 >I> Method "M1" with scope "TEST" in package "mt91.rex_nr_1_via_JSR223". 37 *-* say "arrived in:" .context~name "before reply" arrived in: M2 before reply 31 *-* tsk=1 -- to show up in trace 38 *-* call syssleep getSleepTime() -- randomly sleep 32 *-* call syssleep getSleepTime()-- randomly sleep 39 *-* reply 33 *-* tsk=2 -- to show up in trace 7 *-* t~start("M1")-- asynchroneous! >I> Method "M2" with scope "TEST" in package "mt91.rex_nr_1_via_JSR223". >>> "a Message" 27 *-* counter+=1 -- increase counter 40 *-* call syssleep getSleepTime() -- randomly sleep 8 *-* t~start("M2")-- asynchroneous! 28 *-* say "arrived in:" .context~name "before reply" arrived in: M1 before reply >>> "a Message" 37 *-* say "arrived in:" .context~name "before reply" arrived in: M2 before reply 29 *-* call syssleep getSleepTime()-- randomly sleep 10 *-* t~start("M1") 41 *-* self~counter=self~counter-1 -- decrease counter 38 *-* call syssleep getSleepTime() -- randomly sleep >>> "a Message" 11 *-* t~start("M2") >>> "a Message" 37 *-* say "arrived in:" .context~name "before reply" arrived in: M2 before reply 30 *-* reply 39 *-* reply 38 *-* call syssleep getSleepTime() -- randomly sleep >I> Method "M1" with scope "TEST" in package "mt91.rex_nr_1_via_JSR223". >I> Method "M2" with scope "TEST" in package "mt91.rex_nr_1_via_JSR223". 31 *-* tsk=1 -- to show up in trace 40 *-* call syssleep getSleepTime() -- randomly sleep 39 *-* reply 32 *-* call syssleep getSleepTime()-- randomly sleep >I> Method "M2" with scope "TEST" in package "mt91.rex_nr_1_via_JSR223". 40 *-* call syssleep getSleepTime() -- randomly sleep 41 *-* self~counter=self~counter-1 -- decrease counter 33 *-* tsk=2 -- to show up in trace 27 *-* counter+=1 -- increase counter 28 *-* say "arrived in:" .context~name "before reply" *---> mt91.rex_nr_2_via_JSR223* arrived in: M1 before reply 3 *-* t=.Test~new 29 *-* call syssleep getSleepTime()-- randomly sleep 21 *-* say "arrived in:" .context~name 41 *-* self~counter=self~counter-1 -- decrease counter arrived in: INIT 22 *-* counter=0 >>> "a TEST" 4 *-* t~m1 27 *-* counter+=1 -- increase counter 28 *-* say "arrived in:" .context~name "before reply" arrived in: M1 before reply 29 *-* call syssleep getSleepTime()-- randomly sleep