[Oorexx-devel] Questions, RFI on multithreading

[Rony G. Flatscher]

Hi there,

trying to get BSF4Rexx 4.0 into beta-test shape (32-bit and 64-bit
Windows and Linux versions). Using developed test units I have been able
to assert that it runs stable, with the exception of employing
multithreading heavily.

32-bit means: operating system is 32-bit, hence ooRexx and BSF4Rexx.
Windows is XP SP3, Linux is Ubuntu.
64-bit means: operating system is 64-bit, ooRexx is 64-bit and
BSF4Rexx is 64-bit  Windows is Vista Business, Linux is Ubuntu.

Multithreading in this context means:

* using multiple Rexx threads, each interfacing with Java using
  BSF4Rexx (e.g. 100 Rexx threads per test, in each thread two
  invocations of BSF4Rexx),
* using multiple Java threads, which each uses the same Rexx
  interpreter instance to run off Rexx scripts that themselves
  dispatch multiple Rexx threads, all (Java and Rexx code)
  interfacing with each other using BSF4Rexx (e.g. 23 Java threads,
  each running a Rexx script that creates 29 Rexx threads; summing
  up all threads gives 712 threads, a heavy version may use a total
  of 2,601 threads),
* there are individual Java objects that Rexx interacts with via
  BSF4Rexx,
* there are individual Rexx objects that Java interacts with via
  BSF4Rexx; access to these Rexx objects may be concurrently from
  Rexx threads and Java threads, where the latter get routed via
  BSF4Rexx.


Here are three issues where I would to ask for advice, ideas:

* Scenario 1: 32-bit Windows (not tested on 64-bit Windows),
  exception in rexx.exe:
  o Here I may get the following exception
  o

   rexx.dll!00337328() 
[Frames below may be incorrect and/or missing, no symbols 
loaded for rexx.dll]  
rexx.dll!002ff218() 
rexx.dll!002e9490() 
rexx.dll!00303462() 
rexx.dll!003473c9() 
kernel32.dll!7c80b729() 

Unhandled exception at 0x00337328 in rexx.exe: 0xC005: Access 
violation reading location 0x000c.


or:

   rexx.dll!002e86f8() 
[Frames below may be incorrect and/or missing, no symbols 
loaded for rexx.dll]  
rexx.dll!002e880b() 
rexx.dll!002e9438() 
rexx.dll!0030d2ae() 
rexx.dll!00318639() 
BSF4Rexx.dll!00c44269() 

Unhandled exception at 0x002e86f8 in rexx.exe: 0xC005: Access 
violation reading location 0x028fd522.

  

* Scneario 2: 64-bit Linux, *buffer overflow*: ran the same tests
  as above and in the multithreaded test unit I get a
  non-redirectable output in the shell reading:
  o

r...@ronylinux:/mnt/root_e/rony/dev/bsf/src/source_cc_40/testUnits$ 
rexx testOORexx -R bsf4rexx -s 21 | gvim -
Vim: Reading from stdin...
 buffer overflow detected ***: rexx terminated*
=== Backtrace: =
/lib/libc.so.6(__fortify_fail+0x37)[0x7fb7374ee2c7]
/lib/libc.so.6[0x7fb7374ec170]
/lib/libc.so.6[0x7fb7374eb519]
/lib/libc.so.6(_IO_default_xsputn+0x96)[0x7fb737465426]
/lib/libc.so.6(_IO_vfprintf+0x348d)[0x7fb737437e2d]
/lib/libc.so.6(__vsprintf_chk+0x99)[0x7fb7374eb5b9]
/lib/libc.so.6(__sprintf_chk+0x80)[0x7fb7374eb500]

/mnt/root_e/rony/dev/bsf/src/bsf4rexx/libBSF4Rexx.so(BSF_impl+0x348)[0x7fb734d90fb8]

/mnt/root_e/rony/dev/bsf/src/bsf4rexx/libBSF4Rexx.so(BSF+0x21)[0x7fb734d914e1]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN20RexxNativeActivation17callNativeRoutineEP12RoutineClassP17RexxNativeRoutineP10RexxStringPP10RexxObjectmR15ProtectedObject+0xc2)[0x7fb738634e12]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN12RoutineClass4callEP12RexxActivityP10RexxStringPP10RexxObjectmR15ProtectedObject+0x55)[0x7fb738607495]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN14PackageManager17callNativeRoutineEP12RexxActivityP10RexxStringPP10RexxObjectmR15ProtectedObject+0x62)[0x7fb73864bca2]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN17SystemInterpreter22invokeExternalFunctionEP14RexxActivationP12RexxActivityP10RexxStringPP10RexxObjectmS5_R15ProtectedObject+0x9e)[0x7fb73867e4fe]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN14RexxActivation12externalCallEP10RexxStringmP19RexxExpressionStackS1_R15ProtectedObject+0x13a)[0x7fb738629aba]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN22RexxExpressionFunction8evaluateEP14RexxActivationP19RexxExpressionStack+0x243)[0x7fb73865a003]

/opt/ooRexx/lib/ooRexx/librexx.so.4(_ZN21RexxExpressionMessage8evaluateEP14RexxActivationP19RexxExpressionStack+0x59)[0x7fb73865ac29]
... cut ...
=== Memory map: 
0040-00401000 r-xp  08:01 24936487 

Re: [Oorexx-devel] Questions, RFI on multithreading

[Rony G. Flatscher]

Mark,

thanks.

 You should be able to chase the buffer overflow down fairly easily.

 *** buffer overflow detected ***: rexx terminated
 === Backtrace: =
 /lib/libc.so.6(__fortify_fail+0x37)[0x7fb7374ee2c7]
 /lib/libc.so.6[0x7fb7374ec170]
 /lib/libc.so.6[0x7fb7374eb519]
 /lib/libc.so.6(_IO_default_xsputn+0x96)[0x7fb737465426]
 /lib/libc.so.6(_IO_vfprintf+0x348d)[0x7fb737437e2d]
 /lib/libc.so.6(__vsprintf_chk+0x99)[0x7fb7374eb5b9]
 /lib/libc.so.6(__sprintf_chk+0x80)[0x7fb7374eb500]
 /mnt/root_e/rony/dev/bsf/src/bsf4rexx/libBSF4Rexx.so(BSF_impl+0x348)[0x7fb734d90fb8]

 So it looks like in your BSF() function you are using sprintf() to
 format something
 into a buffer that is too small.  Double your buffer size and see if
 it goes away.

 Or, better, look carefully and what you are formatting and figure out what is
 bigger than you are anticipating.
   
Yes, after having posted it and coming back from lunch I started to go
after that (it may be caused by debug output only).


 For this one:

 Scenario 1: 32-bit Windows (not tested on 64-bit Windows), exception
 in rexx.exe:

 Here I may get the following exception
   
  rexx.dll!00337328() 
 
   [Frames below may be incorrect and/or missing, no symbols loaded for
 rexx.dll]
   rexx.dll!002ff218() 
   rexx.dll!002e9490() 
   rexx.dll!00303462() 
   rexx.dll!003473c9() 
   kernel32.dll!7c80b729()

 You're not going to get anywhere with this, you need to use a debug version
 of the interpreter and BSF so you can see what functions these are.

 If, you have a scenario that produces the exception 100% of the time with
 a non-debug build, and then never produces the exception with a debug
 version - well that's going to be hard.  It also is very rare.  But,
 if it is the case,
 you'll need to at least track the addresses down in the map file and use the
 *.asm files to see exactly where this sequence is.

 That means you'll need to build the intepreter yourself using the release 
 code.
 The map files will be in Win32Rel and the *.asm files are in Win32Rel\ASM.
   
Thanks for this advice!

 If I were tackling this, I'd fix those 2 bugs first, in the order I
 listed, before even thinking about the third one.
   
Again, thanks!

---rony
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july___
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel