Since last August a crash got reported in <https://sourceforge.net/p/oorexx/bugs/1872/> which can be
easily reproduced by following the directions there. Using the supplied simple C++-only program will
work flawlessly if invoked simply as
testTerminate.exe
it will run flawlessly using 250 Rexx interpreter instances to execute a simple script (issuing a
single dot on stdout) and then terminate it on the same thread.
However if supplying an argument to that test program (source got uploaded with #1872 as well), then
the termination of the Rexx interpreter instance occurs on a separate thread and will cause the
crash upon terminating the second Rexx interpreter instance on a separate thread:
testTerminate.exe
anyArgumentSufficesToRunTheRIIOnSeparateThreadAndCauseTheCrash
The relevant Windows code is:
... cut ...
//
from<https://stackoverflow.com/questions/1981459/using-threads-in-c-on-windows-simple-example>,
//
also<https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread>
// #include <windows.h>
DWORD WINAPI ThreadFunc(void* data) {
// Do stuff. This will be the first function called on the new thread.
// When this function returns, the thread goes away. See MSDN for more
details.
RexxInstance *instance = (RexxInstance *) data;
// uncommenting the following statement, makes it run
*// fprintf(stderr, "ThreadFunc(), arrived: terminating instance
[%p]\n", instance);fflush(stderr); *
// Now wait for the interpreter to terminate and we are done.
instance->Terminate();
// fprintf(stderr, "ThreadFunc(), leaving: terminating instance
[%p]\n", instance);fflush(stderr);
return 0;
... cut ...
if (argc>1) // if argument given terminate on separate thread
{
void *data = interpreter;
HANDLE thread = CreateThread(NULL, 0, ThreadFunc, data, 0,
NULL);
if (thread)
{
fprintf(stderr, "Created [%d], thread [%p] for terminating
instance [%p]\n", i, thread, interpreter);fflush(stderr);
}
}
else // terminate on main thread
{
// Now wait for the interpreter to terminate and we are done.
interpreter->Terminate();
}
... cut ...
Uncommenting the blue fprintf statement makes the crash go away *most* of the time such that all 250
Rexx interpreter instances can be terminated on a separate thread. Reducing that to just
fflush(stderr) makes it go away *most* of the time as well (whereas using fflush(stdout) instead
does not have any effect. So accessing stderr in that thread somehow influences the behaviour).
---
Originally this crash occurred in the context of BSF4ooRexx and I reported it in
<https://sourceforge.net/p/oorexx/feature-requests/806/>. It seems that Rick thought it had to do
with the code in BSF4ooRexx and not with ooRexx. Eventually this thread led to a new, small C++
(supplied with #1872 above) only (only creates a RII, runs a simple Rexx script, terminates the RII)
showcase in which no external dependency could be made responsible for the crash, such that it
becomes clear that the cause lies in ooRexx itself.
As the crash occurs rather early, if using "testTerminate.exe someArg" I had hoped that some of the
C++ gurus would be able to locate (and hopfeully fix) the cause relatively quickly (realizing that
multithreaded bugs can be extremely hard to trace down and to fix). Unfortunately, this has not
happened to this day (after almost 16 months). As this is a showstopper and inhibits using this
great ooRexx feature, namely being able to create any number of different Rexx interpreter instances
(each having its own .local) in a process and terminate them.
Therefore I kindly request any of the C++ experts to look into this matter if possible. Maybe the
upcoming vacation time allows for a few time slots to use for analyzing, finding and hopefully
fixing the cause of this showstopper crash!
---rony
P.S.: Have simplified the test code a little bit, also allowing the test program to run with all
three reported scenarios; it is enclosed as (also slightly stripped down) "testTerminate2.cpp" and
"Makefile2.windows" ("nmake /f Makefil2.windows" will create testTerminate2.exe) which after
successful compilation can be invoked as:
* testTerminate2.exe
o no arguments: will create and terminate all 250 Rexx interpreter
instances (RII) on the same
(main) thread without any crash
* testTerminate2.exe oneArg
o any single argument will cause the crash to always occur while
terminating the *second* RII
on a separate thread (as if terminating the previous RII on a separate
thread left something
in an unbalanced state)
* testTerminate2.exe oneArg secondArg
o any two arguments will cause fflush(stderr) in the terminating thread to
be issued and as a
result most of the time all 250 Rexx interpreter instances can be
terminated on a separate
thread without a crash; running it a couple of times may however cause
that crash to occur
at different RIIs
Terminate() will kick off the garbage collector and the crash seems to occur
during its run.
P.P.S.: Would it be possible to have the termination of a RII carried out on the same thread as its
creation (which seems to run stably) by ooRexx itself (e.g. having a specific thread to create and
to terminate RIIs)?
This may need a function to learn about whether a RII is currently running/blocked and only
terminate it on the creation thread if not in use so not to block the creation thread. Maybe also
(an asynchroneous?) a function to force termination of a RII would be helpful in such a scenario,
maybe with an option that indicates whether running/blocking Rexx code on that RII should be
forcefully halted or not.
// Based on: stackOverflow.cpp ooRexx sample
/**
* A simple example that creates an instance of the interpreter and uses that
* instance to execute a Rexx routine defined in this program.
*
* Both routines cause an infinite recursion. One version of the routine traps
* any error, the other does not. This shows how to handle conditions raised
* inside the interpreter execution, i.e., outside of your native code
* execution.
*
* To have the second routine execute, specify any argument on the command line.
* For example:
*
* stackOverflow 1
*
* With no argument, the first routine is used:
*
* starckOverflow
*/
#include "oorexxapi.h"
#include <stdio.h>
#include <string.h>
#if defined(_WIN32)
#define _CDECL __cdecl
#else
#define _CDECL
#endif
/* Prototypes for several simple helper functions that demonstrate usage of some
* of the native C++ APIs. The functions themselves are at the bottom of the
* file.
*/
bool checkForCondition(RexxThreadContext *c, bool clear);
void printInterpreterVersion(RexxInstance *);
int mainArgc = -1;
int totLoops = 250;
char rexxCode[] = ".error~charOut('.');"; // Rexx code to execute on each
interpreter instance
// from
<https://stackoverflow.com/questions/1981459/using-threads-in-c-on-windows-simple-example>,
// also
<https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createthread>
// #include <windows.h>
DWORD WINAPI ThreadFunc(void* data) {
// Do stuff. This will be the first function called on the new thread.
// When this function returns, the thread goes away. See MSDN for more
details.
RexxInstance *instance = (RexxInstance *) data;
// uncommenting the following statement, makes it run
// fprintf(stderr, "ThreadFunc(), arrived: terminating instance [%p]\n",
instance);fflush(stderr);
if (mainArgc==3) // fflush(stderr) suffices (must be stderr !)
{
fflush(stderr);
}
// Now wait for the interpreter to terminate and we are done.
instance->Terminate();
fprintf(stderr, "<---ThreadFunc()\t\t\Rexx interpreter instance (RII)
[%p] got terminated.--->\n", instance);fflush(stderr);
return 0;
}
int _CDECL main(int argc, char **argv)
{
mainArgc = argc;
// show interpreter version
RexxInstance *interpreter0;
RexxThreadContext *threadContext0;
RexxOption *options0 = NULL;
if ( RexxCreateInterpreter(&interpreter0, &threadContext0, options0) == 0 )
{
printf("Failed to create interpreter, aborting.\n");
exit(-1);
}
printInterpreterVersion(interpreter0);
interpreter0->Terminate();
printf("\nThere %s [%d] argument%s, therefore: \n", (argc==1 ? "is":
"are"), argc, (argc==1 ? "" : "s"));
printf("\t%s: creating total of [%d] interpreters, running [%s] \n",
argv[0], totLoops, rexxCode);
printf("\t\tterminating on %s thread%s\n\n",
(argc==1? "SAME" : "SEPARATE"), (argc==3? " (doing a
'fflush(stderr)' in termination thread)" : "" ) );
fflush(stdout);
for (int i=1;i<=totLoops;i++)
{
// These are the arguments to RexxCreateInterpreter(). An array of any
// number of Rexx options can be passed in, but for this example we do
not
// need any options. So, we use NULL.
RexxInstance *interpreter;
RexxThreadContext *threadContext;
RexxOption *options = NULL;
if ( RexxCreateInterpreter(&interpreter, &threadContext, options) == 0 )
{
printf("Failed to create interpreter, aborting.\n");
exit(1);
}
// printInterpreterVersion(interpreter);
// Create a routine object from the source code.
RexxRoutineObject obj = threadContext->NewRoutine("testTerminate",
rexxCode, strlen(rexxCode));
if ( obj == NULL )
{
checkForCondition(threadContext, true);
printf("Error creating routine object, aborting\n");
exit(1);
}
// Execute the routine.
RexxObjectPtr result = threadContext->CallRoutine(obj, NULL);
// Check for a condition raised during CallRoutine().
checkForCondition(threadContext, true);
if (argc>1) // if argument given terminate on separate thread
{
void *data = interpreter;
HANDLE thread = CreateThread(NULL, 0, ThreadFunc, data, 0, NULL);
if (thread)
{
fprintf(stderr, "Created # [%d], thread [%p] for terminating
instance [%p]\n", i, thread, interpreter);fflush(stderr);
}
else // oops, an error occurred!
{
DWORD errnum = GetLastError();
fprintf(stderr, "ERROR at RII # [%d], thread [%p] for
terminating instance [%p]: GetLastError()=[%d]\n", i, thread, interpreter,
errnum);fflush(stderr);
// from: interpreter\platform\windows\SysRexxUtil.cpp-1228
char *errmsg=NULL;
if (FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM, NULL, errnum, 0, (LPSTR)&errmsg, 64, NULL) == 0)
{
fprintf(stderr, " # [%d], thread [%p]: no
information about error via FormatMessage()\n", i, thread);fflush(stderr);
}
else
{ /* succeeded */
fprintf(stderr, " # [%d], thread [%p]:
FormatMessage()=[%s]\n", i, thread, errmsg);fflush(stderr);
LocalFree(errmsg);
}
}
}
else // terminate on main thread
{
// Now wait for the interpreter to terminate and we are done.
interpreter->Terminate();
}
}
printf("\nDone: created [%d] instances, ran [%s], each terminated on %s
thread%s\n",
totLoops, rexxCode, (argc==1? "the SAME" : "SEPARATE"),
(argc==1?"":"s"));
return 0;
}
/**
* Below are several helper functions that demonstrate how to use some of the
* different C++ native APIs.
*/
/**
* Given an interpreter instance, prints out the interpreter version and
* language version. The documentation in the ooRexx programming guide explains
* the byte encoding of the version numbers.
*/
void printInterpreterVersion(RexxInstance *interpreter)
{
wholenumber_t ver = interpreter->InterpreterVersion();
wholenumber_t lang = interpreter->LanguageLevel();
fprintf(stderr, "Created interpreter instance version=%ld.%ld.%ld language
level=%ld.%02ld\n",
(ver & 0xff0000) >> 16, (ver & 0x00ff00) >> 8, ver & 0x0000ff, (lang
& 0xff00) >> 8, lang & 0x00ff);
fflush(stderr);
}
/**
* Given a condition object, extracts and returns as a whole number the subcode
* of the condition.
*/
inline wholenumber_t conditionSubCode(RexxCondition *condition)
{
return (condition->code - (condition->rc * 1000));
}
/**
* Outputs the typical condition message. For example:
*
* 4 *-* say dt~number
* Error 97 running C:\work\qTest.rex line 4: Object method not found
* Error 97.1: Object "a DateTime" does not understand message "NUMBER"
*
* @param c The thread context we are operating in.
* @param condObj The condition information object. The object returned from
* the C++ API GetConditionInfo()
* @param condition The RexxCondition struct. The filled in struct from the
* C++ API DecodeConditionInfo().
*
* @assumes There is a condition and that condObj and condition are valid.
*/
void standardConditionMsg(RexxThreadContext *c, RexxDirectoryObject condObj,
RexxCondition *condition)
{
RexxObjectPtr list = c->SendMessage0(condObj, "TRACEBACK");
if ( list != NULLOBJECT )
{
RexxArrayObject a = (RexxArrayObject)c->SendMessage0(list, "ALLITEMS");
if ( a != NULLOBJECT )
{
size_t count = c->ArrayItems(a);
for ( size_t i = 1; i <= count; i++ )
{
RexxObjectPtr o = c->ArrayAt(a, i);
if ( o != NULLOBJECT )
{
printf("%s\n", c->ObjectToStringValue(o));
}
}
}
}
printf("Error %zd running %s line %zd: %s\n", condition->rc,
c->CString(condition->program),
condition->position, c->CString(condition->errortext));
printf("Error %zd.%03zd: %s\n", condition->rc,
conditionSubCode(condition), c->CString(condition->message));
}
/**
* Given a thread context, checks for a raised condition, and prints out the
* standard condition message if there is a condition.
*
* @param c Thread context we are operating in.
* @param clear Whether to clear the condition or not.
*
* @return True if there was a condition, otherwise false.
*/
bool checkForCondition(RexxThreadContext *c, bool clear)
{
if ( c->CheckCondition() )
{
RexxCondition condition;
RexxDirectoryObject condObj = c->GetConditionInfo();
if ( condObj != NULLOBJECT )
{
c->DecodeConditionInfo(condObj, &condition);
standardConditionMsg(c, condObj, &condition);
if ( clear )
{
c->ClearCondition();
}
return true;
}
}
return false;
}
#/*----------------------------------------------------------------------------*/
#/*
*/
#/* Copyright (c) 2008-2014 Rexx Language Association. All rights reserved.
*/
#/*
*/
#/* This program and the accompanying materials are made available under
*/
#/* the terms of the Common Public License v1.0 which accompanies this
*/
#/* distribution. A copy is also available at the following address:
*/
#/* https://www.oorexx.org/license.html
*/
#/*
*/
#/* Redistribution and use in source and binary forms, with or
*/
#/* without modification, are permitted provided that the following
*/
#/* conditions are met:
*/
#/*
*/
#/* Redistributions of source code must retain the above copyright
*/
#/* notice, this list of conditions and the following disclaimer.
*/
#/* Redistributions in binary form must reproduce the above copyright
*/
#/* notice, this list of conditions and the following disclaimer in
*/
#/* the documentation and/or other materials provided with the distribution.
*/
#/*
*/
#/* Neither the name of Rexx Language Association nor the names
*/
#/* of its contributors may be used to endorse or promote products
*/
#/* derived from this software without specific prior written permission.
*/
#/*
*/
#/* THIS SOFTWARE IS PROVIDED BY THE COPYright HOLDERS AND CONTRIBUTORS
*/
#/* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
*/
#/* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
*/
#/* FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYright
*/
#/* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
*/
#/* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
*/
#/* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
*/
#/* OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
*/
#/* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
*/
#/* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
*/
#/* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#/*
*/
#/*----------------------------------------------------------------------------*/
# This is a Visual C++, nMake compatible make file.
#
# The compiler needs to be able to find the ooRexx native API headers and
# libraries. If REXX_HOME is set on the system, the correct directory will be
# automatically added to the LIB and INCLUDE environment variables by this make
# file.
#
# Otherwise, either uncomment the next line and define the correct REXX_HOME, or
# be sure the LIB and INCLUDE environment variables allow the compiler to find
# the ooRexx native API headers and libraries.
#
# On a build system for ooRexx, you can set REXX_HOME to point to Win32Dbg or
# Win32Rel, as appropriate, in your build directory. For this to work, you
# will have to have built the interpreter.
#REXX_HOME = C:\work.ooRexx\wc\main\Win32Dbg # For example
# Define RELEASE on the command line, or here, to build a non-debug version. By
# default debug versions are built. I.e. nMake RELEASE=1 /F Makefile
#RELEASE = 1
!IF DEFINED(REXX_HOME)
INCLUDE = $(INCLUDE);$(REXX_HOME)\api
LIB = $(LIB);$(REXX_HOME)\api
!ENDIF
REXX_LIBS = rexx.lib rexxapi.lib
WARNINGFLAGS = /W3 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_NONSTDC_NO_DEPRECATE
!IF DEFINED(RELEASE)
OOREXX_CFLAGS = /nologo /EHsc /O2 /Gs /FAcs /MT $(WARNINGFLAGS) /c
EXE_LFLAGS = /nologo /SUBSYSTEM:Console $(REXX_LIBS) user32.lib comdlg32.lib
gdi32.lib kernel32.lib
DLL_LFLAGS = /nologo /SUBSYSTEM:Windows $(REXX_LIBS) /DLL
!ELSE
OOREXX_CFLAGS = /nologo /EHsc /Zi /Od /MTd $(WARNINGFLAGS) /c
EXE_LFLAGS = /nologo /DEBUG -debugtype:cv /SUBSYSTEM:Console $(REXX_LIBS)
user32.lib comdlg32.lib gdi32.lib kernel32.lib
DLL_LFLAGS = /nologo /DEBUG -debugtype:cv /SUBSYSTEM:Windows $(REXX_LIBS) /DLL
!ENDIF
# What we want to build.
all: testTerminate2.exe
testTerminate2.obj: testTerminate2.cpp
cl $(OOREXX_CFLAGS) testTerminate2.cpp
testTerminate2.exe: testTerminate2.obj
link $(EXE_LFLAGS) testTerminate2.obj bufferoverflowu.lib -out:$(@B).exe
clean:
del *.exe *.dll *.obj *.ilk *.pdb *.lib *.exp *.suo *.cod 1>nul 2>&1
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
