[Tcl Java] thread cleanup
I've been investigating the thread cleanup, and it appears that thread cleanup handlers created with Tcl_CreateThreadExitHandler are only called when tcl is shutting down. This is true at least for tcl8.3.2. I haven't looked at tcl8.4. There is an experimental tcl thread interface in tcl8.3.2 that allows a user to control tcl threads from within tcl scripts. I understand that this interface will be included in tcl8.4, and threads created with this interface will call the cleanup handlers created with Tcl_CreateThreadExitHandler. So there should be no problem for threads created within tcl. This is all well and fine, but the problem is that tclblends design conflicts with the order of the cleanup handlers. When a tcl thread exits, it calls these registered cleanup handlers, and then it deletes the interpreter for the thread. The problem in the current design is that the jvm thread is detached, before the interpreter is destroyed. When the intrepter is destroyed it cleans up all of the remainined Tcl_Obj's which results in a call to FreeTclObject. FreeTclObject then tries to call the release method for the TclObject that it represents, and this results in a segfault, because the jvm has already detached from the current thread. I came this while testing in aolserver, and from looking at the tcl source, it appears that you will have the same problem with the regular version of tclblend when using the thread interface in tcl8.4. Threads created from java will not get segfaults, because the cleanup routines will not be called. Of course, you will then have memory leaks. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: possible memory.
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> I ran the tests again, and it looks like everything is working Mo> just great. I can also run some multi-threaded tests that did Mo> not work in the past. Mo> Are there any remaining issues we need to look at before Mo> merging this branch back into the HEAD? We can't fix the ref Mo> count problem right now, we need to get this all merged back Mo> into the HEAD and then we can look at overhauling ref Mo> counting, perhaps on another branch. This sounds promising. As far as threading changes, there was the issue of thread cleanup routine not being called. Other than that and the ref counting changes that you mentioned, I thinks that's it. If I get a chance this weekend, I'll spend some time looking at the thread cleanup issue. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: ajuba branch
>>>>> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes:
Mo> On Thu, 26 Oct 2000, Daniel Wickstrom wrote:
Mo> It could be related to the event queue, I just don't see it on
Mo> my box. A printed stack trace would really help:
Mo> replace:
Mo> catch {$notifier doOneEvent 0} msg
Mo> with:
Mo> java::try {$notifier doOneEvent 0} catch {Exception e} {$e
Mo> printStackTrace}
Mo> If not, could you investigate it a bit more? Perhaps get a
Mo> stack trace and print it to see where things are going wrong.
>> I'll see what I can do. I want to write some tests that will
>> exercise the enw notifer anyway.
Here is the stacktrace:
% source tclblend/javaTimer.test
javaTimer-3.1 JavaTimerProc
Contents of test case:
set t [java::new tests.TimerHandlerTest $notifier 100]
set result [java::field $t value]
java::field $t err true
lappend result [java::try {$notifier doOneEvent 0} catch {Exception e} {$e
printStackTrace}] $msg [java::field $t value]
Result was:
0 1 {unknown java object "java0x1"} 0
Result should have been:
0 1 {java.lang.NullPointerException: TimerHandlerTest} 1
javaTimer-3.1 FAILED
% source tclblend/javaTimer.test
java.lang.NullPointerException: TimerHandlerTest
at tcl.lang.TclPkgInvoker.invokeMethod(TclPkgInvoker.java:81)
at tcl.lang.JavaInvoke.call(JavaInvoke.java:254)
at tcl.lang.JavaInvoke.callMethod(JavaInvoke.java:114)
at tcl.lang.ReflectObject.cmdProc(ReflectObject.java:906)
at tcl.lang.Interp.callCommand(Interp.java:983)
at tcl.lang.Interp.eval(Interp.java:791)
at tcl.lang.JavaTryCmd.cmdProc(JavaTryCmd.java:74)
at tcl.lang.Interp.callCommand(Interp.java:983)
javaTimer-3.1 JavaTimerProc
Contents of test case:
set t [java::new tests.TimerHandlerTest $notifier 100]
set result [java::field $t value]
java::field $t err true
lappend result [java::try {$notifier doOneEvent 0} catch {Exception e} {$e
printStackTrace}] $msg [java::field $t value]
Result was:
0 {} {unknown java object "java0x1"} 1
Result should have been:
0 1 {java.lang.NullPointerException: TimerHandlerTest} 1
javaTimer-3.1 FAILED
%
The TclJava mailing list is sponsored by Scriptics Corporation.
To subscribe:send mail to [EMAIL PROTECTED]
with the word SUBSCRIBE as the subject.
To unsubscribe: send mail to [EMAIL PROTECTED]
with the word UNSUBSCRIBE as the subject.
To send to the list, send email to '[EMAIL PROTECTED]'.
An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: ajuba branch
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes:
Mo> What version of Tcl are you using? This should not show up
Mo> with Tcl 8.3.2 or 8.4. At any rate, it does not matter.
I'm using tcl version 8.3.2.
>> tclblend/javaTimer.test
>>
>>
>> javaTimer-3.1 JavaTimerProc Contents of test case:
>>
>> set t [java::new tests.TimerHandlerTest $notifier 100] set
>> result [java::field $t value] java::field $t err true lappend
>> result [catch {$notifier doOneEvent 0} msg] $msg [java::field
>> $t value]
>>
>> Result was: 0 0 1 0 Result should have been: 0 1
>> {java.lang.NullPointerException: TimerHandlerTest} 1
>> javaTimer-3.1 FAILED
Mo> Humm, I have not seen this error before either. Could this
Mo> have something to do with your Notifier changes?
No these tests were run before I made the notifier changes. This is a
fresh checkout from cvs. Since making the notifier changes, I get the
same errors, so the notifier modifications didn't change anything with
regard to the test results.
Mo> If not, could you investigate it a bit more? Perhaps get a
Mo> stack trace and print it to see where things are going wrong.
I'll see what I can do. I want to write some tests that will exercise
the enw notifer anyway.
-Dan
The TclJava mailing list is sponsored by Scriptics Corporation.
To subscribe:send mail to [EMAIL PROTECTED]
with the word SUBSCRIBE as the subject.
To unsubscribe: send mail to [EMAIL PROTECTED]
with the word UNSUBSCRIBE as the subject.
To send to the list, send email to '[EMAIL PROTECTED]'.
An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: possible memory.
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> Are you up to it? I am going to be working on this object ref Mo> queue thing for some time, so if you could take a whack at the Mo> Notifier it would really help. There is not going to be much Mo> overlap in these changes, so they can be done in parallel. Ok here is a patch to make the notifier thread safe. I've run the test cases on a solaris box, and it still fails with the same test cases that I posted earlier before I made the modifications. Let me know what you think. How is it going with the object ref queue thing? -Dan notifier-patch.txt
[Tcl Java] autoconf 2.14
> "Thomas" == Thomas McKay <[EMAIL PROTECTED]> writes: Thomas> Can someone send me a pointer to where I can download Thomas> autoconf 2.14. I just checked out tcljava from Thomas> sourceforge and would like to make it. There aren't any Thomas> instructions on doing this so I am assuming that I: 1) Thomas> autoconf configure.in 2) configure 3) make You can get autoconf at ftp://ftp.gnu.org, but you shouldn't need it. The source that you checkout from sourceforge already has a valid configure script. You just need to run 1) configure and 2) make. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: ajuba branch
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes:
Mo> I just got a fresh CVS checkout from SF and ran the tests with
Mo> IBM JDK 1.3 (on Linux) and it worked just fine. I don't know
Mo> what to tell ya. You are using the branch tag:
Mo> ajuba-tclblend-contract-2000-08-01-branch
Mo> right?
Ok, I retried my fresh checkout on the solaris box, and it runs ok
except for a few test failures:
585 danw@satdsp14:/home/unix/wickstrom/web/tcljava>make test
pwd is /home/unix/wickstrom/web/tcljava/tests
CLASSPATH is
/usr/local/jdk/jdk1.1.8/lib/classes.zip:/home/unix/wickstrom/web/tcljava/tests:/home/unix/wickstrom/web/tcljava/tclblend.jar:/home/unix/wickstrom/web/tcljava/tcljava.jar
auto_path is /home/unix/wickstrom/web/tcljava /usr/local/tcl/tcl8.3.2/library
/usr/local/tcl/tcl8.3.2 /usr/local/tcl/tcl8.3.2/lib /usr/local/lib
tcljava/ArrayObject.test
tcljava/AutomaticSignature.test
tcljava/BeanEvent.test
tcljava/BeanEventMgr.test
tcljava/ClassRep.test
tcljava/FieldSig.test
tcljava/FuncSig.test
tcljava/JavaBindCmd.test
tcljava/JavaCallCmd.test
tcljava/JavaCastCmd.test
tcljava/JavaDefineClassCmd.test
tcljava/JavaEventCmd.test
tcljava/JavaFieldCmd.test
tcljava/JavaImportCmd.test
tcljava/JavaInfoCmd.test
tcljava/JavaInstanceofCmd.test
tcljava/JavaInvoke.test
tcljava/JavaIsNullCmd.test
tcljava/JavaLoadCmd.test
tcljava/JavaLock.test
java::lock-1.1 error checking
Contents of test case:
list [catch {java::lock} msg] $msg
Result was:
1 {no value given for parameter "javaObj" to "java::lock"}
Result should have been:
1 {wrong # args: should be "java::lock javaObj"}
java::lock-1.1 FAILED
java::lock-1.2 error checking
Contents of test case:
list [catch {java::lock a b} msg] $msg
Result was:
1 {called "java::lock" with too many arguments}
Result should have been:
1 {wrong # args: should be "java::lock javaObj"}
java::lock-1.2 FAILED
java::unlock-1.1 unlock error messages
Contents of test case:
list [catch {java::unlock} msg] $msg
Result was:
1 {no value given for parameter "javaObj" to "java::unlock"}
Result should have been:
1 {wrong # args: should be "java::unlock javaObj"}
java::unlock-1.1 FAILED
java::unlock-1.2 unlock error checking
Contents of test case:
list [catch {java::unlock a b} msg] $msg
Result was:
1 {called "java::unlock" with too many arguments}
Result should have been:
1 {wrong # args: should be "java::unlock javaObj"}
java::unlock-1.2 FAILED
tcljava/JavaNewCmd.test
tcljava/JavaNullCmd.test
tcljava/JavaPropCmd.test
tcljava/JavaThrowCmd.test
tcljava/JavaTryCmd.test
tcljava/PkgInvoker.test
tcljava/PropertySig.test
tcljava/ReflectException.test
tcljava/ReflectObject.test
tcljava/TclEvent.test
tclblend/javaCmd.test
tclblend/javaIdle.test
tclblend/javaInterp.test
tclblend/javaObj.test
tclblend/javaTimer.test
javaTimer-3.1 JavaTimerProc
Contents of test case:
set t [java::new tests.TimerHandlerTest $notifier 100]
set result [java::field $t value]
java::field $t err true
lappend result [catch {$notifier doOneEvent 0} msg] $msg [java::field $t value]
Result was:
0 0 1 0
Result should have been:
0 1 {java.lang.NullPointerException: TimerHandlerTest} 1
javaTimer-3.1 FAILED
tclblend/javaUtil.test
Tests done.
The TclJava mailing list is sponsored by Scriptics Corporation.
To subscribe:send mail to [EMAIL PROTECTED]
with the word SUBSCRIBE as the subject.
To unsubscribe: send mail to [EMAIL PROTECTED]
with the word UNSUBSCRIBE as the subject.
To send to the list, send email to '[EMAIL PROTECTED]'.
An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: ajuba branch
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: >> >> I'm running with the following: >> >> redhat 6.2 tcl8.3.2 tcljava from souceforge ajuba contract >> branch blackdown jdk1.1.8_v1 Mo> That really should work, everything you are using is the same Mo> as my install except I have the CVS version of Tcl. Are you sure that the sourceforge archive is up to date? I tried the same test with the following setup: solaris 2.7 sun jdk1.1.8 tcljava from ajuba branch This time I received an error because the all the files couldn't be found. It seems that there is quite a bit missing from the tests sub-directory. After copying the tests sub-directory from 1.2.6, I get the following: 557 danw@satdsp14:/home/unix/wickstrom/web/tcljava>make test pwd is /home/unix/wickstrom/web/tcljava/tests CLASSPATH is /usr/local/jdk/jdk1.1.8/lib/classes.zip:/home/unix/wickstrom/web/tcljava/tests:/home/unix/wickstrom/web/tcljava/tclblend.jar:/home/unix/wickstrom/web/tcljava/tcljava.jar auto_path is /home/unix/wickstrom/web/tcljava /usr/local/tcl/tcl8.3.2/library /usr/local/tcl/tcl8.3.2 /usr/local/tcl/tcl8.3.2/lib /usr/local/lib tcljava/ArrayObject.test tcljava/AutomaticSignature.test Assertion failed: tsdPtr->initialized, file /home/unix/wickstrom/web/tcljava/src/native/javaCmd.c, line 293 SIGABRT 6* abort (generated by abort(3) routine) si_signo [6]: SIGABRT 6* abort (generated by abort(3) routine) si_errno [0]: Error 0 si_code [-1]: SI_LWP [pid: 24502, uid: 1103] stackbase=FE7C1D5C, stackpointer=FE7C0CE4 Full thread dump: "SIGQUIT handler" (TID:0xfd7002a0, sys_thread_t:0xb3310, state:R, thread_t: t@6, sp:0xfe791a80 threadID:0xfe791dc8, stack_base:0xfe791d5c, stack_size:0x2) prio=0 "Finalizer thread" (TID:0xfd700088, sys_thread_t:0xb3280, state:R, thread_t: t@5, sp:0xfe7c1a80 threadID:0xfe7c1dc8, stack_base:0xfe7c1d5c, stack_size:0x2) prio=1 *current thread* tcl.lang.CObject.finalize(CObject.java:225) "main" (TID:0xfd7000b0, sys_thread_t:0xbcd80, state:R, thread_t: t@1, sp:0xffbedd40 threadID:0x20d88, stack_base:0xffbec080, stack_size:0x80) prio=5 java.lang.StringBuffer.expandCapacity(StringBuffer.java) java.lang.StringBuffer.insert(StringBuffer.java) tcl.lang.JavaInfoCmd.getNameFromClass(JavaInfoCmd.java:655) tcl.lang.FuncSig.getMethodDescription(FuncSig.java:1328) tcl.lang.FuncSig.sortMethods(FuncSig.java:1255) tcl.lang.FuncSig.getAllDeclaredMethods(FuncSig.java:1104) tcl.lang.FuncSig.lookupMethod(FuncSig.java:274) tcl.lang.FuncSig.get(FuncSig.java:241) tcl.lang.JavaInvoke.callStaticMethod(JavaInvoke.java:158) tcl.lang.JavaCallCmd.cmdProc(JavaCallCmd.java:74) tcl.lang.Interp.callCommand(Interp.java:983) Monitor Cache Dump: java.lang.StringBuffer@FD733308/FD7F59F8: owner "main" (0xbcd80, 1 entry) Registered Monitor Dump: Thread queue lock: Name and type hash table lock: String intern lock: JNI pinning lock: JNI global reference lock: BinClass lock: Class loading lock: Java stack lock: Code rewrite lock: Heap lock: owner "main" (0xbcd80, 1 entry) Has finalization queue lock: owner "main" (0xbcd80, 1 entry) Finalize me queue lock: Monitor registry: owner "Finalizer thread" (0xb3280, 1 entry) Abort make: *** [test_tclblend.exec] Error 134 Have you tried doing a clean checkout and build since you started using sourceforge? -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] possible memory.
> "Dan" == Dan Wickstrom <[EMAIL PROTECTED]> writes:
Dan> It appears that the splitList method is only called from
Dan> within the setListFromAny method The setListFromAny method is
Dan> then used in most of the TclList methods. It's not clear to
Dan> me why the TclList object and the underlying Tcl_Obj wouldn't
Dan> get cleaned up.
Further investigation shows that the corresponding setListFromAny call
is made from the TclList.getString method. As an example here is a
section from ArrayObject.java:
case OPT_SET:
if (numArgs != 2) {
throw new TclNumArgsException(interp, optionIdx + 1, argv,
"indexList value");
}
if (convert == false) {
throw new TclException(interp,
"-noconvert flag not allowed for the \"set\" sub-command");
}
indexListObj = argv[optionIdx + 1];
numDims = TclList.getLength(interp, indexListObj);
if (numDims == 0) {
subArrayObj = javaObj;
subArrayClass = dereferenceClassDims(interp, javaClass, 1);
index = 0;
} else {
// Dereference all but the last dimension specified. Set
// the value of index'th element of the "subArrayObj" to
// the value in argv[optionIdx + 2].
subArrayObj = dereferenceArrayDims(interp, javaObj,
numDims, indexListObj);
subArrayClass = dereferenceClassDims(interp, javaClass, numDims);
index = TclInteger.get(interp,
TclList.index(interp, indexListObj, numDims - 1));
}
// Set the value of the index'th element of "subArrayObj" to the value
// in the TclObject argv[optionIdx + 2].
setArrayElt(interp, subArrayObj, subArrayClass, index, argv[optionIdx + 2]);
interp.resetResult();
return;
The TclList.getLength method converts indexListObj to a TclList object
which uses an underlying Tcl_Obj to hold the internal rep. At the end
of this case statement, it returns leaving indexListObj with no
references. The jvm garbage collects indexListObj, but the underlying
Tcl_Obj is left stranded and results in a memory leak.
-Dan
The TclJava mailing list is sponsored by Scriptics Corporation.
To subscribe:send mail to [EMAIL PROTECTED]
with the word SUBSCRIBE as the subject.
To unsubscribe: send mail to [EMAIL PROTECTED]
with the word UNSUBSCRIBE as the subject.
To send to the list, send email to '[EMAIL PROTECTED]'.
An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] thread cleanup - something that works
I was never able to get Tcl_CreateThreadExitHandler to work for either the TclThreadCleanup or the JavaCacheCleanup routines. As a work-around I decided to call both of these cleanup routines at the end of the JavaInterpDeleted function. I've run this setup under load, and other than leaking memory, I've had no problems. I now suspect that the memory leaking is due to a problem with reference counting. I'm currently investigating that issue. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Fwd: TclBlend Ref Counting Bug (GC related)
> "Jiang" == Jiang Wu <[EMAIL PROTECTED]> writes: Thanks, this description helps alot. Jiang> Background: The relationship between the various objects Jiang> are quite tricky. It takes some time to think them Jiang> through. Basically, to reference a Tcl_Obj, the C Jiang> structure representing a Tcl object in the C Tcl Jiang> interpreter, TclBlend creates a TclObject, a Java class Jiang> instance using the following relationship: Jiang> TclObject (tcl.lang.TclObject, Java class) Jiang> CObject (tcl.lang.CObject, Java class) Jiang>a pointer to Tcl_Obj In the case above what does the Tcl_Obj represent? Is it just a native tcl object such as a string, a list, or an array? Jiang> CObject is a subclass of InternalObject Jiang> (tcl.lang.InternalObject). InternalObject is an abstract Jiang> class containing an interface so that TclObject can be used Jiang> as the common interface to not only Tcl_Obj, but also other Jiang> types of TclBlend objects such as ReflectObject, Jiang> TclInteger, etc, all of those are subclass of Jiang> InternalObject. Jiang> On the C side, the C Tcl interpreter needs access to Java Jiang> objects defined in TclBlend, such as ReflectObject, Jiang> TclInteger, etc. TclBlend uses a Tcl_Obj implementation to Jiang> wrap around any TclObject using the following relationship: Jiang> Tcl_Obj (A C struct with function pointers to functions Jiang> that are specifically designed to manipulate TclBlend's Jiang> TclObject. Defined in javaObj.c in TclBlend.) Jiang> reference/pointer to the Java TclObject This makes sense. The Tcl_Obj internal rep always holds a jobject handle to a java TclObject. Jiang> It seems that you can get infinite recusive containment as Jiang> in: Jiang> Tcl_Object Jiang> Tcl_Obj Jiang> Tcl_Object Jiang> Tcl_Obj ... Yeh this was confusing me, especially when I started thinking about the changes you made to the refcounting code. Jiang> TclBlend is smart enough to detect such attempt. When it Jiang> is detected, TclBlend removes the extra level containment Jiang> so that most of the time you will have either: Is that what's going on in the setInternalRep method of the TclObject class? It checks for a CObject internal rep and creates a new reference to the current TclObject instance if the new rep is not of type CObject. Where else is this detection done? Jiang>Tcl_Object Tcl_Obj Jiang> or Jiang>Tcl_Obj Tcl_Object Jiang> The Problem: Jiang> The proposed solution is to never use the GC thread to free Jiang> Tcl_Obj and always use the thread that created the Tcl_Obj Jiang> to free the Tcl_Obj. My initial impression is that this is the correct way to do it, though it can be tricky to get all of the reference counting right. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: merge with aolserver
> "Jiang" == Jiang Wu <[EMAIL PROTECTED]> writes: Jiang> I ran your test on my version of the fix. I don't see any Jiang> crash. One difference in my code is that I didn't remove Jiang> any of the reference counting code that was in the C part Jiang> of TclBlend. I.e. when a Tcl_Obj is created to wrap a Jiang> TclObject, it adds a ref count. Theoretically, this is not Jiang> needed since it already adds a Java global reference. I Jiang> left those code alone because I have not had the time to Jiang> think through the impact on removing these. I've been trying to think about the ref-counting problem, but I find that I get confused about the role of various objects in tclblend. Could someone summarize the roles and relationships for TclObjects, CObjects, ReflectObjects and Tcl_Obj's and if you think there is anything else that is relevant to thinking about reference counting, I would be glad to hear about that also. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> What do you mean by "The jvm attach is done at the start of Mo> the thread by a registered proc"? Are you not using: Mo> (*javaVM)->AttachCurrentThread() Mo> In JavaInitEnv() to attach the Tcl thread to the JVM? This is Mo> also the place where this is called. Mo> Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL); Mo> Perhaps I am just not understanding what you mean. You really Mo> should not need to do your own JVM attach, at least that is Mo> not something I had considered (ugh, we don't really need Mo> another init case). I should have elaborated. To use tclblend in aolserver, I basically had to tear apart javaCmd.c and re-implement it in the structure required by aolserver. aolserver provides functions outside of tcl to register a function at the start of a thread. I'm using this aolserver registered function to call AttachCurrentThread. I'm also calling Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL) in my init routine. Other than javaCmd.c, the rest of the tclblend code is the same as what I pulled out of cvs. At some later time, when I understand everything, I'm going to try and restructure everything to use the code in javaCmd.c if possible. If not, I will have one extra 'c' file to support aolserver. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> There are two cleanup cases. Mo> TclThreadCleanup is called when a Tcl thread (one that was not Mo> started inside a JVM) is terminated. TclThreadCleanup will Mo> just call DetachCurrentThread() to disconnect the Tcl thread Mo> from the JVM. Mo> JavaCacheCleanup should be called when a Java or Tcl thread is Mo> terminated, its job is to clean up the thread local cache or Mo> classes. For me these two cases seem to be one in the same. At the end of a connection thread, I need to do both JavaCacheCleanup and DetachCurrentThread. Mo> I am not sure which of these you are using because the Mo> description below says "Connection thread starts" and then Mo> "jvm attach to the current connection thread", that would make Mo> me think it is a thread created from Tcl. I think what you're saying is correct. I've always thought of it as aolserver starting a connection thread and allocating a tcl interpreter instance to run in that thread. The jvm attach is done at the start of the thread by a registered proc, and after the jvm is attached to the connection thread, tclblend is initialized. >> 1) Connection thread starts. 2) A new tcl interpreter is >> allocated to the connection thread. 3) jvm attach to the >> current connection thread. 4) tclblend is initialized for the >> current connection thread. 4) run tcl scripts with tclblend >> calls. 5) thread completes. 6) cleanup thread function is >> called. <-- this doesn't happen with aolserver Mo> The problem I was seeing is that JavaCacheCleanup was getting Mo> called when I exited Tcl thread but when the JVM exited, I saw Mo> no such callback. This might be because the finalizer was not Mo> called on exit. I need to fix that problem with the Notifier Mo> before I can really test this properly. I would be happy to see it called at the end of a thread, because I don't exit the jvm until aolserver is shutdown. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> That does seem logical. It looks like the Class refs need to Mo> be cleaned up. What do you think of the patch below? The patch looks good. I'll have to wait until tonight to apply it, as I can't seem to apply it to the version that I have available right now. Mo> This would tend to happen if you created a lot of Tcl or Java Mo> threads and loaded Tcl Blend into them. Is that what you are Mo> doing? This also sounds like that mem leak problem from Mo> yesterday, do this patch help? The patch is for the Mo> ajuba-tclblend-contract-2000-08-01-branch branch in the CVS. Exactly. Each time someone accesses a url a new thread is spawned for the connection and tlcblend is loaded into it. In the test case that I ran, there were probably more than 20 concurrent threads with tclblend loaded into each of them. Mo> That seems to happen to me too. Are you saying that you also don't see the cleanup cache method get called at the end of a thread? If not, this would indicate a problem with the tcl implementation not calling these registered cleanup procs. Mo> It seems like the new cache Mo> cleanup method is getting called when Tcl Blend is loaded into Mo> Tcl, but I am not sure what is going on when you load Tcl Mo> Blend and Tcl into a JVM. How could we test this sort of Mo> thing? I'm not sure what your saying here, and it's probably because I'm used to thinking about this stuff in terms of the way it operates in conjunction with aolserver. In aolserver there is only one case: 1) Connection thread starts. 2) A new tcl interpreter is allocated to the connection thread. 3) jvm attach to the current connection thread. 4) tclblend is initialized for the current connection thread. 4) run tcl scripts with tclblend calls. 5) thread completes. 6) cleanup thread function is called. <-- this doesn't happen with aolserver -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] leaking memory
Last weekend I ran the merged tclblend/aolserver combination using apache-bench to make concurrent accesses of multiple urls, and I noticed that the memory size was growing over time. I think this is probably due in part to the java info cache not being cleaned up. It seems that at the end of a thread, the DeleteGlobalRef should be called for each of the items in the cache. Another thing that I've noticed is that the TclThreadCleanup routine is not being called. I register this proc one time from the aolserver startup routine, but I'm wondering if maybe it needs to be registered at the start of each thread. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: merge with aolserver
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> Humm, I think so but I can't remember off hand. The problem Mo> with this "fix" is that it currently does not work. There are Mo> strange crashes when running the test suite. By strange crashes, do you mean that it doesn't consistently crash at the same point in the test suite? -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: merge with aolserver
> "Jiang" == Jiang Wu <[EMAIL PROTECTED]> writes: Jiang> This patch contains my code for the ref count fix. Patch Jiang> is generated against 1.2.6. It passes the test suite Jiang> without any problems. It includes the reflect object bug Jiang> fix, which causes the test suite to core dump. I am Jiang> planning to integrate this into my work product, which is Jiang> using 1.2.6. But with this fix, it makes life much easier Jiang> to write Tcl commands in Java. O.k. Thanks. I'll take a look at these and see if I can apply it to my work. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: merge with aolserver
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Jiang> 1. The work on fixing how TclObject uses Tcl_Obj's Jiang> reference counting. >> Is this detailed earlier in the mailing list archvies? Mo> Humm, I think so but I can't remember off hand. The problem Mo> with this "fix" is that it currently does not work. There are Mo> strange crashes when running the test suite. I was thinking Mo> that it might be best to try using one of the other Mo> implementations we talked about (like the free Tcl object Mo> enque thing) in the short term. After the notifier queue impl Mo> was fixed, things should be working well enough to be merged Mo> back into the HEAD branch. Then, the ref counting stuff could Mo> be worked on in isolation. If you could send me what you have, I would be willing to look into it some more. I've spent a considerable amount of time looking at the reference counting problem this week-end, and I would like to pursue it a little further. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: merge with aolserver
> "Jiang" == Jiang Wu <[EMAIL PROTECTED]> writes: Jiang> I had allocated time to work on this until the end of Jiang> August. Due to various delays, the project was not Jiang> finished in that time frame. I am no longer working on it. Jiang> Looking at the CVS branch, I think the following things Jiang> have to be added before you can get a working version: Has this work been suspended indefinitely? Jiang> 1. The work on fixing how TclObject uses Tcl_Obj's Jiang> reference counting. Is this detailed earlier in the mailing list archvies? -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: global JavaInfo structure in threaded tclblend.
>>>>> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> On Mon, 28 Aug 2000, Daniel Wickstrom wrote: >> I would anticipate some changes to the autoconf files and >> makefile templates, and the addition of some aolserver specific >> 'c' files. Mo> That does not sound too bad. How about a src/aolserver Mo> subdirectory for your .c files? That works for me. >> I don't expect a firm commitment from you to add support for >> aolserver integration, but I'm interested to know if you are >> open to the idea. Your openness to the idea would affect my >> approach to integrating tclblend in aolserver. If however, you >> don't want the additional complication to the tclblend >> distribution, I understand. Mo> Keep in mind that you are going to have to do your own Mo> testing. If (more like when) I change something that breaks Mo> the AOL build, you are going to have to fix it and send in a Mo> patch. If you are interested in taking responsibility for the Mo> AOL parts of the code then I am interested in adding the code. This also works for me. I would be glad to take responsiblity for the aolserver-specific code. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: global JavaInfo structure in threaded tclblend.
>>>>> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> On Mon, 28 Aug 2000, Daniel Wickstrom wrote: >> >>>>> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: >> >> Also, when I get tclblend working in aolserver, would you be >> interested in incorporating the changes in tclblend as a >> conditional compile option to support aolserver? I will try to >> minimize the changes to tclblend as much as possible. Mo> What would you need to change? I can't agree to something Mo> without knowing what it would entail. I would anticipate some changes to the autoconf files and makefile templates, and the addition of some aolserver specific 'c' files. I don't expect a firm commitment from you to add support for aolserver integration, but I'm interested to know if you are open to the idea. Your openness to the idea would affect my approach to integrating tclblend in aolserver. If however, you don't want the additional complication to the tclblend distribution, I understand. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: global JavaInfo structure in threaded tclblend.
>>>>> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> On Wed, 23 Aug 2000, Daniel Wickstrom wrote: >> I'm curious about the JavaInfo structure in javaCmd.c. When I >> looked at this before, I assumed that you would need a seperate >> JavaInfo structure per thread. The code I'm looking at now >> seems to assume one global JavaInfo structure that can be used >> by all tcl threads. Am I missing something here, or is this >> part of the code still work in progress? I thought I read >> somewhere in the java docs that class and method ids can't be >> shared between java threads. >> >> -Dan Mo> Just a quick update, I moved the cache into thread local Mo> storage and reworked a bunch of stuff to support it. Things Mo> should be fully thread safe now. If you want to take another Mo> peek you are welcome to, of course you might have to wait Mo> until monday morning for the CVS mirror to update. Ok thanks. I'll take a look at it tonight. When do you anticipate finishing your work on the ajuba contract branch? Also, when I get tclblend working in aolserver, would you be interested in incorporating the changes in tclblend as a conditional compile option to support aolserver? I will try to minimize the changes to tclblend as much as possible. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: global JavaInfo structure in threaded tclblend.
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes: Mo> So it looks like you are right, we need to put the method Mo> cache in thread local storage. The contract branch is clearly Mo> not finished, there is a lot of work that still needs to be Mo> done. Ok thanks for the sanity check. Could you summarize the changes that you are going to make to the ajub contract branch? Is there anything else that needs to be put into thread-local storage? It seems that the notifier code will need to change also. Is that correct? -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] global JavaInfo structure in threaded tclblend.
After moving back to the u.s., I've started looking into integrating tclblend into aolserver/nsjava again. Last night I checked out the cvs sources from the cvs ajuba contract branch, and started to compare it to version 1.2.6. I'm curious about the JavaInfo structure in javaCmd.c. When I looked at this before, I assumed that you would need a seperate JavaInfo structure per thread. The code I'm looking at now seems to assume one global JavaInfo structure that can be used by all tcl threads. Am I missing something here, or is this part of the code still work in progress? I thought I read somewhere in the java docs that class and method ids can't be shared between java threads. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: [Tcl Java] Threading in tclblend.
> "Mo" == Mo DeJong <[EMAIL PROTECTED]> writes:
Mo> The startup stuff is kind of tricky because we need to support
Mo> two different kinds of loading. Tcl Blend can be loaded from
Mo> Tcl, whick will then load the JVM. Tcl Blend can also be
Mo> loaded from a JVM, this means Tcl Blend will also need to load
Mo> up Tcl.
I can see tclblend being loaded into tcl when you do something like
load tclblend into tclsh, but how do you end up with the reverse case?
Mo> In the case where Tcl Blend is loaded into Tcl, the
Mo> JavaGetEnv() method is called and this bit of code is
Mo> executed.
Mo> if (currentEnv != NULL) { return currentEnv; }
O.k. you only init these structures once. I see that, but what I
don't understand is what happens when another thread comes along and
uses the JNIEnv pointer stored in currentEnv. My understanding about
this is that for each thread the JNIEnv pointer is different. My work
on nsjava where I've integrated a jvm to work with aolserver's tcl
interpreters has shown me that this is true. Also the class, method,
etc., id's stored in the "java" structure need to be different for
each thread. I think I'm missing something here about the over-all
architecture of tclblend. Is there only one tcl interpreter thread in
tclblend or are there multiple tcl interpreters?
Mo> The tricky part is how Tcl is compiled. If Tcl is compiled
Mo> with threads support enabled, then you could use a Tcl mutex
Mo> to protect the resources. Problem is, some people might not
Mo> want to use the thread enabled version of Tcl. I am kind of
Mo> leaning towards requiring the thread enabled version of Tcl
Mo> for Tcl Blend because I think going for a simple solution and
Mo> avoiding using JNI when we can is a very good thing.
Just my opinion, but I would require that tcl be compiled with
threads. It seems that it would make things alot simpler. Of course
I'm biased, as aolserver will only ever use threaded interpreters :).
What is your current user base like? Do you think alot of people
have a need for a non-threaded interpreter? I know the threaded
version is probably slower, but I wonder if it is that significant.
Mo> We would certainly welcome any suggestions or insights you
Mo> might have on the subject. I hope that we can get all these
Mo> threading issues ironed out in the next 3-4 weeks so that we
Mo> can cut a 1.3.0 release of Jacl and Tcl Blend for people to
Mo> bang on.
As I get into this more, I might have something to offer. In nsjava,
I attach the jvm to the connection thread, so I have one tcl
interpreter per thread, I cache the JNIEnv pointer in a cache that is
indexed with the thread id. The only locking that I do is when I'm
accessing the cache to either put a new env pointer into it or to pull
an env pointer out. I've run this setup under high-load situations
using apache bench, and I'm not seeing any lock contention. Basically
I only support one new tcl command called ns_java which is roughly
equivalent to your java::call tcl commmand. I understand enough about
how aolserver works now, so I could probably extend this to give
nsjava functionality equivalent to tclblend, but I would rather see if
I can merge tclblend into nsjava. Providing aolserver with the
ability to script java components from tcl would be a very powerful
feature that I think many people would find useful for web based
development. I think it would be much better than accessing java
through jsp.
Dan
The TclJava mailing list is sponsored by Scriptics Corporation.
To subscribe:send mail to [EMAIL PROTECTED]
with the word SUBSCRIBE as the subject.
To unsubscribe: send mail to [EMAIL PROTECTED]
with the word UNSUBSCRIBE as the subject.
To send to the list, send email to '[EMAIL PROTECTED]'.
An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Threading in tclblend.
I've been experimenting with integrating tclblend into aolserver, and after looking at the tclblend code, I'm a little puzzled about something. In javaCmd.c the variable java declared as type JavaInfo and currentEnv are declared as global variables, yet I would think that these two variables would be overwritten each time a new thread starts up. Am I missing something? I've noticed using of java locks throughout the code. Maybe only one thread can be executing c-code at any given time? Any insights would be appreciated. Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
