[Tcl Java] Re: leaking memory
"Mo" == Mo DeJong [EMAIL PROTECTED] writes: Mo That does seem logical. It looks like the Class refs need to Mo be cleaned up. What do you think of the patch below? The patch looks good. I'll have to wait until tonight to apply it, as I can't seem to apply it to the version that I have available right now. Mo This would tend to happen if you created a lot of Tcl or Java Mo threads and loaded Tcl Blend into them. Is that what you are Mo doing? This also sounds like that mem leak problem from Mo yesterday, do this patch help? The patch is for the Mo ajuba-tclblend-contract-2000-08-01-branch branch in the CVS. Exactly. Each time someone accesses a url a new thread is spawned for the connection and tlcblend is loaded into it. In the test case that I ran, there were probably more than 20 concurrent threads with tclblend loaded into each of them. Mo That seems to happen to me too. Are you saying that you also don't see the cleanup cache method get called at the end of a thread? If not, this would indicate a problem with the tcl implementation not calling these registered cleanup procs. Mo It seems like the new cache Mo cleanup method is getting called when Tcl Blend is loaded into Mo Tcl, but I am not sure what is going on when you load Tcl Mo Blend and Tcl into a JVM. How could we test this sort of Mo thing? I'm not sure what your saying here, and it's probably because I'm used to thinking about this stuff in terms of the way it operates in conjunction with aolserver. In aolserver there is only one case: 1) Connection thread starts. 2) A new tcl interpreter is allocated to the connection thread. 3) jvm attach to the current connection thread. 4) tclblend is initialized for the current connection thread. 4) run tcl scripts with tclblend calls. 5) thread completes. 6) cleanup thread function is called. -- this doesn't happen with aolserver -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
"Mo" == Mo DeJong [EMAIL PROTECTED] writes: Mo There are two cleanup cases. Mo TclThreadCleanup is called when a Tcl thread (one that was not Mo started inside a JVM) is terminated. TclThreadCleanup will Mo just call DetachCurrentThread() to disconnect the Tcl thread Mo from the JVM. Mo JavaCacheCleanup should be called when a Java or Tcl thread is Mo terminated, its job is to clean up the thread local cache or Mo classes. For me these two cases seem to be one in the same. At the end of a connection thread, I need to do both JavaCacheCleanup and DetachCurrentThread. Mo I am not sure which of these you are using because the Mo description below says "Connection thread starts" and then Mo "jvm attach to the current connection thread", that would make Mo me think it is a thread created from Tcl. I think what you're saying is correct. I've always thought of it as aolserver starting a connection thread and allocating a tcl interpreter instance to run in that thread. The jvm attach is done at the start of the thread by a registered proc, and after the jvm is attached to the connection thread, tclblend is initialized. 1) Connection thread starts. 2) A new tcl interpreter is allocated to the connection thread. 3) jvm attach to the current connection thread. 4) tclblend is initialized for the current connection thread. 4) run tcl scripts with tclblend calls. 5) thread completes. 6) cleanup thread function is called. -- this doesn't happen with aolserver Mo The problem I was seeing is that JavaCacheCleanup was getting Mo called when I exited Tcl thread but when the JVM exited, I saw Mo no such callback. This might be because the finalizer was not Mo called on exit. I need to fix that problem with the Notifier Mo before I can really test this properly. I would be happy to see it called at the end of a thread, because I don't exit the jvm until aolserver is shutdown. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
On Thu, 12 Oct 2000, Daniel Wickstrom wrote: "Mo" == Mo DeJong [EMAIL PROTECTED] writes: Mo There are two cleanup cases. Mo TclThreadCleanup is called when a Tcl thread (one that was not Mo started inside a JVM) is terminated. TclThreadCleanup will Mo just call DetachCurrentThread() to disconnect the Tcl thread Mo from the JVM. Mo JavaCacheCleanup should be called when a Java or Tcl thread is Mo terminated, its job is to clean up the thread local cache or Mo classes. For me these two cases seem to be one in the same. At the end of a connection thread, I need to do both JavaCacheCleanup and DetachCurrentThread. That would be the case when you created the thread in Tcl (meaning the JVM did not create the thread). Mo I am not sure which of these you are using because the Mo description below says "Connection thread starts" and then Mo "jvm attach to the current connection thread", that would make Mo me think it is a thread created from Tcl. I think what you're saying is correct. I've always thought of it as aolserver starting a connection thread and allocating a tcl interpreter instance to run in that thread. The jvm attach is done at the start of the thread by a registered proc, and after the jvm is attached to the connection thread, tclblend is initialized. What do you mean by "The jvm attach is done at the start of the thread by a registered proc"? Are you not using: (*javaVM)-AttachCurrentThread() In JavaInitEnv() to attach the Tcl thread to the JVM? This is also the place where this is called. Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL); Perhaps I am just not understanding what you mean. You really should not need to do your own JVM attach, at least that is not something I had considered (ugh, we don't really need another init case). Mo DeJong Red Hat Inc The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
"Mo" == Mo DeJong [EMAIL PROTECTED] writes: Mo What do you mean by "The jvm attach is done at the start of Mo the thread by a registered proc"? Are you not using: Mo (*javaVM)-AttachCurrentThread() Mo In JavaInitEnv() to attach the Tcl thread to the JVM? This is Mo also the place where this is called. Mo Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL); Mo Perhaps I am just not understanding what you mean. You really Mo should not need to do your own JVM attach, at least that is Mo not something I had considered (ugh, we don't really need Mo another init case). I should have elaborated. To use tclblend in aolserver, I basically had to tear apart javaCmd.c and re-implement it in the structure required by aolserver. aolserver provides functions outside of tcl to register a function at the start of a thread. I'm using this aolserver registered function to call AttachCurrentThread. I'm also calling Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL) in my init routine. Other than javaCmd.c, the rest of the tclblend code is the same as what I pulled out of cvs. At some later time, when I understand everything, I'm going to try and restructure everything to use the code in javaCmd.c if possible. If not, I will have one extra 'c' file to support aolserver. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
On Thu, 12 Oct 2000, Daniel Wickstrom wrote: "Mo" == Mo DeJong [EMAIL PROTECTED] writes: Mo What do you mean by "The jvm attach is done at the start of Mo the thread by a registered proc"? Are you not using: Mo (*javaVM)-AttachCurrentThread() Mo In JavaInitEnv() to attach the Tcl thread to the JVM? This is Mo also the place where this is called. Mo Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL); Mo Perhaps I am just not understanding what you mean. You really Mo should not need to do your own JVM attach, at least that is Mo not something I had considered (ugh, we don't really need Mo another init case). I should have elaborated. To use tclblend in aolserver, I basically had to tear apart javaCmd.c and re-implement it in the structure required by aolserver. aolserver provides functions outside of tcl to register a function at the start of a thread. I'm using this aolserver registered function to call AttachCurrentThread. I'm also calling Tcl_CreateThreadExitHandler(TclThreadCleanup, NULL) in my init routine. Other than javaCmd.c, the rest of the tclblend code is the same as what I pulled out of cvs. At some later time, when I understand everything, I'm going to try and restructure everything to use the code in javaCmd.c if possible. If not, I will have one extra 'c' file to support aolserver. If you ask me, you time would be better spent getting the current code into "shape" so that it can be dropped into aolserver as is. The code on the branch is not really set in stone, so if you can provide a reasonable way to add your stuff with only a minimal change (like #ifdef AOLSERVER perhaps), then we could add it in without lots of duplicated functionality. Of course, it is up to you, but in the long run 1 codebase is far far better. One thing that we really should do is document the init cases, we currently have three of four ways a thread can go through the init code. It would be nice to write each one down. Mo DeJong Red Hat Inc The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
If you ask me, you time would be better spent getting the current code into "shape" so that it can be dropped into aolserver as is. The code on the branch is not really set in stone, so if you can provide a reasonable way to add your stuff with only a minimal change (like #ifdef AOLSERVER perhaps), then we could add it in without lots of duplicated functionality. Of course, it is up to you, but in the long run 1 codebase is far far better. I'm all for having one code base. I'll see if I can't work towards merging completely with the tclblend code. When I asked you about this originally, I was under the impression that you didn't really want me to touch any of the tclblend 'c' code. One thing that we really should do is document the init cases, we currently have three of four ways a thread can go through the init code. It would be nice to write each one down. That would be helpful. When I'm sure about the aolserver case, I'll go ahead and document it. -Dan The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
On Thu, 12 Oct 2000, Dan Wickstrom wrote: If you ask me, you time would be better spent getting the current code into "shape" so that it can be dropped into aolserver as is. The code on the branch is not really set in stone, so if you can provide a reasonable way to add your stuff with only a minimal change (like #ifdef AOLSERVER perhaps), then we could add it in without lots of duplicated functionality. Of course, it is up to you, but in the long run 1 codebase is far far better. I'm all for having one code base. I'll see if I can't work towards merging completely with the tclblend code. When I asked you about this originally, I was under the impression that you didn't really want me to touch any of the tclblend 'c' code. I thought you were talking about code that would only apply to aol server stuff. If it is possible to cleanly merge in your Tcl Blend init case, then I am all for that. Of course, I would need to see the patch first :) Mo DeJong Red Hat Inc The TclJava mailing list is sponsored by Scriptics Corporation. To subscribe:send mail to [EMAIL PROTECTED] with the word SUBSCRIBE as the subject. To unsubscribe: send mail to [EMAIL PROTECTED] with the word UNSUBSCRIBE as the subject. To send to the list, send email to '[EMAIL PROTECTED]'. An archive is available at http://www.mail-archive.com/tcljava@scriptics.com
[Tcl Java] Re: leaking memory
On Wed, 11 Oct 2000, Daniel Wickstrom wrote:
Last weekend I ran the merged tclblend/aolserver combination using
apache-bench to make concurrent accesses of multiple urls, and I
noticed that the memory size was growing over time. I think this is
probably due in part to the java info cache not being cleaned up. It
seems that at the end of a thread, the DeleteGlobalRef should be
called for each of the items in the cache.
That does seem logical. It looks like the Class refs need to
be cleaned up. What do you think of the patch below? This
would tend to happen if you created a lot of Tcl or Java
threads and loaded Tcl Blend into them. Is that what
you are doing? This also sounds like that mem leak problem
from yesterday, do this patch help? The patch is for the
ajuba-tclblend-contract-2000-08-01-branch branch in the CVS.
Another thing that I've noticed is that the TclThreadCleanup routine
is not being called. I register this proc one time from the aolserver
startup routine, but I'm wondering if maybe it needs to be registered
at the start of each thread.
-Dan
That seems to happen to me too. It seems like the new cache
cleanup method is getting called when Tcl Blend is loaded into
Tcl, but I am not sure what is going on when you load
Tcl Blend and Tcl into a JVM. How could we test this
sort of thing?
Here is my proposed patch to cleanup the Class refs.
I am also going to attach it to this file because
it is going to get hosed in mail.
Mo DeJong
Red Hat Inc
Index: tcljava/src/native/javaCmd.c
===
RCS file: /home/cvs/external/tcljava/src/native/javaCmd.c,v
retrieving revision 1.9.2.7
diff -u -r1.9.2.7 javaCmd.c
--- javaCmd.c 2000/08/27 08:07:40 1.9.2.7
+++ javaCmd.c 2000/10/12 06:00:15
@@ -104,6 +104,7 @@
static int AddToFieldCache(JNIEnv *env, Tcl_Interp *interp, jfieldID
*addr,
char *name, jclass *class, char *sig);
static voidTclThreadCleanup(ClientData clientData);
+static voidJavaCacheCleanup(ClientData clientData);
/*
*--
@@ -206,7 +207,7 @@
#ifdef TCLBLEND_DEBUG
fprintf(stderr, "TCLBLEND_DEBUG: Tclblend_Init finished\n");
-fprintf(stderr, "TCLBLEND_DEBUG: JavaInitBlend() returned ");
+fprintf(stderr, "TCLBLEND_DEBUG: JavaInitBlend returned ");
if (result == TCL_ERROR) {
fprintf(stderr, "TCL_ERROR");
} else if (result == TCL_OK) {
@@ -393,6 +394,8 @@
* From this point on, deal with the case where Tcl Blend is loaded
from Tcl.
* Check to see if the current process already has a Java VM. If
so, attach
* the current thread to it, otherwise create a new JVM (automatic
thread attach).
+ * In the case where Tcl Blend is loaded into Java, the call to
+ * JNI_GetCreatedJavaVMs will fill in the javaVM pointer with the
JVM pointer.
*/
if (JNI_GetCreatedJavaVMs(javaVM, 1, nVMs) 0) {
@@ -558,7 +561,7 @@
goto error;
}
-} else {
+} else { /* (nVMs == 0) */
#ifdef TCLBLEND_DEBUG
fprintf(stderr, "TCLBLEND_DEBUG: JVM in process, attaching\n");
@@ -734,20 +737,66 @@
static void
TclThreadCleanup(ClientData clientData)
{
-JNIEnv* env;
-ThreadSpecificData *tsdPtr = TCL_TSD_INIT(dataKey);
-
#ifdef TCLBLEND_DEBUG
fprintf(stderr, "TCLBLEND_DEBUG: called TclThreadCleanup\n");
#endif /* TCLBLEND_DEBUG */
-env = tsdPtr-currentEnv;
(*javaVM)-DetachCurrentThread(javaVM);
}
/*
*--
*
+ * JavaCacheCleanup --
+ *
+ * This method will be called when a Tcl or Java thread is finished.
+ * It needs to remove any global cache references so that the
+ * classes and methods can be cleaned up by the JVM.
+ *
+ * Results:
+ * None.
+ *
+ * Side effects:
+ * None.
+ *
+ *--
+ */
+static void
+JavaCacheCleanup(ClientData clientData)
+{
+JNIEnv* env = JavaGetEnv();
+JavaInfo* jcache = JavaGetCache();
+
+/* FIXME: need to add code to check for case where TclThreadCleanup is
called first */
+
+#ifdef TCLBLEND_DEBUG
+fprintf(stderr, "TCLBLEND_DEBUG: called JavaCacheCleanup\n");
+#endif /* TCLBLEND_DEBUG */
+
+/* We need to delete any global refs to Java classes */
+
+(*env)-DeleteGlobalRef(env, jcache-Object);
+(*env)-DeleteGlobalRef(env, jcache-Interp);
+(*env)-DeleteGlobalRef(env, jcache-Command);
+(*env)-DeleteGlobalRef(env, jcache-TclObject);
+(*env)-DeleteGlobalRef(env, jcache-TclException);
+(*env)-DeleteGlobalRef(env, jcache-CommandWithDispose);
+(*env)-DeleteGlobalRef(env, jcache-CObject);
+(*env)-DeleteGlobalRef(env, jcache-Extension);
+(*env)-DeleteGlobalRef(env, jcache-VarTrace);
+(*env)-DeleteGlobalRef(env, jcache-Void);
+
